From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 01:13:48 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3ECC106566B; Sun, 7 Mar 2010 01:13:48 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 438518FC12; Sun, 7 Mar 2010 01:13:47 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AvsEAOOMkkuDaFvH/2dsb2JhbACbSXO2WIR4BIMXiyQ X-IronPort-AV: E=Sophos;i="4.49,596,1262581200"; d="scan'208";a="67794263" Received: from danube.cs.uoguelph.ca ([131.104.91.199]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 06 Mar 2010 20:13:47 -0500 Received: from localhost (localhost.localdomain [127.0.0.1]) by danube.cs.uoguelph.ca (Postfix) with ESMTP id 05C5310843E2; Sat, 6 Mar 2010 20:13:47 -0500 (EST) X-Virus-Scanned: amavisd-new at danube.cs.uoguelph.ca Received: from danube.cs.uoguelph.ca ([127.0.0.1]) by localhost (danube.cs.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id lyEyZ0nar1B2; Sat, 6 Mar 2010 20:13:46 -0500 (EST) Received: from muncher.cs.uoguelph.ca (muncher.cs.uoguelph.ca [131.104.91.102]) by danube.cs.uoguelph.ca (Postfix) with ESMTP id 0A3381084106; Sat, 6 Mar 2010 20:13:46 -0500 (EST) Received: from localhost (rmacklem@localhost) by muncher.cs.uoguelph.ca (8.11.7p3+Sun/8.11.6) with ESMTP id o271Q4s18320; Sat, 6 Mar 2010 20:26:05 -0500 (EST) X-Authentication-Warning: muncher.cs.uoguelph.ca: rmacklem owned process doing -bs Date: Sat, 6 Mar 2010 20:26:04 -0500 (EST) From: Rick Macklem X-X-Sender: rmacklem@muncher.cs.uoguelph.ca To: Daniel Braniss In-Reply-To: Message-ID: References: <20100226174021.8feadad9.gerrit@pmp.uni-hannover.de> <20100226224320.8c4259bf.gerrit@pmp.uni-hannover.de> <4B884757.9040001@digiware.nl> <20100227080220.ac6a2e4d.gerrit@pmp.uni-hannover.de> <4B892918.4080701@digiware.nl> <20100227202105.f31cbef7.gerrit@pmp.uni-hannover.de> <20100227193819.GA60576@icarus.home.lan> <4B89943C.70704@digiware.nl> <20100227220310.GA65110@icarus.home.lan> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: stable@freebsd.org, freebsd-fs@freebsd.org, Willem Jan Withagen , =?utf-8?B?R2Vycml0IEvDvGhu?= , =?utf-8?B?RWlyaWsgw5h2ZXJieQ==?= , rwatson@freebsd.org, Jeremy Chadwick Subject: Re: mbuf leakage with nfs/zfs? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 01:13:48 -0000 On Sat, 6 Mar 2010, Daniel Braniss wrote: > it works ok in 7.2, so it would be interesting to compare changes ... > The sys/rpc in FreeBSD8 is completely different code than what is used in FreeBSD7, so I'm afraid thay're apples vs oranges. rick From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 02:06:12 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9DCDA106564A for ; Sun, 7 Mar 2010 02:06:12 +0000 (UTC) (envelope-from tom@hur.st) Received: from ita.aagh.net (ita.aagh.net [208.86.225.114]) by mx1.freebsd.org (Postfix) with ESMTP id 7079C8FC0A for ; Sun, 7 Mar 2010 02:06:12 +0000 (UTC) Received: from cpc1-hart9-2-0-cust900.11-3.cable.virginmedia.com ([86.30.3.133] helo=voi.aagh.net ident=mailnull) by ita.aagh.net with esmtps (TLSv1:AES256-SHA:256) (Exim 4.71 (FreeBSD)) (envelope-from ) id 1No5I2-000GhP-VB; Sun, 07 Mar 2010 01:28:19 +0000 Received: from freaky by voi.aagh.net with local (Exim 4.71 (FreeBSD)) (envelope-from ) id 1No5Hx-0006Hq-Iz; Sun, 07 Mar 2010 01:28:13 +0000 Date: Sun, 7 Mar 2010 01:28:13 +0000 From: Thomas Hurst To: Steven Hartland Message-ID: <20100307012813.GA23944@voi.aagh.net> Mail-Followup-To: Steven Hartland , Jack Vogel , Nick Rogers , stable@freebsd.org References: <147432021003051648x1a1417dfp3c778922ea2c571f@mail.gmail.com> <2a41acea1003051718r241ac3e9w6ceb37bde0128b43@mail.gmail.com> <20100306180029.GA99452@voi.aagh.net> <5EAE9AE93E2D4232B65D2B7D22874CF2@multiplay.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5EAE9AE93E2D4232B65D2B7D22874CF2@multiplay.co.uk> Organization: Not much. User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Nick Rogers , stable@freebsd.org, Jack Vogel Subject: Re: em(4) interface hangs under 8.0-RELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 02:06:12 -0000 * Steven Hartland (killing@multiplay.co.uk) wrote: > If that's a supermicro then em3 is usually the IPMI shared card so perhaps > that's the cause? No, it's a Tyan K8WE: http://www.tyan.com/archive/products/html/thunderk8we.html With a quad port Intel NIC in one of the 133MHz PCI-X slots. There's a Supermicro Marvell 8x SATA card in another slot that's been experiencing ZFS CKSUM errors since updating to 8 too, which is slightly suspicious. Would like to replace the SATA card with a nice simple AHCI board. Sadly nobody seems to make them. -- Thomas 'Freaky' Hurst http://hur.st/ From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 02:26:31 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CE05B106566C for ; Sun, 7 Mar 2010 02:26:31 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.emeryville.ca.mail.comcast.net (qmta09.emeryville.ca.mail.comcast.net [76.96.30.96]) by mx1.freebsd.org (Postfix) with ESMTP id B2F618FC0A for ; Sun, 7 Mar 2010 02:26:31 +0000 (UTC) Received: from omta10.emeryville.ca.mail.comcast.net ([76.96.30.28]) by qmta09.emeryville.ca.mail.comcast.net with comcast id q2MY1d0040cQ2SLA92SY0N; Sun, 07 Mar 2010 02:26:32 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta10.emeryville.ca.mail.comcast.net with comcast id q2SX1d0053S48mS8W2SXpm; Sun, 07 Mar 2010 02:26:31 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 19A5E1E3035; Sat, 6 Mar 2010 18:26:30 -0800 (PST) Date: Sat, 6 Mar 2010 18:26:30 -0800 From: Jeremy Chadwick To: freebsd-stable@freebsd.org Message-ID: <20100307022630.GA30517@icarus.home.lan> References: <147432021003051648x1a1417dfp3c778922ea2c571f@mail.gmail.com> <2a41acea1003051718r241ac3e9w6ceb37bde0128b43@mail.gmail.com> <20100306180029.GA99452@voi.aagh.net> <5EAE9AE93E2D4232B65D2B7D22874CF2@multiplay.co.uk> <20100307012813.GA23944@voi.aagh.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100307012813.GA23944@voi.aagh.net> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: em(4) interface hangs under 8.0-RELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 02:26:31 -0000 On Sun, Mar 07, 2010 at 01:28:13AM +0000, Thomas Hurst wrote: > * Steven Hartland (killing@multiplay.co.uk) wrote: > > > If that's a supermicro then em3 is usually the IPMI shared card so perhaps > > that's the cause? > > No, it's a Tyan K8WE: > > http://www.tyan.com/archive/products/html/thunderk8we.html > > With a quad port Intel NIC in one of the 133MHz PCI-X slots. There's a > Supermicro Marvell 8x SATA card in another slot that's been experiencing > ZFS CKSUM errors since updating to 8 too, which is slightly suspicious. > > Would like to replace the SATA card with a nice simple AHCI board. > Sadly nobody seems to make them. It's been mentioned in the past that for "simple" SATA expansion cards, a good/affordable choice at this point are cards using the Silicon Image 3124/3132/3531 chips (driven by the siis(4) driver). Avoid the 3112. The reason I say that (despite not having any experience with them) is there's active development on that driver by mav@, and SI is apparently fairly forthcoming with docs/quirks. Otherwise Areca cards are a good choice, but are $$$. I personally stick solely to on-board SATA driven by Intel controllers (ICH7/9/10 or ESB), but if I had to get an expansion card, at this point I'd probably go the SI 3124 route. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 11:29:15 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D9A2C106564A for ; Sun, 7 Mar 2010 11:29:15 +0000 (UTC) (envelope-from torfing@broadpark.no) Received: from eterpe-smout.broadpark.no (eterpe-smout.broadpark.no [80.202.8.16]) by mx1.freebsd.org (Postfix) with ESMTP id 9503D8FC1C for ; Sun, 7 Mar 2010 11:29:15 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from ignis-smin.broadpark.no ([unknown] [80.202.8.11]) by eterpe-smout.broadpark.no (Sun Java(tm) System Messaging Server 7u3-12.01 64bit (built Oct 15 2009)) with ESMTP id <0KYW00J24T8PG870@eterpe-smout.broadpark.no> for freebsd-stable@freebsd.org; Sun, 07 Mar 2010 12:29:13 +0100 (CET) Received: from kg-v2.kg4.no ([unknown] [80.203.92.186]) by ignis-smin.broadpark.no (Sun Java(tm) System Messaging Server 7u3-12.01 64bit (built Oct 15 2009)) with SMTP id <0KYW00F3WT8PDYF0@ignis-smin.broadpark.no> for freebsd-stable@freebsd.org; Sun, 07 Mar 2010 12:29:13 +0100 (CET) Date: Sun, 07 Mar 2010 12:29:13 +0100 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Message-id: <20100307122913.2f634018.torfinn.ingolfsen@broadpark.no> In-reply-to: <20100306141944.95ec8cb6.torfinn.ingolfsen@broadpark.no> References: <20100131144217.ca08e965.torfinn.ingolfsen@broadpark.no> <20100306141944.95ec8cb6.torfinn.ingolfsen@broadpark.no> X-Mailer: Sylpheed 2.7.1 (GTK+ 2.18.7; amd64-portbld-freebsd8.0) X-Face: "t9w2,-X@O^I`jVW\sonI3.,36KBLZE*AL[y9lL[PyFD*r_S:dIL9c[8Y>V42R0"!"yb_zN,f#%.[PYYNq; m"_0v; ~rUM2Yy!zmkh)3&U|u!=T(zyv,MHJv"nDH>OJ`t(@mil461d_B'Uo|'nMwlKe0Mv=kvV?Nh@>Hb<3s_z2jYgZhPb@?Wi^x1a~Hplz1.zH Subject: Re: panic - sleeping thread on FreeBSD 8.0-stable / amd64 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 11:29:15 -0000 On Sat, 06 Mar 2010 14:19:44 +0100 Torfinn Ingolfsen wrote: > However, the storage pool is not: > root@kg-f2# zpool status storage > pool: storage > state: UNAVAIL > status: One or more devices are faulted in response to IO failures. > action: Make sure the affected devices are connected, then run 'zpool clear'. > see: http://www.sun.com/msg/ZFS-8000-HC > scrub: scrub completed after 0h0m with 0 errors on Fri Mar 5 18:36:17 2010 > config: > > NAME STATE READ WRITE CKSUM > storage UNAVAIL 0 3 0 insufficient replicas > raidz1 UNAVAIL 0 0 0 insufficient replicas > ad8 ONLINE 0 0 0 > ad10 REMOVED 0 0 0 > ad12 REMOVED 0 0 0 > ad14 ONLINE 0 0 0 > ada0 ONLINE 0 0 0 > > errors: 2 data errors, use '-v' for a list After a reboot, things look like this: root@kg-f2# atacontrol list ATA channel 0: Master: no device present Slave: no device present ATA channel 2: Master: ad4 SATA revision 2.x Slave: no device present ATA channel 3: Master: ad6 SATA revision 2.x Slave: no device present ATA channel 4: Master: ad8 SATA revision 2.x Slave: no device present ATA channel 5: Master: ad10 SATA revision 2.x Slave: no device present ATA channel 6: Master: ad12 SATA revision 2.x Slave: no device present ATA channel 7: Master: ad14 SATA revision 2.x Slave: no device present root@kg-f2# zpool status storage pool: storage state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 raidz1 ONLINE 0 0 0 ad8 ONLINE 0 0 0 ad10 ONLINE 0 0 0 ad12 ONLINE 0 0 0 ad14 ONLINE 0 0 0 ada0 ONLINE 0 0 0 errors: No known data errors and after a scrub: root@kg-f2# zpool scrub storage root@kg-f2# zpool status storage pool: storage state: ONLINE scrub: scrub completed after 0h0m with 0 errors on Sun Mar 7 12:28:31 2010 config: NAME STATE READ WRITE CKSUM storage ONLINE 0 0 0 raidz1 ONLINE 0 0 0 ad8 ONLINE 0 0 0 ad10 ONLINE 0 0 0 ad12 ONLINE 0 0 0 ad14 ONLINE 0 0 0 ada0 ONLINE 0 0 0 errors: No known data errors -- Torfinn From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 11:59:36 2010 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 771B5106566B; Sun, 7 Mar 2010 11:59:36 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 549A48FC17; Sun, 7 Mar 2010 11:59:36 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id CE53346B51; Sun, 7 Mar 2010 06:59:35 -0500 (EST) Date: Sun, 7 Mar 2010 11:59:35 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: current@FreeBSD.org, stable@FreeBSD.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Cc: Subject: net.inet.tcp.timer_race: does anyone have a non-zero value? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 11:59:36 -0000 Dear all: I'm embarking on some new network stack locking work, which requires me to address a number of loose ends in the current model. A few years ago, my attention was drawn to a largly theoretical race, which had existed in the BSD code since inception. It is detected and handled in practice, but relies on type stability of TCP connection data structures, which will need to change in the future due to on-going virtualization work. I didn't fix it at the time, but did add a counter so that we could see if it was happening in the field -- that counter, net.inet.tcp.timer_race, indicates whether or not the stack has detected it happening (and then handled it). This e-mail is to collect the results of that in-the-field survey. Please check the results of the following command: % sysctl net.inet.tcp.timer_race net.inet.tcp.timer_race: 0 If your system shows a non-zero value, please send me a *private e-mail* with the output of that command, plus also the output of "sysctl kern.smp", "uptime", and a brief description of the workload and network interface configuration. For example: it's a busy 8-core web server with roughly X connections/second, and that has three em network interfaces used to load balance from an upstream source. IPSEC is used for management purposes (but not bulk traffic), and there's a local MySQL database. I've already seen one non-zero report, but would be interested in knowing a bit more about the kinds of situations where it's happening so that I can prioritize fixing it appropriately, but also reason about the frequency at which it happens so we can select a fix that avoids adding significant overhead in the common case. Thanks, Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 12:33:45 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C7264106566C; Sun, 7 Mar 2010 12:33:45 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.26]) by mx1.freebsd.org (Postfix) with ESMTP id 08BCF8FC18; Sun, 7 Mar 2010 12:33:44 +0000 (UTC) Received: by ey-out-2122.google.com with SMTP id 25so745605eya.9 for ; Sun, 07 Mar 2010 04:33:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:to:cc:subject:references :organization:from:date:in-reply-to:message-id:user-agent :mime-version:content-type; bh=kKNIkVOPxR143+A1u2d9tHcanxXLh0RN/+EO8gisn3w=; b=JWIsyueZ6pBTOBaeQaHYMEBBNv4y3T6BnmSjMBBnDYFc24dgTlpVLbc+JrEfxUztIj oSEI7xGmeGyr6BzKsGv7G52n6LlH+WJ1rUA1NH9D75ZDKJjfLCNcr024W+7RwkIjV+RP a/CNDQCBBzC4MxPopq7w45Pov2quNf3exeUew= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=to:cc:subject:references:organization:from:date:in-reply-to :message-id:user-agent:mime-version:content-type; b=MmkDl7Yb6XuqVInhEkvNVSL6beRgzy9wYmPi3LjElrFF3SRipaO9gvQ9L4sjRSqICX w5f2GnRMEaiOxi79RZ+Rj50ehMO3f/0di1uWvcmmm4dParNWDXJ/Cz9w6SCQDtU71Que elNnJWEvlLLe+JKdF1EpXtk20yijW0i4YlSa4= Received: by 10.213.1.215 with SMTP id 23mr2100214ebg.52.1267965223741; Sun, 07 Mar 2010 04:33:43 -0800 (PST) Received: from localhost ([95.69.173.225]) by mx.google.com with ESMTPS id 14sm1852248ewy.2.2010.03.07.04.33.42 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 07 Mar 2010 04:33:43 -0800 (PST) To: Robert Watson References: Organization: TOA Ukraine From: Mikolaj Golub Date: Sun, 07 Mar 2010 14:33:40 +0200 In-Reply-To: (Robert Watson's message of "Sun\, 7 Mar 2010 11\:59\:35 +0000 \(GMT\)") Message-ID: <86d3zgcnvf.fsf@kopusha.onet> User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: stable@FreeBSD.org, current@FreeBSD.org Subject: Re: net.inet.tcp.timer_race: does anyone have a non-zero value? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 12:33:45 -0000 On Sun, 7 Mar 2010 11:59:35 +0000 (GMT) Robert Watson wrote: > Please check the results of the following command: > > % sysctl net.inet.tcp.timer_race > net.inet.tcp.timer_race: 0 Are the results for FreeBSD7 look interesting for you? Because currently we have mostly FreeBSD7.1 hosts in production and I observe nonzero values on 8 hosts (about 15%). I would send more details to you privately if you are interested. -- Mikolaj Golub From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 13:42:38 2010 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D1B29106564A; Sun, 7 Mar 2010 13:42:38 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id ABB148FC16; Sun, 7 Mar 2010 13:42:38 +0000 (UTC) Received: from [192.168.2.102] (host86-181-230-35.range86-181.btcentralplus.com [86.181.230.35]) by cyrus.watson.org (Postfix) with ESMTPSA id C0D9546B03; Sun, 7 Mar 2010 08:42:37 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: <86d3zgcnvf.fsf@kopusha.onet> Date: Sun, 7 Mar 2010 13:42:36 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <86d3zgcnvf.fsf@kopusha.onet> To: Mikolaj Golub X-Mailer: Apple Mail (2.1077) Cc: stable@FreeBSD.org, current@FreeBSD.org Subject: Re: net.inet.tcp.timer_race: does anyone have a non-zero value? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 13:42:38 -0000 On Mar 7, 2010, at 12:33 PM, Mikolaj Golub wrote: > On Sun, 7 Mar 2010 11:59:35 +0000 (GMT) Robert Watson wrote: >=20 >> Please check the results of the following command: >>=20 >> % sysctl net.inet.tcp.timer_race >> net.inet.tcp.timer_race: 0 >=20 > Are the results for FreeBSD7 look interesting for you? Because = currently we > have mostly FreeBSD7.1 hosts in production and I observe nonzero = values on 8 > hosts (about 15%). I would send more details to you privately if you = are > interested. Yes, 7.x is also of interest, thanks! Robert= From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 14:55:32 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82C78106564A for ; Sun, 7 Mar 2010 14:55:32 +0000 (UTC) (envelope-from alexeichi@yahoo.de) Received: from smtp124.mail.ukl.yahoo.com (smtp124.mail.ukl.yahoo.com [77.238.184.55]) by mx1.freebsd.org (Postfix) with SMTP id 0CA7C8FC14 for ; Sun, 7 Mar 2010 14:55:31 +0000 (UTC) Received: (qmail 76931 invoked from network); 7 Mar 2010 14:28:51 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.de; h=Received:X-Yahoo-SMTP:X-YMail-OSG:X-Yahoo-Newman-Property:Subject:From:To:Cc:In-Reply-To:References:Content-Type:Date:Message-ID:Mime-Version:X-Mailer; b=XHWLTMe1bo2/cEvBRwW4+zfI8+CoJHLb51uCin6gVQc5YfrU+g6jUlMkpiHYrd1qusEANn+PEfFvMLaT/BQrdQNezQ/hhIkjy5VZxc/4UqhFTbU47bU5DkhMwLfWtbXEJ0HvevB5d0uX0cP4I1SNejEYZvfPT/J5kJcIqH+I6ys= ; Received: from p57BBFC5A.dip.t-dialin.net (alexeichi@87.187.252.90 with login) by smtp124.mail.ukl.yahoo.com with SMTP; 07 Mar 2010 14:28:51 +0000 GMT X-Yahoo-SMTP: yMol3buswBBNNUOSiWLj.UV.YXoMOg-- X-YMail-OSG: D2RZ7kQVM1mt045BB0oz_5WGz6xhqkq6tzfsxiDD4p_AEXS3ENa3BWN9HUOw9zKUCn4FjVSqP30FbdksKZkYGiSAuvlpNHsn2a0_L3aA1NxUO_k010edZHRHb6wJzkhWbxYwBaT5y5UPRRj_vUPjIhoRD0cWiVpfqB56assAfG1npO6TFU9E2cKKhaTN3TMmqtjaJT6GFpYJ9iJL_43L_yhT3sKhwA29lnixnfnL9pbd6pyB3EpgXbDvQO.QMx0k4dgM0be_0..lLJGN7h29g8c4WGM- X-Yahoo-Newman-Property: ymail-3 From: Alexander Eichner To: Mikolaj Golub In-Reply-To: <86zl2l2w9n.fsf@kopusha.onet> References: <86zl2l2w9n.fsf@kopusha.onet> Content-Type: multipart/mixed; boundary="=-kv3VlgqGB/FHWN5JcU0h" Date: Sun, 07 Mar 2010 15:28:48 +0100 Message-ID: <1267972128.2825.2.camel@Prometheus> Mime-Version: 1.0 X-Mailer: Evolution 2.28.1 Cc: alc@freebsd.org, freebsd-emulation@freebsd.org, freebsd-stable@freebsd.org Subject: Re: virtualbox status on 8.0-STABLE i386 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 14:55:32 -0000 --=-kv3VlgqGB/FHWN5JcU0h Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit Hi, can you try the attached patch please? This should fix the panic you encountered. Please undo your kernel changes befoer testing. Thanks. Regards, Alexander Eichner Mikolaj Golub wrote: > Hi, > > Recently I have updated my 8.0-STABLE i386 system and have learnt that > virtualbox begins to crash my box with the error > > panic: vm_fault: fault on nofault entry, addr: c1608000 > > (kgdb) bt > #0 doadump () at pcpu.h:246 > #1 0xc04ec379 in db_fncall (dummy1=-1064468854, dummy2=0, dummy3=-1, dummy4=0xe865d5bc "пуeÐ¥") > at /usr/src/sys/ddb/db_command.c:548 > #2 0xc04ec7af in db_command (last_cmdp=0xc0e04c9c, cmd_table=0x0, dopager=0) > at /usr/src/sys/ddb/db_command.c:445 > #3 0xc04ec864 in db_command_script (command=0xc0e05bc4 "call doadump") > at /usr/src/sys/ddb/db_command.c:516 > #4 0xc04f09a0 in db_script_exec (scriptname=0xe865d6c8 "kdb.enter.panic", warnifnotfound=Variable "warnifnotfound" is not available. > ) > at /usr/src/sys/ddb/db_script.c:302 > #5 0xc04f0a87 in db_script_kdbenter (eventname=0xc0cc248d "panic") at /usr/src/sys/ddb/db_script.c:324 > #6 0xc04ee768 in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_main.c:228 > #7 0xc08d7d06 in kdb_trap (type=3, code=0, tf=0xe865d804) at /usr/src/sys/kern/subr_kdb.c:535 > #8 0xc0beb39b in trap (frame=0xe865d804) at /usr/src/sys/i386/i386/trap.c:690 > #9 0xc0bccd0b in calltrap () at /usr/src/sys/i386/i386/exception.s:165 > #10 0xc08d7e8a in kdb_enter (why=0xc0cc248d "panic", msg=0xc0cc248d "panic") at cpufunc.h:71 > #11 0xc08a88b6 in panic (fmt=0xc0cecbc4 "vm_fault: fault on nofault entry, addr: %lx") > at /usr/src/sys/kern/kern_shutdown.c:562 > #12 0xc0b0c3d7 in vm_fault (map=0xc1990000, vaddr=3244326912, fault_type=Variable "fault_type" is not available. > ) > at /usr/src/sys/vm/vm_fault.c:283 > #13 0xc0bea7d6 in trap_pfault (frame=0xe865dac0, usermode=0, eva=3244330720) > at /usr/src/sys/i386/i386/trap.c:840 > #14 0xc0beb225 in trap (frame=0xe865dac0) at /usr/src/sys/i386/i386/trap.c:533 > #15 0xc0bccd0b in calltrap () at /usr/src/sys/i386/i386/exception.s:165 > #16 0xc12beed0 in rtR0MemObjNativeGetPagePhysAddr (pMem=0xc5ed3110, iPage=0) at pmap.h:300 > #17 0xc12ac354 in SUPR0LockMem (pSession=0xc5c61c10, pvR3=695959552, cPages=1, paPages=0xc5f83668) > at SUPDrv.c:2307 > #18 0xc12ac8cb in supdrvIOCtl (uIOCtl=536892942, pDevExt=0xc12c9ac0, pSession=0xc5c61c10, > pReqHdr=0xc5f83650) at SUPDrv.c:1245 > #19 0xc12b0c3a in VBoxDrvFreeBSDIOCtl (pDev=0xc665d800, ulCmd=536892942, > pvData=0xe865dd00 "ÑŽ8 )\003╬кюq\002", fFile=3, pTd=0xc69556f0) > at /usr/ports/emulators/virtualbox-ose-kmod/work/VirtualBox-3.1.2_OSE/out/freebsd.x86/debug/bin/src/vboxdrv/freebsd/SUPDrv-freebsd.c:505 > #20 0xc0829658 in devfs_ioctl_f (fp=0xc670fa80, com=536892942, data=0xe865dd00, cred=0xc6bbeb00, > td=0xc69556f0) at /usr/src/sys/fs/devfs/devfs_vnops.c:659 > #21 0xc08eec8d in kern_ioctl (td=0xc69556f0, fd=7, com=536892942, data=0xe865dd00 "ÑŽ8 )\003╬кюq\002") > at file.h:262 > #22 0xc08eee14 in ioctl (td=0xc69556f0, uap=0xe865dcf8) at /usr/src/sys/kern/sys_generic.c:678 > #23 0xc0beaad0 in syscall (frame=0xe865dd38) at /usr/src/sys/i386/i386/trap.c:1111 > #24 0xc0bccda0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:261 > #25 0x00000033 in ?? () > Previous frame inner to this frame (corrupt stack?) > (kgdb) fr 16 > #16 0xc12beed0 in rtR0MemObjNativeGetPagePhysAddr (pMem=0xc5ed3110, iPage=0) at pmap.h:300 > 300 pa = (pa & PG_FRAME) | (va & PAGE_MASK); > (kgdb) list > 295 * access the PTE because it would use the new PDE. It is, > 296 * however, safe to use the old PDE because the page table > 297 * page is preserved by the promotion. > 298 */ > 299 pa = KPTmap[i386_btop(va)]; > 300 pa = (pa & PG_FRAME) | (va & PAGE_MASK); > 301 } > 302 return (pa); > 303 } > 304 > > There were some changes in this part recently (r203182): > > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/i386/include/pmap.h.diff?r1=1.140.2.2;r2=1.140.2.3;only_with_tag=RELENG_8 > > So I removed KPTmap[i386_btop(va)] with *vtopte(va) and have working > virtualbox again, but I suppose this is rather the problem with virualbox and > not with the kernel code. > > In February Alexander Eichner posted the patch to freebsd-emulation@ (thread > with the subject "Patch to fix VirtualBox with recent kernel versions"): > > http://lists.freebsd.org/pipermail/freebsd-emulation/2010-February/007434.html > > But it does not fix my panics. The patch adds additional handling in > rtR0MemObjNativeGetPagePhysAddr() for the case > pMem.enmType == RTR0MEMOBJTYPE_MAPPING, while I am observing the panics > for pMem.enmType == RTR0MEMOBJTYPE_LOCK: > > (kgdb) fr 17 > #17 0xc12ac354 in SUPR0LockMem (pSession=0xc5c61c10, pvR3=695959552, cPages=1, paPages=0xc5f83668) > at SUPDrv.c:2307 > 2307 paPages[iPage] = RTR0MemObjGetPagePhysAddr(Mem.MemObj, iPage); > (kgdb) p Mem.MemObj.enmType > $1 = RTR0MEMOBJTYPE_LOCK > > So, it looks like some additional handling should be added for this case... > --=-kv3VlgqGB/FHWN5JcU0h Content-Disposition: attachment; filename="memobj_lockuser_fix.patch" Content-Type: text/x-patch; name="memobj_lockuser_fix.patch"; charset="UTF-8" Content-Transfer-Encoding: 7bit Index: src/VBox/Runtime/r0drv/freebsd/memobj-r0drv-freebsd.c =================================================================== --- src/VBox/Runtime/r0drv/freebsd/memobj-r0drv-freebsd.c (revision 58182) +++ src/VBox/Runtime/r0drv/freebsd/memobj-r0drv-freebsd.c (working copy) @@ -491,7 +491,7 @@ rc = vm_map_wire(&((struct proc *)R0Process)->p_vmspace->vm_map, /* the map */ (vm_offset_t)R3Ptr, /* start */ (vm_offset_t)R3Ptr + cb, /* end */ - VM_MAP_WIRE_USER | VM_MAP_WIRE_NOHOLES); /* flags */ + VM_MAP_WIRE_SYSTEM | VM_MAP_WIRE_NOHOLES); /* flags */ if (rc == KERN_SUCCESS) { pMemFreeBSD->Core.u.Lock.R0Process = R0Process; --=-kv3VlgqGB/FHWN5JcU0h-- From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 15:05:38 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32D44106566C for ; Sun, 7 Mar 2010 15:05:38 +0000 (UTC) (envelope-from n-XrggvsmYZe5ItdnnOBlBqNvZcN094OVi7TluJDIer1Y9stVyOUl4ifkUQURZKj@bounce.linkedin.com) Received: from mail15-a-ac.linkedin.com (mail15-a-ac.linkedin.com [208.111.169.137]) by mx1.freebsd.org (Postfix) with ESMTP id E53568FC15 for ; Sun, 7 Mar 2010 15:05:37 +0000 (UTC) DomainKey-Signature: s=prod; d=linkedin.com; c=nofws; q=dns; h=Sender:Date:From:To:Message-ID:Subject:MIME-Version: Content-Type:X-LinkedIn-fbl; b=iTGUmhz+k+e1slJMhxqZblmdri984V8RUqDbalWw7cdNScFLA0UVGh8H eBk9kCaipdRM4qP0KUSAbbhsiPwTpW5u7rUjDSoLS5f3naUuIBaOotCpk c/x0pMxcuQVhJ6y; DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=linkedin.com; i=invitations@linkedin.com; q=dns/txt; s=proddkim; t=1267974337; x=1299510337; h=from:sender:reply-to:subject:date:message-id:to:cc: mime-version:content-transfer-encoding:content-id: content-description:resent-date:resent-from:resent-sender: resent-to:resent-cc:resent-message-id:in-reply-to: references:list-id:list-help:list-unsubscribe: list-subscribe:list-post:list-owner:list-archive; z=From:=20"Igor=20Lyapin=20(LinkedIn=20Invitations)"=20|Sender:=20messages-noreply@bounce .linkedin.com|Subject:=20Reminder=20about=20your=20invita tion=20from=20Igor=20Lyapin|Date:=20Sun,=207=20Mar=202010 =2007:05:37=20-0800=20(PST)|Message-ID:=20<1676218004.227 24243.1267974337169.JavaMail.app@ech3-cdn09.prod>|To:=20 =20|MIME-Version:=201.0; bh=v54J9gTDw4hAFpTXtE9ZADyIT1nRH7R1XAvzvhp7XOo=; b=X3xpfwF85ZDlbBuPxphWy+vmUbJqbkyR7ehCpTORlc599Kz9telVkW1Z JDVA5s5t1yeeus2HqiWsXQSCaKYusA/5P+sETYSIB5jbUhMaRg0PqH1fe eSlKsLhP7lmTFEa; Sender: messages-noreply@bounce.linkedin.com Date: Sun, 7 Mar 2010 07:05:37 -0800 (PST) From: "Igor Lyapin (LinkedIn Invitations)" To: Message-ID: <1676218004.22724243.1267974337169.JavaMail.app@ech3-cdn09.prod> MIME-Version: 1.0 X-LinkedIn-fbl: XrggvsmYZe5ItdnnOBlBqNvZcN094OVi7TluJDIer1Y9stVyOUl4ifkUQURZKj Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Reminder about your invitation from Igor Lyapin X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 15:05:38 -0000 LinkedIn ------------ This is a reminder that on February 26, Igor Lyapin sent you an invitation to become part of his or her professional network at LinkedIn. Follow this link to accept Igor Lyapin's invitation. https://www.linkedin.com/e/doi/1105542438/kg5abXYk/gir_18199627_0/EML-inv_17_rem/ Signing up is free and takes less than a minute. This is a reminder that on {0}, {1} sent you an invitation to become part of his or her professional network at LinkedIn. February 26, Igor Lyapin wrote: > To: [freebsd-stable@freebsd.org] > From: Igor Lyapin [igor.lyapin@gmail.com] > Subject: Igor Lyapin wants to stay in touch on LinkedIn > > I'd like to add you to my professional network on LinkedIn. > > - Igor Lyapin The only way to get access to Igor Lyapin's professional network is through the following link: https://www.linkedin.com/e/doi/1105542438/kg5abXYk/gir_18199627_0/EML-inv_17_rem/ You can remove yourself from Igor Lyapin's network at any time. -------------- ------ (c) 2010, LinkedIn Corporation From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 16:19:08 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B90F7106564A for ; Sun, 7 Mar 2010 16:19:08 +0000 (UTC) (envelope-from mailnull@mips.inka.de) Received: from mail-in-14.arcor-online.net (mail-in-14.arcor-online.net [151.189.21.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6F2508FC12 for ; Sun, 7 Mar 2010 16:19:08 +0000 (UTC) Received: from mail-in-20-z2.arcor-online.net (mail-in-20-z2.arcor-online.net [151.189.8.85]) by mx.arcor.de (Postfix) with ESMTP id CE2CB28B040 for ; Sun, 7 Mar 2010 17:19:06 +0100 (CET) Received: from mail-in-04.arcor-online.net (mail-in-04.arcor-online.net [151.189.21.44]) by mail-in-20-z2.arcor-online.net (Postfix) with ESMTP id C631D1080D0 for ; Sun, 7 Mar 2010 17:19:06 +0100 (CET) Received: from lorvorc.mips.inka.de (dslb-092-075-206-049.pools.arcor-ip.net [92.75.206.49]) by mail-in-04.arcor-online.net (Postfix) with ESMTPS id 91C7733AA15 for ; Sun, 7 Mar 2010 17:19:06 +0100 (CET) X-DKIM: Sendmail DKIM Filter v2.8.2 mail-in-04.arcor-online.net 91C7733AA15 Received: from lorvorc.mips.inka.de (localhost [127.0.0.1]) by lorvorc.mips.inka.de (8.14.4/8.14.3) with ESMTP id o27GJ6gn006327 for ; Sun, 7 Mar 2010 17:19:06 +0100 (CET) (envelope-from mailnull@lorvorc.mips.inka.de) Received: (from mailnull@localhost) by lorvorc.mips.inka.de (8.14.4/8.14.4/Submit) id o27GJ5pD006326 for freebsd-stable@freebsd.org; Sun, 7 Mar 2010 17:19:05 +0100 (CET) (envelope-from mailnull) From: naddy@mips.inka.de (Christian Weisgerber) Date: Sun, 7 Mar 2010 16:19:05 +0000 (UTC) Message-ID: Originator: naddy@mips.inka.de (Christian Weisgerber) To: freebsd-stable@freebsd.org Subject: ntpd multicast TTL X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 16:19:08 -0000 ntpd is a convenient source of multicast packets for testing purposes. When I add broadcast 224.0.1.1 to my ntp.conf, ntpd sends a multicast packet with TTL 1 every 64 seconds. Just as expected. However, when I explicitly specify the TTL as in broadcast 224.0.1.1 ttl 1 it sends packets with TTL 32. Trying a few other numbers confirms that it multiplies the specified TTL by 32. That is not expected. (I also don't recall this happening the last time I tried it, but that may have been years ago.) Is this simply a bug in ntpd? 7.3-PRERELEASE from yesterday. -- Christian "naddy" Weisgerber naddy@mips.inka.de From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 20:34:39 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 91394106567D for ; Sun, 7 Mar 2010 20:34:39 +0000 (UTC) (envelope-from sam@errno.com) Received: from ebb.errno.com (ebb.errno.com [69.12.149.25]) by mx1.freebsd.org (Postfix) with ESMTP id 4FDC88FC08 for ; Sun, 7 Mar 2010 20:34:39 +0000 (UTC) Received: from ice.local ([10.0.0.115]) (authenticated bits=0) by ebb.errno.com (8.13.6/8.12.6) with ESMTP id o27KYbsP023714 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 7 Mar 2010 12:34:38 -0800 (PST) (envelope-from sam@errno.com) Message-ID: <4B940DDC.7080707@errno.com> Date: Sun, 07 Mar 2010 12:34:36 -0800 From: Sam Leffler User-Agent: Thunderbird 2.0.0.23 (Macintosh/20090812) MIME-Version: 1.0 To: Jim Pingle References: <5fbf03c21002270427s74d0f067gb76cfe10c2794121@mail.gmail.com> <20100227124308.GA28213@mx.techwires.net> <5fbf03c21002270501l2e388ec9tbdad684d38609861@mail.gmail.com> <4B89E174.1060802@pingle.org> <1B1F52CC-FF36-4432-9345-73BB673EE7E8@gmail.com> <4B8AAA0B.4010402@pingle.org> <3BA70425-5413-4E04-AA91-85B699B1D8EF@gmail.com> <4B8B25E0.4060604@pingle.org> <5E979157-9E68-4C08-81BD-14255B038D7F@gmail.com> <4B8B2EEA.6010500@pingle.org> <4B8B48E9.70104@pingle.org> In-Reply-To: <4B8B48E9.70104@pingle.org> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-DCC-x.dcc-servers-Metrics: ebb.errno.com; whitelist Cc: Rui Paulo , freebsd-stable@freebsd.org Subject: Re: FreeBSD-8.0 802.11n support with ath/mwl X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 20:34:39 -0000 Jim Pingle wrote: > On 2/28/2010 10:16 PM, Rui Paulo wrote: >> On 1 Mar 2010, at 03:05, Jim Pingle wrote: >> >>> On 2/28/2010 9:41 PM, Rui Paulo wrote: >>>> On 1 Mar 2010, at 02:26, Jim Pingle wrote: >>>>> Ah, I wasn't aware of wlandebug(8). However, it doesn't seem to operate >>>>> on this mwl(4) card. It sets the value of the sysctl net.wlan.0.debug >>>>> and that doesn't show up on my system. Another system with a ral(4) card >>>>> does have that sysctl. Judging by the information in the wlandebug(8) >>>>> man page it appears as though this may be a side effect of mwl doing >>>>> much of the work in firmware. >>>> wlandebug takes an -i argument. I seem to recall you created your wlan interface named "mwl_wlan0", so you need to type wlandebug -i mwl_wlan0. >>> I saw that, but that is hardcoded to expect wlan (wlan0, wlan1, etc) >>> for an interface name. Having seen that, I recompiled wlandebug without >>> the hardcoded interface name check and it didn't work either, but it did >>> toss an error for the sysctl it was trying to tweak. >> The whole system was designed for the interfaces to start with "wlan" and be named "wlan". > > It does certainly seem to lean that way. Personally I prefer to keep the > hardware name in there, but I wasn't aware that would cause other > issues. Especially when VAPs come into play, I'd rather have, for > example, mwl0_wlan1, mwl0_wlan2, ath0_wlan0, etc, so I can better tie > what goes where. But that's more of a bikeshed of personal preference. :-) Not following the wlanX naming convention will cause confusion for things like rc config files (I believe). Definitely any tool I touched expects vap's to be named wlanX. sysctl net.wlan.X.%parent gives the parent ifnet of a vap; I've considered many times including this in the ifconfig status. Feel free to offer a patch that does this. > >>> That made me look deeper at the code and see it was really just setting >>> the debug sysctl based on flags that wlandebug was aware of. Handy, but >>> the same thing could be done by hand with sysctl and some bitwise math >>> in a pinch, assuming the interface has the right oids. (Which mine >>> doesn't, for some reason...) >> The purpose of wlandebug is to not do any math by hand. > > Indeed, You're 110% right on that. I was just trying to work around the > other issues I was seeing to get to the root of the issue, which seems > to be the missing sysctl oid. > > I need to run some more tests and straighten my antenna issues out, but > I'll report what I find back to the list in a few days. mwl 11n support broke sometime near the last firmware update and I never fixed it. I know in particular there are issues with AMPDU and seq#'s but possible other things too. The fw is rather finicky in how state is managed and it's likely the host is not in sync causing problems w/ the ampdu support and rate control algorithm that both operate in the fw. You should be able to get >100 Mb/s througput on an HT40 channel but I think I was seeing more like 35-40. Turning off ampdu is usually helpful to stabilize operation. Sam From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 21:28:41 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC40D1065670 for ; Sun, 7 Mar 2010 21:28:40 +0000 (UTC) (envelope-from lists@pingle.org) Received: from willow.pingle.org (willow.pingle.org [68.76.213.30]) by mx1.freebsd.org (Postfix) with ESMTP id A3C328FC0A for ; Sun, 7 Mar 2010 21:28:40 +0000 (UTC) Received: from localhost (unknown [127.0.0.1]) by willow.pingle.org (Postfix) with ESMTP id B31E211427; Sun, 7 Mar 2010 16:28:39 -0500 (EST) X-Virus-Scanned: amavisd-new at pingle.org Received: from willow.pingle.org ([127.0.0.1]) by localhost (willow.pingle.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id LqOwwbyJQl9H; Sun, 7 Mar 2010 16:28:38 -0500 (EST) Received: from [192.168.10.10] (hpcw.hpcisp.com [68.76.213.13]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: jim) by willow.pingle.org (Postfix) with ESMTPSA id DA31D1141C; Sun, 7 Mar 2010 16:28:37 -0500 (EST) Message-ID: <4B941A85.5050809@pingle.org> Date: Sun, 07 Mar 2010 16:28:37 -0500 From: Jim Pingle User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.8) Gecko/20100227 Lightning/1.0b1 Thunderbird/3.0.3 MIME-Version: 1.0 To: Sam Leffler References: <5fbf03c21002270427s74d0f067gb76cfe10c2794121@mail.gmail.com> <20100227124308.GA28213@mx.techwires.net> <5fbf03c21002270501l2e388ec9tbdad684d38609861@mail.gmail.com> <4B89E174.1060802@pingle.org> <1B1F52CC-FF36-4432-9345-73BB673EE7E8@gmail.com> <4B8AAA0B.4010402@pingle.org> <3BA70425-5413-4E04-AA91-85B699B1D8EF@gmail.com> <4B8B25E0.4060604@pingle.org> <5E979157-9E68-4C08-81BD-14255B038D7F@gmail.com> <4B8B2EEA.6010500@pingle.org> <4B8B48E9.70104@pingle.org> <4B940DDC.7080707@errno.com> In-Reply-To: <4B940DDC.7080707@errno.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: Rui Paulo , freebsd-stable@freebsd.org Subject: Re: FreeBSD-8.0 802.11n support with ath/mwl X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 21:28:41 -0000 On 3/7/2010 3:34 PM, Sam Leffler wrote: > Not following the wlanX naming convention will cause confusion for > things like rc config files (I believe). Definitely any tool I touched > expects vap's to be named wlanX. Duly noted. I wasn't aware of just how many issues would arise from straying from that convention. > sysctl net.wlan.X.%parent gives the parent ifnet of a vap; I've > considered many times including this in the ifconfig status. Feel free > to offer a patch that does this. That may be handy in the future. My C skills are a bit rusty but if I decide to go down that road I'll surely share any patches I might come up with. > mwl 11n support broke sometime near the last firmware update and I never > fixed it. I know in particular there are issues with AMPDU and seq#'s > but possible other things too. The fw is rather finicky in how state is > managed and it's likely the host is not in sync causing problems w/ the > ampdu support and rate control algorithm that both operate in the fw. > You should be able to get >100 Mb/s througput on an HT40 channel but I > think I was seeing more like 35-40. Turning off ampdu is usually > helpful to stabilize operation. I'll have to try that next chance I get. I did manage to get more antenna pigtails and now have a total of three attached, but it didn't make a difference. A colleague was able to get an N rate association on his mwl card (108Mbps) with similar settings to mine, but with a different client card. It would seem that the problem may lie in the client card in my case, so I'm also investigating options there at the moment. It's an Intel ABGN 5100 card, in case anyone wonders. I also found the mwlstats and mwldebug programs which are in the src tree but not built with the system. A simple "cd /usr/src/tools/tools/mwl; make all install" built them, but mwldebug didn't seem to work for me. Looking at the code I just need to build a kernel or module while having MWL_DEBUG defined to get the sysctls to show up, but I have not yet tried that as I've had a fairly busy week. I'll see if changing the ampdu settings make any difference and report back, and hopefully the rebuild with MWL_DEBUG might also help me get a lead if tweaking ampdu doesn't help. Thanks for the reply, Sam. I appreciate the insight. Jim From owner-freebsd-stable@FreeBSD.ORG Sun Mar 7 22:42:42 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0FFBF106566C for ; Sun, 7 Mar 2010 22:42:42 +0000 (UTC) (envelope-from max@love2party.net) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.17.10]) by mx1.freebsd.org (Postfix) with ESMTP id 971498FC19 for ; Sun, 7 Mar 2010 22:42:41 +0000 (UTC) Received: from vampire.homelinux.org (dslb-088-066-010-096.pools.arcor-ip.net [88.66.10.96]) by mrelayeu.kundenserver.de (node=mreu2) with ESMTP (Nemesis) id 0McRym-1O64Ya1cVk-00HhGC; Sun, 07 Mar 2010 23:42:40 +0100 Received: (qmail 20453 invoked from network); 7 Mar 2010 22:42:40 -0000 Received: from f8x64.laiers.local (192.168.4.188) by ns1.laiers.local with SMTP; 7 Mar 2010 22:42:40 -0000 From: Max Laier Organization: FreeBSD To: Jeremy Chadwick Date: Sun, 7 Mar 2010 23:42:40 +0100 User-Agent: KMail/1.12.4 (FreeBSD/8.0-RELEASE-p2; KDE/4.3.5; amd64; ; ) References: <147432021003051648x1a1417dfp3c778922ea2c571f@mail.gmail.com> <147432021003061456g7072284eved49a93fb19db24c@mail.gmail.com> <20100306233221.GA27176@icarus.home.lan> In-Reply-To: <20100306233221.GA27176@icarus.home.lan> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201003072342.40044.max@love2party.net> X-Provags-ID: V01U2FsdGVkX19W3djvsy+QXGA12VESvUEB8eK3wmI0eTZF/sD OFfH1+CxRDA8GmdZMqGvPnp9kro71YvczoZkDfcwKWX0/y1IKf kt5Ykk/mO3/gGhBRJwBAA== Cc: Nick Rogers , stable@freebsd.org, Jack Vogel Subject: Re: em(4) interface hangs under 8.0-RELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Mar 2010 22:42:42 -0000 On Sunday 07 March 2010 00:32:21 Jeremy Chadwick wrote: > On Sat, Mar 06, 2010 at 02:56:07PM -0800, Nick Rogers wrote: > > ALTQ + RELENG_8 + em(4) will not work at the moment. It does not matter > > what your PF ruleset looks like or how much traffic you are pushing. The > > packets that transit the em interface simply never make it to the ALTQ > > queues (not even the interface's root queue). Thus any kind of bandwidth > > rate limiting or whatever you are doing will not work. > > > > This was fixed by the following commit. I think its supposed to MFC soon? > > http://svn.freebsd.org/viewvc/base?view=revision&revision=203834 > > Nick, thanks for pointing me to this. I'll hold off on doing our > upgrade (on that one server) until said code is backported to RELENG_8. > > Max, will this be MFC'd in about a week? Usually things of this sort > MFC'd after 4 weeks, but I thought I'd get confirmation. As noted in the commit message, I do indeed plan to MFC this in about a week. There are two remaining issues in some drivers that may result in packet reordering (w/ or w/o ALTQ). I was hoping to address this before the MFC, but I didn't yet find the time. Regards, Max From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 01:06:15 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 93E4A106564A for ; Mon, 8 Mar 2010 01:06:15 +0000 (UTC) (envelope-from tom.hurst@clara.net) Received: from ita.aagh.net (ita.aagh.net [208.86.225.114]) by mx1.freebsd.org (Postfix) with ESMTP id 63A518FC12 for ; Mon, 8 Mar 2010 01:06:15 +0000 (UTC) Received: from cpc1-hart9-2-0-cust900.11-3.cable.virginmedia.com ([86.30.3.133] helo=voi.aagh.net ident=mailnull) by ita.aagh.net with esmtps (TLSv1:AES256-SHA:256) (Exim 4.71 (FreeBSD)) (envelope-from ) id 1NoR0c-000K1h-UT; Mon, 08 Mar 2010 00:39:47 +0000 Received: from freaky by voi.aagh.net with local (Exim 4.71 (FreeBSD)) (envelope-from ) id 1NoR0b-000NxY-FM; Mon, 08 Mar 2010 00:39:45 +0000 Date: Mon, 8 Mar 2010 00:39:45 +0000 From: Thomas Hurst To: Jeremy Chadwick Message-ID: <20100308003945.GA90714@voi.aagh.net> Mail-Followup-To: Jeremy Chadwick , freebsd-stable@freebsd.org References: <147432021003051648x1a1417dfp3c778922ea2c571f@mail.gmail.com> <2a41acea1003051718r241ac3e9w6ceb37bde0128b43@mail.gmail.com> <20100306180029.GA99452@voi.aagh.net> <5EAE9AE93E2D4232B65D2B7D22874CF2@multiplay.co.uk> <20100307012813.GA23944@voi.aagh.net> <20100307022630.GA30517@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100307022630.GA30517@icarus.home.lan> Organization: Not much. User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-stable@freebsd.org Subject: Re: em(4) interface hangs under 8.0-RELEASE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 01:06:15 -0000 * Jeremy Chadwick (freebsd@jdc.parodius.com) wrote: > It's been mentioned in the past that for "simple" SATA expansion cards, > a good/affordable choice at this point are cards using the Silicon Image > 3124/3132/3531 chips (driven by the siis(4) driver). Avoid the 3112. > > The reason I say that (despite not having any experience with them) is > there's active development on that driver by mav@, and SI is apparently > fairly forthcoming with docs/quirks. That's interesting, thanks. I'll see if I can find any. > I personally stick solely to on-board SATA driven by Intel controllers > (ICH7/9/10 or ESB), but if I had to get an expansion card, at this point > I'd probably go the SI 3124 route. Yes, I'm planning on replacing my dual Opteron with a single-socket Xeon for more memory/CPU; it has 6 on-board AHCI interfaces, and I guess a simple two port card would do for the rest. -- Thomas 'Freaky' Hurst http://hur.st/ From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 01:12:54 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2704F1065670 for ; Mon, 8 Mar 2010 01:12:54 +0000 (UTC) (envelope-from howard@leadmon.net) Received: from mail.leadmon.net (mail.leadmon.net [173.13.218.154]) by mx1.freebsd.org (Postfix) with ESMTP id 934D08FC12 for ; Mon, 8 Mar 2010 01:12:53 +0000 (UTC) Received: from HDLDESKTOP64 (hdl-desktop-64.leadmon.net [10.0.0.3]) (authenticated bits=0) by mail.leadmon.net (8.14.4/8.14.4/LNSG+SCOP+PSBL+LUBL+NJABL+SBL+DSBL+SORBS+CBL+RHSBL) with ESMTP id o280iicr033449 for ; Sun, 7 Mar 2010 19:44:44 -0500 (EST) (envelope-from howard@leadmon.net) X-DKIM: Sendmail DKIM Filter v2.8.3 mail.leadmon.net o280iicr033449 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=leadmon.net; s=default; t=1268009084; bh=PBSuoJyB7qURpM9TWE1dRxFxumgM1ItG+kBXL3phjJk=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type; b=ChhHZFhTMOgcIeOPYq/vVGKYAA8ME2sE067OAjdMO/e65RQdBkjS33lhITejfbDJc iILjJHhhczdz07kdQJuhse4V3YUDT9qUNIBdJiii+wOf1xAIUK0pldBPAGwjdYUV4s DedPuQI2anjeebcqwmttnaT7iDAeif4wUhlvbB7Q= X-DomainKeys: Sendmail DomainKeys Filter v1.0.2 mail.leadmon.net o280iicr033449 DomainKey-Signature: a=rsa-sha1; s=default; d=leadmon.net; c=simple; q=dns; h=x-senderid:authentication-results:from:to:subject:date: message-id:mime-version:content-type:x-mailer:thread-index:content-language; b=A99qUqM1DUQvHGUc3GDTf5mpMhvZdrFpdf016cbsStJvMqPML0lphFtUcDyl/o3Z3 hI7oXieQ2j/SmvDep/OWrSb6lsNgPkRfNqJIE8p1YNKLgAeaOyIJSQXPKO27Isxfo8a sCPnWWIQJjv+1pPYJy2j+18fSmG1pFmDGcRJiB8= X-SenderID: Sendmail Sender-ID Filter v1.0.0 mail.leadmon.net o280iicr033449 Authentication-Results: mail.leadmon.net; sender-id=pass header.from=howard@leadmon.net; spf=pass smtp.mfrom=howard@leadmon.net From: "Howard Leadmon" To: Date: Sun, 7 Mar 2010 19:44:41 -0500 Message-ID: <000301cabe58$8bbc1880$a3344980$@net> MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook 12.0 Thread-Index: Acq+WIm+LfQRDkRrT9S8l2YftpnHpg== Content-Language: en-us X-Virus-Scanned: clamav-milter 0.95.3 at vorlon.leadmon.net X-Virus-Status: Clean Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: FBSD 8.0 & ZFS Issue.. default_perms_for_dir X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 01:12:54 -0000 I just did a cvsup and updated my amd64 8.0-STABLE machine, and after rebooting with the new kernel and all I am seeing the following on directory access: default_perms_for_dir: sys_acl_get_file(., ACL_TYPE_DEFAULT): Invalid argument, falling back on umask default_perms_for_dir: sys_acl_get_file(., ACL_TYPE_DEFAULT): Invalid argument, falling back on umask Now I know it wasn't doing this before, so something has apparently changed since the update and rebuild. Has anyone run into this issue, or machine something that is known in the current STABLE source tree? Just not sure if this is something I got to worry about, but not really sure how to go about fixing it either.. --- Howard Leadmon From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 03:27:51 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E085F106564A for ; Mon, 8 Mar 2010 03:27:51 +0000 (UTC) (envelope-from brian.j.conway@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 8BF468FC13 for ; Mon, 8 Mar 2010 03:27:51 +0000 (UTC) Received: by vws14 with SMTP id 14so2852942vws.13 for ; Sun, 07 Mar 2010 19:27:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:received:date:from:to :subject:message-id:x-mailer:mime-version:content-type :content-transfer-encoding; bh=Ab5gce35vWzxdRpD8qGDrZJGffhZqnLtQLygZOSRVhY=; b=qngK4Ko8MRK9hlQ1Y3zu3M07QMIAcyglzHUtneWPcAelCbv1Qm1hRhfFZBiK3cLCpj J8oi1Eg7RfPHeW0D5xvefNKRlXbbV1KncOKVUDelGlQ+5BT8IWlnD/TbcFjeuuws689Y 6t1jtQEDu8ieifFMieDtT63/kWuWsAeD7GYgc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:subject:message-id:x-mailer:mime-version :content-type:content-transfer-encoding; b=d+G7/pUUoRhWnnlYtgY53UHTJeV4jPtVEFQehWyn27MpPRRC88i7+JoP7crUnlhRES MUqTzA1deEnwrJroE2S5kT5F6Tb+oDkEMVI0C0jREBi7k1Ihc8mu7lfuO271PBUAWFoe 7gh6idNm+k2sl0/+9k+OBCGkhc10trFuPzvg0= Received: by 10.220.107.72 with SMTP id a8mr2538634vcp.92.1268017205663; Sun, 07 Mar 2010 19:00:05 -0800 (PST) Received: from ladyluck.zxvf.local (c-98-229-162-102.hsd1.ma.comcast.net [98.229.162.102]) by mx.google.com with ESMTPS id 23sm42828215vws.6.2010.03.07.19.00.03 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 07 Mar 2010 19:00:04 -0800 (PST) Sender: Brian Conway Received: from localhost ([127.0.0.1] helo=ladyluck.zxvf.local) by ladyluck.zxvf.local with smtp (Exim 4.69) (envelope-from ) id 1NoTCM-0007w7-LD for freebsd-stable@freebsd.org; Sun, 07 Mar 2010 22:00:02 -0500 Date: Sun, 7 Mar 2010 22:00:02 -0500 From: Brian Conway To: freebsd-stable@freebsd.org Message-Id: <20100307220002.f4b51436.bconway@alum.wpi.edu> X-Mailer: Sylpheed 2.5.0 (GTK+ 2.12.12; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: 8.0-RELEASE-p2: boot0cfg yields "vnode_pager_getpages: I/O read error" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 03:27:52 -0000 Greetings. I'm testing NanoBSD images for eventual use in an Alix board and have run into an issue with the boot0cfg lines in the update scripts, in multiple environments. Specifically, any use of boot0cfg yields a successful output[1], followed by: vnode_pager_getpages: I/O read error At this point, it looks like the command succeeded, but all reads and writes to disk fail: # man init /usr/bin/man: Device not configured. # touch /var/tmp/test /usr/bin/touch: Input/output error. I've tried the following combinations and it happens under all of them: - VM using a file-loop-backed disk (KVM on Debian Lenny/5.0) - VM using a block-backed disk (same) - Alix board with CF disk in LBA mode - Alix board with CF disk in CHS mode - All of the above in packet and nopacket modes I've tried using the 'sysctl kern.geom.debugflags=0x10' mentioned in the boot0cfg man page, but that results in no change. The NanoBSD images I'm using are mostly vanilla, built on 8.0-RELEASE-p2. Any ideas? Bug or user error? Thanks. Brian Conway [1] # boot0cfg -s 1 -v ad0 # flag start chs type end chs offset size 1 0x80 0: 1: 1 0xa5 791: 15:63 63 798273 2 0x00 792: 1: 1 0xa5 559: 15:63 798399 798273 3 0x00 560: 0: 1 0xa5 562: 15:63 1596672 3024 version=2.0 drive=0x80 mask=0x3 ticks=182 bell=# (0x23) options=packet,update,nosetdrv volume serial ID 9090-9090 default_selection=F1 (Slice 1) From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 08:33:57 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 18003106564A; Mon, 8 Mar 2010 08:33:57 +0000 (UTC) (envelope-from to.my.trociny@gmail.com) Received: from mail-ew0-f226.google.com (mail-ew0-f226.google.com [209.85.219.226]) by mx1.freebsd.org (Postfix) with ESMTP id 46E628FC13; Mon, 8 Mar 2010 08:33:55 +0000 (UTC) Received: by ewy26 with SMTP id 26so3232320ewy.3 for ; Mon, 08 Mar 2010 00:33:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:to:cc:subject:references :organization:from:date:in-reply-to:message-id:user-agent :mime-version:content-type:content-transfer-encoding; bh=srbW6vdDB+eItNSbzlPOYrVl7nEOO68i3Gq36MtNBto=; b=kMowaUBquvWB3bJgZxSZBN2yWWhKW/hsO/KHG1EkdzS8smvTI81vLjqqguYtgDg2ny 1pvespVfh2IkjHXwqELQk9KesAvzwiWhHGlJbTK03R9KykEjAI0gxye8Gfm05f2M9r9y OrnvknuitqNOLaQ+WuJK+Dfxab6utbN1iLYGE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=to:cc:subject:references:organization:from:date:in-reply-to :message-id:user-agent:mime-version:content-type :content-transfer-encoding; b=J/svZU+Pq8phkWtX1A1K4fv5CmVPC2fYLV4SXDwVJR7KklTpghjoG/KraeHT1+zP2g 1fIhQC22CH/ke7vjOhE5m61xVRrFxt55GWq13/ADjiRVF3unElWmeo3Qb+lwpjL0YMcF fixZyIs7FoudWBg3CqGp39UpP604PnPpXLbwk= Received: by 10.213.57.10 with SMTP id a10mr2897500ebh.92.1268037235065; Mon, 08 Mar 2010 00:33:55 -0800 (PST) Received: from localhost ([95.69.173.225]) by mx.google.com with ESMTPS id 14sm2148632ewy.6.2010.03.08.00.33.53 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 08 Mar 2010 00:33:53 -0800 (PST) To: Alexander Eichner References: <86zl2l2w9n.fsf@kopusha.onet> <1267972128.2825.2.camel@Prometheus> Organization: TOA Ukraine From: Mikolaj Golub Date: Mon, 08 Mar 2010 10:33:52 +0200 In-Reply-To: <1267972128.2825.2.camel@Prometheus> (Alexander Eichner's message of "Sun\, 07 Mar 2010 15\:28\:48 +0100") Message-ID: <86ocizgqkv.fsf@kopusha.onet> User-Agent: Gnus/5.11 (Gnus v5.11) Emacs/22.3 (berkeley-unix) MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit Cc: alc@freebsd.org, freebsd-emulation@freebsd.org, freebsd-stable@freebsd.org Subject: Re: virtualbox status on 8.0-STABLE i386 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 08:33:57 -0000 On Sun, 07 Mar 2010 15:28:48 +0100 Alexander Eichner wrote: > Hi, > > can you try the attached patch please? > This should fix the panic you encountered. Please undo your kernel > changes befoer testing. Unfortunately, the same panic: (kgdb) bt #0 doadump () at pcpu.h:246 #1 0xc04ec379 in db_fncall (dummy1=-1064468854, dummy2=0, dummy3=-1, dummy4=0xe866b5b4 "ȵfè") at /usr/src/sys/ddb/db_command.c:548 #2 0xc04ec7af in db_command (last_cmdp=0xc0e04c9c, cmd_table=0x0, dopager=0) at /usr/src/sys/ddb/db_command.c:445 #3 0xc04ec864 in db_command_script (command=0xc0e05bc4 "call doadump") at /usr/src/sys/ddb/db_command.c:516 #4 0xc04f09a0 in db_script_exec (scriptname=0xe866b6c0 "kdb.enter.panic", warnifnotfound=Variable "warnifnotfound" is not available. ) at /usr/src/sys/ddb/db_script.c:302 #5 0xc04f0a87 in db_script_kdbenter (eventname=0xc0cc246d "panic") at /usr/src/sys/ddb/db_script.c:324 #6 0xc04ee768 in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_main.c:228 #7 0xc08d7d06 in kdb_trap (type=3, code=0, tf=0xe866b7fc) at /usr/src/sys/kern/subr_kdb.c:535 #8 0xc0beb38b in trap (frame=0xe866b7fc) at /usr/src/sys/i386/i386/trap.c:690 #9 0xc0bcccfb in calltrap () at /usr/src/sys/i386/i386/exception.s:165 #10 0xc08d7e8a in kdb_enter (why=0xc0cc246d "panic", msg=0xc0cc246d "panic") at cpufunc.h:71 #11 0xc08a88b6 in panic (fmt=0xc0cecba4 "vm_fault: fault on nofault entry, addr: %lx") at /usr/src/sys/kern/kern_shutdown.c:562 #12 0xc0b0c3c7 in vm_fault (map=0xc1990000, vaddr=3244318720, fault_type=Variable "fault_type" is not available. ) at /usr/src/sys/vm/vm_fault.c:283 #13 0xc0bea7c6 in trap_pfault (frame=0xe866bab8, usermode=0, eva=3244322776) at /usr/src/sys/i386/i386/trap.c:840 #14 0xc0beb215 in trap (frame=0xe866bab8) at /usr/src/sys/i386/i386/trap.c:533 #15 0xc0bcccfb in calltrap () at /usr/src/sys/i386/i386/exception.s:165 #16 0xc12beef3 in rtR0MemObjNativeGetPagePhysAddr () from /boot/modules/vboxdrv.ko #17 0xc12ac374 in SUPR0LockMem () from /boot/modules/vboxdrv.ko #18 0xc12ac8eb in supdrvIOCtl () from /boot/modules/vboxdrv.ko #19 0xc12b0c5a in VBoxDrvFreeBSDIOCtl () from /boot/modules/vboxdrv.ko #20 0xc0829658 in devfs_ioctl_f (fp=0xc5f1c8c0, com=3321378576, data=0xe866bd00, cred=0xc6972a00, td=0xc728e250) at /usr/src/sys/fs/devfs/devfs_vnops.c:659 #21 0xc08eec8d in kern_ioctl (td=0xc728e250, fd=7, com=536892942, data=0xe866bd00 "@Ç\023)ø\023üÀ8½fèø\023üÀ,½fè\005\\½À\001") at file.h:262 #22 0xc08eee14 in ioctl (td=0xc728e250, uap=0xe866bcf8) at /usr/src/sys/kern/sys_generic.c:678 #23 0xc0beaac0 in syscall (frame=0xe866bd38) at /usr/src/sys/i386/i386/trap.c:1111 #24 0xc0bccd90 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:261 #25 0x00000033 in ?? () Previous frame inner to this frame (corrupt stack?) (this time I built the modules without debugging symbols). Just to be sure that I did all thing properly below the steps I did: 1) returned original pmap.h (with KPTmap), rebuilt the kernel and rebooted 2) rebuilt with the patch virtualbox drivers and virtualbox (not sure this last was needed bu just in case...): cd emulators/virtualbox-ose-kmod && make patch applied this patch and your previous patch ("Patch to fix VirtualBox with recent kernel versions") built and reinstall the same for emulators/virtualbox-ose 3) rebooted and started vm guest virtualbox-ose-3.1.2_1 A general-purpose full virtualizer for x86 hardware virtualbox-ose-kmod-3.1.2_1 VirtualBox kernel module for FreeBSD -- Mikolaj Golub From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 09:37:59 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD3E11065670 for ; Mon, 8 Mar 2010 09:37:59 +0000 (UTC) (envelope-from john.marshall@riverwillow.com.au) Received: from mail1.riverwillow.net.au (mail1.riverwillow.net.au [203.58.93.36]) by mx1.freebsd.org (Postfix) with ESMTP id 0D9D78FC14 for ; Mon, 8 Mar 2010 09:37:58 +0000 (UTC) Received: from rwpc12.mby.riverwillow.net.au (rwpc12.mby.riverwillow.net.au [172.25.24.168]) (authenticated bits=0) by mail1.riverwillow.net.au (8.14.4/8.14.4) with ESMTP id o289bpdR014659 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK) for ; Mon, 8 Mar 2010 20:37:52 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=riverwillow.com.au; s=m1001; t=1268041072; bh=2FlnYO7DOjeMzLEG3wxqNQ/cCIwvkiYBnKKY90xdaAI=; h=Date:From:To:Subject:Message-ID:References:Mime-Version: Content-Type:In-Reply-To; b=tWjsxgxIqp/+6KHf2gL3Ejed/d8leWO7kHzGEHLrotYgoHJXAaYArXgUarNQu7SWm rpLSMsGccFUbmQWonKzke0L5X7iByO2WF+LuidejGvI0xDRvsEf/RFah9Moa7lR5O4 UjbHHVVSbpHTMfGZCQ8iy40Une6g2vdglrJk0CmI= Received: from rwpc12.mby.riverwillow.net.au (localhost [127.0.0.1]) by rwpc12.mby.riverwillow.net.au (8.14.3/8.14.3) with ESMTP id o289boqW005605 for ; Mon, 8 Mar 2010 20:37:50 +1100 (AEDT) (envelope-from john.marshall@riverwillow.com.au) Received: (from john@localhost) by rwpc12.mby.riverwillow.net.au (8.14.3/8.14.3/Submit) id o289bnKO005604 for freebsd-stable@freebsd.org; Mon, 8 Mar 2010 20:37:49 +1100 (AEDT) (envelope-from john) Date: Mon, 8 Mar 2010 20:37:49 +1100 From: John Marshall To: freebsd-stable@freebsd.org Message-ID: <20100308093749.GB963@rwpc12.mby.riverwillow.net.au> Mail-Followup-To: freebsd-stable@freebsd.org References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gj572EiMnwbLXET9" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i OpenPGP: id=A29A84A2 Subject: Re: ntpd multicast TTL X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 09:37:59 -0000 --gj572EiMnwbLXET9 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, 07 Mar 2010, 16:19 +0000, Christian Weisgerber wrote: > ntpd is a convenient source of multicast packets for testing purposes. > When I add >=20 > broadcast 224.0.1.1 >=20 > to my ntp.conf, ntpd sends a multicast packet with TTL 1 every 64 > seconds. Just as expected. However, when I explicitly specify the > TTL as in >=20 > broadcast 224.0.1.1 ttl 1 >=20 > it sends packets with TTL 32. Trying a few other numbers confirms > that it multiplies the specified TTL by 32. That is not expected. > (I also don't recall this happening the last time I tried it, but > that may have been years ago.) >=20 > Is this simply a bug in ntpd? No, it's just that the ntp's server configuration statements don't use their ttl option to specify network ttl value, but as zero-based index into ntp's ttl value array. The default array is as you describe [1,32,64,96,128,160,192,224] but can be overridden by the ttl configuration statement. So the following lines in your ntp.conf would result in your multicast server transmitting packets with ttl=3D4. ttl 2 4 6 8 broadcast 224.0.1.1 ttl 1 I tripped over this last year when experimenting with ntp multicast. I had to resort to the source code to understand what was happening. It is actually documented. --=20 John Marshall --gj572EiMnwbLXET9 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuUxW0ACgkQw/tAaKKahKKAogCfQzQSQtfYYGMzp7wWN00yiW4m +OEAoIZQzlEddVIGdbRRjojpITYg0/tH =exTv -----END PGP SIGNATURE----- --gj572EiMnwbLXET9-- From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 10:47:38 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F13561065678; Mon, 8 Mar 2010 10:47:38 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from mail-yw0-f181.google.com (mail-yw0-f181.google.com [209.85.211.181]) by mx1.freebsd.org (Postfix) with ESMTP id 7DCCB8FC0C; Mon, 8 Mar 2010 10:47:38 +0000 (UTC) Received: by ywh11 with SMTP id 11so1741168ywh.13 for ; Mon, 08 Mar 2010 02:47:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:cc:content-type; bh=MAxk1zMLGVxNWZAu2yd3HrjKKB9uVgDCFUbDHhLMPnU=; b=UelTaE8luNtXkx6RlJaaRQ3NtjE1tIWGwkzKncSJOwj664rs6v/amu1zu62qROM1lI AzeezAAhKqkyI6gpot+FJQaGIboTBwWuZQ+OzKZ9IrIrkNFatHZQ0KmackwDvO/Hk2jC 9QGSCVMtQP5RgZvYIk5NkVlJEO030/4iXd1xQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:cc:content-type; b=VNBvXeBrfwr1CTFv7dybOexcnlTTAG0rtAGkVdoNRarze6GyfBhQ7VgiFJ7jjhr1h3 lEayNC6lA0axQ2uugTocMCFnJCmp4+izr2/xKK57epQbQSp4fgiSR9i85p5ctZqLUu9s eiHq5AThBpaLJ1RrgVuqRV7hkcvw91IZMZc48= MIME-Version: 1.0 Received: by 10.101.131.22 with SMTP id i22mr9252521ann.92.1268045257689; Mon, 08 Mar 2010 02:47:37 -0800 (PST) Date: Mon, 8 Mar 2010 12:47:37 +0200 Message-ID: From: Dan Naumov To: FreeBSD-STABLE Mailing List , freebsd-questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Attilio Rao Subject: powerd on 8.0, is it considered safe? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 10:47:39 -0000 Hello Is powerd finally considered stable and safe to use on 8.0? At least on 7.2, it consistently caused panics when used on Atom systems with Hyper-Threading enabled, but I recall that Attilio Rao was looking into it. - Sincerely, Dan Naumov From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 11:10:13 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F34941065672 for ; Mon, 8 Mar 2010 11:10:13 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta10.westchester.pa.mail.comcast.net (qmta10.westchester.pa.mail.comcast.net [76.96.62.17]) by mx1.freebsd.org (Postfix) with ESMTP id A0F9F8FC12 for ; Mon, 8 Mar 2010 11:10:13 +0000 (UTC) Received: from omta15.westchester.pa.mail.comcast.net ([76.96.62.87]) by qmta10.westchester.pa.mail.comcast.net with comcast id qb0c1d0031swQuc5AbADhH; Mon, 08 Mar 2010 11:10:13 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta15.westchester.pa.mail.comcast.net with comcast id qbDC1d0043S48mS3bbDCo5; Mon, 08 Mar 2010 11:13:13 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 5FCA01E3035; Mon, 8 Mar 2010 03:10:11 -0800 (PST) Date: Mon, 8 Mar 2010 03:10:11 -0800 From: Jeremy Chadwick To: freebsd-stable@freebsd.org Message-ID: <20100308111011.GA71267@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: powerd on 8.0, is it considered safe? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 11:10:14 -0000 On Mon, Mar 08, 2010 at 12:47:37PM +0200, Dan Naumov wrote: > Is powerd finally considered stable and safe to use on 8.0? At least > on 7.2, it consistently caused panics when used on Atom systems with > Hyper-Threading enabled, but I recall that Attilio Rao was looking > into it. I didn't realise it was unsafe to use on RELENG_8 or RELENG_7...? I've been using powerd(8) without any problems on my Supermicro X7SBA system (CPU = Core2Duo E8400) for quite some time, including back to the RELENG_7 days. I should note that on RELENG_7, EIST/SpeedStep didn't appear to work properly (cpufreq(4) driver reporting that the est piece wouldn't attach), which historically (for me) has resulted in ACPI throttling being done (which works). Based on the sysctl output below, I believe acpi_perf is in use, but I'm not 100% certain. The man page doesn't really disclose how to determine which driver is in use. The X7SBA system in question is below. Be aware I limit the lowest clock frequency to 1500MHz using debug.cpufreq.lowest="1500" in /boot/loader.conf, which is why you don't see anything lower in freq_levels. $ uname -a FreeBSD icarus.home.lan 8.0-STABLE FreeBSD 8.0-STABLE #0: Mon Mar 1 11:51:38 PST 2010 root@icarus.home.lan:/usr/obj/usr/src/sys/X7SBA_RELENG_8_amd64 amd64 $ kenv | grep smbios.planar.product smbios.planar.product="X7SBA" $ uptime 3:00am up 6 days, 14:38, 1 user, load averages: 0.00, 0.00, 0.00 $ ps -auxwwww | grep powerd root 991 0.0 0.0 6836 1284 ?? Ss Mon12pm 2:42.21 /usr/sbin/powerd jdc 71353 0.0 0.0 9036 1524 0 S+ 3:00am 0:00.00 grep powerd $ sysctl dev.cpu dev.est dev.cpufreq dev.p4tcc debug.cpufreq kern.timecounter dev.cpu.0.%desc: ACPI CPU dev.cpu.0.%driver: cpu dev.cpu.0.%location: handle=\_PR_.CPU0 dev.cpu.0.%pnpinfo: _HID=none _UID=0 dev.cpu.0.%parent: acpi0 dev.cpu.0.temperature: 41.0C dev.cpu.0.freq: 1500 dev.cpu.0.freq_levels: 3000/35000 2667/28000 2333/22000 2041/19250 2000/16000 1750/14000 1500/12000 dev.cpu.0.cx_supported: C1/0 C2/85 dev.cpu.0.cx_lowest: C1 dev.cpu.0.cx_usage: 100.00% 0.00% last 500us dev.cpu.1.%desc: ACPI CPU dev.cpu.1.%driver: cpu dev.cpu.1.%location: handle=\_PR_.CPU1 dev.cpu.1.%pnpinfo: _HID=none _UID=0 dev.cpu.1.%parent: acpi0 dev.cpu.1.temperature: 41.0C dev.cpu.1.cx_supported: C1/0 C2/85 dev.cpu.1.cx_lowest: C1 dev.cpu.1.cx_usage: 100.00% 0.00% last 500us dev.est.0.%desc: Enhanced SpeedStep Frequency Control dev.est.0.%driver: est dev.est.0.%parent: cpu0 dev.est.0.freq_settings: 3000/35000 2667/28000 2333/22000 2000/16000 dev.est.1.%desc: Enhanced SpeedStep Frequency Control dev.est.1.%driver: est dev.est.1.%parent: cpu1 dev.est.1.freq_settings: 3000/35000 2667/28000 2333/22000 2000/16000 dev.cpufreq.0.%driver: cpufreq dev.cpufreq.0.%parent: cpu0 dev.cpufreq.1.%driver: cpufreq dev.cpufreq.1.%parent: cpu1 dev.p4tcc.0.%desc: CPU Frequency Thermal Control dev.p4tcc.0.%driver: p4tcc dev.p4tcc.0.%parent: cpu0 dev.p4tcc.0.freq_settings: 10000/-1 8750/-1 7500/-1 6250/-1 5000/-1 3750/-1 2500/-1 1250/-1 dev.p4tcc.1.%desc: CPU Frequency Thermal Control dev.p4tcc.1.%driver: p4tcc dev.p4tcc.1.%parent: cpu1 dev.p4tcc.1.freq_settings: 10000/-1 8750/-1 7500/-1 6250/-1 5000/-1 3750/-1 2500/-1 1250/-1 debug.cpufreq.verbose: 0 debug.cpufreq.lowest: 1500 kern.timecounter.tick: 1 kern.timecounter.choice: TSC(-100) ACPI-fast(1000) i8254(0) dummy(-1000000) kern.timecounter.hardware: ACPI-fast kern.timecounter.stepwarnings: 0 kern.timecounter.tc.i8254.mask: 65535 kern.timecounter.tc.i8254.counter: 7127 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.ACPI-fast.mask: 16777215 kern.timecounter.tc.ACPI-fast.counter: 3275901 kern.timecounter.tc.ACPI-fast.frequency: 3579545 kern.timecounter.tc.ACPI-fast.quality: 1000 kern.timecounter.tc.TSC.mask: 4294967295 kern.timecounter.tc.TSC.counter: 1458979861 kern.timecounter.tc.TSC.frequency: 2992512852 kern.timecounter.tc.TSC.quality: -100 kern.timecounter.smp_tsc: 0 kern.timecounter.invariant_tsc: 1 -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 11:41:01 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 583C2106564A for ; Mon, 8 Mar 2010 11:41:01 +0000 (UTC) (envelope-from milu@dat.pl) Received: from jab.dat.pl (dat.pl [80.51.155.34]) by mx1.freebsd.org (Postfix) with ESMTP id BCBEA8FC15 for ; Mon, 8 Mar 2010 11:41:00 +0000 (UTC) Received: from jab.dat.pl (jsrv.dat.pl [127.0.0.1]) by jab.dat.pl (Postfix) with ESMTP id E45BA5C58; Mon, 8 Mar 2010 12:24:04 +0100 (CET) X-Virus-Scanned: amavisd-new at dat.pl Received: from jab.dat.pl ([127.0.0.1]) by jab.dat.pl (jab.dat.pl [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 9j2Qj13CjLVk; Mon, 8 Mar 2010 12:24:02 +0100 (CET) Received: from snifi.localnet (87-204-241-35.ip.netia.com.pl [87.204.241.35]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by jab.dat.pl (Postfix) with ESMTPSA id 180094054; Mon, 8 Mar 2010 12:24:02 +0100 (CET) From: Maciej Milewski To: freebsd-stable@freebsd.org Date: Mon, 8 Mar 2010 12:25:30 +0100 User-Agent: KMail/1.12.4 (Linux/2.6.32-trunk-686; KDE/4.3.4; i686; ; ) References: <20100307220002.f4b51436.bconway@alum.wpi.edu> In-Reply-To: <20100307220002.f4b51436.bconway@alum.wpi.edu> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-2" Content-Transfer-Encoding: quoted-printable Message-Id: <201003081225.31058.milu@dat.pl> Cc: Brian Conway Subject: Re: 8.0-RELEASE-p2: boot0cfg yields "vnode_pager_getpages: I/O read error" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 11:41:01 -0000 Dnia poniedzia=B3ek, 8 marca 2010 o 04:00:02 Brian Conway napisa=B3(a): > Greetings. I'm testing NanoBSD images for eventual use in an Alix board > and have run into an issue with the boot0cfg lines in the update scripts, > in multiple environments. Specifically, any use of boot0cfg yields a > successful output[1], followed by: > ... > Any ideas? Bug or > user error? Thanks. >=20 > Brian Conway Brian, try changing in updatep1 and updatep2 to use gpart not boot0cfg: #boot0cfg -s 1 -v ${NANO_DRIVE} gpart set -a active -i 1 ${NANO_DRIVE} I think is the problem. I was having similar errors as you while I was usin= g=20 boot0cfg. Maciej Milewski From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 12:01:49 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF1F2106567C; Mon, 8 Mar 2010 12:01:49 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 4776C8FC20; Mon, 8 Mar 2010 12:01:49 +0000 (UTC) Received: by gyg8 with SMTP id 8so1392971gyg.13 for ; Mon, 08 Mar 2010 04:01:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=9x+TWoXGA/FSk/Inpa6GbIjtBwvkno4VWd1BMramNx0=; b=bsCpDrMOD7WCX28yqJS2EXEBOInXue7M7Pn8mY15dZT4uFBKr9heHzjiEaBnrv0U+x 36RHGpjrDGKa0YIrk0ylq/K9tOrkz5W9xb1eZTYeotFHy4FkfAGXttOwhInZnqAC7fft /zJcboa58ca7UswEJzik5GImo7AKf7uQQERPM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=c5kJSr9ior0+Apd5+Dkm1w+BZ7njx9mXvs7/39dIUZ5bhVoPC3AVfXyjhpvYdTqTk7 lphhdoH/zhUn1A1ejdYxhx61x2Z1NleAEQ+zQINHwxvMYoy79tWLMJUnCrU9XghNpe8f 7sQspPcAd0xXFfTnU3JEoVnGH/NxQ6lXK+Sg0= MIME-Version: 1.0 Received: by 10.100.233.20 with SMTP id f20mr9191893anh.245.1268049708499; Mon, 08 Mar 2010 04:01:48 -0800 (PST) Date: Mon, 8 Mar 2010 14:01:48 +0200 Message-ID: From: Dan Naumov To: Jeremy Chadwick , FreeBSD-STABLE Mailing List , freebsd-questions@freebsd.org, Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: RE: powerd on 8.0, is it considered safe? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 12:01:49 -0000 Okay, now I am baffled. Up until this point, I wasn't using powerd on this new Atom D510 system. I ran sysctl and noticed that dev.cpu.0.freq: is actually 1249 and doesn't change no matter what kind of load the system is under. If I boot to BIOS, under BIOS CPU is shown as 1,66 Ghz. Okayy... I guess this explains why my buildworld and buildkernel took over 5 hours if by default, it gets stuck at 1249 Mhz for no obvious reason. I enabled powerd and now according to dev.cpu.0.freq:, the system is permanently stuck at 1666 Mhz, regardless of whether the system is under load or not. atombsd# uname -a FreeBSD atombsd.localdomain 8.0-RELEASE-p2 FreeBSD 8.0-RELEASE-p2 #0: Tue Jan 5 21:11:58 UTC 2010 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 atombsd# kenv | grep smbios.planar.product smbios.planar.product="X7SPA-H" atombsd# sysctl dev.cpu dev.est dev.cpufreq dev.p4tcc debug.cpufreq kern.timecounter dev.cpu.0.%desc: ACPI CPU dev.cpu.0.%driver: cpu dev.cpu.0.%location: handle=\_PR_.P001 dev.cpu.0.%pnpinfo: _HID=none _UID=0 dev.cpu.0.%parent: acpi0 dev.cpu.0.freq: 1666 dev.cpu.0.freq_levels: 1666/-1 1457/-1 1249/-1 1041/-1 833/-1 624/-1 416/-1 208/-1 dev.cpu.0.cx_supported: C1/0 dev.cpu.0.cx_lowest: C1 dev.cpu.0.cx_usage: 100.00% last 500us dev.cpu.1.%desc: ACPI CPU dev.cpu.1.%driver: cpu dev.cpu.1.%location: handle=\_PR_.P002 dev.cpu.1.%pnpinfo: _HID=none _UID=0 dev.cpu.1.%parent: acpi0 dev.cpu.1.cx_supported: C1/0 dev.cpu.1.cx_lowest: C1 dev.cpu.1.cx_usage: 100.00% last 500us dev.cpu.2.%desc: ACPI CPU dev.cpu.2.%driver: cpu dev.cpu.2.%location: handle=\_PR_.P003 dev.cpu.2.%pnpinfo: _HID=none _UID=0 dev.cpu.2.%parent: acpi0 dev.cpu.2.cx_supported: C1/0 dev.cpu.2.cx_lowest: C1 dev.cpu.2.cx_usage: 100.00% last 500us dev.cpu.3.%desc: ACPI CPU dev.cpu.3.%driver: cpu dev.cpu.3.%location: handle=\_PR_.P004 dev.cpu.3.%pnpinfo: _HID=none _UID=0 dev.cpu.3.%parent: acpi0 dev.cpu.3.cx_supported: C1/0 dev.cpu.3.cx_lowest: C1 dev.cpu.3.cx_usage: 100.00% last 500us sysctl: unknown oid 'dev.est' Right. So how do I investigate why does the CPU get stuck at 1249 Mhz after boot by default when not using powerd and why it gets stuck at 1666 Mhz with powerd enabled and doesn't scale back down when IDLE? Out of curiosity, I stopped powerd but the CPU remained at 1666 Mhz. - Sincerely, Dan Naumov From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 12:29:54 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 16BCA1065672; Mon, 8 Mar 2010 12:29:54 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from mail-yx0-f184.google.com (mail-yx0-f184.google.com [209.85.210.184]) by mx1.freebsd.org (Postfix) with ESMTP id B816A8FC1D; Mon, 8 Mar 2010 12:29:53 +0000 (UTC) Received: by yxe14 with SMTP id 14so415631yxe.7 for ; Mon, 08 Mar 2010 04:29:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:date:message-id:subject :from:to:content-type; bh=+HMYnIXBz3tQXYZNIqlHaCZEeaEq8UCwL82ADGVgTCg=; b=VFSySO8H+85zDSnk5roeEo5RkJBhx+1fpE6Xpjud2DHAGATjE2jFVXvurYdnnXReZn pOZ+rkvc86zRlUYTpMqkl37dM6oc1EL6vKdfVuCKxMXQu76JosVbJzagXLUoZqIV8GFr 3xp/cWSz12eUQvNd36wYXjQmKxa9AKp/gCUA0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=umHWR6Y/Sf/QLJ/f6e06U0rskGPhfTvyUCEOma3QHtRvTsPUkmD1SXGE0P0arPchcz 81WAznalpjr0hNsBlbNsTPoCPgQ8mfvrCAXZIs2uAPgBXJTYB6fGsmD4HwPRT5SAZ+Eb 89qThetLN1vekNhniLvgtSRIsZ2oZ+SW2T6SU= MIME-Version: 1.0 Received: by 10.101.131.22 with SMTP id i22mr9523972ann.92.1268051392907; Mon, 08 Mar 2010 04:29:52 -0800 (PST) Date: Mon, 8 Mar 2010 14:29:52 +0200 Message-ID: From: Dan Naumov To: FreeBSD-STABLE Mailing List , Jeremy Chadwick , Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: RE: powerd on 8.0, is it considered safe? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 12:29:54 -0000 >Up until this point, I wasn't using powerd on this new Atom D510 >system. I ran sysctl and noticed that dev.cpu.0.freq: is actually 1249 >and doesn't change no matter what kind of load the system is under. If >I boot to BIOS, under BIOS CPU is shown as 1,66 Ghz. Okayy... I guess >this explains why my buildworld and buildkernel took over 5 hours if >by default, it gets stuck at 1249 Mhz for no obvious reason. I enabled >powerd and now according to dev.cpu.0.freq:, the system is permanently >stuck at 1666 Mhz, regardless of whether the system is under load or >not. OK, a reboot somehow fixed the powerd issue: 1) Disabled powerd 2) Rebooted 3) Upon bootup, checked dev.cpu.0.freq - it's stuck at 1249 (should be 1666 by default) 4) Enabled and started powerd - CPU scales correctly according to load There is some bug somewhere though, because something puts my CPU to 1249 Mhz upon boot with powerd disabled and it gets stuck there, this shouldn't happen. - Sincerely, Dan Naumov From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 13:23:41 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EE856106564A for ; Mon, 8 Mar 2010 13:23:40 +0000 (UTC) (envelope-from uranus@tinlans.org) Received: from tinlans.org (tinlans.org [220.133.199.45]) by mx1.freebsd.org (Postfix) with ESMTP id 1BAA88FC12 for ; Mon, 8 Mar 2010 13:23:39 +0000 (UTC) Received: from tinlans.org (localhost [127.0.0.1]) by tinlans.org (Postfix) with ESMTP id B251DCF042 for ; Mon, 8 Mar 2010 21:08:22 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tinlans.org; h= x-mailer:content-type:content-type:mime-version:date:date :subject:subject:from:from:message-id:received:received; s= tinlans; t=1268053702; bh=vEaOtbW6nHKS1hfJZVdel7iet4n/D2M4JjQgKG E9Hxc=; b=p7RYvh4xasS9qofRnLqWgg0HYbmoRSlHdoni3U6UWpAgNEnFhylBnk BSVw4p89dqfsjlMHvxPZcE7SWunhr/Ig== X-Virus-Scanned: amavisd-new at tinlans.org Received: from tinlans.org ([127.0.0.1]) by tinlans.org (tinlans.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qT5ZdNne7uAp for ; Mon, 8 Mar 2010 21:08:22 +0800 (CST) Received: from TinlansPC (TinlansPC [192.168.1.2]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tinlans.org (Postfix) with ESMTPSA id 7CA59CF040 for ; Mon, 8 Mar 2010 21:08:22 +0800 (CST) Message-ID: From: "Ling-hua Tseng" To: Date: Mon, 8 Mar 2010 21:07:12 +0800 MIME-Version: 1.0 X-Priority: 3 X-MSMail-Priority: Normal Importance: Normal X-Mailer: Microsoft Windows Live Mail 14.0.8089.726 X-MimeOLE: Produced By Microsoft MimeOLE V14.0.8089.726 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Supplementary groups on LDAP cannot work with RELENG_8 + nss_ldap X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 13:23:41 -0000 Today I upgraded 2 of my 4 machines from RELENG_7 to RELENG_8. Both of the 2 machines are just LDAP clients. My LDAP server is still running on RELENG_7, and the remained one is also a LDAP client. All of them were installed OpenLDAP-2.4.21 and nss_ldap-1.265_3. Before I upgrades my system, everything works properly. I added a group named `group1' on LDAP server, and then add a user named `user1' to this group. I can type `id user1' to see the following line: uid=3D3000(user1) gid=3D3000(user1) groups=3D3000(user1),10000(gorup1) Of course, now the following record is already my LDAP server: -- dn: cn=3Dgroup,ou=3Dgroup,dc=3Dmydomain,dc=3Dorg objectClass: posixGroup cn: group1 gidNumber: 10000 memberUid: user1 -- After I upgraded these 2 machines from RELENG_7 to RELENG_8, to type `id user1' could only show the following information: uid=3D3000(user1) gid=3D3000(user1) groups=3D3000(user1) This user's supplementary group was gone, and he couldn't write any group-writable files which had gid 10000 one = the 2 machines. But in my other 2 machines that running on RELENG_7, this problem is still not occured. I have logged the behaviors of RELENG_7 & RELENG_8. Here is the behavior when I type `id user1' on RELENG_7: -- conn=3D1007 op=3D2 SRCH base=3D"ou=3Dpeople,dc=3Dmydomain,dc=3Dorg" = scope=3D1 deref=3D0 = filter=3D"(&(objectClass=3DposixAccount)(uid=3Duser1))" conn=3D1007 op=3D2 SRCH attr=3Duid userPassword uidNumber gidNumber cn = homeDirectory loginShell gecos description objectClass shadowLastChange = shadowMax shadowExpire loginClass conn=3D1007 op=3D3 SRCH base=3D"ou=3Dgroup,dc=3Dmydomain,dc=3Dorg" = scope=3D1 deref=3D0 filter=3D"(&(objectClass=3DposixGroup))" conn=3D1007 op=3D3 SRCH attr=3Dcn userPassword memberUid uniqueMember = gidNumber conn=3D1007 op=3D4 SRCH base=3D"ou=3Dgroup,dc=3Dmydomain,dc=3Dorg" = scope=3D1 deref=3D0 = filter=3D"(&(objectClass=3DposixGroup)(gidNumber=3D3000))" conn=3D1007 op=3D4 SRCH attr=3Dcn userPassword memberUid uniqueMember = gidNumber conn=3D1007 op=3D4 SRCH base=3D"ou=3Dgroup,dc=3Dmydomain,dc=3Dorg" = scope=3D1 deref=3D0 = filter=3D"(&(objectClass=3DposixGroup)(gidNumber=3D3000))" conn=3D1007 op=3D4 SRCH attr=3Dcn userPassword memberUid uniqueMember = gidNumber conn=3D1007 op=3D4 SRCH base=3D"ou=3Dgroup,dc=3Dmydomain,dc=3Dorg" = scope=3D1 deref=3D0 = filter=3D"(&(objectClass=3DposixGroup)(gidNumber=3D10000))" conn=3D1007 op=3D4 SRCH attr=3Dcn userPassword memberUid uniqueMember = gidNumber -- In step 2, it tries to fetch out the full group list from my LDAP = server. According to this information, it can know what user1's supplementary = groups are. RELENG_8: -- conn=3D1008 op=3D2 SRCH base=3D"ou=3Dpeople,dc=3Dmydomain,dc=3Dorg" = scope=3D1 deref=3D0 = filter=3D"(&(objectClass=3DposixAccount)(uid=3Duser1))" conn=3D1008 op=3D2 SRCH attr=3Duid userPassword uidNumber gidNumber cn = homeDirectory loginShell gecos description objectClass shadowLastChange = shadowMax shadowExpire loginClass conn=3D1008 op=3D3 SRCH base=3D"ou=3Dgroup,dc=3Dmydomain,dc=3Dorg" = scope=3D1 deref=3D0 = filter=3D"(&(objectClass=3DposixGroup)(gidNumber=3D3000))" conn=3D1008 op=3D3 SRCH attr=3Dcn userPassword memberUid uniqueMember = gidNumber conn=3D1008 op=3D3 SRCH base=3D"ou=3Dgroup,dc=3Dmydomain,dc=3Dorg" = scope=3D1 deref=3D0 = filter=3D"(&(objectClass=3DposixGroup)(gidNumber=3D3000))" conn=3D1008 op=3D3 SRCH attr=3Dcn userPassword memberUid uniqueMember = gidNumber -- It never tried to get the group list from LDAP server, hence it's impossible to know user1's supplementary groups. The client settings on RELENG_7 & RELENG_8 are fully consistent, so I don't think it's the problem of my config files. Since my 4 machines use the same version of nss_ldap, to downgrade nss_ldap's version for testing is meaningless. Should this problem is a base system's bug? From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 13:34:14 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 568DB1065673; Mon, 8 Mar 2010 13:34:14 +0000 (UTC) (envelope-from tajudd@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 1F1828FC31; Mon, 8 Mar 2010 13:34:13 +0000 (UTC) Received: by pwj1 with SMTP id 1so4144371pwj.13 for ; Mon, 08 Mar 2010 05:34:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=VV4bOB0jWLuLEfulA2ru9XutM1k2+TV+UI1iJRD4cXw=; b=hGXJ77dmtbYFJGg1gmw3Fv85HUo3XNVu3yUDbcVN5QzxXcUuYr4P3zpCmOnr87JtAa spfWddHSWlIFdQMSfuWU9uN1Spc1FLD4gnFveZo2ULTK1ZwFqdmXFraJuI4l/Vxvp9+J ZszqKq+TkWl3ZOITNmZt1sfDBHOdayMQav3mQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=m2Inhn8ef8eixNeMwX7z1yrfn1YzH77IjeIYEbkvXHQ7nr1/jnfx2u+PMO936EJtUi Nb0vzhq5tEuqtmRpVf2jS8CpC2ISU43/jXp7ZgfG0jGkXsp09MBfAS3xDdgl5HZMJAWj +vElObqSO3F+2dym3eFGbra9V5fNVRPF3rFr8= MIME-Version: 1.0 Received: by 10.114.252.39 with SMTP id z39mr3280892wah.93.1268055253480; Mon, 08 Mar 2010 05:34:13 -0800 (PST) In-Reply-To: References: Date: Mon, 8 Mar 2010 06:34:13 -0700 Message-ID: From: Tim Judd To: Dan Naumov Content-Type: text/plain; charset=ISO-8859-1 Cc: Attilio Rao , FreeBSD-STABLE Mailing List , freebsd-questions@freebsd.org, Jeremy Chadwick Subject: Re: powerd on 8.0, is it considered safe? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 13:34:14 -0000 I've been running powerd for a while. Been running it on an ASUS B202. It brought my freq down to 100mhz when I checked on it. Stopping powerd brought the freq up to 1600, and restarting powerd brought it back to 100mhz eventually. You might need to load an ACPI module for your system. Mine would be acpi_asus.ko if I choose to run it. I've never seen powerd to cause panics. I'm sure it's possible, but I've never seen it. Mark +1 for the success here on this side. From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 14:32:16 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B60DC106566B for ; Mon, 8 Mar 2010 14:32:16 +0000 (UTC) (envelope-from brian.j.conway@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id 4012B8FC0A for ; Mon, 8 Mar 2010 14:32:15 +0000 (UTC) Received: by fxm23 with SMTP id 23so4819718fxm.3 for ; Mon, 08 Mar 2010 06:32:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:received:date:from :x-x-sender:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version:content-type; bh=EEjrDua9qdRBWtvbquejb4Kybhnv/uITrs/SVz3Ylr0=; b=ozxdWYWAo3iyjU/HeUY1UHK1zfTDLNaX469UrnJ/dkQj/GL22Cnqc8BqDFzfWMAKH5 KUHPvO0YQRsIAbWva2URBT6np6l8lKngE/hJQzWD/fpREcRhfNwdbu0iT8jHZZCctAUM BHeFO5cpvI5/dQ0PIOhk8aznqg12hjLM8IXB8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:x-x-sender:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version:content-type; b=YPQD7snX0jWvZdKTLcM1KULsYuQPXZWLD5MYGTPz9dMUfqmGcvwxkduoZwxJlt8pEk 65FtcUZmUVdqbI41Lm2KkHrkTkMofg1lghW2OO1nCIXhPLNZW8ifBEQyEZH7LipZkQUB sWYjm0TpkWfTsGio6iU712+FYht1i4YBAWDL0= Received: by 10.223.76.79 with SMTP id b15mr5755511fak.89.1268058734902; Mon, 08 Mar 2010 06:32:14 -0800 (PST) Received: from ladyluck.zxvf.local (c-98-229-162-102.hsd1.ma.comcast.net [98.229.162.102]) by mx.google.com with ESMTPS id 14sm2906848fxm.5.2010.03.08.06.32.12 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 08 Mar 2010 06:32:13 -0800 (PST) Sender: Brian Conway Received: from bconway (helo=localhost) by ladyluck.zxvf.local with local-esmtp (Exim 4.69) (envelope-from ) id 1Noe0B-00082Q-Gi; Mon, 08 Mar 2010 09:32:11 -0500 Date: Mon, 8 Mar 2010 09:32:11 -0500 (EST) From: Brian Conway X-X-Sender: bconway@ladyluck.zxvf.local To: Maciej Milewski In-Reply-To: <201003081225.31058.milu@dat.pl> Message-ID: References: <20100307220002.f4b51436.bconway@alum.wpi.edu> <201003081225.31058.milu@dat.pl> User-Agent: Alpine 2.00 (DEB 1167 2008-08-23) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="8323329-160343530-1268058731=:30892" Cc: freebsd-stable@freebsd.org Subject: Re: 8.0-RELEASE-p2: boot0cfg yields "vnode_pager_getpages: I/O read error" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 14:32:16 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --8323329-160343530-1268058731=:30892 Content-Type: TEXT/PLAIN; charset=iso-8859-2; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE On Mon, 8 Mar 2010, Maciej Milewski wrote: > Dnia poniedzia=B3ek, 8 marca 2010 o 04:00:02 Brian Conway napisa=B3(a): >> Greetings. I'm testing NanoBSD images for eventual use in an Alix board >> and have run into an issue with the boot0cfg lines in the update scripts= , >> in multiple environments. Specifically, any use of boot0cfg yields a >> successful output[1], followed by: >> ... >> Any ideas? Bug or >> user error? Thanks. >> >> Brian Conway > > Brian, > try changing in updatep1 and updatep2 to use gpart not boot0cfg: > > #boot0cfg -s 1 -v ${NANO_DRIVE} > gpart set -a active -i 1 ${NANO_DRIVE} > > I think is the problem. I was having similar errors as you while I was us= ing > boot0cfg. > > Maciej Milewski > Maciej, Thanks, that resolved it and I've updated the scripts on my end. It looks= =20 like nanobsd.sh is using boot0cfg successfully when it builds the images,= =20 so I'll submit a PR. Brian Conway --8323329-160343530-1268058731=:30892-- From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 14:53:20 2010 Return-Path: Delivered-To: stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 644D7106564A; Mon, 8 Mar 2010 14:53:20 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 3FDD38FC1A; Mon, 8 Mar 2010 14:53:20 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id B9D3646B66; Mon, 8 Mar 2010 09:53:19 -0500 (EST) Date: Mon, 8 Mar 2010 14:53:19 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: current@FreeBSD.org, stable@FreeBSD.org In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: Survey results very helpful, thanks! (was: Re: net.inet.tcp.timer_race: does anyone have a non-zero value?) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 14:53:20 -0000 On Sun, 7 Mar 2010, Robert Watson wrote: > If your system shows a non-zero value, please send me a *private e-mail* > with the output of that command, plus also the output of "sysctl kern.smp", > "uptime", and a brief description of the workload and network interface > configuration. For example: it's a busy 8-core web server with roughly X > connections/second, and that has three em network interfaces used to load > balance from an upstream source. IPSEC is used for management purposes (but > not bulk traffic), and there's a local MySQL database. I've now received a number of reports that confirm our suspicion that the race does occur, albeit very rarely, and particularly on systems with many cores or multiple network interfaces. Fixing it is definitely on the TODO for 9.0, both to improve our ability to do multiple virtual network stacks, but with an appropriately scalable fix in mind given our improved TCP scalability for 9.0 as well. Thanks for all the responses, Robert N M Watson Computer Laboratory University of Cambridge From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 14:53:41 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 97CCC106566B for ; Mon, 8 Mar 2010 14:53:41 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id 1DFA98FC17 for ; Mon, 8 Mar 2010 14:53:40 +0000 (UTC) Received: by fxm23 with SMTP id 23so4841700fxm.3 for ; Mon, 08 Mar 2010 06:53:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :content-type:content-transfer-encoding; bh=V7tkZ9z2WVcwPTck8arpXHWyGZAwKXiqPzQP2iEHnck=; b=sqGo2rW4+3ADzv0BI1HR+HBP+Ts1xS2mQy9cCVtcFBvjzx9jSTBtNY4NhJgwngwHrG aQrRvnecBbDKWBvxTS5ksMgNn9pWqeenCyYkpmYzLGcAkNiYjA7I4C/0c6tQqsaPZM4X Rv4879VePzHcEXc5PQXKfEH9Udtqe07JKGoDQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; b=PBiV+mBnc4ursvN/o8WpKgb2mZco3gqqAaeY1U3IFolpYfo7sXH/+RJcu4KPywkEt8 wJkIuJrX7EalwTN/TRho5xWhSIxcdet289EepF/7MXCljSTaUA5xAZLwdblmJrGY/u2M Gyg8jLA+6IqyfIAH+M1AqTk72SLi4YtbsF/Yc= Received: by 10.223.64.141 with SMTP id e13mr5840753fai.101.1268060019937; Mon, 08 Mar 2010 06:53:39 -0800 (PST) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id 14sm2920963fxm.5.2010.03.08.06.53.38 (version=SSLv3 cipher=RC4-MD5); Mon, 08 Mar 2010 06:53:39 -0800 (PST) Sender: Alexander Motin Message-ID: <4B950F70.8030601@FreeBSD.org> Date: Mon, 08 Mar 2010 16:53:36 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Jeremy Chadwick References: <1268058181.00227122.1268045402@10.7.7.3> <1268058195.00227162.1268047203@10.7.7.3> In-Reply-To: <1268058195.00227162.1268047203@10.7.7.3> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: powerd on 8.0, is it considered safe? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 14:53:41 -0000 Jeremy Chadwick wrote: > I should note that on RELENG_7, EIST/SpeedStep didn't appear to work > properly (cpufreq(4) driver reporting that the est piece wouldn't > attach), which historically (for me) has resulted in ACPI throttling > being done (which works). Based on the sysctl output below, I believe > acpi_perf is in use, but I'm not 100% certain. The man page doesn't > really disclose how to determine which driver is in use. > > The X7SBA system in question is below. Be aware I limit the lowest > clock frequency to 1500MHz using debug.cpufreq.lowest="1500" in > /boot/loader.conf, which is why you don't see anything lower in > freq_levels. > > dev.est.0.freq_settings: 3000/35000 2667/28000 2333/22000 2000/16000 This is the real set of EIST supported levels. > dev.cpu.0.freq_levels: 3000/35000 2667/28000 2333/22000 2041/19250 2000/16000 1750/14000 1500/12000 All the rest of these are throttling. I would recommend you to disable it by setting: hint.p4tcc.0.disabled=1 hint.acpi_throttle.0.disabled=1 -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 14:56:20 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A213C1065674 for ; Mon, 8 Mar 2010 14:56:20 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id 2E61F8FC0C for ; Mon, 8 Mar 2010 14:56:19 +0000 (UTC) Received: by fxm23 with SMTP id 23so4844494fxm.3 for ; Mon, 08 Mar 2010 06:56:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=A2jKvFBg9ICww6qwOBVhdGcwNZ1Yd18DUKf3nVYYw6U=; b=FT6mVHPZn9rnuBkM79Bu4rq2aKyCC2uZn9Ht2gI5DC0Rj9twlNT0jtjesxKY5/7SAh 4+x9QAsgWHhSKkBMTK3QdTbqY1ydSyIzKIirMFfGXkcH1IZXRNcM/8w2msR1NfurVHVJ 4a5s+gkZ0KqjgZ1xl1BmZTWgd8wQqAVgTmUkg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=SniGpENa7c9NmuG0QMmljOsp3vQPW16BA0LLjE0k6eBUOcWZwNZUT6z5HVrIFsbDQk iGi/b8Un7mP4+oOr4T4a9aJfm0xx2BVg1ULsDdTztJowFPcpjJtOAdCrXx1ulhLsABKd 7LJECu4joftY13CRJpGBMMQGZJTPk30MTzXI4= Received: by 10.223.64.84 with SMTP id d20mr5918862fai.76.1268060178942; Mon, 08 Mar 2010 06:56:18 -0800 (PST) Received: from mavbook.mavhome.dp.ua (pc.mavhome.dp.ua [212.86.226.226]) by mx.google.com with ESMTPS id 14sm2973457fxm.9.2010.03.08.06.56.18 (version=SSLv3 cipher=RC4-MD5); Mon, 08 Mar 2010 06:56:18 -0800 (PST) Sender: Alexander Motin Message-ID: <4B95100F.1020108@FreeBSD.org> Date: Mon, 08 Mar 2010 16:56:15 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Jeremy Chadwick References: <1268058181.00227122.1268045402@10.7.7.3> <1268058195.00227162.1268047203@10.7.7.3> <4B950F70.8030601@FreeBSD.org> In-Reply-To: <4B950F70.8030601@FreeBSD.org> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: powerd on 8.0, is it considered safe? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 14:56:20 -0000 Alexander Motin wrote: > Jeremy Chadwick wrote: >> I should note that on RELENG_7, EIST/SpeedStep didn't appear to work >> properly (cpufreq(4) driver reporting that the est piece wouldn't >> attach), which historically (for me) has resulted in ACPI throttling >> being done (which works). Based on the sysctl output below, I believe >> acpi_perf is in use, but I'm not 100% certain. The man page doesn't >> really disclose how to determine which driver is in use. >> >> The X7SBA system in question is below. Be aware I limit the lowest >> clock frequency to 1500MHz using debug.cpufreq.lowest="1500" in >> /boot/loader.conf, which is why you don't see anything lower in >> freq_levels. >> >> dev.est.0.freq_settings: 3000/35000 2667/28000 2333/22000 2000/16000 > > This is the real set of EIST supported levels. > >> dev.cpu.0.freq_levels: 3000/35000 2667/28000 2333/22000 2041/19250 > 2000/16000 1750/14000 1500/12000 > > All the rest of these are throttling. > > I would recommend you to disable it by setting: > hint.p4tcc.0.disabled=1 > hint.acpi_throttle.0.disabled=1 > dev.cpu.1.cx_supported: C1/0 C2/85 And you may also enable C2 state, as your ACPI reports it, and it should be safe now. To get maximum effect from it you may wish to reduce HZ. -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 15:34:34 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 091ED1065799; Mon, 8 Mar 2010 15:34:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 54A9D8FC16; Mon, 8 Mar 2010 15:34:33 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 03E5046B8B; Mon, 8 Mar 2010 10:34:33 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 3D9448A026; Mon, 8 Mar 2010 10:34:32 -0500 (EST) From: John Baldwin To: Robert Watson Date: Mon, 8 Mar 2010 10:00:12 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <20100306164451.00001999@unknown> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201003081000.12656.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Mon, 08 Mar 2010 10:34:32 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.5 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: Alexander Leidinger , freebsd-stable@freebsd.org Subject: Re: is dtrace usable? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 15:34:34 -0000 On Saturday 06 March 2010 11:00:12 am Robert Watson wrote: > On Sat, 6 Mar 2010, Alexander Leidinger wrote: > > >> Take a look at the DTrace configuration information here: > >> > >> http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/dtrace.html > > > > I've just reread it (despite the fact that I already used it). Some > > comments: > > > > Last time I tried, I didn't see any problems by adding > > makeoptions WITH_CTF=yes > > to the kernel config instead of doing > > make WITH_CTF=1 kernel > > > > Did I miss something, and if not, shouldn't we tell about the > > makeoptions part instead (a kernel rebuild later will not cause > > trouble when someone forgets to do the WITH_CTF part as it is already > > in the kernel makefile)? > > I'll leave John to answer this one, CC line broadended. I would be very surprised if 'makeoptions WITH_CTF=yes' worked. The many times I and others have tried it it did not work. Do you have a log of your build showing the ctfconvert and ctfmerge command lines? -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 16:40:30 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25822106566B; Mon, 8 Mar 2010 16:40:30 +0000 (UTC) (envelope-from dan.naumov@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id C35018FC3B; Mon, 8 Mar 2010 16:40:29 +0000 (UTC) Received: by gyg8 with SMTP id 8so1518015gyg.13 for ; Mon, 08 Mar 2010 08:40:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=KUNYp6Jw1K6u7vAh3lVGp1YoHkjNVBN8NaSYa9iJrxg=; b=TvnI/fGB/etLTjha2Dq7pxlUPHhMWsMWZIDETXlt+S2U8l7vDTgGX8XwLVI7H9LtmK NzVLhIeuMm675h9NlaQhEHWSosiz0d2pfbvFNTf3YI07dij/hQxiFQ659v01srXZpH3h RNSkgDH6622j8cVR476AivPzpsgnDunk/kwo4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=sG4mx18WCJ/d79T1gMi/xudgnU0DgpNAfZ9v2nfJMMsTBfQDL4joSNnjHvLJuK5HW2 Ba9pRdAzuG9Q11ebnyFwKWPH+HC/673JvxydQTbyW8tdH8wsZkh9nCLv3RkXl/sD9JZ+ QmYGVzweAS/tNDTVNdN/G0h6QoFVkzD4a/S9A= MIME-Version: 1.0 Received: by 10.101.155.38 with SMTP id h38mr7687304ano.131.1268066428994; Mon, 08 Mar 2010 08:40:28 -0800 (PST) In-Reply-To: References: Date: Mon, 8 Mar 2010 18:40:28 +0200 Message-ID: From: Dan Naumov To: FreeBSD-STABLE Mailing List , Jeremy Chadwick , freebsd-questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Subject: Re: powerd on 8.0, is it considered safe? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 16:40:30 -0000 OK, now I feel a bit stupid. The second half of my PR at http://www.freebsd.org/cgi/query-pr.cgi?pr=144551 (anything related to powerd behaviour) can be ignored. For testing purposes, I started powerd in the foreground and observed it's behaviour. It works exactly as advertised and apparently the very act of issuing a "sysctl -a | grep dev.cpu.0.freq" command uses up a high % of CPU time for a fraction of a second, resulting in confusing output, I was always getting the highest cpu frequency state as the output. Testing powerd in foreground however, shows correct behaviour, CPU is downclocked both before and after issuing that command :) Still doesn't explain why the system boots up at 1249 Mhz, but that's not that big of an issue at this point now I see that powerd is behaving correctly. - Sincerely, Dan Naumov From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 18:06:11 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A36441065670 for ; Mon, 8 Mar 2010 18:06:11 +0000 (UTC) (envelope-from korvus@comcast.net) Received: from mx04.pub.collaborativefusion.com (mx04.pub.collaborativefusion.com [206.210.72.84]) by mx1.freebsd.org (Postfix) with ESMTP id 703748FC18 for ; Mon, 8 Mar 2010 18:06:11 +0000 (UTC) Received: from [192.168.2.164] ([206.210.89.202]) by mx04.pub.collaborativefusion.com (StrongMail Enterprise 4.1.1.4(4.1.1.4-47689)); Mon, 08 Mar 2010 13:23:49 -0500 X-VirtualServerGroup: Default X-MailingID: 00000::00000::00000::00000::::3 X-SMHeaderMap: mid="X-MailingID" X-Destination-ID: freebsd-stable@FreeBSD.org X-SMFBL: ZnJlZWJzZC1zdGFibGVARnJlZUJTRC5vcmc= Message-ID: <4B953C92.5080606@comcast.net> Date: Mon, 08 Mar 2010 13:06:10 -0500 From: Steve Polyack User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.7) Gecko/20100211 Thunderbird/3.0.1 MIME-Version: 1.0 To: freebsd-stable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: ZFS hot spares X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 18:06:11 -0000 ZFS in FreeBSD lacks at least one major feature from the Solaris version: hot spares. There is a PR open at http://www.freebsd.org/cgi/query-pr.cgi?pr=134491, but there hasn't been any motion/thoughts posted on it since its creation almost one year ago. I'm aware that on Solaris, hot spare replacement is handled by a few Solaris-specific daemons, zfs-retire and zfs-diagnose, which both plug into the Solaris FMA (Fault Management Architecture). Have there been any thoughts on porting these over or getting something similar running within FreeBSD? With all of the recent SATA/SAS CAM hotplug work now committed, it would be nice to have automatic replacement of hot spares with a future hot-replacement of the failed drive. On the other side, I'd be interested in hearing if anyone has had success in rolling their own scripted solution: i.e. something which polls 'zpool status' looking for failed drives and performing hot-spare replacements automatically. Thanks, Steve Polyack From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 19:55:56 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6B2A7106566C for ; Mon, 8 Mar 2010 19:55:56 +0000 (UTC) (envelope-from eugen@eg.sd.rdtc.ru) Received: from eg.sd.rdtc.ru (eg.sd.rdtc.ru [62.231.161.221]) by mx1.freebsd.org (Postfix) with ESMTP id BD4AA8FC12 for ; Mon, 8 Mar 2010 19:55:55 +0000 (UTC) Received: from eg.sd.rdtc.ru (localhost [127.0.0.1]) by eg.sd.rdtc.ru (8.14.3/8.14.3) with ESMTP id o28JaH0M005864; Tue, 9 Mar 2010 02:36:17 +0700 (KRAT) (envelope-from eugen@eg.sd.rdtc.ru) Received: (from eugen@localhost) by eg.sd.rdtc.ru (8.14.3/8.14.3/Submit) id o28JaHdS005863; Tue, 9 Mar 2010 02:36:17 +0700 (KRAT) (envelope-from eugen) Date: Tue, 9 Mar 2010 02:36:17 +0700 From: Eugene Grosbein To: Brian Conway Message-ID: <20100308193617.GA5826@rdtc.ru> References: <20100307220002.f4b51436.bconway@alum.wpi.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100307220002.f4b51436.bconway@alum.wpi.edu> User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@freebsd.org Subject: Re: 8.0-RELEASE-p2: boot0cfg yields "vnode_pager_getpages: I/O read error" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 19:55:56 -0000 On Sun, Mar 07, 2010 at 10:00:02PM -0500, Brian Conway wrote: > Greetings. I'm testing NanoBSD images for eventual use in an Alix board > and have run into an issue with the boot0cfg lines in the update scripts, > in multiple environments. Specifically, any use of boot0cfg yields a > successful output[1], followed by: > > vnode_pager_getpages: I/O read error > > At this point, it looks like the command succeeded, but all reads and > writes to disk fail: > > # man init > /usr/bin/man: Device not configured. > > # touch /var/tmp/test > /usr/bin/touch: Input/output error. boot0cfg is broken since RELENG_6 (at least) for unknown reason. For RELENG_6, there was a workaround (use kern.geom.debugflags=16) but it does not work for RELENG_8 anymore. Eugene Grosbein From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 20:13:32 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9BB2C106566B; Mon, 8 Mar 2010 20:13:32 +0000 (UTC) (envelope-from bc979@lafn.org) Received: from zoom.lafn.org (zoom.lafn.ORG [206.117.18.8]) by mx1.freebsd.org (Postfix) with ESMTP id 70E1A8FC1D; Mon, 8 Mar 2010 20:13:32 +0000 (UTC) Received: from [10.0.1.4] (pool-71-109-144-133.lsanca.dsl-w.verizon.net [71.109.144.133]) (authenticated bits=0) by zoom.lafn.org (8.14.3/8.14.2) with ESMTP id o28JYvci070955 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Mon, 8 Mar 2010 11:34:58 -0800 (PST) (envelope-from bc979@lafn.org) References: In-Reply-To: Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii Message-Id: Content-Transfer-Encoding: quoted-printable From: Doug Hardie Date: Mon, 8 Mar 2010 11:34:56 -0800 To: Robert Watson X-Mailer: Apple Mail (2.1077) X-Virus-Scanned: clamav-milter 0.95.3 at zoom.lafn.org X-Virus-Status: Clean Cc: stable@freebsd.org, current@freebsd.org Subject: Re: Survey results very helpful, thanks! (was: Re: net.inet.tcp.timer_race: does anyone have a non-zero value?) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 20:13:32 -0000 On 8 March 2010, at 06:53, Robert Watson wrote: >=20 > On Sun, 7 Mar 2010, Robert Watson wrote: >=20 >> If your system shows a non-zero value, please send me a *private = e-mail* with the output of that command, plus also the output of "sysctl = kern.smp", "uptime", and a brief description of the workload and network = interface configuration. For example: it's a busy 8-core web server = with roughly X connections/second, and that has three em network = interfaces used to load balance from an upstream source. IPSEC is used = for management purposes (but not bulk traffic), and there's a local = MySQL database. >=20 > I've now received a number of reports that confirm our suspicion that = the race does occur, albeit very rarely, and particularly on systems = with many cores or multiple network interfaces. Fixing it is definitely = on the TODO for 9.0, both to improve our ability to do multiple virtual = network stacks, but with an appropriately scalable fix in mind given our = improved TCP scalability for 9.0 as well. I run a number of 4 core systems with em interfaces. These are = production systems that are unmanned and located a long way from me. = Under unusual conditions it can take up to 6 hours to get there. I have = been waiting to switch to 8.0 because of the discussions on the em = device and now it sounds like I had better just skip 8.x and wait for 9. = 7.2 is working just fine.= From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 20:33:33 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2EACF1065672; Mon, 8 Mar 2010 20:33:33 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 0625F8FC12; Mon, 8 Mar 2010 20:33:33 +0000 (UTC) Received: from fledge.watson.org (fledge.watson.org [65.122.17.41]) by cyrus.watson.org (Postfix) with ESMTPS id AD7F246B45; Mon, 8 Mar 2010 15:33:32 -0500 (EST) Date: Mon, 8 Mar 2010 20:33:32 +0000 (GMT) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: Doug Hardie In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: stable@freebsd.org, current@freebsd.org Subject: Re: Survey results very helpful, thanks! (was: Re: net.inet.tcp.timer_race: does anyone have a non-zero value?) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 20:33:33 -0000 On Mon, 8 Mar 2010, Doug Hardie wrote: > I run a number of 4 core systems with em interfaces. These are production > systems that are unmanned and located a long way from me. Under unusual > conditions it can take up to 6 hours to get there. I have been waiting to > switch to 8.0 because of the discussions on the em device and now it sounds > like I had better just skip 8.x and wait for 9. 7.2 is working just fine. Not sure that any information in this survey thread should be relevant to that decision. This race has existed since before FreeBSD, having appeared in the original BSD network stack, and is just as present in FreeBSD 7.x as 8.x or 9.x. When I learned about the race during the early 7.x development cycle, I added a counter/statistic to measure how much it happened in practice, but was not able to exercise it in my testing, and so left the counter in to appear in 7.0 and later so that we could perform this survey as core counts/etc increase. The two likely outcomes were "it is never exercised" and "it is exercised but only very infrequently", neither really justifying the quite complex change to correct it given requirements at the time. On-going development work on the virtual network stack is what justifies correcting the bug at this point, moving from detecting and handling the race to preventing it from occuring as an invariant. The motivation here, BTW, is that we'd like to eliminate the type-stable storage requirement for connection state (which ensures that memory once used for a connection block is only ever used for connection blocks in the future), allowing memory to be fully freed when a virtual network stack is destroyed. Using type-stable storage helped address this bug, but was primarily present to reduce the overhead of monitoring using netstat(1). We'll now need to use a slightly more expensive solution (true reference counts) in that context, although in practice it will almost certainly be an unmeasurable cost. Which is to say that while there might be something in the em/altq/... thread to reasonably lead you to avoid 8.0, nothing in the TCP timer race thread should do so, since it affects 7.2 just as much as 8.0. Even if you do see a non-zero counter, that's not a matter for operational concern, just useful from the perspective of a network stack developer to understanding timing and behaviors in the stack. :-) Robert From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 21:06:56 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 902EC1065676 for ; Mon, 8 Mar 2010 21:06:56 +0000 (UTC) (envelope-from karl@denninger.net) Received: from FS.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id 4D9F18FC1D for ; Mon, 8 Mar 2010 21:06:56 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by FS.denninger.net (8.14.3/8.13.1) with SMTP id o28KUjXe012710 for ; Mon, 8 Mar 2010 14:30:45 -0600 (CST) (envelope-from karl@denninger.net) Received: from [127.0.0.1] [192.168.1.40] by Spamblock-sys (LOCAL); Mon Mar 8 14:30:45 2010 Message-ID: <4B955DF7.4010906@denninger.net> Date: Mon, 08 Mar 2010 14:28:39 -0600 From: Karl Denninger User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Doug Hardie References: In-Reply-To: X-Enigmail-Version: 0.96.0 Content-Type: multipart/mixed; boundary="------------090403000808060305000607" X-Antivirus: avast! (VPS 100308-1, 03/08/2010), Outbound message X-Antivirus-Status: Clean X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: stable@freebsd.org, Robert Watson , current@freebsd.org Subject: Re: Survey results very helpful, thanks! X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 21:06:56 -0000 This is a multi-part message in MIME format. --------------090403000808060305000607 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Doug Hardie wrote: > On 8 March 2010, at 06:53, Robert Watson wrote: > > >> On Sun, 7 Mar 2010, Robert Watson wrote: >> >> >>> If your system shows a non-zero value, please send me a *private e-mail* with the output of that command, plus also the output of "sysctl kern.smp", "uptime", and a brief description of the workload and network interface configuration. For example: it's a busy 8-core web server with roughly X connections/second, and that has three em network interfaces used to load balance from an upstream source. IPSEC is used for management purposes (but not bulk traffic), and there's a local MySQL database. >>> >> I've now received a number of reports that confirm our suspicion that the race does occur, albeit very rarely, and particularly on systems with many cores or multiple network interfaces. Fixing it is definitely on the TODO for 9.0, both to improve our ability to do multiple virtual network stacks, but with an appropriately scalable fix in mind given our improved TCP scalability for 9.0 as well. >> > > I run a number of 4 core systems with em interfaces. These are production systems that are unmanned and located a long way from me. Under unusual conditions it can take up to 6 hours to get there. I have been waiting to switch to 8.0 because of the discussions on the em device and now it sounds like I had better just skip 8.x and wait for 9. 7.2 is working just fine._______________________________________________ > I don't think its that simple. I run a number of production systems with "em" interfaces, and they get POUNDED. None have had any trouble with 8.x. $ ifconfig em0 em0: flags=8843 metric 0 mtu 1500 options=19b ether 00:30:48:d2:5a:24 inet 67.23.181.70 netmask 0xffffff00 broadcast 67.23.181.255 media: Ethernet autoselect (100baseTX ) status: active $ uptime 3:27PM up 61 days, 22:34, 1 user, load averages: 5.08, 4.48, 4.28 That's one of the busier ones; it's kinda loafing right now on network I/O (running about 3mbps sustained) but typically operates in the 15-20mbps range to the wild wild net for 6-8 hours in the evening doing what its doing now (handling a very busy forum) plus a few hundred videocast streams.... The last reboot was to replace a power strip in the colo rack with one that had remote management capability. It hasn't actually crashed since, well, pretty much forever (it was running 7.x before 8.x went to production status) The box is a dual quad-core Xeon running the amd64 codebase. -- Karl --------------090403000808060305000607-- From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 21:38:13 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9E2F5106566B for ; Mon, 8 Mar 2010 21:38:13 +0000 (UTC) (envelope-from matheusber@gmail.com) Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com [209.85.212.54]) by mx1.freebsd.org (Postfix) with ESMTP id 49E3D8FC0A for ; Mon, 8 Mar 2010 21:38:13 +0000 (UTC) Received: by vws14 with SMTP id 14so3282189vws.13 for ; Mon, 08 Mar 2010 13:38:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:received:received :message-id:date:subject:from:to:user-agent:mime-version :content-type:content-transfer-encoding:x-priority:importance; bh=hpXTdUm5RUzKOgL7dNCufpcX1kcQKs7inN3CU+0N4m4=; b=mFY4HzWvKcWs5qxivewyV0GVnbbEnsQt7xI0i7HOy/Z7CanZkB8uWtc0LV11T3Utaz V3WadQQspp46cHnNUJhjplSgR+KRSW5hJCHXY40Y52pZmbxN+DvLDxJRMd2TU8ZyeFzE p4YD5+O/+wVKGT25+ha+Wu4rWER7MDFhMqnPU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:subject:from:to:user-agent:mime-version :content-type:content-transfer-encoding:x-priority:importance; b=LcZWZb4mNRvy6O9XtNYRY0PacPdC76KM2MUE6nrVTs4mh6ZTgefdCtxY0LtNcO5FA0 F1ZDq6cs5ECYQMd0Duyx30UVb17/TwQ19NJJEm2JcMzncu/TLx9ej5tvRDxjVoyaua5U SzhwTwZVBjoJ3FdexGZvAMYJnPQ2EI7HfC3h8= Received: by 10.220.124.232 with SMTP id v40mr155938vcr.36.1268084292200; Mon, 08 Mar 2010 13:38:12 -0800 (PST) Received: from cygnus.homeunix.com ([189.71.102.244]) by mx.google.com with ESMTPS id 42sm50316037vws.8.2010.03.08.13.38.07 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 08 Mar 2010 13:38:10 -0800 (PST) Sender: Nenhum_de_Nos Received: by cygnus.homeunix.com (Postfix, from userid 80) id 15835B8A1E; Mon, 8 Mar 2010 18:38:03 -0300 (BRT) Received: from 187.90.85.166 (SquirrelMail authenticated user matheus) by cygnus.homeunix.com with HTTP; Mon, 8 Mar 2010 18:38:02 -0300 (BRT) Message-ID: <80587c73d8c5ee56d8890d04179024b8.squirrel@cygnus.homeunix.com> Date: Mon, 8 Mar 2010 18:38:02 -0300 (BRT) From: "Nenhum_de_Nos" To: freebsd-stable@freebsd.org User-Agent: SquirrelMail/1.4.15 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Subject: ahci errors on 8-stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 21:38:13 -0000 hail, I've seen these errors in a production machine in deep disk load (scp and bsdtar in heavy use): ahcich0: Timeout on slot 1 ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 40 serr 00000000 ahcich0: Timeout on slot 16 ahcich0: is 00000000 cs 0000e000 ss ffffffff rs ffffffff tfd c0 serr 00000000 tun0: link state changed to DOWN tun0: link state changed to UP ahcich0: Timeout on slot 13 ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 40 serr 00000000 ahcich0: Timeout on slot 9 ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 40 serr 00000000 tun0: link state changed to DOWN tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP tun0: link state changed to DOWN tun0: link state changed to UP ahcich0: Timeout on slot 30 ahcich0: is 00000000 cs 38000000 ss ffffffff rs ffffffff tfd c0 serr 00000000 ahcich0: Timeout on slot 11 ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 40 serr 00000000 swap_pager: indefinite wait buffer: bufobj: 0, blkno: 3, size: 4096 ahcich0: Timeout on slot 14 ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 40 serr 00000000 tun0: link state changed to DOWN tun0: link state changed to UP ahcich0: Timeout on slot 9 ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 40 serr 00000000 tun0: link state changed to DOWN tun0: link state changed to UP ahcich0: Timeout on slot 15 ahcich0: is 00000000 cs fc007fff ss ffffffff rs ffffffff tfd c0 serr 00000000 ahcich0: Timeout on slot 14 ahcich0: is 00000000 cs 00003ff8 ss ffffffff rs ffffffff tfd c0 serr 00000000 ahcich0: Timeout on slot 6 ahcich0: is 00000000 cs 8000003f ss ffffffff rs ffffffff tfd c0 serr 00000000 ahcich0: Timeout on slot 1 ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 40 serr 00000000 ahcich0: Timeout on slot 1 ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 40 serr 00000000 ahcich0: Timeout on slot 1 ahcich0: is 00000000 cs 00000000 ss ffffffff rs ffffffff tfd 40 serr 00000000 [matheus@optimus ~]$ uname -a FreeBSD xxx 8.0-STABLE FreeBSD 8.0-STABLE #3: Wed Feb 10 11:09:34 BRT 2010 root@xxx:/usr/obj/usr/src/sys/optimus amd64 this machine has 6GB ram, but just around 5GB are available: Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-STABLE #3: Wed Feb 10 11:09:34 BRT 2010 root@xxx:/usr/obj/usr/src/sys/optimus amd64 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM)2 Quad CPU Q9400 @ 2.66GHz (2666.38-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x1067a Stepping = 10 Features=0xbfebfbff [root@optimus /var/log]# head -20 dmesg.yesterday Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-STABLE #3: Wed Feb 10 11:09:34 BRT 2010 root@xxx:/usr/obj/usr/src/sys/optimus amd64 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM)2 Quad CPU Q9400 @ 2.66GHz (2666.38-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x1067a Stepping = 10 Features=0xbfebfbff Features2=0x408e3fd AMD Features=0x20100800 AMD Features2=0x1 TSC: P-state invariant real memory = 6442450944 (6144 MB) avail memory = 6058479616 (5777 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 I use a small ZFS pool: [root@optimus /var/log]# zpool status pool: pool state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM pool ONLINE 0 0 0 label/zfs1 ONLINE 0 0 0 label/zfs2 ONLINE 0 0 0 errors: No known data errors using two 1TB disks. should I fear these errors ? thanks, matheus -- We will call you cygnus, The God of balance you shall be A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? http://en.wikipedia.org/wiki/Posting_style From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 22:27:06 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 367331065672 for ; Mon, 8 Mar 2010 22:27:06 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta12.emeryville.ca.mail.comcast.net (qmta12.emeryville.ca.mail.comcast.net [76.96.27.227]) by mx1.freebsd.org (Postfix) with ESMTP id 1F1708FC08 for ; Mon, 8 Mar 2010 22:27:05 +0000 (UTC) Received: from omta18.emeryville.ca.mail.comcast.net ([76.96.30.74]) by qmta12.emeryville.ca.mail.comcast.net with comcast id qj6b1d00D1bwxycACmT6fi; Mon, 08 Mar 2010 22:27:06 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta18.emeryville.ca.mail.comcast.net with comcast id qmVu1d0063S48mS8emVwVz; Mon, 08 Mar 2010 22:30:00 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id DF20D1E3035; Mon, 8 Mar 2010 14:26:53 -0800 (PST) Date: Mon, 8 Mar 2010 14:26:53 -0800 From: Jeremy Chadwick To: freebsd-stable@freebsd.org Message-ID: <20100308222653.GA87837@icarus.home.lan> References: <80587c73d8c5ee56d8890d04179024b8.squirrel@cygnus.homeunix.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <80587c73d8c5ee56d8890d04179024b8.squirrel@cygnus.homeunix.com> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: ahci errors on 8-stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 22:27:06 -0000 On Mon, Mar 08, 2010 at 06:38:02PM -0300, Nenhum_de_Nos wrote: > I've seen these errors in a production machine in deep disk load (scp and > bsdtar in heavy use): Please provide the output from the following commands: vmstat -i pciconf -lvc -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Mon Mar 8 23:45:13 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 17CFA106566C for ; Mon, 8 Mar 2010 23:45:13 +0000 (UTC) (envelope-from matheusber@gmail.com) Received: from mail-qy0-f194.google.com (mail-qy0-f194.google.com [209.85.221.194]) by mx1.freebsd.org (Postfix) with ESMTP id BDEFD8FC17 for ; Mon, 8 Mar 2010 23:45:12 +0000 (UTC) Received: by qyk32 with SMTP id 32so4859911qyk.28 for ; Mon, 08 Mar 2010 15:45:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:received:date:from:to :subject:message-id:in-reply-to:references:x-mailer:mime-version :content-type:content-transfer-encoding; bh=Wlrp3QDVKUCkDcpj33p1e/Z9DpWHFwSyrOI7puVDNWI=; b=qUtSa3t4ksVQG48kFN4jjU1V403drMAimyfzkx3lX2V9HDwlaAwivnavcV0UzQtS9A pDEiaXE2JkwjF/RFYcT21WEc6GA+h8VmzwU5F5/r/4FEwsnQdFuSo8zbDorh6lZjuRtX DjYKewaWCeQnaKakGi3xxCcCw2j+LMhNsb094= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:subject:message-id:in-reply-to:references :x-mailer:mime-version:content-type:content-transfer-encoding; b=BlvlkM90FNHKBa2PzO0SdlrvNRLucVRWEiYh9HGkfkb/+sK5tfA+iIvLVWxSR6wZlT DPb9ARmsFzZXDN7O0ONVOuAR1z2EQ5tPnKFb71BRPU+6ZcckHgvchnNFZ639ASlk7eUU QW1BqMk2tde+W5zEdZM3leWn0+3LJkQih85Ak= Received: by 10.224.72.94 with SMTP id l30mr218368qaj.273.1268091911627; Mon, 08 Mar 2010 15:45:11 -0800 (PST) Received: from cygnus.homeunix.com ([189.71.102.244]) by mx.google.com with ESMTPS id 21sm3788015qyk.9.2010.03.08.15.45.07 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 08 Mar 2010 15:45:10 -0800 (PST) Sender: Nenhum_de_Nos Received: from elita (unknown [10.1.1.84]) by cygnus.homeunix.com (Postfix) with ESMTP id 4BF15B8A1D for ; Mon, 8 Mar 2010 20:45:01 -0300 (BRT) Date: Mon, 8 Mar 2010 20:44:58 -0300 From: Nenhum_de_Nos To: freebsd-stable@freebsd.org Message-Id: <20100308204458.9e0d51a8.matheus@eternamente.info> In-Reply-To: <20100308222653.GA87837@icarus.home.lan> References: <80587c73d8c5ee56d8890d04179024b8.squirrel@cygnus.homeunix.com> <20100308222653.GA87837@icarus.home.lan> X-Mailer: Sylpheed 3.0.0 (GTK+ 2.10.14; i686-pc-mingw32) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: Re: ahci errors on 8-stable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 08 Mar 2010 23:45:13 -0000 On Mon, 8 Mar 2010 14:26:53 -0800 Jeremy Chadwick wrote: > On Mon, Mar 08, 2010 at 06:38:02PM -0300, Nenhum_de_Nos wrote: > > I've seen these errors in a production machine in deep disk load (scp and > > bsdtar in heavy use): > > Please provide the output from the following commands: As I had huge disk activity when those messages appeared, I did reboot after and now no more are there. I think the vmstat command should be issued when the problem was happening right ? (if so I can run the backup tar's and see what happens). > vmstat -i [matheus@optimus /usr/home/matheus]$ vmstat -i interrupt total rate irq16: vgapci0+ 25054 1 irq18: uhci2 ehci0+ 587787 23 irq21: ral0 uhci1 963763 39 irq22: fwohci0 1 0 irq23: uhci3 ehci1 157584 6 cpu0: timer 49659956 2012 irq256: em0 55035142 2229 irq257: hdac0 1444588 58 irq258: ahci0 358695 14 cpu1: timer 49649636 2011 cpu2: timer 49722440 2014 cpu3: timer 49722444 2014 Total 257327090 10426 > pciconf -lvc [root@optimus ~]# pciconf -lvc hostb0@pci0:0:0:0: class=0x060000 card=0x50028086 chip=0x2e208086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' class = bridge subclass = HOST-PCI cap 09[e0] = vendor (length 12) Intel cap 1 version 1 features: Quick Resume, SATA RAID-5, 4 PCI-e x1 slots, SATA RAID-0/1/10 pcib1@pci0:0:1:0: class=0x060400 card=0x50028086 chip=0x2e218086 rev=0x03 hdr=0x01 vendor = 'Intel Corporation' class = bridge subclass = PCI-PCI cap 0d[88] = PCI Bridge card=0x50028086 cap 01[80] = powerspec 3 supports D0 D3 current D0 cap 05[90] = MSI supports 1 message cap 10[a0] = PCI-Express 2 root port max data 128(128) link x16(x16) none0@pci0:0:3:0: class=0x078000 card=0x50028086 chip=0x2e248086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' class = simple comms cap 01[50] = powerspec 3 supports D0 D3 current D0 cap 05[8c] = MSI supports 1 message, 64 bit em0@pci0:0:25:0: class=0x020000 card=0x50028086 chip=0x10cd8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' class = network subclass = ethernet cap 01[c8] = powerspec 2 supports D0 D3 current D0 cap 05[d0] = MSI supports 1 message, 64 bit enabled with 1 message cap 09[e0] = vendor (length 6) Intel cap 2 version 0 uhci0@pci0:0:26:0: class=0x0c0300 card=0x50028086 chip=0x3a378086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB UHCI Controller *4' class = serial bus subclass = USB cap 09[50] = vendor (length 6) Intel cap 2 version 0 uhci1@pci0:0:26:1: class=0x0c0300 card=0x50028086 chip=0x3a388086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB UHCI Controller *5' class = serial bus subclass = USB cap 09[50] = vendor (length 6) Intel cap 2 version 0 uhci2@pci0:0:26:2: class=0x0c0300 card=0x50028086 chip=0x3a398086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB UHCI Controller *6' class = serial bus subclass = USB cap 09[50] = vendor (length 6) Intel cap 2 version 0 ehci0@pci0:0:26:7: class=0x0c0320 card=0x50028086 chip=0x3a3c8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB EHCI Controller *2' class = serial bus subclass = USB cap 01[50] = powerspec 2 supports D0 D3 current D0 cap 0a[58] = EHCI Debug Port at offset 0xa0 in map 0x14 cap 09[98] = vendor (length 6) Intel cap 2 version 0 hdac0@pci0:0:27:0: class=0x040300 card=0x50028086 chip=0x3a3e8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'HD Audio Controller' class = multimedia subclass = HDA cap 01[50] = powerspec 2 supports D0 D3 current D0 cap 05[60] = MSI supports 1 message, 64 bit enabled with 1 message cap 10[70] = PCI-Express 1 root endpoint max data 128(128) link x0(x0) uhci3@pci0:0:29:0: class=0x0c0300 card=0x50028086 chip=0x3a348086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB UHCI Controller *1' class = serial bus subclass = USB cap 09[50] = vendor (length 6) Intel cap 2 version 0 uhci4@pci0:0:29:1: class=0x0c0300 card=0x50028086 chip=0x3a358086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB UHCI Controller *2' class = serial bus subclass = USB cap 09[50] = vendor (length 6) Intel cap 2 version 0 uhci5@pci0:0:29:2: class=0x0c0300 card=0x50028086 chip=0x3a368086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB UHCI Controller *3' class = serial bus subclass = USB cap 09[50] = vendor (length 6) Intel cap 2 version 0 ehci1@pci0:0:29:7: class=0x0c0320 card=0x50028086 chip=0x3a3a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'USB EHCI Controller *1' class = serial bus subclass = USB cap 01[50] = powerspec 2 supports D0 D3 current D0 cap 0a[58] = EHCI Debug Port at offset 0xa0 in map 0x14 cap 09[98] = vendor (length 6) Intel cap 2 version 0 pcib2@pci0:0:30:0: class=0x060401 card=0x00000000 chip=0x244e8086 rev=0x90 hdr=0x01 vendor = 'Intel Corporation' device = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge' class = bridge subclass = PCI-PCI cap 0d[50] = PCI Bridge card=0x00000000 isab0@pci0:0:31:0: class=0x060100 card=0x50028086 chip=0x3a168086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'LPC Interface Controller' class = bridge subclass = PCI-ISA cap 09[e0] = vendor (length 12) Intel cap 1 version 0 features: SATA RAID-5, 4 PCI-e x1 slots ahci0@pci0:0:31:2: class=0x010601 card=0x50028086 chip=0x3a228086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '6 port SATA AHCI Controller' class = mass storage subclass = SATA cap 05[80] = MSI supports 16 messages enabled with 1 message cap 01[70] = powerspec 3 supports D0 D3 current D0 cap 12[a8] = SATA Index-Data Pair cap 09[b0] = vendor (length 6) Intel cap 2 version 0 none1@pci0:0:31:3: class=0x0c0500 card=0x50028086 chip=0x3a308086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'SMB controller (50011458)' class = serial bus subclass = SMBus atapci0@pci0:0:31:5: class=0x010185 card=0x50028086 chip=0x3a268086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = 'SATA2(2Port2) (ICH10 Family)' class = mass storage subclass = ATA cap 01[70] = powerspec 3 supports D0 D3 current D0 cap 09[b0] = vendor (length 6) Intel cap 2 version 0 vgapci0@pci0:1:0:0: class=0x030000 card=0x23021682 chip=0x042110de rev=0xa1 hdr=0x00 vendor = 'Nvidia Corp' device = 'GeForce 8500 GT (G86)' class = display subclass = VGA cap 01[60] = powerspec 2 supports D0 D3 current D0 cap 05[68] = MSI supports 1 message, 64 bit cap 10[78] = PCI-Express 1 endpoint max data 128(128) link x16(x16) ral0@pci0:2:0:0: class=0x028000 card=0x3a711186 chip=0x03021814 rev=0x00 hdr=0x00 vendor = 'Ralink Technology, Corp' device = 'wireless a/b (RT2525 2.4GHz transceiver + RT2560 MAC/BBP)' class = network cap 01[40] = powerspec 2 supports D0 D3 current D0 fwohci0@pci0:2:1:0: class=0x0c0010 card=0x50028086 chip=0x581111c1 rev=0x70 hdr=0x00 vendor = 'Lucent/Agere Systems (Was: AT&T MicroElectronics)' device = '1394A PCI PHY/Link Open Host Ctrlr I/F (FW322)' class = serial bus subclass = FireWire cap 01[44] = powerspec 2 supports D0 D1 D2 D3 current D0 [root@optimus ~]# thanks, matheus > -- > | Jeremy Chadwick jdc@parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP: 4BD6C0CB | > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- Nenhum_de_Nos From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 00:08:29 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42B61106566B for ; Tue, 9 Mar 2010 00:08:29 +0000 (UTC) (envelope-from peter@simons-rock.edu) Received: from hedwig.simons-rock.edu (hedwig.simons-rock.edu [208.81.88.14]) by mx1.freebsd.org (Postfix) with ESMTP id 09A288FC18 for ; Tue, 9 Mar 2010 00:08:28 +0000 (UTC) Received: from cesium.hyperfine.info (c2.8d.5646.static.theplanet.com [70.86.141.194]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by hedwig.simons-rock.edu (Postfix) with ESMTP id E67992BB343; Mon, 8 Mar 2010 19:08:27 -0500 (EST) Date: Mon, 8 Mar 2010 19:08:26 -0500 From: "Peter C. Lai" To: Ling-hua Tseng Message-ID: <20100309000826.GF4648@cesium.hyperfine.info> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17 (2007-11-01) Cc: freebsd-stable@freebsd.org Subject: Re: Supplementary groups on LDAP cannot work with RELENG_8 + nss_ldap X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 00:08:29 -0000 Unable to reproduce, at least on a brand new 8-R install. Did you make sure you correctly merged /etc/nsswitch.conf during mergemaster? On 2010-03-08 09:07:12PM +0800, Ling-hua Tseng wrote: > Today I upgraded 2 of my 4 machines from RELENG_7 to RELENG_8. > Both of the 2 machines are just LDAP clients. > My LDAP server is still running on RELENG_7, > and the remained one is also a LDAP client. > All of them were installed OpenLDAP-2.4.21 and nss_ldap-1.265_3. > > Before I upgrades my system, everything works properly. > I added a group named `group1' on LDAP server, > and then add a user named `user1' to this group. > I can type `id user1' to see the following line: > uid=3000(user1) gid=3000(user1) groups=3000(user1),10000(gorup1) > > Of course, now the following record is already my LDAP server: > -- > dn: cn=group,ou=group,dc=mydomain,dc=org > objectClass: posixGroup > cn: group1 > gidNumber: 10000 > memberUid: user1 > -- > > After I upgraded these 2 machines from RELENG_7 to RELENG_8, > to type `id user1' could only show the following information: > uid=3000(user1) gid=3000(user1) groups=3000(user1) > This user's supplementary group was gone, > and he couldn't write any group-writable files which had gid 10000 one the 2 machines. > But in my other 2 machines that running on RELENG_7, > this problem is still not occured. > > I have logged the behaviors of RELENG_7 & RELENG_8. > Here is the behavior when I type `id user1' on RELENG_7: > -- > conn=1007 op=2 SRCH base="ou=people,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixAccount)(uid=user1))" > conn=1007 op=2 SRCH attr=uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass shadowLastChange shadowMax shadowExpire loginClass > > conn=1007 op=3 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup))" > conn=1007 op=3 SRCH attr=cn userPassword memberUid uniqueMember gidNumber > > conn=1007 op=4 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup)(gidNumber=3000))" > conn=1007 op=4 SRCH attr=cn userPassword memberUid uniqueMember gidNumber > > conn=1007 op=4 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup)(gidNumber=3000))" > conn=1007 op=4 SRCH attr=cn userPassword memberUid uniqueMember gidNumber > > conn=1007 op=4 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup)(gidNumber=10000))" > conn=1007 op=4 SRCH attr=cn userPassword memberUid uniqueMember gidNumber > -- > In step 2, it tries to fetch out the full group list from my LDAP server. > According to this information, it can know what user1's supplementary groups are. > > RELENG_8: > -- > conn=1008 op=2 SRCH base="ou=people,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixAccount)(uid=user1))" > conn=1008 op=2 SRCH attr=uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass shadowLastChange shadowMax shadowExpire loginClass > > conn=1008 op=3 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup)(gidNumber=3000))" > conn=1008 op=3 SRCH attr=cn userPassword memberUid uniqueMember gidNumber > > conn=1008 op=3 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup)(gidNumber=3000))" > conn=1008 op=3 SRCH attr=cn userPassword memberUid uniqueMember gidNumber > -- > It never tried to get the group list from LDAP server, > hence it's impossible to know user1's supplementary groups. > > The client settings on RELENG_7 & RELENG_8 are fully consistent, > so I don't think it's the problem of my config files. > Since my 4 machines use the same version of nss_ldap, > to downgrade nss_ldap's version for testing is meaningless. > > Should this problem is a base system's bug? > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" -- =========================================================== Peter C. Lai | Bard College at Simon's Rock Systems Administrator | 84 Alford Rd. Information Technology Svcs. | Gt. Barrington, MA 01230 USA peter AT simons-rock.edu | (413) 528-7428 =========================================================== From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 01:01:08 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B01811065676 for ; Tue, 9 Mar 2010 01:01:08 +0000 (UTC) (envelope-from uranus@tinlans.org) Received: from tinlans.org (tinlans.org [220.133.199.45]) by mx1.freebsd.org (Postfix) with ESMTP id 3ACC88FC0A for ; Tue, 9 Mar 2010 01:01:08 +0000 (UTC) Received: from tinlans.org (localhost [127.0.0.1]) by tinlans.org (Postfix) with ESMTP id 843FCCF052; Tue, 9 Mar 2010 09:01:37 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tinlans.org; h= x-mailer:content-transfer-encoding:content-type:content-type :mime-version:date:date:subject:subject:in-reply-to:references :from:from:message-id:received:received; s=tinlans; t= 1268096497; bh=a1nDdP0MYUbIe9GMwe9DYZ9C6t04ZVJ08RLWDPVns/A=; b=E euMbNPbIn0fLzhoReoYoX+s0MsRfRzGSn4f6soJBn2zQOxiKNengbIEPI7mCt5QY 1Jg2ah6Cs4C7+ynO8yCeg== X-Virus-Scanned: amavisd-new at tinlans.org Received: from tinlans.org ([127.0.0.1]) by tinlans.org (tinlans.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id x14WTWh6fQ7D; Tue, 9 Mar 2010 09:01:37 +0800 (CST) Received: from TinlansPC (TinlansPC [192.168.1.2]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tinlans.org (Postfix) with ESMTPSA id 4FFB1CF040; Tue, 9 Mar 2010 09:01:37 +0800 (CST) Message-ID: <80F42CAF32A14A1FB43B830AAF877A5A@TinlansPC> From: "Linghua Tseng" To: "Peter C. Lai" References: <20100309000826.GF4648@cesium.hyperfine.info> In-Reply-To: <20100309000826.GF4648@cesium.hyperfine.info> Date: Tue, 9 Mar 2010 09:00:49 +0800 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal Importance: Normal X-Mailer: Microsoft Windows Live Mail 14.0.8089.726 X-MimeOLE: Produced By Microsoft MimeOLE V14.0.8089.726 Cc: freebsd-stable@freebsd.org Subject: Re: Supplementary groups on LDAP cannot work with RELENG_8 +nss_ldap X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 01:01:08 -0000 Yes, I'm sure. Here is the output of `diff -u /usr/src/etc/nsswitch.conf /etc/nsswitch.conf'. --- /usr/src/etc/nsswitch.conf 2010-03-08 09:04:25.000000000 +0800 +++ /etc/nsswitch.conf 2010-03-08 18:01:08.000000000 +0800 @@ -1,13 +1,13 @@ # # nsswitch.conf(5) - name service switch configuration file -# $FreeBSD: src/etc/nsswitch.conf,v 1.1.10.1 2009/08/03 08:13:06 kensmith Exp $ +# $FreeBSD: src/etc/nsswitch.conf,v 1.1 2006/05/03 15:14:47 ume Exp $ # group: compat -group_compat: nis +group_compat: ldap nis hosts: files dns networks: files passwd: compat -passwd_compat: nis +passwd_compat: ldap nis shells: files services: compat services_compat: nis The line `+:*::::::::' has already put into /etc/master.passwd, and the line `+:*::' has already put into /etc/group. In fact, my 2 machines were upgraded on different days. The 1st one's uname: 8.0-STABLE FreeBSD 8.0-STABLE #0: Mon Mar 8 10:21:45 CST 2010 The 2nd one's uname: 8.0-STABLE FreeBSD 8.0-STABLE #0: Wed Feb 24 03:46:38 CST 2010 Both of them cannot work properly. It can prove that this problem can be reproduced since 2/24 or earlier. Besides, I precisely followed the 11-step instructions that described in /usr/src/Makefile for upgrading my systems. To do mergemaster is never a big problem for me because I've used it since this script was born. /usr/local/etc/ldap.conf & /usr/local/etc/nss_ldap.conf are also consistent on my 4 machines. These settings works properly for my RELENG_7 machines, but RELENG_8 ones. By the way, I don't use nscd because it always caches users' login shell so that users cannot update it immediately. I also installed pam_ldap, and I have read this old topic: http://lists.freebsd.org/pipermail/freebsd-stable/2008-March/041393.html It says to set `bind_policy' to `hard' can resolve this issue, but it cannot work for me. -------------------------------------------------- From: "Peter C. Lai" Sent: Tuesday, March 09, 2010 8:08 AM To: "Ling-hua Tseng" Cc: Subject: Re: Supplementary groups on LDAP cannot work with RELENG_8 +nss_ldap > Unable to reproduce, at least on a brand new 8-R install. > Did you make sure you correctly merged /etc/nsswitch.conf during mergemaster? > > On 2010-03-08 09:07:12PM +0800, Ling-hua Tseng wrote: >> Today I upgraded 2 of my 4 machines from RELENG_7 to RELENG_8. >> Both of the 2 machines are just LDAP clients. >> My LDAP server is still running on RELENG_7, >> and the remained one is also a LDAP client. >> All of them were installed OpenLDAP-2.4.21 and nss_ldap-1.265_3. >> >> Before I upgrades my system, everything works properly. >> I added a group named `group1' on LDAP server, >> and then add a user named `user1' to this group. >> I can type `id user1' to see the following line: >> uid=3000(user1) gid=3000(user1) groups=3000(user1),10000(gorup1) >> >> Of course, now the following record is already my LDAP server: >> -- >> dn: cn=group,ou=group,dc=mydomain,dc=org >> objectClass: posixGroup >> cn: group1 >> gidNumber: 10000 >> memberUid: user1 >> -- >> >> After I upgraded these 2 machines from RELENG_7 to RELENG_8, >> to type `id user1' could only show the following information: >> uid=3000(user1) gid=3000(user1) groups=3000(user1) >> This user's supplementary group was gone, >> and he couldn't write any group-writable files which had gid 10000 one the 2 machines. >> But in my other 2 machines that running on RELENG_7, >> this problem is still not occured. >> >> I have logged the behaviors of RELENG_7 & RELENG_8. >> Here is the behavior when I type `id user1' on RELENG_7: >> -- >> conn=1007 op=2 SRCH base="ou=people,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixAccount)(uid=user1))" >> conn=1007 op=2 SRCH attr=uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass >> shadowLastChange shadowMax shadowExpire loginClass >> >> conn=1007 op=3 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup))" >> conn=1007 op=3 SRCH attr=cn userPassword memberUid uniqueMember gidNumber >> >> conn=1007 op=4 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup)(gidNumber=3000))" >> conn=1007 op=4 SRCH attr=cn userPassword memberUid uniqueMember gidNumber >> >> conn=1007 op=4 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup)(gidNumber=3000))" >> conn=1007 op=4 SRCH attr=cn userPassword memberUid uniqueMember gidNumber >> >> conn=1007 op=4 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup)(gidNumber=10000))" >> conn=1007 op=4 SRCH attr=cn userPassword memberUid uniqueMember gidNumber >> -- >> In step 2, it tries to fetch out the full group list from my LDAP server. >> According to this information, it can know what user1's supplementary groups are. >> >> RELENG_8: >> -- >> conn=1008 op=2 SRCH base="ou=people,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixAccount)(uid=user1))" >> conn=1008 op=2 SRCH attr=uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass >> shadowLastChange shadowMax shadowExpire loginClass >> >> conn=1008 op=3 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup)(gidNumber=3000))" >> conn=1008 op=3 SRCH attr=cn userPassword memberUid uniqueMember gidNumber >> >> conn=1008 op=3 SRCH base="ou=group,dc=mydomain,dc=org" scope=1 deref=0 filter="(&(objectClass=posixGroup)(gidNumber=3000))" >> conn=1008 op=3 SRCH attr=cn userPassword memberUid uniqueMember gidNumber >> -- >> It never tried to get the group list from LDAP server, >> hence it's impossible to know user1's supplementary groups. >> >> The client settings on RELENG_7 & RELENG_8 are fully consistent, >> so I don't think it's the problem of my config files. >> Since my 4 machines use the same version of nss_ldap, >> to downgrade nss_ldap's version for testing is meaningless. >> >> Should this problem is a base system's bug? >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > > -- > =========================================================== > Peter C. Lai | Bard College at Simon's Rock > Systems Administrator | 84 Alford Rd. > Information Technology Svcs. | Gt. Barrington, MA 01230 USA > peter AT simons-rock.edu | (413) 528-7428 > =========================================================== > > From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 07:51:25 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 871F9106564A for ; Tue, 9 Mar 2010 07:51:25 +0000 (UTC) (envelope-from mail@thomasjepsen.dk) Received: from csmtp1.one.com (csmtp1.one.com [195.47.247.21]) by mx1.freebsd.org (Postfix) with ESMTP id 0AF758FC12 for ; Tue, 9 Mar 2010 07:51:24 +0000 (UTC) Received: from [192.168.0.3] (84-238-113-98.u.parknet.dk [84.238.113.98]) by csmtp1.one.com (Postfix) with ESMTP id 9735E1BC00239 for ; Tue, 9 Mar 2010 07:31:32 +0000 (UTC) Message-ID: <4B95F953.3080004@thomasjepsen.dk> Date: Tue, 09 Mar 2010 08:31:31 +0100 From: Thomas Jepsen User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; da; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <20100308120015.00979106573C@hub.freebsd.org> In-Reply-To: <20100308120015.00979106573C@hub.freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Please unsubscribe X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 07:51:25 -0000 Den 08-03-2010 13:00, freebsd-stable-request@freebsd.org skrev: > Send freebsd-stable mailing list submissions to > freebsd-stable@freebsd.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > or, via email, send a message with subject or body 'help' to > freebsd-stable-request@freebsd.org > > You can reach the person managing the list at > freebsd-stable-owner@freebsd.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of freebsd-stable digest..." > > > Today's Topics: > > 1. Re: net.inet.tcp.timer_race: does anyone have a non-zero > value? (Mikolaj Golub) > 2. Re: net.inet.tcp.timer_race: does anyone have a non-zero > value? (Robert N. M. Watson) > 3. Re: virtualbox status on 8.0-STABLE i386 (Alexander Eichner) > 4. Reminder about your invitation from Igor Lyapin > (Igor Lyapin (LinkedIn Invitations)) > 5. ntpd multicast TTL (Christian Weisgerber) > 6. Re: FreeBSD-8.0 802.11n support with ath/mwl (Sam Leffler) > 7. Re: FreeBSD-8.0 802.11n support with ath/mwl (Jim Pingle) > 8. Re: em(4) interface hangs under 8.0-RELEASE (Max Laier) > 9. Re: em(4) interface hangs under 8.0-RELEASE (Thomas Hurst) > 10. FBSD 8.0& ZFS Issue.. default_perms_for_dir (Howard Leadmon) > 11. 8.0-RELEASE-p2: boot0cfg yields "vnode_pager_getpages: I/O > read error" (Brian Conway) > 12. Re: virtualbox status on 8.0-STABLE i386 (Mikolaj Golub) > 13. Re: ntpd multicast TTL (John Marshall) > 14. powerd on 8.0, is it considered safe? (Dan Naumov) > 15. Re: powerd on 8.0, is it considered safe? (Jeremy Chadwick) > 16. Re: 8.0-RELEASE-p2: boot0cfg yields "vnode_pager_getpages: > I/O read error" (Maciej Milewski) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sun, 07 Mar 2010 14:33:40 +0200 > From: Mikolaj Golub > Subject: Re: net.inet.tcp.timer_race: does anyone have a non-zero > value? > To: Robert Watson > Cc: stable@FreeBSD.org, current@FreeBSD.org > Message-ID:<86d3zgcnvf.fsf@kopusha.onet> > Content-Type: text/plain; charset=us-ascii > > On Sun, 7 Mar 2010 11:59:35 +0000 (GMT) Robert Watson wrote: > > >> Please check the results of the following command: >> >> % sysctl net.inet.tcp.timer_race >> net.inet.tcp.timer_race: 0 >> > Are the results for FreeBSD7 look interesting for you? Because currently we > have mostly FreeBSD7.1 hosts in production and I observe nonzero values on 8 > hosts (about 15%). I would send more details to you privately if you are > interested. > > > > > > No virus found in this incoming message. > Checked by AVG - www.avg.com > Version: 9.0.733 / Virus Database: 271.1.1/2730 - Release Date: 03/08/10 08:34:00 > > From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 08:07:56 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E3708106566B for ; Tue, 9 Mar 2010 08:07:55 +0000 (UTC) (envelope-from kamikaze@bsdforen.de) Received: from mail.bsdforen.de (bsdforen.de [212.204.60.79]) by mx1.freebsd.org (Postfix) with ESMTP id 6EF5D8FC08 for ; Tue, 9 Mar 2010 08:07:55 +0000 (UTC) Received: from mobileKamikaze.norad (vpn-cl-162-232.rz.uni-karlsruhe.de [141.3.162.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.bsdforen.de (Postfix) with ESMTP id E8A128A178D for ; Tue, 9 Mar 2010 09:07:53 +0100 (CET) Message-ID: <4B9601D9.5030101@bsdforen.de> Date: Tue, 09 Mar 2010 09:07:53 +0100 From: Dominic Fandrey User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-GB; rv:1.9.1.8) Gecko/20100302 Thunderbird/3.0.3 MIME-Version: 1.0 To: freebsd-stable@freebsd.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Subject: ntpd does not re-query servers, when a new interface appears X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 08:07:56 -0000 ntpd tracks interface updates, however it does not requery servers, when they occur. This was less than an hour ago, at my university, the notebook boots and is not connected to anything: 9 Mar 08:07:17 ntpd[1510]: logging to file /var/log/ntpd 9 Mar 08:07:17 ntpd[1510]: precision = 2.234 usec 9 Mar 08:07:17 ntpd[1510]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled 9 Mar 08:07:17 ntpd[1510]: Listening on interface #1 wildcard, ::#123 Disabled 9 Mar 08:07:17 ntpd[1510]: Listening on interface #2 bge0, 192.168.1.12#123 Enabled 9 Mar 08:07:17 ntpd[1510]: Listening on interface #3 lo0, fe80::1#123 Enabled 9 Mar 08:07:17 ntpd[1510]: Listening on interface #4 lo0, ::1#123 Enabled 9 Mar 08:07:17 ntpd[1510]: Listening on interface #5 lo0, 127.0.0.1#123 Enabled 9 Mar 08:07:17 ntpd[1510]: Listening on routing socket on fd #26 for interface updates 9 Mar 08:07:17 ntpd[1510]: kernel time sync status 2040 9 Mar 08:07:17 ntpd[1510]: frequency initialized 3.155 PPM from /var/db/ntpd.drift 9 Mar 08:07:20 ntpd[1542]: host name not found: 0.de.pool.ntp.org 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `0.de.pool.ntp.org', giving up on it 9 Mar 08:07:20 ntpd[1542]: host name not found: 1.de.pool.ntp.org 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `1.de.pool.ntp.org', giving up on it 9 Mar 08:07:20 ntpd[1542]: host name not found: 2.de.pool.ntp.org 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `2.de.pool.ntp.org', giving up on it 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp3.rz.uni-karlsruhe.de 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp3.rz.uni-karlsruhe.de', giving up on it 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp4.rz.uni-karlsruhe.de 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp4.rz.uni-karlsruhe.de', giving up on it So ntpd has given up on all the servers listed in the ntp.conf file. I then proceed to connect to the wireless network and proceed to log into two VPNs: 9 Mar 08:08:58 ntpd[1510]: Listening on interface #6 wlan0, 192.168.75.58#123 Enabled 9 Mar 08:09:00 ntpd[1510]: Listening on interface #7 tun0, 193.196.120.15#123 Enabled 9 Mar 08:09:04 ntpd[1510]: Listening on interface #8 tun1, 141.3.162.67#123 Enabled Over interface #8 some of the servers are actually available, but ntpq -p still states: No association ID's returned Only when I restart ntpd, it operates as expected: remote refid st t when poll reach delay offset jitter ============================================================================== zit-net2.uni-pa .STEP. 16 u - 512 0 0.000 0.000 0.000 alpha.rueckgr.a .STEP. 16 u - 512 0 0.000 0.000 0.000 ntp.goneco.de .STEP. 16 u - 512 0 0.000 0.000 0.000 +proxy4.rz.uni-k 129.13.64.17 2 u 30 128 271 2.937 2.530 1.891 +proxy2.rz.uni-k 129.13.64.17 2 u 58 128 375 3.593 -8.981 1.837 *proxy1.rz.uni-k 129.13.64.17 2 u 15 128 271 3.297 8.244 1.487 -- A: Because it fouls the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 08:27:20 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F34D21065670; Tue, 9 Mar 2010 08:27:19 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 786368FC14; Tue, 9 Mar 2010 08:27:19 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2C358.dip.t-dialin.net [217.226.195.88]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 2332F844482; Tue, 9 Mar 2010 09:27:13 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 324E65274; Tue, 9 Mar 2010 09:27:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1268123230; bh=nrvQ0OrG6NtXRBKbYj32uFjjOlhoOR6ctUOQcVy/rms=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=B7pLfbEcQ4cV25WQNQxdzBk2HUQ29K+EVdK8XH27WHdGxUCdHTTW8WCEBlApyZaFR F1lBRnDd39Dvne+LD8pSmXFJSwouRurHYtTRyTm77RdiM+wy26gtHeiZ6sesQ3ZYAc oVj0ysInoX6WuxSpYq1HwGnaWgXk69Z0a837I/9rn3Aqvdn5kXZJHX++FVPFpTNSRL 6/CBwRgJtofY7F/bv9Qn1zUUbNoyy0IypFGSgP3RtOxc9uvqMdYmS3ptyygNt1SCyO qoYLWjCRqzcgwOZpqQWHKrIrPW+a3JgrWpV+S6pooFGVrbKDNiuFVVCXQ3b+J5E0eg 8RBs8XV3zi5RQ== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o298R9ep001911; Tue, 9 Mar 2010 09:27:09 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Tue, 09 Mar 2010 09:27:09 +0100 Message-ID: <20100309092709.16456y55vq0kaa88@webmail.leidinger.net> Date: Tue, 09 Mar 2010 09:27:09 +0100 From: Alexander Leidinger To: John Baldwin References: <20100306164451.00001999@unknown> <201003081000.12656.jhb@freebsd.org> In-Reply-To: <201003081000.12656.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 2332F844482.E042E X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.44, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1268728035.05964@zZ0jkXE+o1bopUDkIU/TQw X-EBL-Spam-Status: No Cc: freebsd-stable@freebsd.org, Robert Watson Subject: Re: is dtrace usable? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 08:27:20 -0000 Quoting John Baldwin (from Mon, 8 Mar 2010 10:00:12 -0500): > On Saturday 06 March 2010 11:00:12 am Robert Watson wrote: >> On Sat, 6 Mar 2010, Alexander Leidinger wrote: >> >> >> Take a look at the DTrace configuration information here: >> >> >> >> http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/dtrace.html >> > >> > I've just reread it (despite the fact that I already used it). Some >> > comments: >> > >> > Last time I tried, I didn't see any problems by adding >> > makeoptions WITH_CTF=yes >> > to the kernel config instead of doing >> > make WITH_CTF=1 kernel >> > >> > Did I miss something, and if not, shouldn't we tell about the >> > makeoptions part instead (a kernel rebuild later will not cause >> > trouble when someone forgets to do the WITH_CTF part as it is already >> > in the kernel makefile)? >> >> I'll leave John to answer this one, CC line broadended. > > I would be very surprised if 'makeoptions WITH_CTF=yes' worked. The many > times I and others have tried it it did not work. Do you have a log of your > build showing the ctfconvert and ctfmerge command lines? I do not have a log around, it has been a while since I did something with dtrace (a year ago) and I can not remember that I always added WITH_CTF on a build (but it was about SDT probes, not FBT probes, in case it matters). I had a look again, WITH_CTF=yes is one of the first lines in the Makefile, and /usr/share/mk/sys.mk has "if !defined(WITH_CTF)". "make -V WITH_CTF" shows "yes", but "make -V NO_CTF" shows "1". This is strange, isn't it? I would expect that NO_CTF is undefined. Is this a bug in make, or a bug in the man page (neither in the description of the different kinds of variables, nor in the description of "defined" is something mentioned explaining this behavior). The current kernel on my test machine is compiled with "makeoptions ...", and a "dtrace -l" causes the watchdog to trigger a panic. Ugh... that's not a nice behavior. :( Bye, Alexander. -- Adding sound to movies would be like putting lipstick on the Venus de Milo. -- actress Mary Pickford, 1925 http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 08:53:36 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 103F21065675; Tue, 9 Mar 2010 08:53:36 +0000 (UTC) (envelope-from bc979@lafn.org) Received: from zoom.lafn.org (zoom.lafn.ORG [206.117.18.8]) by mx1.freebsd.org (Postfix) with ESMTP id CFB328FC21; Tue, 9 Mar 2010 08:53:35 +0000 (UTC) Received: from [10.0.1.4] (pool-71-109-144-133.lsanca.dsl-w.verizon.net [71.109.144.133]) (authenticated bits=0) by zoom.lafn.org (8.14.3/8.14.2) with ESMTP id o298rYvW093739 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 9 Mar 2010 00:53:35 -0800 (PST) (envelope-from bc979@lafn.org) References: In-Reply-To: Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii Message-Id: <80C9B3BA-C498-419B-BD5E-6C2111F24F64@lafn.org> Content-Transfer-Encoding: quoted-printable From: Doug Hardie Date: Tue, 9 Mar 2010 00:53:34 -0800 To: Robert Watson X-Mailer: Apple Mail (2.1077) X-Virus-Scanned: clamav-milter 0.95.3 at zoom.lafn.org X-Virus-Status: Clean Cc: stable@freebsd.org, current@freebsd.org Subject: Re: Survey results very helpful, thanks! (was: Re: net.inet.tcp.timer_race: does anyone have a non-zero value?) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 08:53:36 -0000 On 8 March 2010, at 12:33, Robert Watson wrote: >=20 > On Mon, 8 Mar 2010, Doug Hardie wrote: >=20 >> I run a number of 4 core systems with em interfaces. These are = production systems that are unmanned and located a long way from me. = Under unusual conditions it can take up to 6 hours to get there. I have = been waiting to switch to 8.0 because of the discussions on the em = device and now it sounds like I had better just skip 8.x and wait for 9. = 7.2 is working just fine. >=20 > Not sure that any information in this survey thread should be relevant = to that decision. This race has existed since before FreeBSD, having = appeared in the original BSD network stack, and is just as present in = FreeBSD 7.x as 8.x or 9.x. When I learned about the race during the = early 7.x development cycle, I added a counter/statistic to measure how = much it happened in practice, but was not able to exercise it in my = testing, and so left the counter in to appear in 7.0 and later so that = we could perform this survey as core counts/etc increase. >=20 > The two likely outcomes were "it is never exercised" and "it is = exercised but only very infrequently", neither really justifying the = quite complex change to correct it given requirements at the time. = On-going development work on the virtual network stack is what justifies = correcting the bug at this point, moving from detecting and handling the = race to preventing it from occuring as an invariant. The motivation = here, BTW, is that we'd like to eliminate the type-stable storage = requirement for connection state (which ensures that memory once used = for a connection block is only ever used for connection blocks in the = future), allowing memory to be fully freed when a virtual network stack = is destroyed. Using type-stable storage helped address this bug, but = was primarily present to reduce the overhead of monitoring using = netstat(1). We'll now need to use a slightly more expensive solution = (true reference counts) in that context, although in practice it will = almost certainly be an unmeasurable cost. >=20 > Which is to say that while there might be something in the em/altq/... = thread to reasonably lead you to avoid 8.0, nothing in the TCP timer = race thread should do so, since it affects 7.2 just as much as 8.0. = Even if you do see a non-zero counter, that's not a matter for = operational concern, just useful from the perspective of a network stack = developer to understanding timing and behaviors in the stack. :-) Thanks for the complete explanation. I don't believe the ALTQ issue = will affect me. I am not currently using it and do not expect to in the = near future. In addition, there was a posting that a fix for at least = part of that will be added in a week or so. Given all that it appears = its time to start the planning/testing process for 8. From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 09:05:05 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 80EA7106564A for ; Tue, 9 Mar 2010 09:05:05 +0000 (UTC) (envelope-from auryn@zirakzigil.org) Received: from mail.zirakzigil.org (mail.zirakzigil.org [82.63.178.63]) by mx1.freebsd.org (Postfix) with ESMTP id DB9458FC0C for ; Tue, 9 Mar 2010 09:05:04 +0000 (UTC) Received: from localhost (unknown [192.168.1.2]) by mail.zirakzigil.org (Postfix) with ESMTP id 8BC4E948A0; Tue, 9 Mar 2010 09:46:26 +0100 (CET) X-Virus-Scanned: amavisd-new at zirakzigil.org Received: from mail.zirakzigil.org ([192.168.1.2]) by localhost (ext.zirakzigil.org [192.168.1.2]) (amavisd-new, port 10024) with ESMTP id cK3G6JxA+h8X; Tue, 9 Mar 2010 09:46:24 +0100 (CET) Received: from aurynmob2.giulioferro.it (unknown [192.168.1.2]) (Authenticated sender: auryn@zirakzigil.org) by mail.zirakzigil.org (Postfix) with ESMTPA id ED4AD9488C; Tue, 9 Mar 2010 09:46:23 +0100 (CET) Message-ID: <4B960ADE.2070303@zirakzigil.org> Date: Tue, 09 Mar 2010 09:46:22 +0100 From: Giulio Ferro User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.7) Gecko/20100223 Thunderbird/3.0.1 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: volker@vwsoft.com Subject: Fwd: Re: NFS Client error X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 09:05:05 -0000 Thanks for your kind reply, I'm forwarding it there... -------- Original Message -------- Subject: Re: NFS Client error Date: Mon, 08 Mar 2010 23:59:29 +0100 From: volker@vwsoft.com To: Giulio Ferro CC: freebsd-hackers@freebsd.org, freebsd-net@freebsd.org On 03/08/10 12:16, Giulio Ferro wrote: > Freebsd 8 stable amd64 > > It mounts different file systems by NFS (with locking) on a > data server directly connected (gigabit) to the server > > Apache running in a several jails on those nfs folders. > > Now and then I get huge slow-down. When I look in the logs > I get thousand of lines like these: > Mar 5 11:50:52 virt2 kernel: vm_fault: pager read error, pid 46487 (httpd) > Mar 5 11:50:52 virt2 kernel: pid 46487 (httpd), uid 80: exited on > signal 11 > > > What should I do? Giulio, it seems this is anyhow not related to network (nfs) operations. It's looking like a problem in the VM. I think it makes sense to have a look at the httpd.core file if the binary has been linked with debugging symbols turned on. Also I think at first, it may not hurt to look at vmstat -m output. You may want to change ${subject} and post to stable@ to drive more attention to your problem. Volker From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 09:14:20 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1232B106566B for ; Tue, 9 Mar 2010 09:14:20 +0000 (UTC) (envelope-from danny@cs.huji.ac.il) Received: from kabab.cs.huji.ac.il (kabab.cs.huji.ac.il [132.65.16.84]) by mx1.freebsd.org (Postfix) with ESMTP id BEE258FC12 for ; Tue, 9 Mar 2010 09:14:19 +0000 (UTC) Received: from pampa.cs.huji.ac.il ([132.65.80.32]) by kabab.cs.huji.ac.il with esmtp id 1NovW3-0002JF-Pg; Tue, 09 Mar 2010 11:14:15 +0200 X-Mailer: exmh version 2.7.2 01/07/2005 with nmh-1.2 To: Giulio Ferro In-reply-to: <4B960ADE.2070303@zirakzigil.org> References: <4B960ADE.2070303@zirakzigil.org> Comments: In-reply-to Giulio Ferro message dated "Tue, 09 Mar 2010 09:46:22 +0100." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Tue, 09 Mar 2010 11:14:15 +0200 From: Daniel Braniss Message-ID: Cc: volker@vwsoft.com, freebsd-stable@freebsd.org Subject: Re: Fwd: Re: NFS Client error X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 09:14:20 -0000 > Thanks for your kind reply, I'm forwarding it there... > > > -------- Original Message -------- > Subject: Re: NFS Client error > Date: Mon, 08 Mar 2010 23:59:29 +0100 > From: volker@vwsoft.com > To: Giulio Ferro > CC: freebsd-hackers@freebsd.org, freebsd-net@freebsd.org > > > > On 03/08/10 12:16, Giulio Ferro wrote: > > Freebsd 8 stable amd64 > > > > It mounts different file systems by NFS (with locking) on a > > data server directly connected (gigabit) to the server > > > > Apache running in a several jails on those nfs folders. > > > > Now and then I get huge slow-down. When I look in the logs > > I get thousand of lines like these: > > Mar 5 11:50:52 virt2 kernel: vm_fault: pager read error, pid 46487 (httpd) > > Mar 5 11:50:52 virt2 kernel: pid 46487 (httpd), uid 80: exited on > > signal 11 > > > > > > What should I do? If the binary (httpd) is on a nfs server, then if the binary got modified this is what usualy happens my 2c danny > > Giulio, > > it seems this is anyhow not related to network (nfs) operations. It's > looking like a problem in the VM. I think it makes sense to have a look > at the httpd.core file if the binary has been linked with debugging > symbols turned on. Also I think at first, it may not hurt to look at > vmstat -m output. > > You may want to change ${subject} and post to stable@ to drive more > attention to your problem. > > Volker > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 09:15:55 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9021E106566B for ; Tue, 9 Mar 2010 09:15:55 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [212.12.50.234]) by mx1.freebsd.org (Postfix) with ESMTP id 5BC298FC15 for ; Tue, 9 Mar 2010 09:15:55 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 1AC1D4D7E0 for ; Tue, 9 Mar 2010 09:15:54 +0000 (UTC) From: Stefan Bethke Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Date: Tue, 9 Mar 2010 10:15:53 +0100 Message-Id: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> To: FreeBSD Stable Mime-Version: 1.0 (Apple Message framework v1077) X-Mailer: Apple Mail (2.1077) Subject: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 09:15:55 -0000 Over the past couple of months, I've more or less regularly observed = machines having more and more processes stuck in the zfs wchan. The = processes never recover from that, and trying to reboot only gets the = entire system stuck, without any console messages. I can enter the = debugger, and I have saved a couple of dumps. The situation seems to be triggered by zfs receive'ing snapshots from = the sister machine (both synchronize their active ZFS filesystems to = each other, using zfs send and zfs receive). It appears it's the = receiving causing trouble. Both machines run 8-stable from mid-February, with a single-disk ZFS = pool, with ARC limited to 512M, prefetch and ZIL disabled via = loader.conf. What should I be looking at to further diagnose? Thanks, Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 09:18:24 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1A87F106575E; Tue, 9 Mar 2010 09:18:23 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 28EDD8FC0C; Tue, 9 Mar 2010 09:18:22 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 903B045C9F; Tue, 9 Mar 2010 10:18:20 +0100 (CET) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 86E5B45C9B; Tue, 9 Mar 2010 10:18:15 +0100 (CET) Date: Tue, 9 Mar 2010 10:18:16 +0100 From: Pawel Jakub Dawidek To: Steve Polyack Message-ID: <20100309091815.GB3155@garage.freebsd.pl> References: <4B953C92.5080606@comcast.net> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="s9fJI615cBHmzTOP" Content-Disposition: inline In-Reply-To: <4B953C92.5080606@comcast.net> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, Michel Bouissou , freebsd-stable Subject: Re: ZFS hot spares X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 09:18:24 -0000 --s9fJI615cBHmzTOP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Mar 08, 2010 at 01:06:10PM -0500, Steve Polyack wrote: > ZFS in FreeBSD lacks at least one major feature from the Solaris=20 > version: hot spares. There is a PR open at=20 > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D134491, but there hasn't bee= n=20 > any motion/thoughts posted on it since its creation almost one year ago. >=20 > I'm aware that on Solaris, hot spare replacement is handled by a few=20 > Solaris-specific daemons, zfs-retire and zfs-diagnose, which both plug=20 > into the Solaris FMA (Fault Management Architecture). Have there been=20 > any thoughts on porting these over or getting something similar running= =20 > within FreeBSD? With all of the recent SATA/SAS CAM hotplug work now=20 > committed, it would be nice to have automatic replacement of hot spares= =20 > with a future hot-replacement of the failed drive. >=20 > On the other side, I'd be interested in hearing if anyone has had=20 > success in rolling their own scripted solution: i.e. something which=20 > polls 'zpool status' looking for failed drives and performing hot-spare= =20 > replacements automatically. Currently FreeBSD's ZFS sends various events to devd. It should be possible to implement some scripts (or maybe reuse zfs-retire/zfs-diagnose?) to perform 'zpool replace' when disk disappears, etc. This shouldn't be very hard modulo bugs in FreeBSD/ZFS as this functionality, because unused, wasn't tested. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --s9fJI615cBHmzTOP Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuWElcACgkQForvXbEpPzQuMACfTjbHEMV5MsAbZmmP7rK4yw0y aIMAoO1gVToaXGpeGNsUwvS3SPC5KuzC =bzbO -----END PGP SIGNATURE----- --s9fJI615cBHmzTOP-- From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 09:28:08 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9D5F6106564A for ; Tue, 9 Mar 2010 09:28:08 +0000 (UTC) (envelope-from auryn@zirakzigil.org) Received: from mail.zirakzigil.org (mail.zirakzigil.org [82.63.178.63]) by mx1.freebsd.org (Postfix) with ESMTP id 0B77C8FC15 for ; Tue, 9 Mar 2010 09:28:07 +0000 (UTC) Received: from localhost (unknown [192.168.1.2]) by mail.zirakzigil.org (Postfix) with ESMTP id EEC7C94F18; Tue, 9 Mar 2010 10:28:06 +0100 (CET) X-Virus-Scanned: amavisd-new at zirakzigil.org Received: from mail.zirakzigil.org ([192.168.1.2]) by localhost (ext.zirakzigil.org [192.168.1.2]) (amavisd-new, port 10024) with ESMTP id SYF6lv-NqLpr; Tue, 9 Mar 2010 10:28:04 +0100 (CET) Received: from aurynmob2.giulioferro.it (unknown [192.168.1.2]) (Authenticated sender: auryn@zirakzigil.org) by mail.zirakzigil.org (Postfix) with ESMTPA id 620B894F0F; Tue, 9 Mar 2010 10:28:04 +0100 (CET) Message-ID: <4B9614A3.4090404@zirakzigil.org> Date: Tue, 09 Mar 2010 10:28:03 +0100 From: Giulio Ferro User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.7) Gecko/20100223 Thunderbird/3.0.1 MIME-Version: 1.0 To: Daniel Braniss , freebsd-stable@freebsd.org References: <4B960ADE.2070303@zirakzigil.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Fwd: Re: NFS Client error X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 09:28:08 -0000 On 09.03.2010 10:14, Daniel Braniss wrote: >> Thanks for your kind reply, I'm forwarding it there... >> >> >> -------- Original Message -------- >> Subject: Re: NFS Client error >> Date: Mon, 08 Mar 2010 23:59:29 +0100 >> From: volker@vwsoft.com >> To: Giulio Ferro >> CC: freebsd-hackers@freebsd.org, freebsd-net@freebsd.org >> >> >> >> On 03/08/10 12:16, Giulio Ferro wrote: >> >>> Freebsd 8 stable amd64 >>> >>> It mounts different file systems by NFS (with locking) on a >>> data server directly connected (gigabit) to the server >>> >>> Apache running in a several jails on those nfs folders. >>> >>> Now and then I get huge slow-down. When I look in the logs >>> I get thousand of lines like these: >>> Mar 5 11:50:52 virt2 kernel: vm_fault: pager read error, pid 46487 (httpd) >>> Mar 5 11:50:52 virt2 kernel: pid 46487 (httpd), uid 80: exited on >>> signal 11 >>> >>> >>> What should I do? >>> > If the binary (httpd) is on a nfs server, then if the binary got > modified this is what usualy happens > Nope. The binary is on the jails on the local machine. Only the configuration dir (etc/apache22) and data dir (www) in on the nfs server. ------------------------ |NFS CLIENT | | jail 1 : httpd | | jail 2 : httpd | ------> NFS SERVER | jail 3 : httpd | |... | ----------------------- Giulio. > my 2c > danny > > >> Giulio, >> >> it seems this is anyhow not related to network (nfs) operations. It's >> looking like a problem in the VM. I think it makes sense to have a look >> at the httpd.core file if the binary has been linked with debugging >> symbols turned on. Also I think at first, it may not hurt to look at >> vmstat -m output. >> >> You may want to change ${subject} and post to stable@ to drive more >> attention to your problem. >> >> Volker >> >> >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >> >> > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 10:27:46 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 17047106564A for ; Tue, 9 Mar 2010 10:27:46 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id 681588FC17 for ; Tue, 9 Mar 2010 10:27:44 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id o29ARZiV068726; Tue, 9 Mar 2010 21:27:35 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Tue, 9 Mar 2010 21:27:35 +1100 (EST) From: Ian Smith To: Dominic Fandrey In-Reply-To: <4B9601D9.5030101@bsdforen.de> Message-ID: <20100309210949.F85436@sola.nimnet.asn.au> References: <4B9601D9.5030101@bsdforen.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-stable@freebsd.org Subject: Re: ntpd does not re-query servers, when a new interface appears X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 10:27:46 -0000 On Tue, 9 Mar 2010, Dominic Fandrey wrote: > ntpd tracks interface updates, however it does not requery > servers, when they occur. This was less than an hour ago, > at my university, the notebook boots and is not connected > to anything: > > 9 Mar 08:07:17 ntpd[1510]: logging to file /var/log/ntpd > 9 Mar 08:07:17 ntpd[1510]: precision = 2.234 usec > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #1 wildcard, ::#123 Disabled > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #2 bge0, 192.168.1.12#123 Enabled > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #3 lo0, fe80::1#123 Enabled > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #4 lo0, ::1#123 Enabled > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #5 lo0, 127.0.0.1#123 Enabled > 9 Mar 08:07:17 ntpd[1510]: Listening on routing socket on fd #26 for interface updates > 9 Mar 08:07:17 ntpd[1510]: kernel time sync status 2040 > 9 Mar 08:07:17 ntpd[1510]: frequency initialized 3.155 PPM from /var/db/ntpd.drift > 9 Mar 08:07:20 ntpd[1542]: host name not found: 0.de.pool.ntp.org > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `0.de.pool.ntp.org', giving up on it > 9 Mar 08:07:20 ntpd[1542]: host name not found: 1.de.pool.ntp.org > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `1.de.pool.ntp.org', giving up on it > 9 Mar 08:07:20 ntpd[1542]: host name not found: 2.de.pool.ntp.org > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `2.de.pool.ntp.org', giving up on it > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp3.rz.uni-karlsruhe.de > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp3.rz.uni-karlsruhe.de', giving up on it > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp4.rz.uni-karlsruhe.de > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp4.rz.uni-karlsruhe.de', giving up on it > > So ntpd has given up on all the servers listed in the ntp.conf file. Yes, but it looks more like name service that's not operating, ntpd seems to be doing its best but can't resolve the hostnames? > I then proceed to connect to the wireless network and proceed to log > into two VPNs: > > 9 Mar 08:08:58 ntpd[1510]: Listening on interface #6 wlan0, 192.168.75.58#123 Enabled > 9 Mar 08:09:00 ntpd[1510]: Listening on interface #7 tun0, 193.196.120.15#123 Enabled > 9 Mar 08:09:04 ntpd[1510]: Listening on interface #8 tun1, 141.3.162.67#123 Enabled > > Over interface #8 some of the servers are actually available, but > ntpq -p still states: > No association ID's returned > > Only when I restart ntpd, it operates as expected: > remote refid st t when poll reach delay offset jitter > ============================================================================== > zit-net2.uni-pa .STEP. 16 u - 512 0 0.000 0.000 0.000 > alpha.rueckgr.a .STEP. 16 u - 512 0 0.000 0.000 0.000 > ntp.goneco.de .STEP. 16 u - 512 0 0.000 0.000 0.000 > +proxy4.rz.uni-k 129.13.64.17 2 u 30 128 271 2.937 2.530 1.891 > +proxy2.rz.uni-k 129.13.64.17 2 u 58 128 375 3.593 -8.981 1.837 > *proxy1.rz.uni-k 129.13.64.17 2 u 15 128 271 3.297 8.244 1.487 I've always had to restart named after losing / regaining an interface, most noticeably after a suspend/resume (eg a low battery suspend), so I run /etc/rc.d/named restart from rc.resume. This looks like a similar issue perhaps, though I don't see why restarting only ntpd would fix it. HTH, Ian From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 10:54:00 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DA44B106566B for ; Tue, 9 Mar 2010 10:54:00 +0000 (UTC) (envelope-from peterjeremy@acm.org) Received: from mail35.syd.optusnet.com.au (mail35.syd.optusnet.com.au [211.29.133.51]) by mx1.freebsd.org (Postfix) with ESMTP id 6B53A8FC13 for ; Tue, 9 Mar 2010 10:54:00 +0000 (UTC) Received: from server.vk2pj.dyndns.org (c122-106-253-149.belrs3.nsw.optusnet.com.au [122.106.253.149]) by mail35.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id o29AruGW020916 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 9 Mar 2010 21:53:57 +1100 X-Bogosity: Ham, spamicity=0.000000 Received: from server.vk2pj.dyndns.org (localhost.vk2pj.dyndns.org [127.0.0.1]) by server.vk2pj.dyndns.org (8.14.3/8.14.3) with ESMTP id o29ArtMo005360; Tue, 9 Mar 2010 21:53:55 +1100 (EST) (envelope-from peter@server.vk2pj.dyndns.org) Received: (from peter@localhost) by server.vk2pj.dyndns.org (8.14.3/8.14.3/Submit) id o29Artts005359; Tue, 9 Mar 2010 21:53:55 +1100 (EST) (envelope-from peter) Date: Tue, 9 Mar 2010 21:53:55 +1100 From: Peter Jeremy To: Stefan Bethke Message-ID: <20100309105355.GD3978@server.vk2pj.dyndns.org> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="4zI0WCX1RcnW9Hbu" Content-Disposition: inline In-Reply-To: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> X-PGP-Key: http://members.optusnet.com.au/peterjeremy/pubkey.asc User-Agent: Mutt/1.5.20 (2009-06-14) Cc: FreeBSD Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 10:54:00 -0000 --4zI0WCX1RcnW9Hbu Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2010-Mar-09 10:15:53 +0100, Stefan Bethke wrote: >Over the past couple of months, I've more or less regularly observed machi= nes having more and more processes stuck in the zfs wchan. The processes n= ever recover from that, How long have you waited? There seems to be a problem with low free memory handling that causes ZFS to turn into cold molasses. The work-around is to run a program that allocates a decent size chunk of memory and then exits. The original suggestion was something like: perl -e '@x =3D (0) x 1000000;' I've written a short program that allocates and dirties ~100MB and then exits and run it from cron. --=20 Peter Jeremy --4zI0WCX1RcnW9Hbu Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuWKMMACgkQ/opHv/APuIfMnACfXdRRLq7lnwMLvxpAIXxBySj/ QtgAmwUTQ5iSzx0QuxXcjq3kT0VKn8fb =L+vJ -----END PGP SIGNATURE----- --4zI0WCX1RcnW9Hbu-- From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 10:58:08 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B6B89106564A for ; Tue, 9 Mar 2010 10:58:08 +0000 (UTC) (envelope-from defree@gmail.com) Received: from mail-fx0-f215.google.com (mail-fx0-f215.google.com [209.85.220.215]) by mx1.freebsd.org (Postfix) with ESMTP id 471D68FC16 for ; Tue, 9 Mar 2010 10:58:08 +0000 (UTC) Received: by fxm7 with SMTP id 7so2798903fxm.14 for ; Tue, 09 Mar 2010 02:58:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:subject :message-id:in-reply-to:references:x-mailer:mime-version :content-type:content-transfer-encoding; bh=FOzyRN18gfzdKTEp0+bYzNi0oT3ZaR1LVMK+I6xivyI=; b=F16kxqp6W203FbGb2ZyQuKgQSp4rzLAzzOt7exMSy9jITsUg9aBvr8NUim+hPNsmNo 2HY/uutmfnPN2ZFyBv+SgFGfUBIKLo54cd3E2c6jCo9m/tf6GxIQKu7Vm8bbUswbp8m9 ulmo+XF2D2rPqqvyC193wgX03Q1aQxEyUoOn0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:subject:message-id:in-reply-to:references :x-mailer:mime-version:content-type:content-transfer-encoding; b=tQ3eSBTxQaF2cBwZ69q0JoajeJORRWH6GcqHm479Dt6rAJVCiw3wJYw9jATmSb4o8u gQHbVcwAjmd1ZBHwfkLJpmFPWn8BTy9cYKj6GmFUZ538jeBJhb3dszmdm4Jphwe5ZI66 mWnJSdgeKDxukvUWN7ywqTNItGDaSLNAv1hIY= Received: by 10.103.78.30 with SMTP id f30mr4396505mul.53.1268130477255; Tue, 09 Mar 2010 02:27:57 -0800 (PST) Received: from localhost (www.spiderboy.fr [88.184.64.205]) by mx.google.com with ESMTPS id e9sm28526563muf.10.2010.03.09.02.27.56 (version=SSLv3 cipher=RC4-MD5); Tue, 09 Mar 2010 02:27:57 -0800 (PST) Sender: =?UTF-8?B?RnLDqWTDqXJpYyBCb3Vy?= Date: Tue, 9 Mar 2010 11:27:41 +0100 From: =?UTF-8?B?RnLDqWTDqXJpYw==?= Bour To: freebsd-stable@freebsd.org Message-ID: <20100309112741.66108c8e@ygg-drasil.fr> In-Reply-To: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> X-Mailer: Claws Mail 3.7.5 (GTK+ 2.18.7; amd64-portbld-freebsd8.0) Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 10:58:08 -0000 Le Tue, 9 Mar 2010 10:15:53 +0100, Stefan Bethke a =C3=A9crit : > Over the past couple of months, I've more or less regularly observed > machines having more and more processes stuck in the zfs wchan. The > processes never recover from that, and trying to reboot only gets the > entire system stuck, without any console messages. I can enter the > debugger, and I have saved a couple of dumps. >=20 > The situation seems to be triggered by zfs receive'ing snapshots from > the sister machine (both synchronize their active ZFS filesystems to > each other, using zfs send and zfs receive). It appears it's the > receiving causing trouble. >=20 > Both machines run 8-stable from mid-February, with a single-disk ZFS > pool, with ARC limited to 512M, prefetch and ZIL disabled via > loader.conf. >=20 > What should I be looking at to further diagnose? >=20 >=20 > Thanks, > Stefan >=20 Hi, I encounter almost the same problem with a 8-STABLE build from the same time. When working a lot on files inside ~/, the directory get locked, any command trying to access it (from "ls" to application reading their configuration files) get stuck. The system is an amd64 desktop computer, 4GiB of memory and vfs.zfs.prefetch_disable is set to 0. From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 10:58:12 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 00C251065670 for ; Tue, 9 Mar 2010 10:58:12 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [212.12.50.234]) by mx1.freebsd.org (Postfix) with ESMTP id BDDCF8FC18 for ; Tue, 9 Mar 2010 10:58:11 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 8FB0F4F117; Tue, 9 Mar 2010 10:58:10 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <20100309105355.GD3978@server.vk2pj.dyndns.org> Date: Tue, 9 Mar 2010 11:58:10 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309105355.GD3978@server.vk2pj.dyndns.org> To: Peter Jeremy X-Mailer: Apple Mail (2.1077) Cc: FreeBSD Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 10:58:12 -0000 Am 09.03.2010 um 11:53 schrieb Peter Jeremy: > On 2010-Mar-09 10:15:53 +0100, Stefan Bethke wrote: >> Over the past couple of months, I've more or less regularly observed = machines having more and more processes stuck in the zfs wchan. The = processes never recover from that, >=20 > How long have you waited? Many hours, sometimes up to 48 hours (when I didn't notice the stuck = processes at first). > There seems to be a problem with low free memory handling that causes = ZFS > to turn into cold molasses. The work-around is to run a program that > allocates a decent size chunk of memory and then exits. The original > suggestion was something like: > perl -e '@x =3D (0) x 1000000;' > I've written a short program that allocates and dirties ~100MB and = then > exits and run it from cron. I'll try that the next time I encounter the stuck processes. I'm recording ZFS ARC stats with munin, would I be able to identify such = a low memory situation from there? Would it make sense to monitor other = stats? Thanks, Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 12:19:37 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 82BBD106566C for ; Tue, 9 Mar 2010 12:19:37 +0000 (UTC) (envelope-from korad@ua.fm) Received: from web01.mi6.kiev.ua (web01.mi6.kiev.ua [91.198.36.2]) by mx1.freebsd.org (Postfix) with ESMTP id 095128FC19 for ; Tue, 9 Mar 2010 12:19:36 +0000 (UTC) Received: from web05.mi6 ([10.0.0.12] helo=web05.mi6.kiev.ua) by web01.mi6.kiev.ua with esmtp (Exim 4.69) (envelope-from ) id 1NoxLv-00011f-SU for freebsd-stable@freebsd.org; Tue, 09 Mar 2010 13:11:56 +0200 Received: from web by web05.mi6.kiev.ua with local (Exim 4.63) (envelope-from ) id 1NoxLy-0003Wr-I6 for freebsd-stable@freebsd.org; Tue, 09 Mar 2010 13:11:58 +0200 To: freebsd-stable@freebsd.org Date: Tue, 09 Mar 2010 13:11:58 +0200 From: Sergey Radashevskiy MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=windows-1251 Content-Transfer-Encoding: QUOTED-PRINTABLE X-User-Agent: Opera/9.80 (Windows NT 5.1; U; ru) Presto/2.2.15 Version/10.10 X-Sender-IP: 91.199.115.115 X-Mailer: I.UA Mail System Message-Id: Subject: freebsd 7.2stable em0: discard frame w/o packet header X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 12:19:37 -0000 The system panic with error kernel: em0: discard frame w/o packet header Crach dump one kgdb /boot/kernel/kernel /var/crash/vmcore.5 Fatal trap 12: page fault while in kernel mode cpuid =3D 0; apic id =3D 00 fault virtual address =3D 0xd9c6f105 fault code =3D supervisor read, page not present instruction pointer =3D 0x20:0x808a021f stack pointer =3D 0x28:0x85909c10 frame pointer =3D 0x28:0x85909c44 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, def32 1, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 14 (swi4: clock sio) trap number =3D 12 panic: page fault cpuid =3D 0 Uptime: 11h51m2s Physical memory: 1936 MB Dumping 331 MB: 316 300 284 268 252 236 220 204 188 172 156 140 124 108 92 = 76 60 44 28 12 (kgdb) where #0 doadump () at pcpu.h:196 #1 0x8069a618 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:4= 18 #2 0x8069a8f5 in panic (fmt=3DVariable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0x8095f294 in trap_fatal (frame=3D0x85909bd0, eva=3D3653693701) at /usr= /src/sys/i386/i386/trap.c:950 #4 0x8095f4fd in trap_pfault (frame=3D0x85909bd0, usermode=3D0, eva=3D3653= 693701) at /usr/src/sys/i386/i386/trap.c:863 #5 0x8095febd in trap (frame=3D0x85909bd0) at /usr/src/sys/i386/i386/trap.= c:541 #6 0x8094479b in calltrap () at /usr/src/sys/i386/i386/exception.s:166 #7 0x808a021f in uma_zfree_arg (zone=3D0x86a3b000, item=3D0x8cdcd580, udat= a=3D0x868fa000) at /usr/src/sys/vm/uma_core.c:2253 #8 0x8075492b in ng_netflow_expire (arg=3D0x868fa000) at /usr/src/sys/netg= raph/netflow/netflow.c:215 #9 0x806ac441 in softclock (dummy=3D0x0) at /usr/src/sys/kern/kern_timeout= .c:274 #10 0x806789c9 in ithread_loop (arg=3D0x85c9d3d0) at /usr/src/sys/kern/kern= _intr.c:1181 #11 0x80675271 in fork_exit (callout=3D0x8067882d , arg=3D0x8= 5c9d3d0, frame=3D0x85909d38) at /usr/src/sys/kern/kern_fork.c:811 #12 0x80944810 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:= 271 Crach dump two kgdb /boot/kernel.old/kernel /var/crash/vmcore.4 Fatal trap 12: page fault while in kernel mode cpuid =3D 1; apic id =3D 01 fault virtual address =3D 0xc fault code =3D supervisor read, page not present instruction pointer =3D 0x20:0x806ec864 stack pointer =3D 0x28:0x85912a64 frame pointer =3D 0x28:0x85912a90 code segment =3D base 0x0, limit 0xfffff, type 0x1b =3D DPL 0, pres 1, def32 1, gran 1 processor eflags =3D interrupt enabled, resume, IOPL =3D 0 current process =3D 3 (ng_queue1) trap number =3D 12 panic: page fault cpuid =3D 1 Uptime: 12h4m33s Physical memory: 1936 MB Dumping 154 MB: 139 123 107 91 75 59 43 27 11 (kgdb) where #0 doadump () at pcpu.h:196 #1 0x8069a618 in boot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:4= 18 #2 0x8069a8f5 in panic (fmt=3DVariable "fmt" is not available. ) at /usr/src/sys/kern/kern_shutdown.c:574 #3 0x8095f294 in trap_fatal (frame=3D0x85912a24, eva=3D12) at /usr/src/sys= /i386/i386/trap.c:950 #4 0x8095f4fd in trap_pfault (frame=3D0x85912a24, usermode=3D0, eva=3D12) = at /usr/src/sys/i386/i386/trap.c:863 #5 0x8095febd in trap (frame=3D0x85912a24) at /usr/src/sys/i386/i386/trap.= c:541 #6 0x8094479b in calltrap () at /usr/src/sys/i386/i386/exception.s:166 #7 0x806ec864 in m_copym (m=3D0x0, off0=3D1496, len=3D1496, wait=3D1) at /= usr/src/sys/kern/uipc_mbuf.c:539 #8 0x807912a3 in ip_fragment (ip=3D0x862146d6, m_frag=3D0x85912b5c, mtu=3D= 1500, if_hwassist_flags=3D38, sw_csum=3D1) at /usr/src/sys/netinet/ip_output.c:731 #9 0x80791f4d in ip_output (m=3D0x86214600, opt=3D0x0, ro=3D0x85912b30, fl= ags=3D32, imo=3D0x0, inp=3D0x88518bf4) at /usr/src/sys/netinet/ip_output.c:570 #10 0x8079332f in rip_output (m=3D0x86341d00, so=3D0x87a86d00, dst=3D354052= 1226) at /usr/src/sys/netinet/raw_ip.c:408 #11 0x807933f4 in rip_send (so=3D0x87a86d00, flags=3D0, m=3D0x86341d00, nam= =3D0x0, control=3D0x0, td=3D0x85cff480) at /usr/src/sys/netinet/raw_ip.c:880 #12 0x806f590d in sosend_generic (so=3D0x87a86d00, addr=3D0x0, uio=3D0x0, t= op=3D0x86341d00, control=3D0x0, flags=3D0, td=3D0x85cff480) at /usr/src/sys/kern/uipc_socket.c:1243 #13 0x806f1767 in sosend (so=3D0x87a86d00, addr=3D0x0, uio=3D0x0, top=3D0x8= 6341d00, control=3D0x0, flags=3D0, td=3D0x85cff480) at /usr/src/sys/kern/uipc_socket.c:1285 #14 0x807652e4 in ng_ksocket_rcvdata (hook=3D0x89164180, item=3D0x86925a50)= at /usr/src/sys/netgraph/ng_ksocket.c:927 #15 0x8075986f in ng_apply_item (node=3D0x88f33900, item=3D0x86925a50, rw= =3D0) at /usr/src/sys/netgraph/ng_base.c:2336 #16 0x8075acf5 in ngthread (arg=3D0x0) at /usr/src/sys/netgraph/ng_base.c:3= 304 #17 0x80675271 in fork_exit (callout=3D0x8075aaa9 , arg=3D0x0, fr= ame=3D0x85912d38) at /usr/src/sys/kern/kern_fork.c:811 #18 0x80944810 in fork_trampoline () at /usr/src/sys/i386/i386/exception.s:= 271 -- =F0=E5=EA=EB=E0=EC=E0 --------------------------------------------------= --------- =C0=EA=F6=E8=FF! =CF=F0=E8 =EF=EE=EA=F3=EF=EA=E5 =F5=EE=F1=F2=E8=ED=E3=E0 = =E1=EE=EB=FC=F8=E5 =EC=E5=F1=F2=E0 =E8 =E4=EE=EC=E5=ED =E2 =EF=EE=E4=E0=F0= =EE=EA. http://FREEhost.UA From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 12:30:03 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 762FE106566B for ; Tue, 9 Mar 2010 12:30:03 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id AE2868FC15 for ; Tue, 9 Mar 2010 12:30:02 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4719A45CA0; Tue, 9 Mar 2010 13:30:00 +0100 (CET) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id B8ECD45683; Tue, 9 Mar 2010 13:29:53 +0100 (CET) Date: Tue, 9 Mar 2010 13:29:54 +0100 From: Pawel Jakub Dawidek To: Stefan Bethke Message-ID: <20100309122954.GE3155@garage.freebsd.pl> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+jhVVhN62yS6hEJ8" Content-Disposition: inline In-Reply-To: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: FreeBSD Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 12:30:03 -0000 --+jhVVhN62yS6hEJ8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 09, 2010 at 10:15:53AM +0100, Stefan Bethke wrote: > Over the past couple of months, I've more or less regularly observed mach= ines having more and more processes stuck in the zfs wchan. The processes = never recover from that, and trying to reboot only gets the entire system s= tuck, without any console messages. I can enter the debugger, and I have s= aved a couple of dumps. >=20 > The situation seems to be triggered by zfs receive'ing snapshots from the= sister machine (both synchronize their active ZFS filesystems to each othe= r, using zfs send and zfs receive). It appears it's the receiving causing = trouble. >=20 > Both machines run 8-stable from mid-February, with a single-disk ZFS pool= , with ARC limited to 512M, prefetch and ZIL disabled via loader.conf. >=20 > What should I be looking at to further diagnose? What kind of hardware do you have there? There is 3-way deadlock I've a fix for which would be hard to trigger on single or dual core machines. Feel free to try the fix: http://people.freebsd.org/~pjd/patches/zfs_3way_deadlock.patch --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --+jhVVhN62yS6hEJ8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuWP0EACgkQForvXbEpPzQCcQCfQqvddvCccbLw3EUv2m6jCi1u 1Y8An3FZEheJd24loScxPr/nrEy1kkM7 =wzoV -----END PGP SIGNATURE----- --+jhVVhN62yS6hEJ8-- From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 12:44:00 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D3E53106564A; Tue, 9 Mar 2010 12:44:00 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [212.12.50.234]) by mx1.freebsd.org (Postfix) with ESMTP id 9D4BE8FC1E; Tue, 9 Mar 2010 12:44:00 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 501FB4F63B; Tue, 9 Mar 2010 12:43:59 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <20100309122954.GE3155@garage.freebsd.pl> Date: Tue, 9 Mar 2010 13:43:58 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> To: Pawel Jakub Dawidek X-Mailer: Apple Mail (2.1077) Cc: FreeBSD Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 12:44:00 -0000 Am 09.03.2010 um 13:29 schrieb Pawel Jakub Dawidek: > On Tue, Mar 09, 2010 at 10:15:53AM +0100, Stefan Bethke wrote: >> Over the past couple of months, I've more or less regularly observed = machines having more and more processes stuck in the zfs wchan. The = processes never recover from that, and trying to reboot only gets the = entire system stuck, without any console messages. I can enter the = debugger, and I have saved a couple of dumps. >>=20 >> The situation seems to be triggered by zfs receive'ing snapshots from = the sister machine (both synchronize their active ZFS filesystems to = each other, using zfs send and zfs receive). It appears it's the = receiving causing trouble. >>=20 >> Both machines run 8-stable from mid-February, with a single-disk ZFS = pool, with ARC limited to 512M, prefetch and ZIL disabled via = loader.conf. >>=20 >> What should I be looking at to further diagnose? >=20 > What kind of hardware do you have there? There is 3-way deadlock I've = a > fix for which would be hard to trigger on single or dual core = machines. FreeBSD lokschuppen.zs64.net 8.0-STABLE FreeBSD 8.0-STABLE #24: Sat Feb = 13 11:20:03 UTC 2010 = root@lokschuppen.zs64.net:/usr/obj/usr/src/sys/EISENBOOT amd64 Copyrig ht (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-STABLE #24: Sat Feb 13 11:20:03 UTC 2010 root@lokschuppen.zs64.net:/usr/obj/usr/src/sys/EISENBOOT amd64 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM)2 Duo CPU E7300 @ 2.66GHz (2666.65-MHz = K8-class CPU) Origin =3D "GenuineIntel" Id =3D 0x10676 Stepping =3D 6 = Features=3D0xbfebfbff = Features2=3D0x8e39d AMD Features=3D0x20100800 AMD Features2=3D0x1 TSC: P-state invariant real memory =3D 4294967296 (4096 MB) avail memory =3D 4081422336 (3892 MB) > Feel free to try the fix: >=20 > http://people.freebsd.org/~pjd/patches/zfs_3way_deadlock.patch I'll give it a shot on one of the two boxes. Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 12:46:11 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AABB1065674 for ; Tue, 9 Mar 2010 12:46:11 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta14.emeryville.ca.mail.comcast.net (qmta14.emeryville.ca.mail.comcast.net [76.96.27.212]) by mx1.freebsd.org (Postfix) with ESMTP id 82ACC8FC19 for ; Tue, 9 Mar 2010 12:46:11 +0000 (UTC) Received: from omta01.emeryville.ca.mail.comcast.net ([76.96.30.11]) by qmta14.emeryville.ca.mail.comcast.net with comcast id r0Wx1d0020EPchoAE0mCFe; Tue, 09 Mar 2010 12:46:12 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta01.emeryville.ca.mail.comcast.net with comcast id r0mB1d0013S48mS8M0mBZK; Tue, 09 Mar 2010 12:46:11 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id C31561E3035; Tue, 9 Mar 2010 04:46:09 -0800 (PST) Date: Tue, 9 Mar 2010 04:46:09 -0800 From: Jeremy Chadwick To: freebsd-stable@freebsd.org Message-ID: <20100309124609.GA6331@icarus.home.lan> References: <4B9601D9.5030101@bsdforen.de> <20100309210949.F85436@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100309210949.F85436@sola.nimnet.asn.au> User-Agent: Mutt/1.5.20 (2009-06-14) Subject: Re: ntpd does not re-query servers, when a new interface appears X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 12:46:11 -0000 On Tue, Mar 09, 2010 at 09:27:35PM +1100, Ian Smith wrote: > On Tue, 9 Mar 2010, Dominic Fandrey wrote: > > ntpd tracks interface updates, however it does not requery > > servers, when they occur. This was less than an hour ago, > > at my university, the notebook boots and is not connected > > to anything: > > > > 9 Mar 08:07:17 ntpd[1510]: logging to file /var/log/ntpd > > 9 Mar 08:07:17 ntpd[1510]: precision = 2.234 usec > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #1 wildcard, ::#123 Disabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #2 bge0, 192.168.1.12#123 Enabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #3 lo0, fe80::1#123 Enabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #4 lo0, ::1#123 Enabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #5 lo0, 127.0.0.1#123 Enabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on routing socket on fd #26 for interface updates > > 9 Mar 08:07:17 ntpd[1510]: kernel time sync status 2040 > > 9 Mar 08:07:17 ntpd[1510]: frequency initialized 3.155 PPM from /var/db/ntpd.drift > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 0.de.pool.ntp.org > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `0.de.pool.ntp.org', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 1.de.pool.ntp.org > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `1.de.pool.ntp.org', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 2.de.pool.ntp.org > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `2.de.pool.ntp.org', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp3.rz.uni-karlsruhe.de > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp3.rz.uni-karlsruhe.de', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp4.rz.uni-karlsruhe.de > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp4.rz.uni-karlsruhe.de', giving up on it > > > > So ntpd has given up on all the servers listed in the ntp.conf file. > > Yes, but it looks more like name service that's not operating, ntpd > seems to be doing its best but can't resolve the hostnames? > > > I then proceed to connect to the wireless network and proceed to log > > into two VPNs: > > > > 9 Mar 08:08:58 ntpd[1510]: Listening on interface #6 wlan0, 192.168.75.58#123 Enabled > > 9 Mar 08:09:00 ntpd[1510]: Listening on interface #7 tun0, 193.196.120.15#123 Enabled > > 9 Mar 08:09:04 ntpd[1510]: Listening on interface #8 tun1, 141.3.162.67#123 Enabled > > > > Over interface #8 some of the servers are actually available, but > > ntpq -p still states: > > No association ID's returned > > > > Only when I restart ntpd, it operates as expected: > > remote refid st t when poll reach delay offset jitter > > ============================================================================== > > zit-net2.uni-pa .STEP. 16 u - 512 0 0.000 0.000 0.000 > > alpha.rueckgr.a .STEP. 16 u - 512 0 0.000 0.000 0.000 > > ntp.goneco.de .STEP. 16 u - 512 0 0.000 0.000 0.000 > > +proxy4.rz.uni-k 129.13.64.17 2 u 30 128 271 2.937 2.530 1.891 > > +proxy2.rz.uni-k 129.13.64.17 2 u 58 128 375 3.593 -8.981 1.837 > > *proxy1.rz.uni-k 129.13.64.17 2 u 15 128 271 3.297 8.244 1.487 > > I've always had to restart named after losing / regaining an interface, > most noticeably after a suspend/resume (eg a low battery suspend), so I > run /etc/rc.d/named restart from rc.resume. This looks like a similar > issue perhaps, though I don't see why restarting only ntpd would fix it. named is supposed to auto-probe for interfaces at a specific interval; see the "interface-interval" option. I forget what the default is, but on our servers we explicitly disable it by setting it to 0. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 12:57:09 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6B0C106566B; Tue, 9 Mar 2010 12:57:09 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from proxypop1.sarenet.es (proxypop1.sarenet.es [194.30.0.99]) by mx1.freebsd.org (Postfix) with ESMTP id 8A6BC8FC12; Tue, 9 Mar 2010 12:57:09 +0000 (UTC) Received: from [172.16.1.204] (unknown [192.148.167.2]) by proxypop1.sarenet.es (Postfix) with ESMTP id C740C5FDA; Tue, 9 Mar 2010 13:57:07 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <20100309122954.GE3155@garage.freebsd.pl> Date: Tue, 9 Mar 2010 13:57:07 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> To: Pawel Jakub Dawidek X-Mailer: Apple Mail (2.1077) Cc: FreeBSD Stable , Stefan Bethke Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 12:57:09 -0000 On Mar 9, 2010, at 1:29 PM, Pawel Jakub Dawidek wrote: > On Tue, Mar 09, 2010 at 10:15:53AM +0100, Stefan Bethke wrote: >> Over the past couple of months, I've more or less regularly observed = machines having more and more processes stuck in the zfs wchan. The = processes never recover from that, and trying to reboot only gets the = entire system stuck, without any console messages. I can enter the = debugger, and I have saved a couple of dumps. >>=20 >> The situation seems to be triggered by zfs receive'ing snapshots from = the sister machine (both synchronize their active ZFS filesystems to = each other, using zfs send and zfs receive). It appears it's the = receiving causing trouble. >>=20 >> Both machines run 8-stable from mid-February, with a single-disk ZFS = pool, with ARC limited to 512M, prefetch and ZIL disabled via = loader.conf. >>=20 >> What should I be looking at to further diagnose? >=20 > What kind of hardware do you have there? There is 3-way deadlock I've = a > fix for which would be hard to trigger on single or dual core = machines. >=20 > Feel free to try the fix: >=20 > http://people.freebsd.org/~pjd/patches/zfs_3way_deadlock.patch Maybe related to the deadlock I reported when I was receiving an = incremental snapshot while the target dataset was being read? Borja. From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 12:58:23 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 026391065670 for ; Tue, 9 Mar 2010 12:58:23 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 414F48FC2D for ; Tue, 9 Mar 2010 12:58:22 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4EF5045CAC; Tue, 9 Mar 2010 13:58:20 +0100 (CET) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id DD09C45CBA; Tue, 9 Mar 2010 13:58:14 +0100 (CET) Date: Tue, 9 Mar 2010 13:58:15 +0100 From: Pawel Jakub Dawidek To: Borja Marcos Message-ID: <20100309125815.GF3155@garage.freebsd.pl> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="+sHJum3is6Tsg7/J" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: FreeBSD Stable , Stefan Bethke Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 12:58:23 -0000 --+sHJum3is6Tsg7/J Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 09, 2010 at 01:57:07PM +0100, Borja Marcos wrote: >=20 > On Mar 9, 2010, at 1:29 PM, Pawel Jakub Dawidek wrote: >=20 > > On Tue, Mar 09, 2010 at 10:15:53AM +0100, Stefan Bethke wrote: > >> Over the past couple of months, I've more or less regularly observed m= achines having more and more processes stuck in the zfs wchan. The process= es never recover from that, and trying to reboot only gets the entire syste= m stuck, without any console messages. I can enter the debugger, and I hav= e saved a couple of dumps. > >>=20 > >> The situation seems to be triggered by zfs receive'ing snapshots from = the sister machine (both synchronize their active ZFS filesystems to each o= ther, using zfs send and zfs receive). It appears it's the receiving causi= ng trouble. > >>=20 > >> Both machines run 8-stable from mid-February, with a single-disk ZFS p= ool, with ARC limited to 512M, prefetch and ZIL disabled via loader.conf. > >>=20 > >> What should I be looking at to further diagnose? > >=20 > > What kind of hardware do you have there? There is 3-way deadlock I've a > > fix for which would be hard to trigger on single or dual core machines. > >=20 > > Feel free to try the fix: > >=20 > > http://people.freebsd.org/~pjd/patches/zfs_3way_deadlock.patch >=20 > Maybe related to the deadlock I reported when I was receiving an incremen= tal snapshot while the target dataset was being read? Could be. This deadlock is in general related to zfs recv functionality. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --+sHJum3is6Tsg7/J Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuWReYACgkQForvXbEpPzQXUgCff7LzvckBJCEu/KzhxEwApHCe hXcAoPS1vGVYm+6SnLr4LHP3k9+tdXWq =GWQu -----END PGP SIGNATURE----- --+sHJum3is6Tsg7/J-- From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 13:17:50 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1A0A1065670 for ; Tue, 9 Mar 2010 13:17:50 +0000 (UTC) (envelope-from kamikaze@bsdforen.de) Received: from mail.bsdforen.de (bsdforen.de [212.204.60.79]) by mx1.freebsd.org (Postfix) with ESMTP id 3077B8FC08 for ; Tue, 9 Mar 2010 13:17:50 +0000 (UTC) Received: from mobileKamikaze.norad (vpn-cl-162-232.rz.uni-karlsruhe.de [141.3.162.232]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.bsdforen.de (Postfix) with ESMTP id DEE968A178D; Tue, 9 Mar 2010 14:17:48 +0100 (CET) Message-ID: <4B964A7C.6010709@bsdforen.de> Date: Tue, 09 Mar 2010 14:17:48 +0100 From: Dominic Fandrey User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-GB; rv:1.9.1.8) Gecko/20100302 Thunderbird/3.0.3 MIME-Version: 1.0 To: Ian Smith References: <4B9601D9.5030101@bsdforen.de> <20100309210949.F85436@sola.nimnet.asn.au> In-Reply-To: <20100309210949.F85436@sola.nimnet.asn.au> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: ntpd does not re-query servers, when a new interface appears X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 13:17:50 -0000 On 09/03/2010 11:27, Ian Smith wrote: > On Tue, 9 Mar 2010, Dominic Fandrey wrote: > > ntpd tracks interface updates, however it does not requery > > servers, when they occur. This was less than an hour ago, > > at my university, the notebook boots and is not connected > > to anything: > > > > 9 Mar 08:07:17 ntpd[1510]: logging to file /var/log/ntpd > > 9 Mar 08:07:17 ntpd[1510]: precision = 2.234 usec > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #1 wildcard, ::#123 Disabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #2 bge0, 192.168.1.12#123 Enabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #3 lo0, fe80::1#123 Enabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #4 lo0, ::1#123 Enabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #5 lo0, 127.0.0.1#123 Enabled > > 9 Mar 08:07:17 ntpd[1510]: Listening on routing socket on fd #26 for interface updates > > 9 Mar 08:07:17 ntpd[1510]: kernel time sync status 2040 > > 9 Mar 08:07:17 ntpd[1510]: frequency initialized 3.155 PPM from /var/db/ntpd.drift > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 0.de.pool.ntp.org > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `0.de.pool.ntp.org', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 1.de.pool.ntp.org > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `1.de.pool.ntp.org', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 2.de.pool.ntp.org > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `2.de.pool.ntp.org', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp3.rz.uni-karlsruhe.de > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp3.rz.uni-karlsruhe.de', giving up on it > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp4.rz.uni-karlsruhe.de > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp4.rz.uni-karlsruhe.de', giving up on it > > > > So ntpd has given up on all the servers listed in the ntp.conf file. > > Yes, but it looks more like name service that's not operating, ntpd > seems to be doing its best but can't resolve the hostnames? Why would I have named running on a notebook? This is a notebook, which is not connected to the internet. > > I then proceed to connect to the wireless network and proceed to log > > into two VPNs: > > > > 9 Mar 08:08:58 ntpd[1510]: Listening on interface #6 wlan0, 192.168.75.58#123 Enabled > > 9 Mar 08:09:00 ntpd[1510]: Listening on interface #7 tun0, 193.196.120.15#123 Enabled > > 9 Mar 08:09:04 ntpd[1510]: Listening on interface #8 tun1, 141.3.162.67#123 Enabled > > > > Over interface #8 some of the servers are actually available, but > > ntpq -p still states: > > No association ID's returned > > > > Only when I restart ntpd, it operates as expected: > > remote refid st t when poll reach delay offset jitter > > ============================================================================== > > zit-net2.uni-pa .STEP. 16 u - 512 0 0.000 0.000 0.000 > > alpha.rueckgr.a .STEP. 16 u - 512 0 0.000 0.000 0.000 > > ntp.goneco.de .STEP. 16 u - 512 0 0.000 0.000 0.000 > > +proxy4.rz.uni-k 129.13.64.17 2 u 30 128 271 2.937 2.530 1.891 > > +proxy2.rz.uni-k 129.13.64.17 2 u 58 128 375 3.593 -8.981 1.837 > > *proxy1.rz.uni-k 129.13.64.17 2 u 15 128 271 3.297 8.244 1.487 > > I've always had to restart named after losing / regaining an interface, > most noticeably after a suspend/resume (eg a low battery suspend), so I > run /etc/rc.d/named restart from rc.resume. This looks like a similar > issue perhaps, though I don't see why restarting only ntpd would fix it. As I said, named doesn't run at all. When the notebook gets an internet connection, ntpd recognizes this. It somehow doesn't occur to it, though, that it might be able to resolve the servers, now. -- A: Because it fouls the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 13:30:49 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50078106566B for ; Tue, 9 Mar 2010 13:30:49 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta04.westchester.pa.mail.comcast.net (qmta04.westchester.pa.mail.comcast.net [76.96.62.40]) by mx1.freebsd.org (Postfix) with ESMTP id F1EB08FC0A for ; Tue, 9 Mar 2010 13:30:47 +0000 (UTC) Received: from omta01.westchester.pa.mail.comcast.net ([76.96.62.11]) by qmta04.westchester.pa.mail.comcast.net with comcast id qzgY1d0040EZKEL541Wo4b; Tue, 09 Mar 2010 13:30:48 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta01.westchester.pa.mail.comcast.net with comcast id r1Wm1d00C3S48mS3M1Wmhd; Tue, 09 Mar 2010 13:30:48 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 58C461E3035; Tue, 9 Mar 2010 05:30:45 -0800 (PST) Date: Tue, 9 Mar 2010 05:30:45 -0800 From: Jeremy Chadwick To: Dominic Fandrey Message-ID: <20100309133045.GA7232@icarus.home.lan> References: <4B9601D9.5030101@bsdforen.de> <20100309210949.F85436@sola.nimnet.asn.au> <4B964A7C.6010709@bsdforen.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B964A7C.6010709@bsdforen.de> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-stable@freebsd.org, Ian Smith Subject: Re: ntpd does not re-query servers, when a new interface appears X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 13:30:49 -0000 On Tue, Mar 09, 2010 at 02:17:48PM +0100, Dominic Fandrey wrote: > On 09/03/2010 11:27, Ian Smith wrote: > > On Tue, 9 Mar 2010, Dominic Fandrey wrote: > > > ntpd tracks interface updates, however it does not requery > > > servers, when they occur. This was less than an hour ago, > > > at my university, the notebook boots and is not connected > > > to anything: > > > > > > 9 Mar 08:07:17 ntpd[1510]: logging to file /var/log/ntpd > > > 9 Mar 08:07:17 ntpd[1510]: precision = 2.234 usec > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #1 wildcard, ::#123 Disabled > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #2 bge0, 192.168.1.12#123 Enabled > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #3 lo0, fe80::1#123 Enabled > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #4 lo0, ::1#123 Enabled > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #5 lo0, 127.0.0.1#123 Enabled > > > 9 Mar 08:07:17 ntpd[1510]: Listening on routing socket on fd #26 for interface updates > > > 9 Mar 08:07:17 ntpd[1510]: kernel time sync status 2040 > > > 9 Mar 08:07:17 ntpd[1510]: frequency initialized 3.155 PPM from /var/db/ntpd.drift > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 0.de.pool.ntp.org > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `0.de.pool.ntp.org', giving up on it > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 1.de.pool.ntp.org > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `1.de.pool.ntp.org', giving up on it > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 2.de.pool.ntp.org > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `2.de.pool.ntp.org', giving up on it > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp3.rz.uni-karlsruhe.de > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp3.rz.uni-karlsruhe.de', giving up on it > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp4.rz.uni-karlsruhe.de > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp4.rz.uni-karlsruhe.de', giving up on it > > > > > > So ntpd has given up on all the servers listed in the ntp.conf file. > > > > Yes, but it looks more like name service that's not operating, ntpd > > seems to be doing its best but can't resolve the hostnames? > > Why would I have named running on a notebook? This is a notebook, > which is not connected to the internet. > > > > I then proceed to connect to the wireless network and proceed to log > > > into two VPNs: > > > > > > 9 Mar 08:08:58 ntpd[1510]: Listening on interface #6 wlan0, 192.168.75.58#123 Enabled > > > 9 Mar 08:09:00 ntpd[1510]: Listening on interface #7 tun0, 193.196.120.15#123 Enabled > > > 9 Mar 08:09:04 ntpd[1510]: Listening on interface #8 tun1, 141.3.162.67#123 Enabled > > > > > > Over interface #8 some of the servers are actually available, but > > > ntpq -p still states: > > > No association ID's returned > > > > > > Only when I restart ntpd, it operates as expected: > > > remote refid st t when poll reach delay offset jitter > > > ============================================================================== > > > zit-net2.uni-pa .STEP. 16 u - 512 0 0.000 0.000 0.000 > > > alpha.rueckgr.a .STEP. 16 u - 512 0 0.000 0.000 0.000 > > > ntp.goneco.de .STEP. 16 u - 512 0 0.000 0.000 0.000 > > > +proxy4.rz.uni-k 129.13.64.17 2 u 30 128 271 2.937 2.530 1.891 > > > +proxy2.rz.uni-k 129.13.64.17 2 u 58 128 375 3.593 -8.981 1.837 > > > *proxy1.rz.uni-k 129.13.64.17 2 u 15 128 271 3.297 8.244 1.487 > > > > I've always had to restart named after losing / regaining an interface, > > most noticeably after a suspend/resume (eg a low battery suspend), so I > > run /etc/rc.d/named restart from rc.resume. This looks like a similar > > issue perhaps, though I don't see why restarting only ntpd would fix it. > > As I said, named doesn't run at all. When the notebook gets an > internet connection, ntpd recognizes this. It somehow doesn't > occur to it, though, that it might be able to resolve the > servers, now. I believe this is the problem. Note that you'll need to add an SSL cert. exception for this site due to them using self-signed certs. https://support.ntp.org/bugs/show_bug.cgi?id=987 -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 13:33:24 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 453A0106564A for ; Tue, 9 Mar 2010 13:33:24 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta06.emeryville.ca.mail.comcast.net (qmta06.emeryville.ca.mail.comcast.net [76.96.30.56]) by mx1.freebsd.org (Postfix) with ESMTP id 2B0CF8FC12 for ; Tue, 9 Mar 2010 13:33:23 +0000 (UTC) Received: from omta10.emeryville.ca.mail.comcast.net ([76.96.30.28]) by qmta06.emeryville.ca.mail.comcast.net with comcast id r0pp1d00A0cQ2SLA61ZQqH; Tue, 09 Mar 2010 13:33:24 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta10.emeryville.ca.mail.comcast.net with comcast id r1ZP1d0033S48mS8W1ZPVp; Tue, 09 Mar 2010 13:33:24 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 35B7C1E3035; Tue, 9 Mar 2010 05:33:22 -0800 (PST) Date: Tue, 9 Mar 2010 05:33:22 -0800 From: Jeremy Chadwick To: Dominic Fandrey Message-ID: <20100309133322.GA7362@icarus.home.lan> References: <4B9601D9.5030101@bsdforen.de> <20100309210949.F85436@sola.nimnet.asn.au> <4B964A7C.6010709@bsdforen.de> <20100309133045.GA7232@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100309133045.GA7232@icarus.home.lan> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-stable@freebsd.org, Ian Smith Subject: Re: ntpd does not re-query servers, when a new interface appears X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 13:33:24 -0000 On Tue, Mar 09, 2010 at 05:30:45AM -0800, Jeremy Chadwick wrote: > On Tue, Mar 09, 2010 at 02:17:48PM +0100, Dominic Fandrey wrote: > > On 09/03/2010 11:27, Ian Smith wrote: > > > On Tue, 9 Mar 2010, Dominic Fandrey wrote: > > > > ntpd tracks interface updates, however it does not requery > > > > servers, when they occur. This was less than an hour ago, > > > > at my university, the notebook boots and is not connected > > > > to anything: > > > > > > > > 9 Mar 08:07:17 ntpd[1510]: logging to file /var/log/ntpd > > > > 9 Mar 08:07:17 ntpd[1510]: precision = 2.234 usec > > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled > > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #1 wildcard, ::#123 Disabled > > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #2 bge0, 192.168.1.12#123 Enabled > > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #3 lo0, fe80::1#123 Enabled > > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #4 lo0, ::1#123 Enabled > > > > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #5 lo0, 127.0.0.1#123 Enabled > > > > 9 Mar 08:07:17 ntpd[1510]: Listening on routing socket on fd #26 for interface updates > > > > 9 Mar 08:07:17 ntpd[1510]: kernel time sync status 2040 > > > > 9 Mar 08:07:17 ntpd[1510]: frequency initialized 3.155 PPM from /var/db/ntpd.drift > > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 0.de.pool.ntp.org > > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `0.de.pool.ntp.org', giving up on it > > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 1.de.pool.ntp.org > > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `1.de.pool.ntp.org', giving up on it > > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: 2.de.pool.ntp.org > > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `2.de.pool.ntp.org', giving up on it > > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de > > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it > > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de > > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it > > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp3.rz.uni-karlsruhe.de > > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp3.rz.uni-karlsruhe.de', giving up on it > > > > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp4.rz.uni-karlsruhe.de > > > > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp4.rz.uni-karlsruhe.de', giving up on it > > > > > > > > So ntpd has given up on all the servers listed in the ntp.conf file. > > > > > > Yes, but it looks more like name service that's not operating, ntpd > > > seems to be doing its best but can't resolve the hostnames? > > > > Why would I have named running on a notebook? This is a notebook, > > which is not connected to the internet. > > > > > > I then proceed to connect to the wireless network and proceed to log > > > > into two VPNs: > > > > > > > > 9 Mar 08:08:58 ntpd[1510]: Listening on interface #6 wlan0, 192.168.75.58#123 Enabled > > > > 9 Mar 08:09:00 ntpd[1510]: Listening on interface #7 tun0, 193.196.120.15#123 Enabled > > > > 9 Mar 08:09:04 ntpd[1510]: Listening on interface #8 tun1, 141.3.162.67#123 Enabled > > > > > > > > Over interface #8 some of the servers are actually available, but > > > > ntpq -p still states: > > > > No association ID's returned > > > > > > > > Only when I restart ntpd, it operates as expected: > > > > remote refid st t when poll reach delay offset jitter > > > > ============================================================================== > > > > zit-net2.uni-pa .STEP. 16 u - 512 0 0.000 0.000 0.000 > > > > alpha.rueckgr.a .STEP. 16 u - 512 0 0.000 0.000 0.000 > > > > ntp.goneco.de .STEP. 16 u - 512 0 0.000 0.000 0.000 > > > > +proxy4.rz.uni-k 129.13.64.17 2 u 30 128 271 2.937 2.530 1.891 > > > > +proxy2.rz.uni-k 129.13.64.17 2 u 58 128 375 3.593 -8.981 1.837 > > > > *proxy1.rz.uni-k 129.13.64.17 2 u 15 128 271 3.297 8.244 1.487 > > > > > > I've always had to restart named after losing / regaining an interface, > > > most noticeably after a suspend/resume (eg a low battery suspend), so I > > > run /etc/rc.d/named restart from rc.resume. This looks like a similar > > > issue perhaps, though I don't see why restarting only ntpd would fix it. > > > > As I said, named doesn't run at all. When the notebook gets an > > internet connection, ntpd recognizes this. It somehow doesn't > > occur to it, though, that it might be able to resolve the > > servers, now. > > I believe this is the problem. Note that you'll need to add an SSL > cert. exception for this site due to them using self-signed certs. > > https://support.ntp.org/bugs/show_bug.cgi?id=987 Oh, and/or this one too, which mentions how to get the FORCE_DNSRETRY feature -- you get to rebuild ntpd. :-) https://support.ntp.org/bugs/show_bug.cgi?id=1178 Doesn't sound like this issue is FreeBSD-specific, so you might want to chime in on those NTP bugs. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 13:53:12 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95B2C1065676; Tue, 9 Mar 2010 13:53:11 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 53A068FC29; Tue, 9 Mar 2010 13:53:11 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id DB21646B2C; Tue, 9 Mar 2010 08:53:10 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id 2AA8C8A025; Tue, 9 Mar 2010 08:53:10 -0500 (EST) From: John Baldwin To: Alexander Leidinger Date: Tue, 9 Mar 2010 07:47:00 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <201003081000.12656.jhb@freebsd.org> <20100309092709.16456y55vq0kaa88@webmail.leidinger.net> In-Reply-To: <20100309092709.16456y55vq0kaa88@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201003090747.00192.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Tue, 09 Mar 2010 08:53:10 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-stable@freebsd.org, Robert Watson Subject: Re: is dtrace usable? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 13:53:12 -0000 On Tuesday 09 March 2010 3:27:09 am Alexander Leidinger wrote: > Quoting John Baldwin (from Mon, 8 Mar 2010 10:00:12 -05= 00): >=20 > > On Saturday 06 March 2010 11:00:12 am Robert Watson wrote: > >> On Sat, 6 Mar 2010, Alexander Leidinger wrote: > >> > >> >> Take a look at the DTrace configuration information here: > >> >> > >> >> http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/dtrace= =2Ehtml > >> > > >> > I've just reread it (despite the fact that I already used it). Some > >> > comments: > >> > > >> > Last time I tried, I didn't see any problems by adding > >> > makeoptions WITH_CTF=3Dyes > >> > to the kernel config instead of doing > >> > make WITH_CTF=3D1 kernel > >> > > >> > Did I miss something, and if not, shouldn't we tell about the > >> > makeoptions part instead (a kernel rebuild later will not cause > >> > trouble when someone forgets to do the WITH_CTF part as it is already > >> > in the kernel makefile)? > >> > >> I'll leave John to answer this one, CC line broadended. > > > > I would be very surprised if 'makeoptions WITH_CTF=3Dyes' worked. The = many > > times I and others have tried it it did not work. Do you have a log of= your > > build showing the ctfconvert and ctfmerge command lines? >=20 > I do not have a log around, it has been a while since I did something =20 > with dtrace (a year ago) and I can not remember that I always added =20 > WITH_CTF on a build (but it was about SDT probes, not FBT probes, in =20 > case it matters). >=20 > I had a look again, WITH_CTF=3Dyes is one of the first lines in the =20 > Makefile, and /usr/share/mk/sys.mk has "if !defined(WITH_CTF)". "make =20 > -V WITH_CTF" shows "yes", but "make -V NO_CTF" shows "1". This is =20 > strange, isn't it? I would expect that NO_CTF is undefined. Is this a =20 > bug in make, or a bug in the man page (neither in the description of =20 > the different kinds of variables, nor in the description of "defined" =20 > is something mentioned explaining this behavior). It is defined behavior. From the 2nd and 3rd paragraphs of the make(1) manual page: First of all, the initial list of specifications will be read from the system makefile, sys.mk, unless inhibited with the -r option. The sta= n- dard sys.mk as shipped with FreeBSD also handles make.conf(5), the default path to which can be altered via the make variable __MAKE_CONF. Then the first of BSDmakefile, makefile, and Makefile that can be found in the current directory, object directory (see .OBJDIR), or search pa= th (see the -I option) will be read for the main list of dependency speci= fi- cations. A different makefile or list of them can be supplied via the= -f option(s). Finally, if the file .depend can be found in any of the aforesaid locations, it will also be read (see mkdep(1)). =46rom this you can see that sys.mk is included and parsed before 'Makefile= ', so the WITH_CTF=3Dyes is not set until after sys.mk has been parsed. =2D-=20 John Baldwin From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 14:16:19 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7D097106564A; Tue, 9 Mar 2010 14:16:19 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 0C40A8FC0C; Tue, 9 Mar 2010 14:16:18 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2DC6A.dip.t-dialin.net [217.226.220.106]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 9BCD9844482; Tue, 9 Mar 2010 15:16:12 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id BE01D529C; Tue, 9 Mar 2010 15:16:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1268144169; bh=/gpok8WeHeB2b9eZnjvpf+cF5CcxmcyqQSCrUqPrwSM=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=z33vjZ2HV4m6edhwTLHBg1kTW5F8SZWALDF7qpxyU0rmEmoBFVC2lD86mozHKgWBB ia6b+nK6dY10Ve1YsazNSLCwI3vlaknSWviI5jPHDoxClugplJ3SLl2idO1Lpjoi9E qejgbacD+aLfvAAtGBIsrPwVrFZ3OaCmBcHCX4jVgj/dGPgeDl3PJqbluTW+PpOqBA 2MaDTMw2DsEE5nzvqV7N6lDqR89LxqWuLL303eIjUL2+lYv+jIxweEHXyXW1ePZWiT beY3i0+QCi3ckVOfZlP5hXg/WZGdB63F1KnxoJf9ArTVtnd8DpCwKvQyIOLA6uBDCR euKefv5r8a8RA== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o29EG9S6081029; Tue, 9 Mar 2010 15:16:09 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from psbru.cec.eu.int (psbru.cec.eu.int [158.169.131.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Tue, 09 Mar 2010 15:16:09 +0100 Message-ID: <20100309151609.19673jkwuoptzy0w@webmail.leidinger.net> Date: Tue, 09 Mar 2010 15:16:09 +0100 From: Alexander Leidinger To: John Baldwin References: <201003081000.12656.jhb@freebsd.org> <20100309092709.16456y55vq0kaa88@webmail.leidinger.net> <201003090747.00192.jhb@freebsd.org> In-Reply-To: <201003090747.00192.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 9BCD9844482.05FA2 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.104, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DIET_1 0.34, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1268748973.97182@Z9TGXwEev8tP27NXs31byA X-EBL-Spam-Status: No Cc: freebsd-stable@freebsd.org, Robert Watson Subject: Re: is dtrace usable? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 14:16:19 -0000 Quoting John Baldwin (from Tue, 9 Mar 2010 07:47:00 -0500): > On Tuesday 09 March 2010 3:27:09 am Alexander Leidinger wrote: >> Quoting John Baldwin (from Mon, 8 Mar 2010 >> 10:00:12 -0500): >> >> > On Saturday 06 March 2010 11:00:12 am Robert Watson wrote: >> >> On Sat, 6 Mar 2010, Alexander Leidinger wrote: >> >> >> >> >> Take a look at the DTrace configuration information here: >> >> >> >> >> >> >> http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/dtrace.html >> >> > >> >> > I've just reread it (despite the fact that I already used it). Some >> >> > comments: >> >> > >> >> > Last time I tried, I didn't see any problems by adding >> >> > makeoptions WITH_CTF=yes >> >> > to the kernel config instead of doing >> >> > make WITH_CTF=1 kernel >> >> > >> >> > Did I miss something, and if not, shouldn't we tell about the >> >> > makeoptions part instead (a kernel rebuild later will not cause >> >> > trouble when someone forgets to do the WITH_CTF part as it is already >> >> > in the kernel makefile)? >> >> >> >> I'll leave John to answer this one, CC line broadended. >> > >> > I would be very surprised if 'makeoptions WITH_CTF=yes' worked. The many >> > times I and others have tried it it did not work. Do you have a >> log of your >> > build showing the ctfconvert and ctfmerge command lines? >> >> I do not have a log around, it has been a while since I did something >> with dtrace (a year ago) and I can not remember that I always added >> WITH_CTF on a build (but it was about SDT probes, not FBT probes, in >> case it matters). >> >> I had a look again, WITH_CTF=yes is one of the first lines in the >> Makefile, and /usr/share/mk/sys.mk has "if !defined(WITH_CTF)". "make >> -V WITH_CTF" shows "yes", but "make -V NO_CTF" shows "1". This is >> strange, isn't it? I would expect that NO_CTF is undefined. Is this a >> bug in make, or a bug in the man page (neither in the description of >> the different kinds of variables, nor in the description of "defined" >> is something mentioned explaining this behavior). > > It is defined behavior. From the 2nd and 3rd paragraphs of the make(1) > manual page: > > First of all, the initial list of specifications will be read from the > system makefile, sys.mk, unless inhibited with the -r option. The stan- > dard sys.mk as shipped with FreeBSD also handles make.conf(5), the > default path to which can be altered via the make variable __MAKE_CONF. > > Then the first of BSDmakefile, makefile, and Makefile that can be found > in the current directory, object directory (see .OBJDIR), or search path > (see the -I option) will be read for the main list of > dependency specifi- > cations. A different makefile or list of them can be supplied > via the -f > option(s). Finally, if the file .depend can be found in any of the > aforesaid locations, it will also be read (see mkdep(1)). > > From this you can see that sys.mk is included and parsed before 'Makefile', > so the WITH_CTF=yes is not set until after sys.mk has been parsed. I think we need to find a different solution for this. The need to specify WITH_CTF at the command line is very error prone. :( Bye, Alexander. -- Every time I lose weight, it finds me again! http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 14:18:47 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0F5C1065673; Tue, 9 Mar 2010 14:18:47 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from proxypop2.sarenet.es (proxypop2.sarenet.es [194.30.0.95]) by mx1.freebsd.org (Postfix) with ESMTP id 63AD98FC23; Tue, 9 Mar 2010 14:18:47 +0000 (UTC) Received: from [172.16.1.204] (unknown [192.148.167.2]) by proxypop2.sarenet.es (Postfix) with ESMTP id 4E485732EF; Tue, 9 Mar 2010 15:18:46 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <20100309125815.GF3155@garage.freebsd.pl> Date: Tue, 9 Mar 2010 15:18:45 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> To: Pawel Jakub Dawidek X-Mailer: Apple Mail (2.1077) Cc: FreeBSD Stable , Stefan Bethke Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 14:18:47 -0000 On Mar 9, 2010, at 1:58 PM, Pawel Jakub Dawidek wrote: >>> What kind of hardware do you have there? There is 3-way deadlock = I've a >>> fix for which would be hard to trigger on single or dual core = machines. >>>=20 >>> Feel free to try the fix: >>>=20 >>> http://people.freebsd.org/~pjd/patches/zfs_3way_deadlock.patch >>=20 >> Maybe related to the deadlock I reported when I was receiving an = incremental snapshot while the target dataset was being read? >=20 > Could be. This deadlock is in general related to zfs recv = functionality. Aye aye, Sir set fingers -position crossed testing :) Borja From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 16:39:11 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89F46106564A; Tue, 9 Mar 2010 16:39:11 +0000 (UTC) (envelope-from rwatson@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id 62EEB8FC19; Tue, 9 Mar 2010 16:39:11 +0000 (UTC) Received: from lemongrass.sec.cl.cam.ac.uk (lemongrass.sec.cl.cam.ac.uk [128.232.18.47]) by cyrus.watson.org (Postfix) with ESMTPSA id 89B5146B2A; Tue, 9 Mar 2010 11:39:10 -0500 (EST) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: "Robert N. M. Watson" In-Reply-To: <20100309151609.19673jkwuoptzy0w@webmail.leidinger.net> Date: Tue, 9 Mar 2010 16:39:09 +0000 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201003081000.12656.jhb@freebsd.org> <20100309092709.16456y55vq0kaa88@webmail.leidinger.net> <201003090747.00192.jhb@freebsd.org> <20100309151609.19673jkwuoptzy0w@webmail.leidinger.net> To: Alexander Leidinger X-Mailer: Apple Mail (2.1077) Cc: freebsd-stable@freebsd.org, John Baldwin Subject: Re: is dtrace usable? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 16:39:11 -0000 On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: >> =46rom this you can see that sys.mk is included and parsed before = 'Makefile', >> so the WITH_CTF=3Dyes is not set until after sys.mk has been parsed. >=20 > I think we need to find a different solution for this. The need to = specify WITH_CTF at the command line is very error prone. :( You are neither the first person to have made this observation, nor the = first person to have failed to propose a solution in the form of a patch = :-). Robert= From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 18:03:46 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 950A61065678 for ; Tue, 9 Mar 2010 18:03:46 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) by mx1.freebsd.org (Postfix) with ESMTP id 15C9F8FC39 for ; Tue, 9 Mar 2010 18:03:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id o29I3hio091743; Wed, 10 Mar 2010 05:03:43 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Wed, 10 Mar 2010 05:03:42 +1100 (EST) From: Ian Smith To: Jeremy Chadwick In-Reply-To: <20100309124609.GA6331@icarus.home.lan> Message-ID: <20100310042709.W85436@sola.nimnet.asn.au> References: <4B9601D9.5030101@bsdforen.de> <20100309210949.F85436@sola.nimnet.asn.au> <20100309124609.GA6331@icarus.home.lan> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Cc: freebsd-stable@freebsd.org Subject: Re: ntpd does not re-query servers, when a new interface appears X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 18:03:46 -0000 On Tue, 9 Mar 2010, Jeremy Chadwick wrote: > On Tue, Mar 09, 2010 at 09:27:35PM +1100, Ian Smith wrote: [..] > > Yes, but it looks more like name service that's not operating, ntpd > > seems to be doing its best but can't resolve the hostnames? Right smell, wrong pooch :) Thanks for the pointer to the ntp buglist. > > I've always had to restart named after losing / regaining an interface, > > most noticeably after a suspend/resume (eg a low battery suspend), so I > > run /etc/rc.d/named restart from rc.resume. This looks like a similar > > issue perhaps, though I don't see why restarting only ntpd would fix it. > > named is supposed to auto-probe for interfaces at a specific interval; > see the "interface-interval" option. I forget what the default is, > but on our servers we explicitly disable it by setting it to 0. // We have no dynamic interfaces, so BIND shouldn't need to // poll for interface state {UP|DOWN}. // (will this fix need to reload after suspend/resume?) interface-interval 0; It's rare, maybe twice a year, that this laptop cum server suspends from a 2hr+ lack of power - inverter failures, rewiring etc - so restarting named on resume makes more sense than constant iface polling 'in case'. cheers, Ian From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 18:33:29 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B30BA1065674 for ; Tue, 9 Mar 2010 18:33:29 +0000 (UTC) (envelope-from korvus@comcast.net) Received: from mx04.pub.collaborativefusion.com (mx04.pub.collaborativefusion.com [206.210.72.84]) by mx1.freebsd.org (Postfix) with ESMTP id 6F9DE8FC14 for ; Tue, 9 Mar 2010 18:33:29 +0000 (UTC) Received: from [192.168.2.164] ([206.210.89.202]) by mx04.pub.collaborativefusion.com (StrongMail Enterprise 4.1.1.4(4.1.1.4-47689)); Tue, 09 Mar 2010 13:50:54 -0500 X-VirtualServerGroup: Default X-MailingID: 00000::00000::00000::00000::::380 X-SMHeaderMap: mid="X-MailingID" X-Destination-ID: freebsd-stable@freebsd.org X-SMFBL: ZnJlZWJzZC1zdGFibGVAZnJlZWJzZC5vcmc= Message-ID: <4B969477.70706@comcast.net> Date: Tue, 09 Mar 2010 13:33:27 -0500 From: Steve Polyack User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.7) Gecko/20100211 Thunderbird/3.0.1 MIME-Version: 1.0 To: Ivan Voras References: <4B953C92.5080606@comcast.net> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: ZFS hot spares X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 18:33:29 -0000 On 03/09/10 05:11, Ivan Voras wrote: > On 03/08/10 19:06, Steve Polyack wrote: >> ZFS in FreeBSD lacks at least one major feature from the Solaris >> version: hot spares. There is a PR open at >> http://www.freebsd.org/cgi/query-pr.cgi?pr=134491, but there hasn't been >> any motion/thoughts posted on it since its creation almost one year ago. >> >> I'm aware that on Solaris, hot spare replacement is handled by a few >> Solaris-specific daemons, zfs-retire and zfs-diagnose, which both plug >> into the Solaris FMA (Fault Management Architecture). Have there been >> any thoughts on porting these over or getting something similar running >> within FreeBSD? With all of the recent SATA/SAS CAM hotplug work now >> committed, it would be nice to have automatic replacement of hot spares >> with a future hot-replacement of the failed drive. >> >> On the other side, I'd be interested in hearing if anyone has had >> success in rolling their own scripted solution: i.e. something which >> polls 'zpool status' looking for failed drives and performing hot-spare >> replacements automatically. > > You don't have to exactly poll it. See /etc/devd.conf: > > # Sample ZFS problem reports handling. > notify 10 { > match "system" "ZFS"; > match "type" "zpool"; > action "logger -p kern.err 'ZFS: failed to load zpool $pool'"; > }; > > notify 10 { > match "system" "ZFS"; > match "type" "vdev"; > action "logger -p kern.err 'ZFS: vdev failure, zpool=$pool > type=$type'"; > }; > > notify 10 { > match "system" "ZFS"; > match "type" "data"; > action "logger -p kern.warn 'ZFS: zpool I/O failure, > zpool=$pool error=$zio_err'"; > }; > > notify 10 { > match "system" "ZFS"; > match "type" "io"; > action "logger -p kern.warn 'ZFS: vdev I/O failure, > zpool=$pool path=$vdev_path offset=$zio_offset size=$zio_size > error=$zio_err'"; > }; > > notify 10 { > match "system" "ZFS"; > match "type" "checksum"; > action "logger -p kern.warn 'ZFS: checksum mismatch, > zpool=$pool path=$vdev_path offset=$zio_offset size=$zio_size'"; > }; > > I don't really know if these notifications actually work since I don't > have hot-plug test machines, but if they do, this looks like a decent > starting point. > Thanks for the suggestions. I received a similar one from someone else. If I get time to build a ZFS lab machine then I will certainly try these out and provide feedback on how they work. From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 19:11:05 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC45C106566C for ; Tue, 9 Mar 2010 19:11:05 +0000 (UTC) (envelope-from byshenknet@byshenk.net) Received: from core.byshenk.net (core.byshenk.net [62.58.73.230]) by mx1.freebsd.org (Postfix) with ESMTP id 711E88FC16 for ; Tue, 9 Mar 2010 19:11:05 +0000 (UTC) Received: from core.byshenk.net (localhost [127.0.0.1]) by core.byshenk.net (8.14.4/8.14.4) with ESMTP id o29JBvHa027075; Tue, 9 Mar 2010 20:11:57 +0100 (CET) (envelope-from byshenknet@core.byshenk.net) Received: (from byshenknet@localhost) by core.byshenk.net (8.14.4/8.14.4/Submit) id o29JBv0P027074; Tue, 9 Mar 2010 20:11:57 +0100 (CET) (envelope-from byshenknet) Date: Tue, 9 Mar 2010 20:11:57 +0100 From: Greg Byshenk To: Linghua Tseng Message-ID: <20100309191157.GG2303@core.byshenk.net> References: <20100309000826.GF4648@cesium.hyperfine.info> <80F42CAF32A14A1FB43B830AAF877A5A@TinlansPC> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <80F42CAF32A14A1FB43B830AAF877A5A@TinlansPC> User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=1.9 required=5.0 tests=ALL_TRUSTED, FH_DATE_PAST_20XX autolearn=no version=3.2.5 X-Spam-Level: * X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on core.byshenk.net Cc: "Peter C. Lai" , freebsd-stable@freebsd.org Subject: Re: Supplementary groups on LDAP cannot work with RELENG_8 +nss_ldap X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 19:11:05 -0000 On Tue, Mar 09, 2010 at 09:00:49AM +0800, Linghua Tseng wrote: > Here is the output of `diff -u /usr/src/etc/nsswitch.conf > /etc/nsswitch.conf'. > --- /usr/src/etc/nsswitch.conf 2010-03-08 09:04:25.000000000 +0800 > +++ /etc/nsswitch.conf 2010-03-08 18:01:08.000000000 +0800 > @@ -1,13 +1,13 @@ > # > # nsswitch.conf(5) - name service switch configuration file > -# $FreeBSD: src/etc/nsswitch.conf,v 1.1.10.1 2009/08/03 08:13:06 kensmith > Exp $ > +# $FreeBSD: src/etc/nsswitch.conf,v 1.1 2006/05/03 15:14:47 ume Exp $ > # > group: compat > -group_compat: nis > +group_compat: ldap nis > hosts: files dns > networks: files > passwd: compat > -passwd_compat: nis > +passwd_compat: ldap nis > shells: files > services: compat > services_compat: nis > > The line `+:*::::::::' has already put into /etc/master.passwd, > and the line `+:*::' has already put into /etc/group. I may be completely wrong (I can't seem to find the source), and I don't know if it is the source of your problem, but I recall it being reported that 'passwd_compat' and 'group_compat' require a *single* source entry. -- greg byshenk - gbyshenk@byshenk.net - Leiden, NL From owner-freebsd-stable@FreeBSD.ORG Tue Mar 9 23:43:05 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 900AD106564A for ; Tue, 9 Mar 2010 23:43:05 +0000 (UTC) (envelope-from oberman@es.net) Received: from mailgw.es.net (mail1.es.net [IPv6:2001:400:201:1::2]) by mx1.freebsd.org (Postfix) with ESMTP id 0CB1B8FC0C for ; Tue, 9 Mar 2010 23:43:04 +0000 (UTC) Received: from ptavv.es.net (ptavv.es.net [IPv6:2001:400:910::29]) by mailgw.es.net (8.14.3/8.14.3) with ESMTP id o29Nh2Nb006202 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 9 Mar 2010 15:43:02 -0800 Received: from ptavv.es.net (localhost [127.0.0.1]) by ptavv.es.net (Tachyon Server) with ESMTP id 1D03F1CC18; Tue, 9 Mar 2010 15:43:02 -0800 (PST) To: Jeremy Chadwick In-reply-to: Your message of "Fri, 05 Mar 2010 12:33:09 PST." <20100305203309.GA91525@icarus.home.lan> Date: Tue, 09 Mar 2010 15:43:02 -0800 From: "Kevin Oberman" Message-Id: <20100309234302.1D03F1CC18@ptavv.es.net> X-Proofpoint-Virus-Version: vendor=fsecure engine=1.12.8161:2.4.5, 1.2.40, 4.0.166 definitions=2010-03-10_01:2010-02-06, 2010-03-10, 2010-03-09 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=5.0.0-0908210000 definitions=main-1003090232 Cc: freebsd-stable@freebsd.org Subject: Re: Cron output mail lost with update to RELENG_7 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Mar 2010 23:43:05 -0000 > Date: Fri, 5 Mar 2010 12:33:09 -0800 > From: Jeremy Chadwick > Sender: owner-freebsd-stable@freebsd.org > > On Fri, Mar 05, 2010 at 11:32:47AM -0800, Kevin Oberman wrote: > > I have discovered a problem with the mail sent by cron jobs (I refer > > only to logs, not invocations of mail from scripts.) They never are > > delivered. > > Mar 5 10:32:30 noc5 postfix/sendmail[1175]: fatal: root(0): No recipient addresses found in message header > > Mar 5 10:32:30 noc5 postfix/sendmail[1175]: fatal: root(0): No recipient addresses found in message header > > Mar 5 10:37:00 noc5 postfix/sendmail[1268]: fatal: root(0): No recipient addresses found in message header > > Mar 5 10:37:00 noc5 postfix/sendmail[1268]: fatal: root(0): No recipient addresses found in message header > > > > This showed up when I upgraded the system to RELENG_7 yesterday. My > > previous install was RELENG_7 of May 2, 2009 and it delivered the logs > > without any problems. No other changes were made. postfix was 2.6.5. > > > > I have this same issue on all 8.0 systems I have, but I was blaming a > > fault in postfix config. Now I realize that this is not the problem. > > > > I really don't know quite where to look for this. Any clues would be > > appreciated. > > I don't have this issue on any of our RELENG_7 or RELENG_8 systems, all > of which use postfix and WITHOUT_SENDMAIL in /etc/src.conf. > > It sounds like cron is trying to spawn something like mail(1) (more > likely /usr/sbin/sendmail; would have to look at the code) and passing > it either incorrect flags or actual content within the header itself, > e.g. a missing To: line. > > Since postfix is involved, have you verified your /etc/mail > configuration to make sure mailwrapper is referring to the correct > postfix binaries? > > The only other thing I can think of would be, possibly, some sort of > cronjob root has (either crontab -l or /etc/crontab) which makes use of > the MAILTO environment variable. See cron(8) for what I'm talking > about. > > You might have to run cron in debug mode (see -x flag; your argument > list will probably be quite long :-) ) to see what it's doing. > Otherwise truss or ktrace might be the only way to track down what's > going on underneath. After a lot of testing, I created a dummy sendmail that simply captured the arguments and the data from STDIN. #!/usr/local/bin/perl open OUT, ">/home/oberman/cronout.txt"; foreach (@ARGV) {print OUT "$_\n";} print OUT "Mailcat ran!\n"; sleep 5; while () { print OUT $_; } close OUT; It looks like cron is sending an empty message. I see MAILARG of '-FCronDaemon -odi -oem -oi -t' but that is followed by EOF with no content at all. I'm looking at the cron source, but I am baffled for the moment. I see no recent updates to cron in RELENG_7, though there are in RELENG_8. I'm running out of ideas. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: oberman@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 00:18:30 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD6DC106564A for ; Wed, 10 Mar 2010 00:18:30 +0000 (UTC) (envelope-from oberman@es.net) Received: from mailgw.es.net (mail1.es.net [IPv6:2001:400:201:1::2]) by mx1.freebsd.org (Postfix) with ESMTP id 6C57B8FC0A for ; Wed, 10 Mar 2010 00:18:29 +0000 (UTC) Received: from ptavv.es.net (ptavv.es.net [IPv6:2001:400:910::29]) by mailgw.es.net (8.14.3/8.14.3) with ESMTP id o2A0IPpk007321 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 9 Mar 2010 16:18:25 -0800 Received: from ptavv.es.net (localhost [127.0.0.1]) by ptavv.es.net (Tachyon Server) with ESMTP id 4B2061CC0D; Tue, 9 Mar 2010 16:18:25 -0800 (PST) To: Peter Jeremy In-reply-to: Your message of "Tue, 09 Mar 2010 21:53:55 +1100." <20100309105355.GD3978@server.vk2pj.dyndns.org> Date: Tue, 09 Mar 2010 16:18:25 -0800 From: "Kevin Oberman" Message-Id: <20100310001825.4B2061CC0D@ptavv.es.net> X-Proofpoint-Virus-Version: vendor=fsecure engine=1.12.8161:2.4.5, 1.2.40, 4.0.166 definitions=2010-03-10_01:2010-02-06, 2010-03-10, 2010-03-09 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=5.0.0-0908210000 definitions=main-1003090242 Cc: FreeBSD Stable , Stefan Bethke Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 00:18:30 -0000 > Date: Tue, 9 Mar 2010 21:53:55 +1100 > From: Peter Jeremy > Sender: owner-freebsd-stable@freebsd.org > > On 2010-Mar-09 10:15:53 +0100, Stefan Bethke wrote: > >Over the past couple of months, I've more or less regularly observed machines having more and more processes stuck in the zfs wchan. The processes never recover from that, > > How long have you waited? > > There seems to be a problem with low free memory handling that causes ZFS > to turn into cold molasses. The work-around is to run a program that > allocates a decent size chunk of memory and then exits. The original > suggestion was something like: > perl -e '@x = (0) x 1000000;' > I've written a short program that allocates and dirties ~100MB and then > exits and run it from cron. Sigh! I found it. I build my systems without NIS and I had the stock nsswitch.conf file. Fixed. /me banging my head against the desk. Thanks! -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: oberman@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 00:25:58 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 973D71065674 for ; Wed, 10 Mar 2010 00:25:58 +0000 (UTC) (envelope-from oberman@es.net) Received: from mailgw.es.net (mail1.es.net [IPv6:2001:400:201:1::2]) by mx1.freebsd.org (Postfix) with ESMTP id 421B98FC1E for ; Wed, 10 Mar 2010 00:25:58 +0000 (UTC) Received: from ptavv.es.net (ptavv.es.net [IPv6:2001:400:910::29]) by mailgw.es.net (8.14.3/8.14.3) with ESMTP id o2A0Ps9f007589 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 9 Mar 2010 16:25:55 -0800 Received: from ptavv.es.net (localhost [127.0.0.1]) by ptavv.es.net (Tachyon Server) with ESMTP id 9E8E81CC18; Tue, 9 Mar 2010 16:25:54 -0800 (PST) To: Peter Jeremy In-reply-to: Your message of "Tue, 09 Mar 2010 16:18:25 PST." <20100310001825.4B2061CC0D@ptavv.es.net> Date: Tue, 09 Mar 2010 16:25:54 -0800 From: "Kevin Oberman" Message-Id: <20100310002554.9E8E81CC18@ptavv.es.net> X-Proofpoint-Virus-Version: vendor=fsecure engine=1.12.8161:2.4.5, 1.2.40, 4.0.166 definitions=2010-03-10_01:2010-02-06, 2010-03-10, 2010-03-09 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=5.0.0-0908210000 definitions=main-1003090242 Cc: FreeBSD Stable , Stefan Bethke Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 00:25:58 -0000 Sigh. My brain is fried. I replied to the wrong thread. Pleas ignore this. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: oberman@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 > Date: Tue, 09 Mar 2010 16:18:25 -0800 > From: "Kevin Oberman" > Sender: owner-freebsd-stable@freebsd.org > > > Date: Tue, 9 Mar 2010 21:53:55 +1100 > > From: Peter Jeremy > > Sender: owner-freebsd-stable@freebsd.org > > > > On 2010-Mar-09 10:15:53 +0100, Stefan Bethke wrote: > > >Over the past couple of months, I've more or less regularly observed machines having more and more processes stuck in the zfs wchan. The processes never recover from that, > > > > How long have you waited? > > > > There seems to be a problem with low free memory handling that causes ZFS > > to turn into cold molasses. The work-around is to run a program that > > allocates a decent size chunk of memory and then exits. The original > > suggestion was something like: > > perl -e '@x = (0) x 1000000;' > > I've written a short program that allocates and dirties ~100MB and then > > exits and run it from cron. > > Sigh! I found it. I build my systems without NIS and I had the stock > nsswitch.conf file. Fixed. > > /me banging my head against the desk. > > Thanks! > -- > R. Kevin Oberman, Network Engineer > Energy Sciences Network (ESnet) > Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) > E-mail: oberman@es.net Phone: +1 510 486-8634 > Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 00:34:52 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A282106564A for ; Wed, 10 Mar 2010 00:34:52 +0000 (UTC) (envelope-from oberman@es.net) Received: from mailgw.es.net (mail1.es.net [IPv6:2001:400:201:1::2]) by mx1.freebsd.org (Postfix) with ESMTP id 4C3148FC1D for ; Wed, 10 Mar 2010 00:34:52 +0000 (UTC) Received: from ptavv.es.net (ptavv.es.net [IPv6:2001:400:910::29]) by mailgw.es.net (8.14.3/8.14.3) with ESMTP id o2A0Yn0O007856 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 9 Mar 2010 16:34:50 -0800 Received: from ptavv.es.net (localhost [127.0.0.1]) by ptavv.es.net (Tachyon Server) with ESMTP id D606F1CC0D; Tue, 9 Mar 2010 16:34:49 -0800 (PST) To: Jeremy Chadwick In-reply-to: Your message of "Fri, 05 Mar 2010 12:33:09 PST." <20100305203309.GA91525@icarus.home.lan> Date: Tue, 09 Mar 2010 16:34:49 -0800 From: "Kevin Oberman" Message-Id: <20100310003449.D606F1CC0D@ptavv.es.net> X-Proofpoint-Virus-Version: vendor=fsecure engine=1.12.8161:2.4.5, 1.2.40, 4.0.166 definitions=2010-03-10_01:2010-02-06, 2010-03-10, 2010-03-09 signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 ipscore=0 phishscore=0 bulkscore=0 adultscore=0 classifier=spam adjust=0 reason=mlx engine=5.0.0-0908210000 definitions=main-1003090245 Cc: freebsd-stable@freebsd.org Subject: Re: Cron output mail lost with update to RELENG_7 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 00:34:52 -0000 > Date: Fri, 5 Mar 2010 12:33:09 -0800 > From: Jeremy Chadwick > Sender: owner-freebsd-stable@freebsd.org > > On Fri, Mar 05, 2010 at 11:32:47AM -0800, Kevin Oberman wrote: > > I have discovered a problem with the mail sent by cron jobs (I refer > > only to logs, not invocations of mail from scripts.) They never are > > delivered. > > Mar 5 10:32:30 noc5 postfix/sendmail[1175]: fatal: root(0): No recipient addresses found in message header > > Mar 5 10:32:30 noc5 postfix/sendmail[1175]: fatal: root(0): No recipient addresses found in message header > > Mar 5 10:37:00 noc5 postfix/sendmail[1268]: fatal: root(0): No recipient addresses found in message header > > Mar 5 10:37:00 noc5 postfix/sendmail[1268]: fatal: root(0): No recipient addresses found in message header > > > > This showed up when I upgraded the system to RELENG_7 yesterday. My > > previous install was RELENG_7 of May 2, 2009 and it delivered the logs > > without any problems. No other changes were made. postfix was 2.6.5. > > > > I have this same issue on all 8.0 systems I have, but I was blaming a > > fault in postfix config. Now I realize that this is not the problem. > > > > I really don't know quite where to look for this. Any clues would be > > appreciated. > > I don't have this issue on any of our RELENG_7 or RELENG_8 systems, all > of which use postfix and WITHOUT_SENDMAIL in /etc/src.conf. > > It sounds like cron is trying to spawn something like mail(1) (more > likely /usr/sbin/sendmail; would have to look at the code) and passing > it either incorrect flags or actual content within the header itself, > e.g. a missing To: line. > > Since postfix is involved, have you verified your /etc/mail > configuration to make sure mailwrapper is referring to the correct > postfix binaries? > > The only other thing I can think of would be, possibly, some sort of > cronjob root has (either crontab -l or /etc/crontab) which makes use of > the MAILTO environment variable. See cron(8) for what I'm talking > about. > > You might have to run cron in debug mode (see -x flag; your argument > list will probably be quite long :-) ) to see what it's doing. > Otherwise truss or ktrace might be the only way to track down what's > going on underneath. > > -- > | Jeremy Chadwick jdc@parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP: 4BD6C0CB | > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > > Date: Fri, 5 Mar 2010 12:33:09 -0800 > From: Jeremy Chadwick > Sender: owner-freebsd-stable@freebsd.org > > On Fri, Mar 05, 2010 at 11:32:47AM -0800, Kevin Oberman wrote: > > I have discovered a problem with the mail sent by cron jobs (I refer > > only to logs, not invocations of mail from scripts.) They never are > > delivered. > > Mar 5 10:32:30 noc5 postfix/sendmail[1175]: fatal: root(0): No recipient addresses found in message header > > Mar 5 10:32:30 noc5 postfix/sendmail[1175]: fatal: root(0): No recipient addresses found in message header > > Mar 5 10:37:00 noc5 postfix/sendmail[1268]: fatal: root(0): No recipient addresses found in message header > > Mar 5 10:37:00 noc5 postfix/sendmail[1268]: fatal: root(0): No recipient addresses found in message header > > > > This showed up when I upgraded the system to RELENG_7 yesterday. My > > previous install was RELENG_7 of May 2, 2009 and it delivered the logs > > without any problems. No other changes were made. postfix was 2.6.5. > > > > I have this same issue on all 8.0 systems I have, but I was blaming a > > fault in postfix config. Now I realize that this is not the problem. > > > > I really don't know quite where to look for this. Any clues would be > > appreciated. > > I don't have this issue on any of our RELENG_7 or RELENG_8 systems, all > of which use postfix and WITHOUT_SENDMAIL in /etc/src.conf. > > It sounds like cron is trying to spawn something like mail(1) (more > likely /usr/sbin/sendmail; would have to look at the code) and passing > it either incorrect flags or actual content within the header itself, > e.g. a missing To: line. > > Since postfix is involved, have you verified your /etc/mail > configuration to make sure mailwrapper is referring to the correct > postfix binaries? > > The only other thing I can think of would be, possibly, some sort of > cronjob root has (either crontab -l or /etc/crontab) which makes use of > the MAILTO environment variable. See cron(8) for what I'm talking > about. > > You might have to run cron in debug mode (see -x flag; your argument > list will probably be quite long :-) ) to see what it's doing. > Otherwise truss or ktrace might be the only way to track down what's > going on underneath. Sigh! I found it. I build my systems without NIS and I had the stock nsswitch.conf file. Fixed. /me banging my head on my desk. Can't hurt...the brain is clearly dead already. Thanks! -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: oberman@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 00:37:02 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 11FCB106566B for ; Wed, 10 Mar 2010 00:37:02 +0000 (UTC) (envelope-from msnkipa@mail.ru) Received: from fallback4.mail.ru (fallback4.mail.ru [94.100.176.42]) by mx1.freebsd.org (Postfix) with ESMTP id BEBC38FC19 for ; Wed, 10 Mar 2010 00:37:01 +0000 (UTC) Received: from f292.mail.ru (f292.mail.ru [217.69.129.74]) by fallback4.mail.ru (mPOP.Fallback_MX) with ESMTP id E8ECE13AD9E0 for ; Wed, 10 Mar 2010 02:49:50 +0300 (MSK) Received: from mail by f292.mail.ru with local id 1Np9BN-0000EL-00 for freebsd-stable@freebsd.org; Wed, 10 Mar 2010 02:49:49 +0300 Received: from [188.134.3.101] by win.mail.ru with HTTP; Wed, 10 Mar 2010 02:49:49 +0300 From: =?koi8-r?Q?=ED=C9=C8=C1=C9=CC_=EB=C9=D0=C1?= To: freebsd-stable@freebsd.org Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: 192.168.7.79 via proxy [188.134.3.101] Date: Wed, 10 Mar 2010 02:49:49 +0300 Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit Message-Id: X-Spam: Not detected X-Mras: Ok Subject: ugen kernel module? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: =?koi8-r?Q?=ED=C9=C8=C1=C9=CC_=EB=C9=D0=C1?= List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 00:37:02 -0000 If FreeBSD7 there was ugen.ko kernel module and I can use apcupsd with USB devices, but in FreeBSD there is no such module, how can I use APC power supply with usb interface (I mean usage of the apcupsd port)? From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 04:20:55 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6DE52106564A for ; Wed, 10 Mar 2010 04:20:55 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from email1.allantgroup.com (email1.emsphone.com [199.67.51.115]) by mx1.freebsd.org (Postfix) with ESMTP id 355098FC19 for ; Wed, 10 Mar 2010 04:20:54 +0000 (UTC) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by email1.allantgroup.com (8.14.0/8.14.0) with ESMTP id o2A4KsaH001333 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 9 Mar 2010 22:20:54 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (smmsp@localhost [127.0.0.1]) by dan.emsphone.com (8.14.4/8.14.3) with ESMTP id o2A4KqZt044635 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO) for ; Tue, 9 Mar 2010 22:20:54 -0600 (CST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.4/8.14.3/Submit) id o2A4KmPL044634; Tue, 9 Mar 2010 22:20:48 -0600 (CST) (envelope-from dan) Date: Tue, 9 Mar 2010 22:20:48 -0600 From: Dan Nelson To: msnkipa@mail.ru Message-ID: <20100310042046.GJ12122@dan.emsphone.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-OS: FreeBSD 7.2-STABLE User-Agent: Mutt/1.5.20 (2009-06-14) X-Virus-Scanned: clamav-milter 0.95.3 at email1.allantgroup.com X-Virus-Status: Clean X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (email1.allantgroup.com [199.67.51.78]); Tue, 09 Mar 2010 22:20:54 -0600 (CST) X-Scanned-By: MIMEDefang 2.45 Cc: freebsd-stable@freebsd.org Subject: Re: ugen kernel module? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 04:20:55 -0000 In the last episode (Mar 10): > In FreeBSD7 there was ugen.ko kernel module and I can use apcupsd with USB > devices, but in FreeBSD there is no such module, how can I use APC power > supply with usb interface (I mean usage of the apcupsd port)? It's built into the usb subsystem now. All USB devices (including USB hubs and devices controlled by other drivers) now have a ugen device. Try running "usbconfig list" to show them. I bet your UPS has just moved to a different ugen number. -- Dan Nelson dnelson@allantgroup.com From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 06:58:19 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 555FD106567C for ; Wed, 10 Mar 2010 06:58:19 +0000 (UTC) (envelope-from uranus@tinlans.org) Received: from tinlans.org (tinlans.org [220.133.199.45]) by mx1.freebsd.org (Postfix) with ESMTP id 091BE8FC0C for ; Wed, 10 Mar 2010 06:58:18 +0000 (UTC) Received: from tinlans.org (localhost [127.0.0.1]) by tinlans.org (Postfix) with ESMTP id 6C297CF042; Wed, 10 Mar 2010 14:58:50 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tinlans.org; h= x-mailer:content-transfer-encoding:content-type:content-type :mime-version:date:date:subject:subject:in-reply-to:references :from:from:message-id:received:received; s=tinlans; t= 1268204330; bh=6dObHm1rENMHePETW0BPx+dEK/XQ+jRt/4ckW3plId8=; b=K GB3uLxsdVDij11UZ9p1zNvSNk6E6duaSXsTwcH0z+QFp3ty44V+42saoDx4cebXK dYQQOGMuZFCgvtsvrGEEA== X-Virus-Scanned: amavisd-new at tinlans.org Received: from tinlans.org ([127.0.0.1]) by tinlans.org (tinlans.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id B9ARNCBdtE0P; Wed, 10 Mar 2010 14:58:50 +0800 (CST) Received: from TinlansPC (TinlansPC [192.168.1.2]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tinlans.org (Postfix) with ESMTPSA id 22836CF040; Wed, 10 Mar 2010 14:58:50 +0800 (CST) Message-ID: From: "Linghua Tseng" To: "Greg Byshenk" References: <20100309000826.GF4648@cesium.hyperfine.info> <80F42CAF32A14A1FB43B830AAF877A5A@TinlansPC> <20100309191157.GG2303@core.byshenk.net> In-Reply-To: <20100309191157.GG2303@core.byshenk.net> Date: Wed, 10 Mar 2010 14:58:04 +0800 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal Importance: Normal X-Mailer: Microsoft Windows Live Mail 14.0.8089.726 X-Mimeole: Produced By Microsoft MimeOLE V14.0.8089.726 Cc: freebsd-stable@freebsd.org Subject: Re: Supplementary groups on LDAP cannot work with RELENG_8 +nss_ldap X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 06:58:19 -0000 Thanks. I have tried to modify my /etc/nsswitch.conf to: group: compat group_compat: ldap hosts: files dns networks: files passwd: compat passwd_compat: ldap shells: files services: compat services_compat: nis protocols: files rpc: files But the problem is still occurred. -------------------------------------------------- From: "Greg Byshenk" Sent: Wednesday, March 10, 2010 3:11 AM To: "Linghua Tseng" Cc: "Peter C. Lai" ; Subject: Re: Supplementary groups on LDAP cannot work with RELENG_8 +nss_ldap > On Tue, Mar 09, 2010 at 09:00:49AM +0800, Linghua Tseng wrote: > >> Here is the output of `diff -u /usr/src/etc/nsswitch.conf >> /etc/nsswitch.conf'. >> --- /usr/src/etc/nsswitch.conf 2010-03-08 09:04:25.000000000 +0800 >> +++ /etc/nsswitch.conf 2010-03-08 18:01:08.000000000 +0800 >> @@ -1,13 +1,13 @@ >> # >> # nsswitch.conf(5) - name service switch configuration file >> -# $FreeBSD: src/etc/nsswitch.conf,v 1.1.10.1 2009/08/03 08:13:06 kensmith >> Exp $ >> +# $FreeBSD: src/etc/nsswitch.conf,v 1.1 2006/05/03 15:14:47 ume Exp $ >> # >> group: compat >> -group_compat: nis >> +group_compat: ldap nis >> hosts: files dns >> networks: files >> passwd: compat >> -passwd_compat: nis >> +passwd_compat: ldap nis >> shells: files >> services: compat >> services_compat: nis >> >> The line `+:*::::::::' has already put into /etc/master.passwd, >> and the line `+:*::' has already put into /etc/group. > > I may be completely wrong (I can't seem to find the source), and I > don't know if it is the source of your problem, but I recall it being > reported that 'passwd_compat' and 'group_compat' require a *single* > source entry. > > > -- > greg byshenk - gbyshenk@byshenk.net - Leiden, NL > > From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 08:25:11 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95850106566C for ; Wed, 10 Mar 2010 08:25:11 +0000 (UTC) (envelope-from kamikaze@bsdforen.de) Received: from mail.bsdforen.de (bsdforen.de [212.204.60.79]) by mx1.freebsd.org (Postfix) with ESMTP id E72208FC0C for ; Wed, 10 Mar 2010 08:25:10 +0000 (UTC) Received: from mobileKamikaze.norad (vpn-cl-163-18.rz.uni-karlsruhe.de [141.3.163.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.bsdforen.de (Postfix) with ESMTP id 9F2E38A17F9; Wed, 10 Mar 2010 09:25:09 +0100 (CET) Message-ID: <4B975765.1080907@bsdforen.de> Date: Wed, 10 Mar 2010 09:25:09 +0100 From: Dominic Fandrey User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-GB; rv:1.9.1.8) Gecko/20100302 Thunderbird/3.0.3 MIME-Version: 1.0 To: Jeremy Chadwick References: <4B9601D9.5030101@bsdforen.de> <20100309210949.F85436@sola.nimnet.asn.au> <4B964A7C.6010709@bsdforen.de> <20100309133045.GA7232@icarus.home.lan> <20100309133322.GA7362@icarus.home.lan> In-Reply-To: <20100309133322.GA7362@icarus.home.lan> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, Ian Smith Subject: Re: ntpd does not re-query servers, when a new interface appears X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 08:25:11 -0000 On 09/03/2010 14:33, Jeremy Chadwick wrote: > On Tue, Mar 09, 2010 at 05:30:45AM -0800, Jeremy Chadwick wrote: >> On Tue, Mar 09, 2010 at 02:17:48PM +0100, Dominic Fandrey wrote: >>> On 09/03/2010 11:27, Ian Smith wrote: >>>> On Tue, 9 Mar 2010, Dominic Fandrey wrote: >>>> > ntpd tracks interface updates, however it does not requery >>>> > servers, when they occur. This was less than an hour ago, >>>> > at my university, the notebook boots and is not connected >>>> > to anything: >>>> > >>>> > 9 Mar 08:07:17 ntpd[1510]: logging to file /var/log/ntpd >>>> > 9 Mar 08:07:17 ntpd[1510]: precision = 2.234 usec >>>> > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #0 wildcard, 0.0.0.0#123 Disabled >>>> > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #1 wildcard, ::#123 Disabled >>>> > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #2 bge0, 192.168.1.12#123 Enabled >>>> > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #3 lo0, fe80::1#123 Enabled >>>> > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #4 lo0, ::1#123 Enabled >>>> > 9 Mar 08:07:17 ntpd[1510]: Listening on interface #5 lo0, 127.0.0.1#123 Enabled >>>> > 9 Mar 08:07:17 ntpd[1510]: Listening on routing socket on fd #26 for interface updates >>>> > 9 Mar 08:07:17 ntpd[1510]: kernel time sync status 2040 >>>> > 9 Mar 08:07:17 ntpd[1510]: frequency initialized 3.155 PPM from /var/db/ntpd.drift >>>> > 9 Mar 08:07:20 ntpd[1542]: host name not found: 0.de.pool.ntp.org >>>> > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `0.de.pool.ntp.org', giving up on it >>>> > 9 Mar 08:07:20 ntpd[1542]: host name not found: 1.de.pool.ntp.org >>>> > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `1.de.pool.ntp.org', giving up on it >>>> > 9 Mar 08:07:20 ntpd[1542]: host name not found: 2.de.pool.ntp.org >>>> > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `2.de.pool.ntp.org', giving up on it >>>> > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de >>>> > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it >>>> > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp1.rz.uni-karlsruhe.de >>>> > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp1.rz.uni-karlsruhe.de', giving up on it >>>> > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp3.rz.uni-karlsruhe.de >>>> > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp3.rz.uni-karlsruhe.de', giving up on it >>>> > 9 Mar 08:07:20 ntpd[1542]: host name not found: ntp4.rz.uni-karlsruhe.de >>>> > 9 Mar 08:07:20 ntpd[1542]: couldn't resolve `ntp4.rz.uni-karlsruhe.de', giving up on it >>>> > >>>> > So ntpd has given up on all the servers listed in the ntp.conf file. >>>> >>>> Yes, but it looks more like name service that's not operating, ntpd >>>> seems to be doing its best but can't resolve the hostnames? >>> >>> Why would I have named running on a notebook? This is a notebook, >>> which is not connected to the internet. >>> >>>> > I then proceed to connect to the wireless network and proceed to log >>>> > into two VPNs: >>>> > >>>> > 9 Mar 08:08:58 ntpd[1510]: Listening on interface #6 wlan0, 192.168.75.58#123 Enabled >>>> > 9 Mar 08:09:00 ntpd[1510]: Listening on interface #7 tun0, 193.196.120.15#123 Enabled >>>> > 9 Mar 08:09:04 ntpd[1510]: Listening on interface #8 tun1, 141.3.162.67#123 Enabled >>>> > >>>> > Over interface #8 some of the servers are actually available, but >>>> > ntpq -p still states: >>>> > No association ID's returned >>>> > >>>> > Only when I restart ntpd, it operates as expected: >>>> > remote refid st t when poll reach delay offset jitter >>>> > ============================================================================== >>>> > zit-net2.uni-pa .STEP. 16 u - 512 0 0.000 0.000 0.000 >>>> > alpha.rueckgr.a .STEP. 16 u - 512 0 0.000 0.000 0.000 >>>> > ntp.goneco.de .STEP. 16 u - 512 0 0.000 0.000 0.000 >>>> > +proxy4.rz.uni-k 129.13.64.17 2 u 30 128 271 2.937 2.530 1.891 >>>> > +proxy2.rz.uni-k 129.13.64.17 2 u 58 128 375 3.593 -8.981 1.837 >>>> > *proxy1.rz.uni-k 129.13.64.17 2 u 15 128 271 3.297 8.244 1.487 >>>> >>>> I've always had to restart named after losing / regaining an interface, >>>> most noticeably after a suspend/resume (eg a low battery suspend), so I >>>> run /etc/rc.d/named restart from rc.resume. This looks like a similar >>>> issue perhaps, though I don't see why restarting only ntpd would fix it. >>> >>> As I said, named doesn't run at all. When the notebook gets an >>> internet connection, ntpd recognizes this. It somehow doesn't >>> occur to it, though, that it might be able to resolve the >>> servers, now. >> >> I believe this is the problem. Note that you'll need to add an SSL >> cert. exception for this site due to them using self-signed certs. >> >> https://support.ntp.org/bugs/show_bug.cgi?id=987 > > Oh, and/or this one too, which mentions how to get the FORCE_DNSRETRY > feature -- you get to rebuild ntpd. :-) > > https://support.ntp.org/bugs/show_bug.cgi?id=1178 > > Doesn't sound like this issue is FreeBSD-specific, so you might want to > chime in on those NTP bugs. OK, thanks for all the replies. I'll visit the NTP folks and make a little noise there. In the meantime, your comments made me realize, that I can circumvent this problem by adding the ntp pools to my /etc/hosts file. Regards -- A: Because it fouls the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 09:24:53 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 127DD106566B; Wed, 10 Mar 2010 09:24:53 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from proxypop2.sarenet.es (proxypop2.sarenet.es [194.30.0.95]) by mx1.freebsd.org (Postfix) with ESMTP id C3CB98FC0A; Wed, 10 Mar 2010 09:24:52 +0000 (UTC) Received: from [172.16.1.204] (unknown [192.148.167.2]) by proxypop2.sarenet.es (Postfix) with ESMTP id 5EC0A733A3; Wed, 10 Mar 2010 10:24:51 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: Date: Wed, 10 Mar 2010 10:24:49 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> To: Borja Marcos X-Mailer: Apple Mail (2.1077) Cc: FreeBSD Stable , Pawel Jakub Dawidek , Stefan Bethke Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 09:24:53 -0000 On Mar 9, 2010, at 3:18 PM, Borja Marcos wrote: >=20 > On Mar 9, 2010, at 1:58 PM, Pawel Jakub Dawidek wrote: >=20 >>>> What kind of hardware do you have there? There is 3-way deadlock = I've a >>>> fix for which would be hard to trigger on single or dual core = machines. >>>>=20 >>>> Feel free to try the fix: >>>>=20 >>>> http://people.freebsd.org/~pjd/patches/zfs_3way_deadlock.patch >>>=20 >>> Maybe related to the deadlock I reported when I was receiving an = incremental snapshot while the target dataset was being read? >>=20 >> Could be. This deadlock is in general related to zfs recv = functionality. >=20 > Aye aye, Sir >=20 > set fingers -position crossed >=20 > testing :) Tested. Same deadlock remains. Borja. From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 10:34:34 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66470106564A; Wed, 10 Mar 2010 10:34:34 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 04A3D8FC26; Wed, 10 Mar 2010 10:34:33 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2DEEF.dip.t-dialin.net [217.226.222.239]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id DB856844588; Wed, 10 Mar 2010 11:34:25 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id AE13C534B; Wed, 10 Mar 2010 11:34:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1268217262; bh=kZ4kDKDOxCSF6Dxp1KU/CpimnGA3mPs6rtk5z5XqVTc=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=zi2km/FjOrzOkNVAgNBosBwnYizunoy4WoZ1D/JF/GrjxmapXO7iCi5uJdZZCyVrj 4wZs+y2TNoBhhNcGjzrxIb9IBWp+LK8CUTN+5YN+2I7Au3u1/3KEqVyHITzYo9p135 azbHPjnLdPZoV9Hvdd/suH5AQn3G+JcpIZu57vYwj20w5Va203UAB/kgrU+bn2eglK PKLwZQdC2P44G1l6V7JudVVAbdBzllA8m9auVFWh5EjXE7aX0RhyhyhQEHSHTMzkmo fcI3yfZsosW0i4L2SVZLgixcT2CCmK9FR7aNmGyrNFuGW9Y01GK6nMJsOdcvji2hL+ YqqBt5ALnBVxA== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2AAYM60063101; Wed, 10 Mar 2010 11:34:22 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Wed, 10 Mar 2010 11:34:22 +0100 Message-ID: <20100310113422.95932h10tv1qh2o0@webmail.leidinger.net> Date: Wed, 10 Mar 2010 11:34:22 +0100 From: Alexander Leidinger To: "Robert N. M. Watson" References: <201003081000.12656.jhb@freebsd.org> <20100309092709.16456y55vq0kaa88@webmail.leidinger.net> <201003090747.00192.jhb@freebsd.org> <20100309151609.19673jkwuoptzy0w@webmail.leidinger.net> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: DB856844588.3125A X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-0.84, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, J_CHICKENPOX_73 0.60) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1268822068.63472@Ea0IGgC/rhqwcBnRPhkvOw X-EBL-Spam-Status: No Cc: freebsd-stable@freebsd.org, John Baldwin Subject: Re: is dtrace usable? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 10:34:34 -0000 Quoting "Robert N. M. Watson" (from Tue, 9 Mar 2010 16:39:09 +0000): > > On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: > >>> From this you can see that sys.mk is included and parsed before 'Makefile', >>> so the WITH_CTF=yes is not set until after sys.mk has been parsed. >> >> I think we need to find a different solution for this. The need to >> specify WITH_CTF at the command line is very error prone. :( > > You are neither the first person to have made this observation, nor > the first person to have failed to propose a solution in the form of > a patch :-). It is not a problem to provide a patch, the problem is something else. Is it correct that the result of the ctfmerge/cftconvert stuff is not covered by the CDDL? If yes, why not use it by default if the programs are available (I've read the comment for the NO_CTF part, but IMO we have a chicken&egg situation here, dtrace will not become popular if it is not easy to use it)? This default can be made only for the kernel (by making a copy of the definition of CTFCONVERT into bsd.prog.mk and bsd.lib.mk, or by undefining it there), or for kernel+userland (removing the !WITH_CTF -> NO_CTF part from sys.mk). Theoretically I have a patch for that, but somehow I stumple over something strange which I don't understand. I have the following (/usr/share/mk and /usr/src/share/mk are the same): ---snip--- --- bsd.lib.mk (revision 204031) +++ bsd.lib.mk (working copy) @@ -36,6 +36,18 @@ .if defined(DEBUG_FLAGS) CFLAGS+= ${DEBUG_FLAGS} +# Turn CTF conversion off by default for now. This default could be +# changed later if DTrace becomes popular. +NO_CTF= 1 +.if defined(WITH_CTF) +.undef NO_CTF +.endif + +.if defined(NO_CTF) +.undef CTFCONVERT +.undef CTFMERGE +.endif + .if !defined(NO_CTF) && (${DEBUG_FLAGS:M-g} != "") CTFFLAGS+= -g .endif [some more stuff to make the CTFCONVERT/CTFMERGE stuff working] --- sys.mk (revision 204031) +++ sys.mk (working copy) @@ -46,12 +46,6 @@ .endif PO_CFLAGS ?= ${CFLAGS} -# Turn CTF conversion off by default for now. This default could be -# changed later if DTrace becomes popular. -.if !defined(WITH_CTF) -NO_CTF = 1 -.endif - # C Type Format data is required for DTrace CTFFLAGS ?= -L VERSION [some more stuff to make the CTFCONVERT/CTFMERGE stuff working] --- bsd.prog.mk (revision 204031) +++ bsd.prog.mk (working copy) @@ -19,6 +19,18 @@ CFLAGS+=${DEBUG_FLAGS} CXXFLAGS+=${DEBUG_FLAGS} +# Turn CTF conversion off by default for now. This default could be +# changed later if DTrace becomes popular. +NO_CTF= 1 +.if defined(WITH_CTF) +.undef NO_CTF +.endif + +.if defined(NO_CTF) +.undef CTFCONVERT +.undef CTFMERGE +.endif + .if !defined(NO_CTF) && (${DEBUG_FLAGS:M-g} != "") CTFFLAGS+= -g .endif [some more stuff to make the CTFCONVERT/CTFMERGE stuff working] ---snip--- When I go to /usr/src/bin/mv and run "make -V NO_CTF -V WITH_CTF" I only get empty output. I would expect to see NO_CTF set to 1. And when grepping for NO_CTF in /usr/share/mk I only see the undef of NO_CTF which I added myself in the above patch. Bye, Alexander. -- Life is just a bowl of cherries, but why do I always get the pits? http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 11:02:11 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBE551065670 for ; Wed, 10 Mar 2010 11:02:11 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 05B5F8FC1E for ; Wed, 10 Mar 2010 11:02:09 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 4009045CD9; Wed, 10 Mar 2010 12:02:07 +0100 (CET) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 5AD8645685; Wed, 10 Mar 2010 12:02:03 +0100 (CET) Date: Wed, 10 Mar 2010 12:02:02 +0100 From: Pawel Jakub Dawidek To: Borja Marcos Message-ID: <20100310110202.GA1715@garage.freebsd.pl> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="0F1p//8PRICkK4MW" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: FreeBSD Stable , Stefan Bethke Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 11:02:11 -0000 --0F1p//8PRICkK4MW Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Mar 10, 2010 at 10:24:49AM +0100, Borja Marcos wrote: > Tested. Same deadlock remains. Ok, to track this down I need the following: Uncomment 'CFLAGS+=3D-DDEBUG=3D1' line in sys/modules/zfs/Makefile. Add the following lines to your kernel config: options WITNESS options WITNESS_SKIPSPIN options INVARIANTS options INVARIANT_SUPPORT options DEBUG_VFS_LOCKS options DEBUG_LOCKS options KDB options DDB Recompile your kernel and modules and reboot. Once the deadlock occur, enter DDB and send me the output of: ps show alllocks show lockedvnods show allchains alltrace Thanks. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --0F1p//8PRICkK4MW Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuXfCoACgkQForvXbEpPzRhqQCg0UXbWyt1zs9AGxPJGgU5Dg2m mTAAoPcK3HVV1pG4nEGGiMtm9dLgzAbR =lVA9 -----END PGP SIGNATURE----- --0F1p//8PRICkK4MW-- From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 11:35:23 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D605106564A for ; Wed, 10 Mar 2010 11:35:23 +0000 (UTC) (envelope-from roberto@keltia.net) Received: from keltia.net (unknown [IPv6:2a01:240:fe5c::41]) by mx1.freebsd.org (Postfix) with ESMTP id E87C18FC1D for ; Wed, 10 Mar 2010 11:35:22 +0000 (UTC) Received: from roberto-al.eurocontrol.fr (aran.keltia.net [88.191.250.24]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: roberto) by keltia.net (Postfix/TLS) with ESMTPSA id 524726F3C for ; Wed, 10 Mar 2010 12:35:21 +0100 (CET) Date: Wed, 10 Mar 2010 12:35:16 +0100 From: Ollivier Robert To: freebsd-stable@freebsd.org Message-ID: <20100310113516.GA8848@roberto-al.eurocontrol.fr> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> X-Operating-System: MacOS X / Macbook Pro - FreeBSD 7.2 / Dell D820 SMP User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (keltia.net); Wed, 10 Mar 2010 12:35:21 +0100 (CET) Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 11:35:23 -0000 According to Stefan Bethke: > The situation seems to be triggered by zfs receive'ing snapshots from the sister machine (both synchronize their active ZFS filesystems to each other, using zfs send and zfs receive). It appears it's the receiving causing trouble. Have you tuned kern.maxvnodes in /etc/sysctl.conf? When I move to this new machine, I forgot to get it much higher than the default (now I use 200000) and it was locking up pretty soon. Had not a single lockup now. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr In memoriam to Ondine : http://ondine.keltia.net/ From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 11:55:15 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8206106564A for ; Wed, 10 Mar 2010 11:55:15 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [212.12.50.234]) by mx1.freebsd.org (Postfix) with ESMTP id B1E338FC0A for ; Wed, 10 Mar 2010 11:55:15 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 55D30523A9; Wed, 10 Mar 2010 11:55:14 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <20100310113516.GA8848@roberto-al.eurocontrol.fr> Date: Wed, 10 Mar 2010 12:55:12 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100310113516.GA8848@roberto-al.eurocontrol.fr> To: Ollivier Robert X-Mailer: Apple Mail (2.1077) Cc: freebsd-stable@freebsd.org Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 11:55:16 -0000 Am 10.03.2010 um 12:35 schrieb Ollivier Robert: > According to Stefan Bethke: >> The situation seems to be triggered by zfs receive'ing snapshots from = the sister machine (both synchronize their active ZFS filesystems to = each other, using zfs send and zfs receive). It appears it's the = receiving causing trouble. >=20 > Have you tuned kern.maxvnodes in /etc/sysctl.conf? >=20 > When I move to this new machine, I forgot to get it much higher than = the default (now I use 200000) and it was locking up pretty soon. Had = not a single lockup now. I haven't, it's at the default of 100000. How would I be able to tell = if that limit is being reached? Right now: $ sysctl kern.maxvnodes vfs.numvnodes vfs.freevnodes kern.maxvnodes: 100000 vfs.numvnodes: 87287 vfs.freevnodes: 24993 and on the sister host: $ sysctl kern.maxvnodes vfs.numvnodes vfs.freevnodes kern.maxvnodes: 100000 vfs.numvnodes: 87681 vfs.freevnodes: 7600 Is there a rule of thumb what maxvnodes should be tuned to? Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 11:56:14 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AF9F10656A9 for ; Wed, 10 Mar 2010 11:56:14 +0000 (UTC) (envelope-from riccp@ige.unicamp.br) Received: from saturno.ige.unicamp.br (saturno.ige.unicamp.br [143.106.76.34]) by mx1.freebsd.org (Postfix) with ESMTP id 074F08FC28 for ; Wed, 10 Mar 2010 11:56:13 +0000 (UTC) Received: from saturno.ige.unicamp.br (localhost [127.0.0.1]) by saturno.ige.unicamp.br (Postfix) with ESMTP id B9198206F18; Wed, 10 Mar 2010 08:36:38 -0300 (BRT) X-Virus-Scanned: amavisd-new at ige.unicamp.br Received: from saturno.ige.unicamp.br ([127.0.0.1]) by saturno.ige.unicamp.br (saturno.ige.unicamp.br [127.0.0.1]) (amavisd-new, port 10024) with LMTP id e5mc44NXa7gn; Wed, 10 Mar 2010 08:36:38 -0300 (BRT) Received: from netuno.ige.unicamp.br (netuno.ige.unicamp.br [143.106.76.2]) by saturno.ige.unicamp.br (Postfix) with ESMTP id 76A25206EFC; Wed, 10 Mar 2010 08:36:38 -0300 (BRT) Received: by netuno.ige.unicamp.br (Postfix, from userid 1000) id 63A66A35; Wed, 10 Mar 2010 08:36:38 -0300 (BRT) Date: Wed, 10 Mar 2010 08:36:38 -0300 From: Ricardo Campos Passanezi To: Linghua Tseng Message-ID: <20100310113638.GE94966@ige.unicamp.br> References: <20100309000826.GF4648@cesium.hyperfine.info> <80F42CAF32A14A1FB43B830AAF877A5A@TinlansPC> <20100309191157.GG2303@core.byshenk.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-stable@freebsd.org, Greg Byshenk Subject: Re: Supplementary groups on LDAP cannot work with RELENG_8 +nss_ldap X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 11:56:14 -0000 On Wed, Mar 10, 2010 at 02:58:04PM +0800, Linghua Tseng wrote: > Thanks. > > I have tried to modify my /etc/nsswitch.conf to: > > group: compat > group_compat: ldap > hosts: files dns > networks: files > passwd: compat > passwd_compat: ldap > shells: files > services: compat > services_compat: nis > protocols: files > rpc: files Have you tried with group: files ldap group_compat: nis passwd: files ldap passwd_compat: nis ???? > >> group: compat > >> -group_compat: nis > >> +group_compat: ldap nis > >> hosts: files dns > >> networks: files > >> passwd: compat > >> -passwd_compat: nis > >> +passwd_compat: ldap nis > >> shells: files > >> services: compat > >> services_compat: nis > >> > >> The line `+:*::::::::' has already put into /etc/master.passwd, > >> and the line `+:*::' has already put into /etc/group. I don't use the "+:*::...." lines into master.passwd nor into group. -- Ricardo Campos Passanezi - Network Analyst PGP & GPG public key at: http://www.ige.unicamp.br/~riccp Institute of Geosciences - http://www.ige.unicamp.br - UNICAMP From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 12:25:05 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1E707106567A for ; Wed, 10 Mar 2010 12:25:05 +0000 (UTC) (envelope-from uranus@tinlans.org) Received: from tinlans.org (tinlans.org [220.133.199.45]) by mx1.freebsd.org (Postfix) with ESMTP id C9FAB8FC17 for ; Wed, 10 Mar 2010 12:25:04 +0000 (UTC) Received: from tinlans.org (localhost [127.0.0.1]) by tinlans.org (Postfix) with ESMTP id E991ACF042; Wed, 10 Mar 2010 20:25:36 +0800 (CST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=tinlans.org; h= x-mailer:content-transfer-encoding:content-type:content-type :mime-version:date:date:subject:subject:in-reply-to:references :from:from:message-id:received:received; s=tinlans; t= 1268223936; bh=iLvx1WArW7Hm+1hfip+qu0714cK0RqFpO+WhODkTpyU=; b=W gRb6MOm4YRZrDuqspCY+SKQ/DusoNYz1xrfJICw93iq+jDjT+/ex67+abimgwerk xDHASrXbTfg6ZIF5RA/1A== X-Virus-Scanned: amavisd-new at tinlans.org Received: from tinlans.org ([127.0.0.1]) by tinlans.org (tinlans.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Y5Rp6+-XsRiH; Wed, 10 Mar 2010 20:25:36 +0800 (CST) Received: from TinlansPC (TinlansPC [192.168.1.2]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by tinlans.org (Postfix) with ESMTPSA id A2406CF040; Wed, 10 Mar 2010 20:25:36 +0800 (CST) Message-ID: <6F9DD7495BB24976AEC591FE8C228D70@TinlansPC> From: "Linghua Tseng" To: "Ricardo Campos Passanezi" References: <20100309000826.GF4648@cesium.hyperfine.info> <80F42CAF32A14A1FB43B830AAF877A5A@TinlansPC> <20100309191157.GG2303@core.byshenk.net> <20100310113638.GE94966@ige.unicamp.br> In-Reply-To: <20100310113638.GE94966@ige.unicamp.br> Date: Wed, 10 Mar 2010 20:24:55 +0800 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal Importance: Normal X-Mailer: Microsoft Windows Live Mail 14.0.8089.726 X-Mimeole: Produced By Microsoft MimeOLE V14.0.8089.726 Cc: freebsd-stable@freebsd.org Subject: Re: Supplementary groups on LDAP cannot work with RELENG_8 +nss_ldap X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 12:25:05 -0000 > Have you tried with > > group: files ldap > group_compat: nis > passwd: files ldap > passwd_compat: nis > > > ???? Yes, this setting works properly. Does it mean that I cannot put `ldap' into passwd_compat & group_compat? Sometimes I need the support of +/- in my passwd & group database. Was this feature gone since RELENG_8? > I don't use the "+:*::...." lines into master.passwd nor into group. Of course, these lines are required only if you put `ldap' into passwd_compat & group_compat. To put `ldap' into group & passwd doesn't need them. From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 12:32:05 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2413106564A; Wed, 10 Mar 2010 12:32:05 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from proxypop1.sarenet.es (proxypop1.sarenet.es [194.30.0.99]) by mx1.freebsd.org (Postfix) with ESMTP id AF2828FC21; Wed, 10 Mar 2010 12:32:04 +0000 (UTC) Received: from [172.16.1.204] (unknown [192.148.167.2]) by proxypop1.sarenet.es (Postfix) with ESMTP id 2D7B66078; Wed, 10 Mar 2010 13:32:03 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <20100310110202.GA1715@garage.freebsd.pl> Date: Wed, 10 Mar 2010 13:32:02 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> To: Pawel Jakub Dawidek X-Mailer: Apple Mail (2.1077) Cc: freebsd-fs@freebsd.org, FreeBSD Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 12:32:05 -0000 =09 On Mar 10, 2010, at 12:02 PM, Pawel Jakub Dawidek wrote: > On Wed, Mar 10, 2010 at 10:24:49AM +0100, Borja Marcos wrote: >> Tested. Same deadlock remains. >=20 > Ok, to track this down I need the following: >=20 > Uncomment 'CFLAGS+=3D-DDEBUG=3D1' line in sys/modules/zfs/Makefile. >=20 > Add the following lines to your kernel config: >=20 > options WITNESS > options WITNESS_SKIPSPIN > options INVARIANTS > options INVARIANT_SUPPORT > options DEBUG_VFS_LOCKS > options DEBUG_LOCKS > options KDB > options DDB >=20 > Recompile your kernel and modules and reboot. >=20 > Once the deadlock occur, enter DDB and send me the output of: >=20 > ps > show alllocks > show lockedvnods > show allchains > alltrace Trying. I started my typical test: Machine 1 doing a make buildworld on a dataset with src and obj on it. Machine 1 replicating incremental snapshots of the dataset to machine 2. Machine 2 running some "tar cf - . | ( cd /pool/anotherdataset && tar xf = - )" from the dataset being replicated, ie, doing read operations on the = target dataset. This time, with all those debug options, there was no deadlock, but = almost an instant trap entering DDB. Unfortunately, I tried to capture = the output of "alltrace", etc using the capture option. But couldn't = come back to the system to read it. Any ideas? I'm using VMWare Fusion to run FreeBSD for these tests and = seems I'm out of luck, I don't see any console output mechanism. When rebooting I was greeted by some LORs lock order reversal: 1st 0xffffff000286c2e8 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:549 2nd 0xffffff000286b0d8 dn->dn_mtx (dn->dn_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:1173 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_block_freed() at dnode_block_freed+0x8e dbuf_read() at dbuf_read+0x155 dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0x12a dmu_read() at dmu_read+0x80 load_nvlist() at load_nvlist+0x85 spa_load() at spa_load+0x49a spa_open_common() at spa_open_common+0x12d spa_get_stats() at spa_get_stats+0x42 zfs_ioc_pool_stats() at zfs_ioc_pool_stats+0x2c zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffffd808, rbp =3D 0x801224140 --- lock order reversal: 1st 0xffffff0002864e70 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode_sync.c:381 2nd 0xffffff00026e5140 osi->os_lock (osi->os_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:323 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_destroy() at dnode_destroy+0xa6 dnode_buf_pageout() at dnode_buf_pageout+0xb2 dbuf_evict_user() at dbuf_evict_user+0x55 dbuf_clear() at dbuf_clear+0x5e dnode_evict_dbufs() at dnode_evict_dbufs+0x98 dmu_objset_evict_dbufs() at dmu_objset_evict_dbufs+0x11c dmu_objset_evict() at dmu_objset_evict+0xbf dsl_pool_close() at dsl_pool_close+0x52 spa_unload() at spa_unload+0xb2 spa_load() at spa_load+0x4da spa_open_common() at spa_open_common+0x12d spa_get_stats() at spa_get_stats+0x42 zfs_ioc_pool_stats() at zfs_ioc_pool_stats+0x2c zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffffd808, rbp =3D 0x801224140 --- lock order reversal: 1st 0xffffff000286c058 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1116 2nd 0xffffff0002591a38 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1120 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_dirty() at dbuf_dirty+0x892 dnode_setdirty() at dnode_setdirty+0x1a9 dbuf_dirty() at dbuf_dirty+0xa53 bplist_vacate() at bplist_vacate+0x4d spa_sync() at spa_sync+0x297 txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002905638 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 2nd 0xffffff000250c2f0 spa->spa_sync_bplist.bpl_lock = (spa->spa_sync_bplist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:235 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 bplist_enqueue_deferred() at bplist_enqueue_deferred+0x47 zio_free() at zio_free+0x105 arc_free() at arc_free+0x11c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x483 dbuf_write() at dbuf_write+0x24c dbuf_sync_list() at dbuf_sync_list+0x3eb dbuf_sync_list() at dbuf_sync_list+0x17f dnode_sync() at dnode_sync+0xc12 dmu_objset_sync() at dmu_objset_sync+0x134 dsl_pool_sync() at dsl_pool_sync+0x200 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff000293bcc8 zfs (zfs) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/gfs.c:437 2nd 0xffffff0002916310 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zget() at zfs_zget+0x23c zfs_root() at zfs_root+0x50 zfsctl_create() at zfsctl_create+0x82 zfs_mount() at zfs_mount+0x7ef vfs_donmount() at vfs_donmount+0xcde nmount() at nmount+0x63 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (378, FreeBSD ELF64, nmount), rip =3D 0x800f48f1c, rsp =3D = 0x7fffffffced8, rbp =3D 0x7fffffffcef8 --- lock order reversal: 1st 0xffffff00025e4078 zp->z_name_lock (zp->z_name_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_dir.c:212 2nd 0xffffff0002916330 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zget() at zfs_zget+0x23c zfs_dirent_lock() at zfs_dirent_lock+0x4a0 zfs_dirlook() at zfs_dirlook+0x90 zfs_lookup() at zfs_lookup+0x256 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x8d VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xaf vfs_cache_lookup() at vfs_cache_lookup+0xf0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x4a9 kern_statat_vnhook() at kern_statat_vnhook+0x8f kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x800fd8acc, rsp =3D = 0x7fffffffcf38, rbp =3D 0x7fffffffd3d0 --- lock order reversal: 1st 0xffffff0002916210 zfsvfs->z_teardown_inactive_lock = (zfsvfs->z_teardown_inactive_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vnops.c:3724 2nd 0xffffff0002916330 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:1027 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zinactive() at zfs_zinactive+0x95 zfs_inactive() at zfs_inactive+0x7e zfs_freebsd_inactive() at zfs_freebsd_inactive+0x1a VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xb5 vinactive() at vinactive+0x90 vputx() at vputx+0x2fc kern_statat_vnhook() at kern_statat_vnhook+0xfa kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x800fd8acc, rsp =3D = 0x7fffffffcf38, rbp =3D 0x7fffffffd3d0 --- lock order reversal: 1st 0xffffff0002861958 buf->b_lock (buf->b_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/arc.c:2509 2nd 0xffffff0002956430 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:421 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_read_done() at dbuf_read_done+0x3b arc_read_done() at arc_read_done+0x1d2 zio_done() at zio_done+0x308 zio_execute() at zio_execute+0xb1 arc_read_nolock() at arc_read_nolock+0x3d0 arc_read() at arc_read+0xaf dbuf_read() at dbuf_read+0x62b dmu_buf_hold() at dmu_buf_hold+0xcc zap_lockdir() at zap_lockdir+0x6e zap_cursor_retrieve() at zap_cursor_retrieve+0x1bc zfs_unlinked_drain() at zfs_unlinked_drain+0xd8 zfsvfs_setup() at zfsvfs_setup+0xfa zfs_mount() at zfs_mount+0x7df vfs_donmount() at vfs_donmount+0xcde nmount() at nmount+0x63 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (378, FreeBSD ELF64, nmount), rip =3D 0x800f48f1c, rsp =3D = 0x7fffffffced8, rbp =3D 0x7fffffffcef8 --- Expensive timeout(9) function: 0xffffffff80329580(0xffffff8000284000) = 0.012686807 s ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based = forwarding disabled, default to deny, logging disabled lock order reversal: 1st 0xffffff800a2f5d08 bufwait (bufwait) @ = /pool/newsrc/src/sys/kern/vfs_bio.c:2559 2nd 0xffffff0002c69800 dirhash (dirhash) @ = /pool/newsrc/src/sys/ufs/ufs/ufs_dirhash.c:285 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 ufsdirhash_acquire() at ufsdirhash_acquire+0x44 ufsdirhash_add() at ufsdirhash_add+0x19 ufs_direnter() at ufs_direnter+0x88b ufs_mkdir() at ufs_mkdir+0x623 VOP_MKDIR_APV() at VOP_MKDIR_APV+0xb9 kern_mkdirat() at kern_mkdirat+0x264 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (136, FreeBSD ELF64, mkdir), rip =3D 0x80072bb0c, rsp =3D = 0x7fffffffec88, rbp =3D 0x7fffffffef66 --- lock order reversal: 1st 0xffffff0002745038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 2nd 0xffffff000286ab88 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:130 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 dnode_verify() at dnode_verify+0x70 dnode_hold_impl() at dnode_hold_impl+0x73 dmu_bonus_hold() at dmu_bonus_hold+0x31 bplist_hold() at bplist_hold+0x48 bplist_enqueue() at bplist_enqueue+0x4c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dmu_objset_sync() at dmu_objset_sync+0x1fe dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002745038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 2nd 0xffffff0002f6b0d8 dn->dn_mtx (dn->dn_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:606 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_hold_impl() at dnode_hold_impl+0x184 dmu_bonus_hold() at dmu_bonus_hold+0x31 bplist_hold() at bplist_hold+0x48 bplist_enqueue() at bplist_enqueue+0x4c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dmu_objset_sync() at dmu_objset_sync+0x1fe dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002745038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 2nd 0xffffff00028842e8 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1724 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_rele() at dbuf_rele+0x2d dnode_hold_impl() at dnode_hold_impl+0x20f dmu_bonus_hold() at dmu_bonus_hold+0x31 bplist_hold() at bplist_hold+0x48 bplist_enqueue() at bplist_enqueue+0x4c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dmu_objset_sync() at dmu_objset_sync+0x1fe dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002745038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 2nd 0xffffff00027d5d40 osi->os_lock (osi->os_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:687 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_setdirty() at dnode_setdirty+0xbc dbuf_dirty() at dbuf_dirty+0x516 bplist_enqueue() at bplist_enqueue+0xbd dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dmu_objset_sync() at dmu_objset_sync+0x1fe dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002c84938 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 2nd 0xffffff0002f6b7b0 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:543 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 dbuf_read() at dbuf_read+0x2ad dbuf_will_dirty() at dbuf_will_dirty+0x53 dsl_dataset_block_kill() at dsl_dataset_block_kill+0xe9 dbuf_write() at dbuf_write+0x24c dbuf_sync_list() at dbuf_sync_list+0x159 dbuf_sync_list() at dbuf_sync_list+0x17f dnode_sync() at dnode_sync+0xc12 dmu_objset_sync() at dmu_objset_sync+0x134 dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff000291a210 zfsvfs->z_teardown_inactive_lock = (zfsvfs->z_teardown_inactive_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vfsops.c:917 2nd 0xffffff00027450f8 ds->ds_rwlock (ds->ds_rwlock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dsl_dataset.c:2864 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dsl_dataset_clone_swap() at dsl_dataset_clone_swap+0x5a dmu_recv_end() at dmu_recv_end+0x94 zfs_ioc_recv() at zfs_ioc_recv+0x29d zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffff8e98, rbp =3D 0x7fffffff9bd0 --- lock order reversal: 1st 0xffffff0002745038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:94 2nd 0xffffff0002f6b330 dn->dn_dbufs_mtx (dn->dn_dbufs_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1518 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_destroy() at dbuf_destroy+0x58 bplist_close() at bplist_close+0x37 dsl_dataset_clone_swap_sync() at dsl_dataset_clone_swap_sync+0x506 dsl_sync_task_group_sync() at dsl_sync_task_group_sync+0x173 dsl_pool_sync() at dsl_pool_sync+0x122 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002745038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:94 2nd 0xffffffff81152650 h->hash_mutexes[i] (h->hash_mutexes[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:191 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_destroy() at dbuf_destroy+0x111 bplist_close() at bplist_close+0x37 dsl_dataset_clone_swap_sync() at dsl_dataset_clone_swap_sync+0x506 dsl_sync_task_group_sync() at dsl_sync_task_group_sync+0x173 dsl_pool_sync() at dsl_pool_sync+0x122 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff000291a250 zfsvfs->z_znodes_lock (zfsvfs->z_znodes_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vfsops.c:1317 2nd 0xffffff000291a310 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:966 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_rezget() at zfs_rezget+0x4a zfs_resume_fs() at zfs_resume_fs+0x158 zfs_ioc_recv() at zfs_ioc_recv+0x2b4 zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffff8e98, rbp =3D 0x7fffffff9bd0 --- #=20 Borja. From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 12:35:52 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 057F01065672 for ; Wed, 10 Mar 2010 12:35:52 +0000 (UTC) (envelope-from roberto@keltia.net) Received: from keltia.net (unknown [IPv6:2a01:240:fe5c::41]) by mx1.freebsd.org (Postfix) with ESMTP id AF6768FC26 for ; Wed, 10 Mar 2010 12:35:51 +0000 (UTC) Received: from roberto-al.eurocontrol.fr (aran.keltia.net [88.191.250.24]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: roberto) by keltia.net (Postfix/TLS) with ESMTPSA id B677F42 for ; Wed, 10 Mar 2010 13:35:50 +0100 (CET) Date: Wed, 10 Mar 2010 13:35:49 +0100 From: Ollivier Robert To: freebsd-stable@freebsd.org Message-ID: <20100310123549.GB8848@roberto-al.eurocontrol.fr> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100310113516.GA8848@roberto-al.eurocontrol.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Operating-System: MacOS X / Macbook Pro - FreeBSD 7.2 / Dell D820 SMP User-Agent: Mutt/1.5.20 (2009-06-14) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.3 (keltia.net); Wed, 10 Mar 2010 13:35:50 +0100 (CET) Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 12:35:52 -0000 According to Stefan Bethke: > $ sysctl kern.maxvnodes vfs.numvnodes vfs.freevnodes > kern.maxvnodes: 100000 > vfs.numvnodes: 87681 > vfs.freevnodes: 7600 > > Is there a rule of thumb what maxvnodes should be tuned to? Not sure, I max'ed it to 200000 and the machine has not locked up since. Try that, 300000 if not and so on. The thing is, receiving snapshots is going to generate a lot of inodes/directories so vnodes shortage may be your problem. -- Ollivier ROBERT -=- FreeBSD: The Power to Serve! -=- roberto@keltia.freenix.fr In memoriam to Ondine : http://ondine.keltia.net/ From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 12:56:48 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 455521065672; Wed, 10 Mar 2010 12:56:48 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 824E08FC15; Wed, 10 Mar 2010 12:56:46 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 2CD0545C99; Wed, 10 Mar 2010 13:56:45 +0100 (CET) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id E663745683; Wed, 10 Mar 2010 13:56:39 +0100 (CET) Date: Wed, 10 Mar 2010 13:56:39 +0100 From: Pawel Jakub Dawidek To: Borja Marcos Message-ID: <20100310125639.GB1715@garage.freebsd.pl> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qlTNgmc+xy1dBmNv" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, FreeBSD Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 12:56:48 -0000 --qlTNgmc+xy1dBmNv Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Mar 10, 2010 at 01:32:02PM +0100, Borja Marcos wrote: > Trying. I started my typical test: >=20 > Machine 1 doing a make buildworld on a dataset with src and obj on it. >=20 > Machine 1 replicating incremental snapshots of the dataset to machine 2. >=20 > Machine 2 running some "tar cf - . | ( cd /pool/anotherdataset && tar xf = - )" from the dataset being replicated, ie, doing read operations on the ta= rget dataset. >=20 > This time, with all those debug options, there was no deadlock, but almos= t an instant trap entering DDB. Unfortunately, I tried to capture the outpu= t of "alltrace", etc using the capture option. But couldn't come back to th= e system to read it. >=20 > Any ideas? I'm using VMWare Fusion to run FreeBSD for these tests and see= ms I'm out of luck, I don't see any console output mechanism. You should be able to use text dumps to capture output of those commands on panic. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --qlTNgmc+xy1dBmNv Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuXlwcACgkQForvXbEpPzRzNACfTNHrGE+pPTvYnZlXaRaPpxGc 2u8An3VA/URKFHEEeca5ItcVTdP5G+Uy =z7Mk -----END PGP SIGNATURE----- --qlTNgmc+xy1dBmNv-- From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 13:49:33 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2AE841065672; Wed, 10 Mar 2010 13:49:33 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id F23008FC14; Wed, 10 Mar 2010 13:49:32 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id AE45C46B66; Wed, 10 Mar 2010 08:49:32 -0500 (EST) Received: from jhbbsd.localnet (smtp.hudson-trading.com [209.249.190.9]) by bigwig.baldwin.cx (Postfix) with ESMTPA id C8D298A026; Wed, 10 Mar 2010 08:49:31 -0500 (EST) From: John Baldwin To: Alexander Leidinger Date: Wed, 10 Mar 2010 08:12:29 -0500 User-Agent: KMail/1.12.1 (FreeBSD/7.3-CBSD-20100217; KDE/4.3.1; amd64; ; ) References: <20100310113422.95932h10tv1qh2o0@webmail.leidinger.net> In-Reply-To: <20100310113422.95932h10tv1qh2o0@webmail.leidinger.net> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201003100812.29749.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.0.1 (bigwig.baldwin.cx); Wed, 10 Mar 2010 08:49:31 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at bigwig.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-1.6 required=4.2 tests=AWL,BAYES_00 autolearn=ham version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on bigwig.baldwin.cx Cc: freebsd-stable@freebsd.org, "Robert N. M. Watson" Subject: Re: is dtrace usable? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 13:49:33 -0000 On Wednesday 10 March 2010 5:34:22 am Alexander Leidinger wrote: > Quoting "Robert N. M. Watson" (from Tue, 9 Mar > 2010 16:39:09 +0000): > > > > > On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: > > > >>> From this you can see that sys.mk is included and parsed before 'Makefile', > >>> so the WITH_CTF=yes is not set until after sys.mk has been parsed. > >> > >> I think we need to find a different solution for this. The need to > >> specify WITH_CTF at the command line is very error prone. :( > > > > You are neither the first person to have made this observation, nor > > the first person to have failed to propose a solution in the form of > > a patch :-). > > It is not a problem to provide a patch, the problem is something else. > > Is it correct that the result of the ctfmerge/cftconvert stuff is not > covered by the CDDL? > > If yes, why not use it by default if the programs are available (I've > read the comment for the NO_CTF part, but IMO we have a chicken&egg > situation here, dtrace will not become popular if it is not easy to > use it)? This default can be made only for the kernel (by making a > copy of the definition of CTFCONVERT into bsd.prog.mk and bsd.lib.mk, > or by undefining it there), or for kernel+userland (removing the > !WITH_CTF -> NO_CTF part from sys.mk). Unfortunately the ctf stuff breaks static binaries. I think that if that were fixed we would simply enable it by default and be done. -- John Baldwin From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 14:28:19 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 60397106566B; Wed, 10 Mar 2010 14:28:19 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from freebsd-current.sentex.ca (freebsd-current.sentex.ca [64.7.128.98]) by mx1.freebsd.org (Postfix) with ESMTP id 0D5198FC18; Wed, 10 Mar 2010 14:28:18 +0000 (UTC) Received: from freebsd-current.sentex.ca (localhost [127.0.0.1]) by freebsd-current.sentex.ca (8.14.4/8.14.3) with ESMTP id o2AESIiW020154; Wed, 10 Mar 2010 09:28:18 -0500 (EST) (envelope-from tinderbox@freebsd.org) Received: (from tinderbox@localhost) by freebsd-current.sentex.ca (8.14.4/8.14.3/Submit) id o2AESIHH020139; Wed, 10 Mar 2010 14:28:18 GMT (envelope-from tinderbox@freebsd.org) Date: Wed, 10 Mar 2010 14:28:18 GMT Message-Id: <201003101428.o2AESIHH020139@freebsd-current.sentex.ca> X-Authentication-Warning: freebsd-current.sentex.ca: tinderbox set sender to FreeBSD Tinderbox using -f Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Cc: Subject: [releng_8 tinderbox] failure on arm/arm X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 14:28:19 -0000 TB --- 2010-03-10 14:00:00 - tinderbox 2.6 running on freebsd-current.sentex.ca TB --- 2010-03-10 14:00:00 - starting RELENG_8 tinderbox run for arm/arm TB --- 2010-03-10 14:00:00 - cleaning the object tree TB --- 2010-03-10 14:00:08 - cvsupping the source tree TB --- 2010-03-10 14:00:08 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/RELENG_8/arm/arm/supfile TB --- 2010-03-10 14:00:26 - building world TB --- 2010-03-10 14:00:26 - MAKEOBJDIRPREFIX=/obj TB --- 2010-03-10 14:00:26 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2010-03-10 14:00:26 - TARGET=arm TB --- 2010-03-10 14:00:26 - TARGET_ARCH=arm TB --- 2010-03-10 14:00:26 - TZ=UTC TB --- 2010-03-10 14:00:26 - __MAKE_CONF=/dev/null TB --- 2010-03-10 14:00:26 - cd /src TB --- 2010-03-10 14:00:26 - /usr/bin/make -B buildworld >>> World build started on Wed Mar 10 14:00:26 UTC 2010 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree >>> stage 2.2: rebuilding the object tree >>> stage 2.3: build tools >>> stage 3: cross tools >>> stage 4.1: building includes >>> stage 4.2: building libraries >>> stage 4.3: make dependencies >>> stage 4.4: building everything [...] gzip -cn /src/lib/libc/sys/sched_setparam.2 > sched_setparam.2.gz gzip -cn /src/lib/libc/sys/sched_setscheduler.2 > sched_setscheduler.2.gz gzip -cn /src/lib/libc/sys/sched_yield.2 > sched_yield.2.gz gzip -cn /src/lib/libc/sys/sctp_generic_recvmsg.2 > sctp_generic_recvmsg.2.gz gzip -cn /src/lib/libc/sys/sctp_generic_sendmsg.2 > sctp_generic_sendmsg.2.gz gzip -cn /src/lib/libc/sys/sctp_peeloff.2 > sctp_peeloff.2.gz gzip -cn /src/lib/libc/sys/select.2 > select.2.gz /libexec/ld-elf.so.1: Cannot open "/lib/libncurses.so.8" *** Error code 1 Stop in /src/lib/libc. *** Error code 1 Stop in /src/lib. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2010-03-10 14:28:18 - WARNING: /usr/bin/make returned exit code 1 TB --- 2010-03-10 14:28:18 - ERROR: failed to build world TB --- 2010-03-10 14:28:18 - 1165.76 user 320.11 system 1697.95 real http://tinderbox.freebsd.org/tinderbox-releng_8-RELENG_8-arm-arm.full From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 14:30:22 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7412106568F; Wed, 10 Mar 2010 14:30:22 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from freebsd-current.sentex.ca (freebsd-current.sentex.ca [64.7.128.98]) by mx1.freebsd.org (Postfix) with ESMTP id 6F3FC8FC15; Wed, 10 Mar 2010 14:30:22 +0000 (UTC) Received: from freebsd-current.sentex.ca (localhost [127.0.0.1]) by freebsd-current.sentex.ca (8.14.4/8.14.3) with ESMTP id o2AEULMa049221; Wed, 10 Mar 2010 09:30:21 -0500 (EST) (envelope-from tinderbox@freebsd.org) Received: (from tinderbox@localhost) by freebsd-current.sentex.ca (8.14.4/8.14.3/Submit) id o2AEULsZ049210; Wed, 10 Mar 2010 14:30:21 GMT (envelope-from tinderbox@freebsd.org) Date: Wed, 10 Mar 2010 14:30:21 GMT Message-Id: <201003101430.o2AEULsZ049210@freebsd-current.sentex.ca> X-Authentication-Warning: freebsd-current.sentex.ca: tinderbox set sender to FreeBSD Tinderbox using -f Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Cc: Subject: [releng_8 tinderbox] failure on i386/pc98 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 14:30:22 -0000 TB --- 2010-03-10 14:28:19 - tinderbox 2.6 running on freebsd-current.sentex.ca TB --- 2010-03-10 14:28:19 - starting RELENG_8 tinderbox run for i386/pc98 TB --- 2010-03-10 14:28:19 - cleaning the object tree TB --- 2010-03-10 14:28:38 - cvsupping the source tree TB --- 2010-03-10 14:28:38 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/RELENG_8/i386/pc98/supfile TB --- 2010-03-10 14:29:22 - building world TB --- 2010-03-10 14:29:22 - MAKEOBJDIRPREFIX=/obj TB --- 2010-03-10 14:29:22 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2010-03-10 14:29:22 - TARGET=pc98 TB --- 2010-03-10 14:29:22 - TARGET_ARCH=i386 TB --- 2010-03-10 14:29:22 - TZ=UTC TB --- 2010-03-10 14:29:22 - __MAKE_CONF=/dev/null TB --- 2010-03-10 14:29:22 - cd /src TB --- 2010-03-10 14:29:22 - /usr/bin/make -B buildworld >>> World build started on Wed Mar 10 14:29:23 UTC 2010 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree [...] ===> usr.bin/rpcgen (cleandir) rm -f rpcgen rpc_main.o rpc_clntout.o rpc_cout.o rpc_hout.o rpc_parse.o rpc_sample.o rpc_scan.o rpc_svcout.o rpc_tblout.o rpc_util.o rpcgen.1.gz rpcgen.1.cat.gz rm -f .depend GPATH GRTAGS GSYMS GTAGS ===> usr.bin/rpcinfo (cleandir) rm -f rpcinfo rpcinfo.o rpcinfo.8.gz rpcinfo.8.cat.gz rm -f .depend GPATH GRTAGS GSYMS GTAGS ===> usr.bin/rs (cleandir) /usr/bin/make: Permission denied *** Error code 126 Stop in /src/usr.bin. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2010-03-10 14:30:21 - WARNING: /usr/bin/make returned exit code 1 TB --- 2010-03-10 14:30:21 - ERROR: failed to build world TB --- 2010-03-10 14:30:21 - 32.54 user 21.50 system 122.76 real http://tinderbox.freebsd.org/tinderbox-releng_8-RELENG_8-i386-pc98.full From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 14:55:04 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5B9CF106564A; Wed, 10 Mar 2010 14:55:04 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from freebsd-current.sentex.ca (freebsd-current.sentex.ca [64.7.128.98]) by mx1.freebsd.org (Postfix) with ESMTP id 20ED38FC0C; Wed, 10 Mar 2010 14:55:03 +0000 (UTC) Received: from freebsd-current.sentex.ca (localhost [127.0.0.1]) by freebsd-current.sentex.ca (8.14.4/8.14.3) with ESMTP id o2AEt3Ga044652; Wed, 10 Mar 2010 09:55:03 -0500 (EST) (envelope-from tinderbox@freebsd.org) Received: (from tinderbox@localhost) by freebsd-current.sentex.ca (8.14.4/8.14.3/Submit) id o2AEt3tx044637; Wed, 10 Mar 2010 14:55:03 GMT (envelope-from tinderbox@freebsd.org) Date: Wed, 10 Mar 2010 14:55:03 GMT Message-Id: <201003101455.o2AEt3tx044637@freebsd-current.sentex.ca> X-Authentication-Warning: freebsd-current.sentex.ca: tinderbox set sender to FreeBSD Tinderbox using -f Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Cc: Subject: [releng_8 tinderbox] failure on i386/i386 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 14:55:04 -0000 TB --- 2010-03-10 14:25:03 - tinderbox 2.6 running on freebsd-current.sentex.ca TB --- 2010-03-10 14:25:03 - starting RELENG_8 tinderbox run for i386/i386 TB --- 2010-03-10 14:25:03 - cleaning the object tree TB --- 2010-03-10 14:25:28 - cvsupping the source tree TB --- 2010-03-10 14:25:28 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/RELENG_8/i386/i386/supfile TB --- 2010-03-10 14:25:42 - building world TB --- 2010-03-10 14:25:42 - MAKEOBJDIRPREFIX=/obj TB --- 2010-03-10 14:25:42 - PATH=/usr/bin:/usr/sbin:/bin:/sbin TB --- 2010-03-10 14:25:42 - TARGET=i386 TB --- 2010-03-10 14:25:42 - TARGET_ARCH=i386 TB --- 2010-03-10 14:25:42 - TZ=UTC TB --- 2010-03-10 14:25:42 - __MAKE_CONF=/dev/null TB --- 2010-03-10 14:25:42 - cd /src TB --- 2010-03-10 14:25:42 - /usr/bin/make -B buildworld >>> World build started on Wed Mar 10 14:25:43 UTC 2010 >>> Rebuilding the temporary build tree >>> stage 1.1: legacy release compatibility shims >>> stage 1.2: bootstrap tools >>> stage 2.1: cleaning up the object tree >>> stage 2.2: rebuilding the object tree >>> stage 2.3: build tools >>> stage 3: cross tools >>> stage 4.1: building includes >>> stage 4.2: building libraries >>> stage 4.3: make dependencies [...] ===> gnu/usr.bin/cc (depend) ===> gnu/usr.bin/cc/cc_tools (depend) cc -O2 -pipe -I. -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr\" -I/obj/i386/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/config -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/include -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libcpp/include -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libdecnumber -g -DGENERATOR_FILE -DHAVE_CONFIG_H -std=gnu89 -fstack-protector -c /src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/gengtype.c cc -O2 -pipe -I. -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr\" -I/obj/i386/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/config -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/include -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libcpp/include -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libdecnumber -g -DGENERATOR_FILE -DHAVE_CONFIG_H -std=gnu89 -fstack-protector -c gengtype-yacc+%DIKED.c cc -O2 -pipe -I. -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr\" -I/obj/i386/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/config -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/include -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libcpp/include -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libdecnumber -g -DGENERATOR_FILE -DHAVE_CONFIG_H -std=gnu89 -fstack-protector -c gengtype-lex.c cc -O2 -pipe -I. -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr\" -I/obj/i386/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/config -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/include -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libcpp/include -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libdecnumber -g -DGENERATOR_FILE -DHAVE_CONFIG_H -std=gnu89 -fstack-protector -c /src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/errors.c cc -O2 -pipe -I. -DIN_GCC -DHAVE_CONFIG_H -DPREFIX=\"/usr\" -I/obj/i386/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/src/gnu/usr.bin/cc/cc_tools/../cc_tools -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcc/config -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/include -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libcpp/include -I/src/gnu/usr.bin/cc/cc_tools/../../../../contrib/gcclibs/libdecnumber -g -DGENERATOR_FILE -DHAVE_CONFIG_H -std=gnu89 -fstack-protector -o gengtype gengtype.o gengtype-yacc+%DIKED.o gengtype-lex.o errors.o libiberty.a libiberty.a: could not read symbols: File format not recognized *** Error code 1 Stop in /src/gnu/usr.bin/cc/cc_tools. *** Error code 1 Stop in /src/gnu/usr.bin/cc. *** Error code 1 Stop in /src/gnu/usr.bin. *** Error code 1 Stop in /src/gnu. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. *** Error code 1 Stop in /src. TB --- 2010-03-10 14:55:03 - WARNING: /usr/bin/make returned exit code 1 TB --- 2010-03-10 14:55:03 - ERROR: failed to build world TB --- 2010-03-10 14:55:03 - 1296.30 user 276.52 system 1799.38 real http://tinderbox.freebsd.org/tinderbox-releng_8-RELENG_8-i386-i386.full From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 14:37:50 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBFA0106564A; Wed, 10 Mar 2010 14:37:50 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from proxypop1.sarenet.es (proxypop1.sarenet.es [194.30.0.99]) by mx1.freebsd.org (Postfix) with ESMTP id D9EDC8FC13; Wed, 10 Mar 2010 14:37:48 +0000 (UTC) Received: from [172.16.1.204] (unknown [192.148.167.2]) by proxypop1.sarenet.es (Postfix) with ESMTP id 9C7375FC5; Wed, 10 Mar 2010 15:37:47 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1077) From: Borja Marcos In-Reply-To: <20100310110202.GA1715@garage.freebsd.pl> Date: Wed, 10 Mar 2010 15:37:46 +0100 Message-Id: <548451D6-E62C-4B7F-A846-E9CF08A180C8@sarenet.es> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> To: Pawel Jakub Dawidek X-Mailer: Apple Mail (2.1077) X-Mailman-Approved-At: Wed, 10 Mar 2010 15:08:14 +0000 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, FreeBSD Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 14:37:51 -0000 =09 On Mar 10, 2010, at 12:02 PM, Pawel Jakub Dawidek wrote: > Recompile your kernel and modules and reboot. >=20 > Once the deadlock occur, enter DDB and send me the output of: >=20 > ps > show alllocks > show lockedvnods > show allchains > alltrace (copying to freebsd-fs, as this is filesystem-related) There we go. Thanks for the textdump hint :) Instead of a deadlock I had a crash. Maybe some of the additional = debugging options triggered it. It is a 8-stable cvsupped on 2nd March, = and with this patch applied: http://people.freebsd.org/~pjd/patches/zfs_3way_deadlock.patch This is ddb.txt: #=20 #=20 # cat ddb.txt=20 db> set $terminal=3D0 Unknown variable db> set $lines=3D0 db> ps pid ppid pgrp uid state wmesg wchan cmd 1425 1423 1423 0 S+ zfsvfs-> 0xffffff000256d310 bsdtar 1424 1423 1423 0 S+ pipdwt 0xffffff00025d6000 bsdtar 1423 1384 1423 0 S+ wait 0xffffff0005106000 sh 1422 1420 1420 0 R+ bsdtar 1421 1420 1420 0 S+ pipdwt 0xffffff00025d65b0 bsdtar 1420 1384 1420 0 S+ wait 0xffffff0005107000 sh 1419 1417 1417 0 S+ zfs 0xffffff00057ac578 bsdtar 1418 1417 1417 0 S+ pipdwt 0xffffff00025d4000 bsdtar 1417 1384 1417 0 S+ wait 0xffffff0002e79000 sh 1416 1414 1414 0 S+ zio->io_ 0xffffff00026c5b28 bsdtar 1415 1414 1414 0 S+ pipdwt 0xffffff00025d6b60 bsdtar 1414 1384 1414 0 S+ wait 0xffffff0002e788c0 sh 1413 1409 1413 0 S+ select 0xffffff00029d56c0 top 1409 1408 1409 0 S+ pause 0xffffff0002d83500 csh 1408 1405 1408 1001 S+ wait 0xffffff0002e7a000 su 1405 1404 1405 1001 Ss+ pause 0xffffff0002e78500 csh 1404 1401 1401 1001 S select 0xffffff0002cce1c0 sshd 1401 1280 1401 0 Ss sbwait 0xffffff0002f63694 sshd 1384 1383 1384 0 S+ ttyin 0xffffff00024b54a8 csh 1383 1377 1383 1001 S+ wait 0xffffff0002e77460 su 1377 1376 1377 1001 Ss+ pause 0xffffff0002e79500 csh 1376 1373 1373 1001 S select 0xffffff0002916440 sshd 1373 1280 1373 0 Ss sbwait 0xffffff0002cff3ec sshd 1372 1 1372 0 Ss+ ttyin 0xffffff00024daca8 getty 1371 1 1371 0 Ss+ ttyin 0xffffff00024dc0a8 getty 1370 1 1370 0 Ss+ ttyin 0xffffff00024cdca8 getty 1369 1 1369 0 Ss+ ttyin 0xffffff00024c94a8 getty 1368 1 1368 0 Ss+ ttyin 0xffffff00024dd4a8 getty 1367 1 1367 0 Ss+ ttyin 0xffffff00024c28a8 getty 1366 1 1366 0 Ss+ ttyin 0xffffff00024db4a8 getty 1365 1 1365 0 Ss+ ttyin 0xffffff00024dc8a8 getty 1301 1 1301 0 Ss nanslp 0xffffffff80c4d168 cron 1294 1 1294 25 Ss pause 0xffffff0002623960 sendmail 1288 1 1288 0 Ss select 0xffffff0002917cc0 sendmail 1280 1 1280 0 Ss select 0xffffff00029d5340 sshd 1005 1 1005 0 Rs syslogd 827 1 827 0 Ss select 0xffffff0002ccf2c0 devd 20 0 0 0 SL flowclea 0xffffffff80e0e950 = [flowcleaner] 19 0 0 0 SL sdflush 0xffffffff80e1d8b8 = [softdepflush] 18 0 0 0 RL [syncer] 17 0 0 0 SL vlruwt 0xffffff000256a8c0 [vnlru] 16 0 0 0 SL psleep 0xffffffff80e0e1c8 [bufdaemon] 15 0 0 0 SL pgzero 0xffffffff80e1f34c [pagezero] 14 0 0 0 SL psleep 0xffffffff80e1e6e8 [vmdaemon] 9 0 0 0 RL = [pagedaemon] 8 0 0 0 SL ccb_scan 0xffffffff80c16060 [xpt_thrd] 7 0 0 0 RL (threaded) zfskern 100085 Run CPU 0 = [txg_thread_enter] 100084 RunQ = [txg_thread_enter] 100082 D vgeom:io 0xffffff00026acd90 [vdev da2] 100036 D l2arc_fe 0xffffffff81150620 = [l2arc_feed_thread] 100035 D arc_recl 0xffffffff81147fe0 = [arc_reclaim_thread] 6 0 0 0 SL - 0xffffff000244e648 [fdc0] 5 0 0 0 SL idle 0xffffff800023c000 = [mpt_recovery0] 13 0 0 0 SL - 0xffffffff80c4ce44 [yarrow] 4 0 0 0 RL [g_down] 3 0 0 0 RL [g_up] 2 0 0 0 RL [g_event] 12 0 0 0 WL (threaded) intr 100033 I [swi0: uart = uart] 100032 I [irq7: = ppc0] 100031 I [irq12: = psm0] 100030 I [irq1: = atkbd0] 100024 I [irq17: em2 = mpt0] 100023 I [irq15: = ata1] 100022 I [irq14: = ata0] 100021 I [irq9: = acpi0] 100019 I [swi5: +] 100018 I [swi2: = cambio] 100013 I [swi6: task = queue] 100012 I [swi6: = Giant taskq] 100006 I [swi1: = netisr 0] 100005 I [swi4: = clock] 100004 I [swi3: vm] 11 0 0 0 RL [idle: = cpu0] 1 0 1 0 SLs wait 0xffffff00021e18c0 [init] 10 0 0 0 SL audit_wo 0xffffffff80e1cc10 [audit] 0 0 0 0 SLs (threaded) kernel 100088 D - 0xffffff0002914880 [zil_clean] 100087 D - 0xffffff0002916780 [zil_clean] 100086 D - 0xffffff0002913900 [zil_clean] 100083 D - 0xffffff0002628480 = [zfs_vn_rele_taskq] 100081 D - 0xffffff00026acc00 [spa_zio] 100080 D - 0xffffff00026acb80 [spa_zio] 100079 D - 0xffffff00026acb00 [spa_zio] 100078 D - 0xffffff00026aca80 [spa_zio] 100077 D - 0xffffff00026aca00 [spa_zio] 100076 D - 0xffffff00026ac980 [spa_zio] 100075 D - 0xffffff00026ac900 [spa_zio] 100074 D - 0xffffff00026ac880 [spa_zio_7] 100073 D - 0xffffff00026ac880 [spa_zio_6] 100072 D - 0xffffff00026ac880 [spa_zio_5] 100071 D - 0xffffff00026ac880 [spa_zio_4] 100070 D - 0xffffff00026ac880 [spa_zio_3] 100069 D - 0xffffff00026ac880 [spa_zio_2] 100068 D - 0xffffff00026ac880 [spa_zio_1] 100067 D - 0xffffff00026ac880 [spa_zio_0] 100066 D - 0xffffff00026ac800 [spa_zio_7] 100065 D - 0xffffff00026ac800 [spa_zio_6] 100064 D - 0xffffff00026ac800 [spa_zio_5] 100063 D - 0xffffff00026ac800 [spa_zio_4] 100062 D - 0xffffff00026ac800 [spa_zio_3] 100061 D - 0xffffff00026ac800 [spa_zio_2] 100060 D - 0xffffff00026ac800 [spa_zio_1] 100059 D - 0xffffff00026ac800 [spa_zio_0] 100058 D - 0xffffff00026ac780 [spa_zio] 100057 D - 0xffffff00026ac700 [spa_zio] 100056 D - 0xffffff00026ac680 [spa_zio] 100029 D - 0xffffff0002441980 [em3 taskq] 100028 D - 0xffffff0002440d80 [em2 taskq] 100027 D - 0xffffff0002436200 [em1 taskq] 100026 D - 0xffffff000242fb80 [em0 taskq] 100020 D - 0xffffff0002358600 [thread = taskq] 100017 D - 0xffffff0002316b80 = [acpi_task_2] 100016 D - 0xffffff0002316b80 = [acpi_task_1] 100015 D - 0xffffff0002316b80 = [acpi_task_0] 100014 D - 0xffffff0002316c00 [kqueue = taskq] 100010 D - 0xffffff00021e0180 [firmware = taskq] 100000 D sched 0xffffffff80c49740 [swapper] db> show alllocks Process 1425 (bsdtar) thread 0xffffff000528b3a0 (100115) Process 1422 (bsdtar) thread 0xffffff000528c000 (100112) Process 1419 (bsdtar) thread 0xffffff0002d81000 (100109) Process 1416 (bsdtar) thread 0xffffff000262a3a0 (100091) Process 1401 (sshd) thread 0xffffff0002510740 (100055) Process 1373 (sshd) thread 0xffffff000262a740 (100090) Process 7 (zfskern) thread 0xffffff000262bae0 (100085) db> show lockedvnods Locked vnodes db> show allchains db> alltrace Tracing command bsdtar pid 1425 tid 100115 td 0xffffff000528b3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sx_xlock_hard() at _sx_xlock_hard+0x267 _sx_xlock() at _sx_xlock+0xc1 zfs_zinactive() at zfs_zinactive+0x95 zfs_inactive() at zfs_inactive+0x7e zfs_freebsd_inactive() at zfs_freebsd_inactive+0x1a VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xb5 vinactive() at vinactive+0x90 vputx() at vputx+0x2fc kern_statat_vnhook() at kern_statat_vnhook+0xfa kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x800e35acc, rsp =3D = 0x7fffffffe768, rbp =3D 0x801025380 --- Tracing command bsdtar pid 1424 tid 100114 td 0xffffff000528b740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_write() at pipe_write+0x13fc dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x800e44d5c, rsp =3D = 0x7fffffffe0d8, rbp =3D 0x801004140 --- Tracing command sh pid 1423 tid 100113 td 0xffffff000528bae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip =3D 0x80093554c, rsp =3D = 0x7fffffffe8b8, rbp =3D 0x58f --- Tracing command bsdtar pid 1422 tid 100112 td 0xffffff000528c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a txg_wait_open() at txg_wait_open+0x95 zfs_freebsd_setattr() at zfs_freebsd_setattr+0x862 VOP_SETATTR_APV() at VOP_SETATTR_APV+0xaf setutimes() at setutimes+0x214 kern_lutimes() at kern_lutimes+0xf2 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (276, FreeBSD ELF64, lutimes), rip =3D 0x800da66dc, rsp =3D = 0x7fffffffe7e8, rbp =3D 0x801025380 --- Tracing command bsdtar pid 1421 tid 100111 td 0xffffff000528c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_write() at pipe_write+0x13fc dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x800e44d5c, rsp =3D = 0x7fffffffe0d8, rbp =3D 0x801004140 --- Tracing command sh pid 1420 tid 100110 td 0xffffff000262dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip =3D 0x80093554c, rsp =3D = 0x7fffffffe8b8, rbp =3D 0x58c --- Tracing command bsdtar pid 1419 tid 100109 td 0xffffff0002d81000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x7a3 vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b cache_lookup() at cache_lookup+0x4e0 vfs_cache_lookup() at vfs_cache_lookup+0xc0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x4a9 kern_statat_vnhook() at kern_statat_vnhook+0x8f kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x800e35acc, rsp =3D = 0x7fffffffe768, rbp =3D 0x801025380 --- Tracing command bsdtar pid 1418 tid 100108 td 0xffffff0002d813a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_write() at pipe_write+0x13fc dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x800e44d5c, rsp =3D = 0x7fffffffe0d8, rbp =3D 0x801004140 --- Tracing command sh pid 1417 tid 100101 td 0xffffff0002cfb000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip =3D 0x80093554c, rsp =3D = 0x7fffffffe8b8, rbp =3D 0x589 --- Tracing command bsdtar pid 1416 tid 100091 td 0xffffff000262a3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a zio_wait() at zio_wait+0x7d dbuf_read() at dbuf_read+0x2bb dnode_hold_impl() at dnode_hold_impl+0x114 dmu_bonus_hold() at dmu_bonus_hold+0x31 zfs_zget() at zfs_zget+0x259 zfs_dirent_lock() at zfs_dirent_lock+0x4a0 zfs_dirlook() at zfs_dirlook+0x90 zfs_lookup() at zfs_lookup+0x256 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x8d VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xaf vfs_cache_lookup() at vfs_cache_lookup+0xf0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x4a9 kern_mkdirat() at kern_mkdirat+0xc3 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (136, FreeBSD ELF64, mkdir), rip =3D 0x800e35b0c, rsp =3D = 0x7fffffffe748, rbp =3D 0x1ed --- Tracing command bsdtar pid 1415 tid 100105 td 0xffffff0002cfa000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_write() at pipe_write+0x13fc dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x800e44d5c, rsp =3D = 0x7fffffffe0d8, rbp =3D 0x801004140 --- Tracing command sh pid 1414 tid 100102 td 0xffffff0002cfaae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip =3D 0x80093554c, rsp =3D = 0x7fffffffe8b8, rbp =3D 0x586 --- Tracing command top pid 1413 tid 100054 td 0xffffff0002510ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x800ab7cfc, rsp =3D = 0x7fffffffea08, rbp =3D 0x11 --- Tracing command csh pid 1409 tid 100093 td 0xffffff0002645ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x80094d19c, rsp =3D = 0x7fffffffe638, rbp =3D 0x800c5a400 --- Tracing command su pid 1408 tid 100098 td 0xffffff0002cfbae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip =3D 0x8009e854c, rsp =3D = 0x7fffffffe598, rbp =3D 0x581 --- Tracing command csh pid 1405 tid 100103 td 0xffffff0002cfa740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x80094d19c, rsp =3D = 0x7fffffffe558, rbp =3D 0x800c0f900 --- Tracing command sshd pid 1404 tid 100104 td 0xffffff0002cfa3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x8013d7cfc, rsp =3D = 0x7fffffffdcb8, rbp =3D 0x7fffffffdd40 --- Tracing command sshd pid 1401 tid 100055 td 0xffffff0002510740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x8013d7d7c, rsp =3D = 0x7fffffffdcd8, rbp =3D 0 --- Tracing command csh pid 1384 tid 100107 td 0xffffff0002d81740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x8009ebd7c, rsp =3D = 0x7fffffffe738, rbp =3D 0x1 --- Tracing command su pid 1383 tid 100106 td 0xffffff0002d81ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip =3D 0x8009e854c, rsp =3D = 0x7fffffffe598, rbp =3D 0x568 --- Tracing command csh pid 1377 tid 100100 td 0xffffff0002cfb3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x80094d19c, rsp =3D = 0x7fffffffe558, rbp =3D 0x800c0f600 --- Tracing command sshd pid 1376 tid 100099 td 0xffffff0002cfb740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x8013d7cfc, rsp =3D = 0x7fffffffdcb8, rbp =3D 0x7fffffffdd40 --- Tracing command sshd pid 1373 tid 100090 td 0xffffff000262a740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x8013d7d7c, rsp =3D = 0x7fffffffdcd8, rbp =3D 0 --- Tracing command getty pid 1372 tid 100046 td 0xffffff00025b1ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1371 tid 100095 td 0xffffff00026453a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1370 tid 100096 td 0xffffff0002da63a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1369 tid 100097 td 0xffffff0002da6000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1368 tid 100092 td 0xffffff000262a000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1367 tid 100094 td 0xffffff0002645740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1366 tid 100089 td 0xffffff000262aae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1365 tid 100052 td 0xffffff00025b03a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command cron pid 1301 tid 100053 td 0xffffff00025b0000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip =3D 0x80093efac, rsp =3D = 0x7fffffffeb28, rbp =3D 0x3b --- Tracing command sendmail pid 1294 tid 100048 td 0xffffff00025b13a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x800d3a19c, rsp =3D = 0x7fffffffcd68, rbp =3D 0x1 --- Tracing command sendmail pid 1288 tid 100051 td 0xffffff00025b0740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x800dd8cfc, rsp =3D = 0x7fffffffc1e8, rbp =3D 0x7fffffffc280 --- Tracing command sshd pid 1280 tid 100047 td 0xffffff00025b1740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x8013d7cfc, rsp =3D = 0x7fffffffddd8, rbp =3D 0x2 --- Tracing command syslogd pid 1005 tid 100050 td 0xffffff00025b0ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x800851cfc, rsp =3D = 0x7fffffffe118, rbp =3D 0x800a310c8 --- Tracing command devd pid 827 tid 100049 td 0xffffff00025b1000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x440ecc, rsp =3D = 0x7fffffffe898, rbp =3D 0x7fffffffe8b0 --- Tracing command flowcleaner pid 20 tid 100045 td 0xffffff0002485ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c flowtable_cleaner() at flowtable_cleaner+0xde fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800018cd30, rbp =3D 0 --- Tracing command softdepflush pid 19 tid 100044 td 0xffffff000250d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 softdep_flush() at softdep_flush+0x259 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000187d30, rbp =3D 0 --- Tracing command syncer pid 18 tid 100043 td 0xffffff000250d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c sched_sync() at sched_sync+0x4db fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000182d30, rbp =3D 0 --- Tracing command vnlru pid 17 tid 100042 td 0xffffff000250d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vnlru_proc() at vnlru_proc+0x5cf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800017dd30, rbp =3D 0 --- Tracing command bufdaemon pid 16 tid 100041 td 0xffffff000250dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 buf_daemon() at buf_daemon+0x14a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000178d30, rbp =3D 0 --- Tracing command pagezero pid 15 tid 100040 td 0xffffff000250e000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vm_pagezero() at vm_pagezero+0x73 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000173d30, rbp =3D 0 --- Tracing command vmdaemon pid 14 tid 100039 td 0xffffff000250e3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 vm_daemon() at vm_daemon+0x4d fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800016ed30, rbp =3D 0 --- Tracing command pagedaemon pid 9 tid 100038 td 0xffffff000250e740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vm_pageout() at vm_pageout+0x855 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000169d30, rbp =3D 0 --- Tracing command xpt_thrd pid 8 tid 100037 td 0xffffff000250eae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 xpt_scanner_thread() at xpt_scanner_thread+0xd5 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000164d30, rbp =3D 0 --- Tracing command zfskern pid 7 tid 100085 td 0xffffff000262bae0 kdb_enter() at kdb_enter+0x3d panic() at panic+0x17b dblfault_handler() at dblfault_handler+0x96 Xdblfault() at Xdblfault+0xac --- trap 0x17, rip =3D 0xffffffff805d322d, rsp =3D 0xffffffff80e5ecd0, = rbp =3D 0xffffff8012415090 --- witness_checkorder() at witness_checkorder+0x2d _mtx_lock_spin_flags() at _mtx_lock_spin_flags+0x94 callout_tick() at callout_tick+0x41 hardclock_cpu() at hardclock_cpu+0xc8 hardclock() at hardclock+0x23 lapic_handle_timer() at lapic_handle_timer+0xd2 Xtimerint() at Xtimerint+0x8c --- interrupt, rip =3D 0xffffffff80866093, rsp =3D 0xffffff8012415270, = rbp =3D 0xffffff8012415290 --- spinlock_exit() at spinlock_exit+0x33 putcons() at putcons+0x40 putchar() at putchar+0x6a kvprintf() at kvprintf+0x81 vprintf() at vprintf+0x85 printf() at printf+0x67 db_putc() at db_putc+0x81 kvprintf() at kvprintf+0x81 db_printf() at db_printf+0x86 db_backtrace() at db_backtrace+0x162 db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 dnode_verify() at dnode_verify+0x70 dnode_hold_impl() at dnode_hold_impl+0x73 dmu_bonus_hold() at dmu_bonus_hold+0x31 bplist_hold() at bplist_hold+0x48 bplist_enqueue() at bplist_enqueue+0x4c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dbuf_write() at dbuf_write+0x24c dbuf_sync_list() at dbuf_sync_list+0x3eb dbuf_sync_list() at dbuf_sync_list+0x17f dbuf_sync_list() at dbuf_sync_list+0x17f dbuf_sync_list() at dbuf_sync_list+0x17f dbuf_sync_list() at dbuf_sync_list+0x17f dbuf_sync_list() at dbuf_sync_list+0x17f dbuf_sync_list() at dbuf_sync_list+0x17f dnode_sync() at dnode_sync+0xc12 dmu_objset_sync() at dmu_objset_sync+0x134 dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- Tracing command zfskern pid 7 tid 100084 td 0xffffff000262d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a txg_thread_wait() at txg_thread_wait+0x5b txg_quiesce_thread() at txg_quiesce_thread+0xb4 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012413d30, rbp =3D 0 --- Tracing command zfskern pid 7 tid 100082 td 0xffffff0002640000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 vdev_geom_worker() at vdev_geom_worker+0xf3 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123ffd30, rbp =3D 0 --- Tracing command zfskern pid 7 tid 100036 td 0xffffff0002510000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c l2arc_feed_thread() at l2arc_feed_thread+0x162 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800015fd30, rbp =3D 0 --- Tracing command zfskern pid 7 tid 100035 td 0xffffff0002356740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c arc_reclaim_thread() at arc_reclaim_thread+0x2b7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800015ad30, rbp =3D 0 --- Tracing command fdc0 pid 6 tid 100034 td 0xffffff0002356ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 fdc_thread() at fdc_thread+0x774 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000155d30, rbp =3D 0 --- Tracing command mpt_recovery0 pid 5 tid 100025 td 0xffffff0002353ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 mpt_recovery_thread() at mpt_recovery_thread+0x251 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800011ed30, rbp =3D 0 --- Tracing command yarrow pid 13 tid 100011 td 0xffffff00021e53a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 random_kthread() at random_kthread+0x1ad fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800004cd30, rbp =3D 0 --- Tracing command g_down pid 4 tid 100009 td 0xffffff00021e5ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 g_io_schedule_down() at g_io_schedule_down+0x236 g_down_procbody() at g_down_procbody+0x6f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000042d30, rbp =3D 0 --- Tracing command g_up pid 3 tid 100008 td 0xffffff00021f3000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 g_io_schedule_up() at g_io_schedule_up+0x133 g_up_procbody() at g_up_procbody+0x6f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800003dd30, rbp =3D 0 --- Tracing command g_event pid 2 tid 100007 td 0xffffff00021f33a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 g_event_procbody() at g_event_procbody+0xa1 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000038d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100033 td 0xffffff0002483000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100032 td 0xffffff00024833a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100031 td 0xffffff0002483740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100030 td 0xffffff0002483ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000137d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100024 td 0xffffff0002354000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80000d7d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100023 td 0xffffff00023543a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80000d2d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100022 td 0xffffff0002354740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100021 td 0xffffff0002354ae0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100019 td 0xffffff00023563a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000078d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100018 td 0xffffff00021f3ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000073d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100013 td 0xffffff0002353000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800005ad30, rbp =3D 0 --- Tracing command intr pid 12 tid 100012 td 0xffffff00023533a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100006 td 0xffffff00021f3740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000033d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100005 td 0xffffff00021e4000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800002ed30, rbp =3D 0 --- Tracing command intr pid 12 tid 100004 td 0xffffff00021e43a0 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xffffff00021e4740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d critical_exit() at critical_exit+0xa3 intr_event_handle() at intr_event_handle+0xd7 intr_execute_handlers() at intr_execute_handlers+0x5f lapic_handle_intr() at lapic_handle_intr+0x37 Xapic_isr1() at Xapic_isr1+0xa4 --- interrupt, rip =3D 0xffffffff808545f6, rsp =3D 0xffffff8000024b50, = rbp =3D 0xffffff8000024b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000024d30, rbp =3D 0 --- Tracing command init pid 1 tid 100002 td 0xffffff00021e4ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip =3D 0x40c86c, rsp =3D = 0x7fffffffe808, rbp =3D 0x401d40 --- Tracing command audit pid 10 tid 100001 td 0xffffff00021e5000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a audit_worker() at audit_worker+0x77 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800001ad30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100088 td 0xffffff000262b000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012426d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100087 td 0xffffff000262b3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012421d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100086 td 0xffffff000262b740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff801241cd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100083 td 0xffffff000263eae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012404d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100081 td 0xffffff00026403a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123fad30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100080 td 0xffffff000263d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123f5d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100079 td 0xffffff000263dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123f0d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100078 td 0xffffff000263e000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleep#=20 And this is msgbuf.txt: # cat msgbuf.txt=20 Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-STABLE #3: Wed Mar 10 14:41:54 CET 2010 = root@pruebazfs1.sarenet.es:/pool/newsrc/obj/pool/newsrc/src/sys/DEBUG = amd64 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM)2 Duo CPU T8100 @ 2.10GHz (2116.63-MHz = K8-class CPU) Origin =3D "GenuineIntel" Id =3D 0x10676 Stepping =3D 6 = Features=3D0xfebfbff Features2=3D0x80082201> AMD Features=3D0x20100800 AMD Features2=3D0x1 TSC: P-state invariant real memory =3D 536870912 (512 MB) avail memory =3D 495988736 (473 MB) ACPI APIC Table: MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port = 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x10c0-0x10cf at device 7.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] pci0: at device 7.3 (no driver attached) pci0: at device 7.7 (no driver attached) vgapci0: port 0x10d0-0x10df mem = 0xd0000000-0xd7ffffff,0xd8000000-0xd87fffff irq 16 at device 15.0 on = pci0 mpt0: port 0x1400-0x14ff mem = 0xd8820000-0xd883ffff,0xd8800000-0xd881ffff irq 17 at device 16.0 on = pci0 mpt0: [ITHREAD] mpt0: MPI Version=3D1.2.0.0 pcib2: at device 17.0 on pci0 pci2: on pcib2 em0: port 0x2000-0x203f = mem 0xd8940000-0xd895ffff,0xd8900000-0xd890ffff irq 18 at device 0.0 on = pci2 em0: Memory Access and/or Bus Master bits were not set! em0: [FILTER] em0: Ethernet address: 00:0c:29:11:4f:d5 em1: port 0x2040-0x207f = mem 0xd8960000-0xd897ffff,0xd8910000-0xd891ffff irq 19 at device 1.0 on = pci2 em1: Memory Access and/or Bus Master bits were not set! em1: [FILTER] em1: Ethernet address: 00:0c:29:11:4f:cb pci2: at device 2.0 (no driver attached) em2: port 0x20c0-0x20ff = mem 0xd8980000-0xd899ffff,0xd8920000-0xd892ffff irq 17 at device 3.0 on = pci2 em2: Memory Access and/or Bus Master bits were not set! em2: [FILTER] em2: Ethernet address: 00:0c:29:11:4f:df em3: port 0x2400-0x243f = mem 0xd89a0000-0xd89bffff,0xd8930000-0xd893ffff irq 19 at device 5.0 on = pci2 em3: Memory Access and/or Bus Master bits were not set! em3: [FILTER] em3: Ethernet address: 00:0c:29:11:4f:e9 pcib3: at device 21.0 on pci0 pci3: on pcib3 pcib4: at device 21.1 on pci0 pci4: on pcib4 pcib5: at device 21.2 on pci0 pci5: on pcib5 pcib6: at device 21.3 on pci0 pci6: on pcib6 pcib7: at device 21.4 on pci0 pci7: on pcib7 pcib8: at device 21.5 on pci0 pci8: on pcib8 pcib9: at device 21.6 on pci0 pci9: on pcib9 pcib10: at device 21.7 on pci0 pci10: on pcib10 pcib11: at device 22.0 on pci0 pci11: on pcib11 pcib12: at device 22.1 on pci0 pci12: on pcib12 pcib13: at device 22.2 on pci0 pci13: on pcib13 pcib14: at device 22.3 on pci0 pci14: on pcib14 pcib15: at device 22.4 on pci0 pci15: on pcib15 pcib16: at device 22.5 on pci0 pci16: on pcib16 pcib17: at device 22.6 on pci0 pci17: on pcib17 pcib18: at device 22.7 on pci0 pci18: on pcib18 pcib19: at device 23.0 on pci0 pci19: on pcib19 pcib20: at device 23.1 on pci0 pci20: on pcib20 pcib21: at device 23.2 on pci0 pci21: on pcib21 pcib22: at device 23.3 on pci0 pci22: on pcib22 pcib23: at device 23.4 on pci0 pci23: on pcib23 pcib24: at device 23.5 on pci0 pci24: on pcib24 pcib25: at device 23.6 on pci0 pci25: on pcib25 pcib26: at device 23.7 on pci0 pci26: on pcib26 pcib27: at device 24.0 on pci0 pci27: on pcib27 pcib28: at device 24.1 on pci0 pci28: on pcib28 pcib29: at device 24.2 on pci0 pci29: on pcib29 pcib30: at device 24.3 on pci0 pci30: on pcib30 pcib31: at device 24.4 on pci0 pci31: on pcib31 pcib32: at device 24.5 on pci0 pci32: on pcib32 pcib33: at device 24.6 on pci0 pci33: on pcib33 pcib34: at device 24.7 on pci0 pci34: on pcib34 acpi_acad0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse, device ID 3 ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppc0: [ITHREAD] ppbus0: on ppc0 plip0: on ppbus0 plip0: [ITHREAD] lpt0: on ppbus0 lpt0: [ITHREAD] lpt0: Interrupt-driven port ppi0: on ppbus0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: [FILTER] fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on = acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 cpu0: on acpi0 acpi_throttle0: on cpu0 orm0: at iomem = 0xc0000-0xc7fff,0xca000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff,0xcd000-0x= cdfff,0xdc000-0xdffff,0xe0000-0xe3fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on = isa0 ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is = present; to enable, add "vfs.zfs.prefetch_disable=3D0" to = /boot/loader.conf. ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable = behavior. Consider tuning vm.kmem_size and vm.kmem_size_max in /boot/loader.conf. ZFS filesystem version 3 ZFS storage pool version 14 Timecounter "TSC" frequency 2116630847 Hz quality 800 Timecounters tick every 10.000 msec Expensive timeout(9) function: 0xffffffff805ed840(0) 0.002412596 s acd0: DVDR at ata1-master = UDMA33=20 da0 at mpt0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device=20 da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da0: Command Queueing enabled da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) da1 at mpt0 bus 0 scbus0 target 1 lun 0 da1: Fixed Direct Access SCSI-2 device=20 da1: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da1: Command Queueing enabled da1: 4096MB (8388608 512 byte sectors: 255H 63S/T 522C) da2 at mpt0 bus 0 scbus0 target 2 lun 0 da2: Fixed Direct Access SCSI-2 device=20 da2: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da2: Command Queueing enabled da2: 4096MB (8388608 512 byte sectors: 255H 63S/T 522C) da3 at mpt0 bus 0 scbus0 target 3 lun 0 da3: Fixed Direct Access SCSI-2 device=20 da3: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da3: Command Queueing enabled da3: 12288MB (25165824 512 byte sectors: 255H 63S/T 1566C) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Trying to mount root from ufs:/dev/da0s1a WARNING: / was not properly dismounted /: mount pending error: blocks 4 files 1 <118>Setting hostuuid: 564d8c47-d7e5-fea4-92ef-f17760cfe4d2. <118>Setting hostid: 0x4d345a2c. lock order reversal: 1st 0xffffff000286d2e8 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:549 2nd 0xffffff000286c0d8 dn->dn_mtx (dn->dn_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:1173 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_block_freed() at dnode_block_freed+0x8e dbuf_read() at dbuf_read+0x155 dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0x12a dmu_read() at dmu_read+0x80 load_nvlist() at load_nvlist+0x85 spa_load() at spa_load+0x49a spa_open_common() at spa_open_common+0x12d spa_get_stats() at spa_get_stats+0x42 zfs_ioc_pool_stats() at zfs_ioc_pool_stats+0x2c zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffffd808, rbp =3D 0x801224140 --- Expensive timeout(9) function: 0xffffffff80892b50(0xffffff000244fe00) = 0.005049803 s lock order reversal: 1st 0xffffff0002868e70 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode_sync.c:381 2nd 0xffffff00026e7940 osi->os_lock (osi->os_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:323 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_destroy() at dnode_destroy+0xa6 dnode_buf_pageout() at dnode_buf_pageout+0xb2 dbuf_evict_user() at dbuf_evict_user+0x55 dbuf_clear() at dbuf_clear+0x5e dnode_evict_dbufs() at dnode_evict_dbufs+0x98 dmu_objset_evict_dbufs() at dmu_objset_evict_dbufs+0x11c dmu_objset_evict() at dmu_objset_evict+0xbf dsl_pool_close() at dsl_pool_close+0x52 spa_unload() at spa_unload+0xb2 spa_load() at spa_load+0x4da spa_open_common() at spa_open_common+0x12d spa_get_stats() at spa_get_stats+0x42 zfs_ioc_pool_stats() at zfs_ioc_pool_stats+0x2c zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffffd808, rbp =3D 0x801224140 --- lock order reversal: 1st 0xffffff0002864910 buf->b_lock (buf->b_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/arc.c:2509 2nd 0xffffff00028681a0 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:421 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_read_done() at dbuf_read_done+0x3b arc_read_done() at arc_read_done+0x1d2 zio_done() at zio_done+0x308 zio_execute() at zio_execute+0xb1 arc_read_nolock() at arc_read_nolock+0x3d0 arc_read() at arc_read+0xaf dbuf_read() at dbuf_read+0x62b dmu_buf_hold() at dmu_buf_hold+0xcc zap_lockdir() at zap_lockdir+0x6e zap_lookup_norm() at zap_lookup_norm+0x45 zap_lookup() at zap_lookup+0x2e dsl_prop_get_dd() at dsl_prop_get_dd+0xfc dsl_dataset_get_ref() at dsl_dataset_get_ref+0x4a7 dsl_dataset_hold() at dsl_dataset_hold+0xf2 dmu_objset_open() at dmu_objset_open+0xcf zil_check_log_chain() at zil_check_log_chain+0x2a dmu_objset_find_spa() at dmu_objset_find_spa+0x23f dmu_objset_find_spa() at dmu_objset_find_spa+0x324 dmu_objset_find_spa() at dmu_objset_find_spa+0x324 dmu_objset_find() at dmu_objset_find+0x2a spa_check_logs() at spa_check_logs+0x2a spa_load() at spa_load+0x6ca spa_load() at spa_load+0x4fe spa_open_common() at spa_open_common+0x12d spa_get_stats() at spa_get_stats+0x42 zfs_ioc_pool_stats() at zfs_ioc_pool_stats+0x2c zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffffd808, rbp =3D 0x801224140 --- lock order reversal: 1st 0xffffff000286d058 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1116 2nd 0xffffff000259ac38 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1120 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_dirty() at dbuf_dirty+0x892 dnode_setdirty() at dnode_setdirty+0x1a9 dbuf_dirty() at dbuf_dirty+0xa53 bplist_vacate() at bplist_vacate+0x4d spa_sync() at spa_sync+0x297 txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff000292d538 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 2nd 0xffffff000250c2f0 spa->spa_sync_bplist.bpl_lock = (spa->spa_sync_bplist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:235 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 bplist_enqueue_deferred() at bplist_enqueue_deferred+0x47 zio_free() at zio_free+0x105 arc_free() at arc_free+0x11c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x483 dbuf_write() at dbuf_write+0x24c dbuf_sync_list() at dbuf_sync_list+0x3eb dbuf_sync_list() at dbuf_sync_list+0x17f dnode_sync() at dnode_sync+0xc12 dmu_objset_sync() at dmu_objset_sync+0x134 dsl_pool_sync() at dsl_pool_sync+0x200 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002948c38 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 2nd 0xffffff00028a7000 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1859 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 dbuf_check_blkptr() at dbuf_check_blkptr+0x11e dbuf_sync_list() at dbuf_sync_list+0x34a dbuf_sync_list() at dbuf_sync_list+0x17f dnode_sync() at dnode_sync+0xc12 dmu_objset_sync_dnodes() at dmu_objset_sync_dnodes+0x60 dmu_objset_sync() at dmu_objset_sync+0x162 dsl_pool_sync() at dsl_pool_sync+0x200 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- <118>Entropy harvesting: <118> interrupts <118> ethernet <118> point_to_point <118> kickstart <118>. <118>Starting file system checks: <118>/dev/da0s1a: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED) <118>/dev/da0s1a: SUMMARY INFORMATION BAD (SALVAGED) <118>/dev/da0s1a: BLK(S) MISSING IN BIT MAPS (SALVAGED) <118>/dev/da0s1a: 28961 files, 517324 used, 1004493 free (7877 frags, = 124577 blocks, 0.5% fragmentation) <118>Mounting local file systems: <118>. lock order reversal: 1st 0xffffff000263acc8 zfs (zfs) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/gfs.c:437 2nd 0xffffff000256e310 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zget() at zfs_zget+0x23c zfs_root() at zfs_root+0x50 zfsctl_create() at zfsctl_create+0x82 zfs_mount() at zfs_mount+0x7ef vfs_donmount() at vfs_donmount+0xcde nmount() at nmount+0x63 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (378, FreeBSD ELF64, nmount), rip =3D 0x800f48f1c, rsp =3D = 0x7fffffffced8, rbp =3D 0x7fffffffcef8 --- lock order reversal: 1st 0xffffff0002996078 zp->z_name_lock (zp->z_name_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_dir.c:212 2nd 0xffffff000256e330 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zget() at zfs_zget+0x23c zfs_dirent_lock() at zfs_dirent_lock+0x4a0 zfs_dirlook() at zfs_dirlook+0x90 zfs_lookup() at zfs_lookup+0x256 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x8d VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xaf vfs_cache_lookup() at vfs_cache_lookup+0xf0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x4a9 kern_statat_vnhook() at kern_statat_vnhook+0x8f kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x800fd8acc, rsp =3D = 0x7fffffffcf38, rbp =3D 0x7fffffffd3d0 --- lock order reversal: 1st 0xffffff000256e210 zfsvfs->z_teardown_inactive_lock = (zfsvfs->z_teardown_inactive_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vnops.c:3724 2nd 0xffffff000256e330 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:1027 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zinactive() at zfs_zinactive+0x95 zfs_inactive() at zfs_inactive+0x7e zfs_freebsd_inactive() at zfs_freebsd_inactive+0x1a VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xb5 vinactive() at vinactive+0x90 vputx() at vputx+0x2fc kern_statat_vnhook() at kern_statat_vnhook+0xfa kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x800fd8acc, rsp =3D = 0x7fffffffcf38, rbp =3D 0x7fffffffd3d0 --- <118>/etc/rc: WARNING: $hostname is not set -- see rc.conf(5). <118>Starting Network: lo0 em0 em1 em2 em3. <118>lo0: flags=3D8049 metric 0 mtu 16384 <118> options=3D3 <118> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6=20 <118> inet6 ::1 prefixlen 128=20 <118> inet 127.0.0.1 netmask 0xff000000=20 <118> nd6 options=3D3 <118>em0: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:d5 <118> inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>em1: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:cb <118> inet 10.0.1.2 netmask 0xffffff00 broadcast 10.0.1.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>em2: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:df <118> inet 10.0.2.2 netmask 0xffffff00 broadcast 10.0.2.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>em3: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:e9 <118> inet 172.16.2.118 netmask 0xffff0000 broadcast 172.16.255.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>add net default: gateway 172.16.0.2 Expensive timeout(9) function: 0xffffffff80329580(0xffffff8000272000) = 0.006700013 s <118>Starting devd. ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based = forwarding disabled, default to deny, logging disabled <118>Flushed all rules. <118>00100 allow ip from any to any via lo0 <118>00200 deny ip from any to 127.0.0.0/8 <118>00300 deny ip from 127.0.0.0/8 to any <118>65000 allow ip from any to any <118>Firewall rules loaded. <118>Creating and/or trimming log files <118>. <118>Starting syslogd. <118>No core dumps found. <118>ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib <118>32-bit compatibility ldconfig path: /usr/lib32 lock order reversal: 1st 0xffffff800a2f62e8 bufwait (bufwait) @ = /pool/newsrc/src/sys/kern/vfs_bio.c:2559 2nd 0xffffff0002c71400 dirhash (dirhash) @ = /pool/newsrc/src/sys/ufs/ufs/ufs_dirhash.c:285 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 ufsdirhash_acquire() at ufsdirhash_acquire+0x44 ufsdirhash_add() at ufsdirhash_add+0x19 ufs_direnter() at ufs_direnter+0x88b ufs_mkdir() at ufs_mkdir+0x623 VOP_MKDIR_APV() at VOP_MKDIR_APV+0xb9 kern_mkdirat() at kern_mkdirat+0x264 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (136, FreeBSD ELF64, mkdir), rip =3D 0x80072bb0c, rsp =3D = 0x7fffffffec88, rbp =3D 0x7fffffffef66 --- <118>Clearing /tmp (X related). <118>Updating motd: <118>. <118>Configuring syscons: <118> blanktime <118>. <118>Starting sshd. <118>Starting cron. <118>Starting background file system checks in 60 seconds. <118> <118>Wed Mar 10 17:29:09 CET 2010 <118>Mar 10 17:29:21 su: BAD SU borjam to root on /dev/pts/0 <118>Mar 10 17:29:24 su: BAD SU borjam to root on /dev/pts/0 <118>Mar 10 17:29:26 su: borjam to root on /dev/pts/0 <118>Mar 10 17:30:36 su: borjam to root on /dev/pts/1 lock order reversal: 1st 0xffffff00027e2438 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 2nd 0xffffff000286bb88 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:130 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 dnode_verify() at dnode_verify+0x70 dnode_hold_impl() at dnode_hold_impl+0x73 dmu_bonus_hold() at dmu_bonus_hold+0x31 bplist_hold() at bplist_hold+0x48 bplist_enqueue() at bplist_enqueue+0x4c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dbuf_write() at dbuf_write+0x24c dbuf_sync_list() at dbuf_sync_list+0x3eb dbuf_sync_list() at dbuf_sync_list+0x17f Fatal double fault rip =3D 0xffffffff805d322d rsp =3D 0xffffff8012414fe0 rbp =3D 0xffffff8012415090 cpuid =3D 0; apic id =3D 00 panic: double fault cpuid =3D 0 KDB: enter: panic shared sx zfsvfs->z_teardown_inactive_lock = (zfsvfs->z_teardown_inactive_lock) r =3D 0 (0xffffff000256d210) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vnops.c:3724 exclusive lockmgr zfs (zfs) r =3D 0 (0xffffff00055b3a58) locked @ = /pool/newsrc/src/sys/kern/vfs_subr.c:2091 exclusive lockmgr zfs (zfs) r =3D 0 (0xffffff0005618308) locked @ = /pool/newsrc/src/sys/kern/vfs_syscalls.c:3174 shared lockmgr zfs (zfs) r =3D 0 (0xffffff00057bda58) locked @ = /pool/newsrc/src/sys/kern/vfs_subr.c:2091 exclusive sx zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) r =3D 0 = (0xffffff000256d310) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 shared sx zp->z_name_lock (zp->z_name_lock) r =3D 0 (0xffffff00057c7c38) = locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_dir.c:212 exclusive lockmgr zfs (zfs) r =3D 0 (0xffffff00057ac578) locked @ = /pool/newsrc/src/sys/kern/vfs_subr.c:2091 exclusive sx so_rcv_sx (so_rcv_sx) r =3D 0 (0xffffff0002f63648) locked @ = /pool/newsrc/src/sys/kern/uipc_sockbuf.c:148 exclusive sx so_rcv_sx (so_rcv_sx) r =3D 0 (0xffffff0002cff3a0) locked @ = /pool/newsrc/src/sys/kern/uipc_sockbuf.c:148 exclusive sx ds->ds_deadlist.bpl_lock (ds->ds_deadlist.bpl_lock) r =3D 0 = (0xffffff00027e2438) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002948238) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002963538) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002949438) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002948b38) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002966238) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002963438) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 0xffffff00055b39c0: tag zfs, type VDIR usecount 0, writecount 0, refcount 1 mountedhere 0 flags (VI_DOINGINACT) lock type zfs: EXCL by thread 0xffffff000528b3a0 (pid 1425) #0 0xffffffff80578cef at __lockmgr_args+0x4ef #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff8062d957 at _vn_lock+0x57 #4 0xffffffff806210bb at vget+0x7b #5 0xffffffff8060e570 at cache_lookup+0x4e0 #6 0xffffffff8060e890 at vfs_cache_lookup+0xc0 #7 0xffffffff808cbdb7 at VOP_LOOKUP_APV+0xb7 #8 0xffffffff80615513 at lookup+0x3d3 #9 0xffffffff806164e9 at namei+0x4a9 #10 0xffffffff806255cf at kern_statat_vnhook+0x8f #11 0xffffffff80625775 at kern_statat+0x15 #12 0xffffffff8062583a at lstat+0x2a #13 0xffffffff80876b48 at syscall+0x118 #14 0xffffffff8085cf91 at Xfast_syscall+0xe1 0xffffff0005618270: tag zfs, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags () lock type zfs: EXCL by thread 0xffffff000528c000 (pid 1422) #0 0xffffffff80578f77 at __lockmgr_args+0x777 #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff8062d957 at _vn_lock+0x57 #4 0xffffffff806299ba at setutimes+0x7a #5 0xffffffff80629cb2 at kern_lutimes+0xf2 #6 0xffffffff80876b48 at syscall+0x118 #7 0xffffffff8085cf91 at Xfast_syscall+0xe1 0xffffff00057bd9c0: tag zfs, type VDIR usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type zfs: SHARED (count 1) #0 0xffffffff80578cef at __lockmgr_args+0x4ef #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff8062d957 at _vn_lock+0x57 #4 0xffffffff806210bb at vget+0x7b #5 0xffffffff8060e570 at cache_lookup+0x4e0 #6 0xffffffff8060e890 at vfs_cache_lookup+0xc0 #7 0xffffffff808cbdb7 at VOP_LOOKUP_APV+0xb7 #8 0xffffffff80615513 at lookup+0x3d3 #9 0xffffffff806164e9 at namei+0x4a9 #10 0xffffffff806255cf at kern_statat_vnhook+0x8f #11 0xffffffff80625775 at kern_statat+0x15 #12 0xffffffff8062583a at lstat+0x2a #13 0xffffffff80876b48 at syscall+0x118 #14 0xffffffff8085cf91 at Xfast_syscall+0xe1 0xffffff00057ac4e0: tag zfs, type VDIR usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type zfs: EXCL by thread 0xffffff000262a3a0 (pid 1416) with shared waiters pending #0 0xffffffff80578cef at __lockmgr_args+0x4ef #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff8062d957 at _vn_lock+0x57 #4 0xffffffff806210bb at vget+0x7b #5 0xffffffff8060e570 at cache_lookup+0x4e0 #6 0xffffffff8060e890 at vfs_cache_lookup+0xc0 #7 0xffffffff808cbdb7 at VOP_LOOKUP_APV+0xb7 #8 0xffffffff80615513 at lookup+0x3d3 #9 0xffffffff806164e9 at namei+0x4a9 #10 0xffffffff80624df3 at kern_mkdirat+0xc3 #11 0xffffffff80876b48 at syscall+0x118 #12 0xffffffff8085cf91 at Xfast_syscall+0xe1 Uptime: 2m16s #=20 Kernel configuration follows: # cat config.txt=20 options CONFIG_AUTOGENERATED ident DEBUG machine amd64 cpu HAMMER makeoptions DEBUG=3D-g options AH_SUPPORT_AR5416 options IEEE80211_SUPPORT_MESH options IEEE80211_AMPDU_AGE options IEEE80211_DEBUG options AHD_REG_PRETTY_PRINT options AHC_REG_PRETTY_PRINT options ATA_STATIC_ID options SMP options DEBUG_LOCKS options DEBUG_VFS_LOCKS options DIAGNOSTIC options INVARIANT_SUPPORT options INVARIANTS options DDB options KDB options WITNESS_SKIPSPIN options WITNESS options INCLUDE_CONFIG_FILE options FLOWTABLE options MAC options AUDIT options HWPMC_HOOKS options KBD_INSTALL_CDEV options PRINTF_BUFR_SIZE=3D128 options _KPOSIX_PRIORITY_SCHEDULING options P1003_1B_SEMAPHORES options SYSVSEM options SYSVMSG options SYSVSHM options STACK options KTRACE options SCSI_DELAY=3D5000 options COMPAT_FREEBSD7 options COMPAT_FREEBSD6 options COMPAT_FREEBSD5 options COMPAT_FREEBSD4 options COMPAT_IA32 options COMPAT_43TTY options GEOM_LABEL options GEOM_PART_GPT options PSEUDOFS options PROCFS options CD9660 options MSDOSFS options NFS_ROOT options NFSLOCKD options NFSSERVER options NFSCLIENT options MD_ROOT options UFS_GJOURNAL options UFS_DIRHASH options UFS_ACL options SOFTUPDATES options FFS options SCTP options INET6 options INET options PREEMPTION options SCHED_ULE options GEOM_PART_MBR options GEOM_PART_EBR_COMPAT options GEOM_PART_EBR options GEOM_PART_BSD device isa device mem device io device uart_ns8250 device cpufreq device acpi device pci device fdc device ata device atadisk device ataraid device atapicd device atapifd device atapist device ahc device ahd device amd device hptiop device isp device mpt device sym device trm device adv device adw device aic device bt device scbus device ch device da device sa device cd device pass device ses device amr device arcmsr device ciss device dpt device hptmv device hptrr device iir device ips device mly device twa device aac device aacp device ida device mfi device mlx device twe device atkbdc device atkbd device psm device kbdmux device vga device splash device sc device agp device cbb device pccard device cardbus device uart device ppc device ppbus device lpt device plip device ppi device de device em device igb device ixgbe device le device ti device txp device vx device miibus device ae device age device alc device ale device bce device bfe device bge device dc device et device fxp device jme device lge device msk device nfe device nge device pcn device re device rl device sf device sis device sk device ste device stge device tl device tx device vge device vr device wb device xl device cs device ed device ex device ep device fe device sn device xe device wlan device wlan_wep device wlan_ccmp device wlan_tkip device wlan_amrr device an device ath device ath_hal device ath_rate_sample device ral device wi device loop device random device ether device vlan device tun device pty device md device gif device faith device firmware device bpf device uhci device ohci device ehci device usb device uhid device ukbd device ulpt device umass device ums device urio device uark device ubsa device uftdi device uipaq device uplcom device uslcom device uvisor device uvscom device aue device axe device cdce device cue device kue device rue device udav device rum device uath device ural device zyd device firewire device fwe device fwip device dcons device dcons_crom #=20 From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 15:12:44 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F6301065670; Wed, 10 Mar 2010 15:12:44 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from proxypop2.sarenet.es (proxypop2.sarenet.es [194.30.0.95]) by mx1.freebsd.org (Postfix) with ESMTP id E44F58FC12; Wed, 10 Mar 2010 15:12:38 +0000 (UTC) Received: from [172.16.1.204] (unknown [192.148.167.2]) by proxypop2.sarenet.es (Postfix) with ESMTP id 955B0732A4; Wed, 10 Mar 2010 16:12:37 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <20100310110202.GA1715@garage.freebsd.pl> Date: Wed, 10 Mar 2010 16:12:36 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> To: Pawel Jakub Dawidek X-Mailer: Apple Mail (2.1077) Cc: freebsd-fs@freebsd.org, FreeBSD Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 15:12:44 -0000 =09 On Mar 10, 2010, at 12:02 PM, Pawel Jakub Dawidek wrote: > Once the deadlock occur, enter DDB and send me the output of: >=20 > ps > show alllocks > show lockedvnods > show allchains > alltrace (Again, crossposted to -fs, ZFS related) Previous one was a panic when performing the test with several tar jobs = running in parallel. Now this is a capture of the deadlock itself, instead of a panic. (I = called panic from the debugger to generate a dump) ddb.txt: # cat ddb.txt db> set $lines=3D0 db> ps pid ppid pgrp uid state wmesg wchan cmd 1821 1385 1821 0 R+ CPU 0 sysctl 1820 1818 1818 0 S tx->tx_s 0xffffff00027da650 zfs 1818 1816 1818 0 Ss pause 0xffffff0002e33960 csh 1816 1284 1816 0 Ss select 0xffffff001c6d3cc0 sshd 1699 1697 1697 0 S+ zfsvfs-> 0xffffff0002893210 bsdtar 1698 1697 1697 0 S+ pipdwt 0xffffff00025d62d8 bsdtar 1697 1385 1697 0 S+ wait 0xffffff0002d47000 sh 1385 1384 1385 0 R+ csh 1384 1381 1384 1001 S+ wait 0xffffff0002e33460 su 1381 1380 1381 1001 Ss+ pause 0xffffff0002e36500 csh 1380 1377 1377 1001 R sshd 1377 1284 1377 0 Ss sbwait 0xffffff0002c763ec sshd 1376 1 1376 0 Ss+ ttyin 0xffffff00024daca8 getty 1375 1 1375 0 Ss+ ttyin 0xffffff00024dc0a8 getty 1374 1 1374 0 Ss+ ttyin 0xffffff00024cdca8 getty 1373 1 1373 0 Ss+ ttyin 0xffffff00024c94a8 getty 1372 1 1372 0 Ss+ ttyin 0xffffff00024dd4a8 getty 1371 1 1371 0 Ss+ ttyin 0xffffff00024c28a8 getty 1370 1 1370 0 Ss+ ttyin 0xffffff00024db4a8 getty 1369 1 1369 0 Ss+ ttyin 0xffffff00024dc8a8 getty 1305 1 1305 0 Ss nanslp 0xffffffff80c4d168 cron 1298 1 1298 25 Ss pause 0xffffff000257d500 sendmail 1292 1 1292 0 Ss select 0xffffff000287f5c0 sendmail 1284 1 1284 0 Ss select 0xffffff000287f940 sshd 1005 1 1005 0 Ss select 0xffffff0002c6e2c0 syslogd 827 1 827 0 Ss select 0xffffff000296f440 devd 20 0 0 0 SL flowclea 0xffffffff80e0e950 = [flowcleaner] 19 0 0 0 SL sdflush 0xffffffff80e1d8b8 = [softdepflush] 18 0 0 0 SL rrl->rr_ 0xffffff0002893110 [syncer] 17 0 0 0 SL zfsvfs-> 0xffffff0002893210 [vnlru] 16 0 0 0 SL psleep 0xffffffff80e0e1c8 [bufdaemon] 15 0 0 0 SL pgzero 0xffffffff80e1f34c [pagezero] 14 0 0 0 SL psleep 0xffffffff80e1e6e8 [vmdaemon] 9 0 0 0 SL psleep 0xffffffff80e1e6ac = [pagedaemon] 8 0 0 0 SL ccb_scan 0xffffffff80c16060 [xpt_thrd] 7 0 0 0 SL (threaded) zfskern 100085 D tx->tx_q 0xffffff00027da670 = [txg_thread_enter] 100084 D tx->tx_c 0xffffff0002599d20 = [txg_thread_enter] 100082 D vgeom:io 0xffffff00026acd90 [vdev da2] 100036 D l2arc_fe 0xffffffff81150620 = [l2arc_feed_thread] 100035 D arc_recl 0xffffffff81147fe0 = [arc_reclaim_thread] 6 0 0 0 SL - 0xffffff000244e648 [fdc0] 5 0 0 0 SL idle 0xffffff800023c000 = [mpt_recovery0] 13 0 0 0 SL - 0xffffffff80c4ce44 [yarrow] 4 0 0 0 SL - 0xffffffff80c49648 [g_down] 3 0 0 0 SL - 0xffffffff80c49640 [g_up] 2 0 0 0 SL - 0xffffffff80c49630 [g_event] 12 0 0 0 WL (threaded) intr 100033 I [swi0: uart = uart] 100032 I [irq7: = ppc0] 100031 I [irq12: = psm0] 100030 I [irq1: = atkbd0] 100024 I [irq17: em2 = mpt0] 100023 I [irq15: = ata1] 100022 I [irq14: = ata0] 100021 I [irq9: = acpi0] 100019 I [swi5: +] 100018 I [swi2: = cambio] 100013 I [swi6: task = queue] 100012 I [swi6: = Giant taskq] 100006 I [swi1: = netisr 0] 100005 I [swi4: = clock] 100004 I [swi3: vm] 11 0 0 0 RL [idle: = cpu0] 1 0 1 0 SLs wait 0xffffff00021e18c0 [init] 10 0 0 0 SL audit_wo 0xffffffff80e1cc10 [audit] 0 0 0 0 SLs (threaded) kernel 100087 D - 0xffffff000287c180 [zil_clean] 100086 D - 0xffffff000287b200 [zil_clean] 100083 D - 0xffffff0002640480 = [zfs_vn_rele_taskq] 100081 D - 0xffffff00026acc00 [spa_zio] 100080 D - 0xffffff00026acb80 [spa_zio] 100079 D - 0xffffff00026acb00 [spa_zio] 100078 D - 0xffffff00026aca80 [spa_zio] 100077 D - 0xffffff00026aca00 [spa_zio] 100076 D - 0xffffff00026ac980 [spa_zio] 100075 D - 0xffffff00026ac900 [spa_zio] 100074 D - 0xffffff00026ac880 [spa_zio_7] 100073 D - 0xffffff00026ac880 [spa_zio_6] 100072 D - 0xffffff00026ac880 [spa_zio_5] 100071 D - 0xffffff00026ac880 [spa_zio_4] 100070 D - 0xffffff00026ac880 [spa_zio_3] 100069 D - 0xffffff00026ac880 [spa_zio_2] 100068 D - 0xffffff00026ac880 [spa_zio_1] 100067 D - 0xffffff00026ac880 [spa_zio_0] 100066 D - 0xffffff00026ac800 [spa_zio_7] 100065 D - 0xffffff00026ac800 [spa_zio_6] 100064 D - 0xffffff00026ac800 [spa_zio_5] 100063 D - 0xffffff00026ac800 [spa_zio_4] 100062 D - 0xffffff00026ac800 [spa_zio_3] 100061 D - 0xffffff00026ac800 [spa_zio_2] 100060 D - 0xffffff00026ac800 [spa_zio_1] 100059 D - 0xffffff00026ac800 [spa_zio_0] 100058 D - 0xffffff00026ac780 [spa_zio] 100057 D - 0xffffff00026ac700 [spa_zio] 100056 D - 0xffffff00026ac680 [spa_zio] 100029 D - 0xffffff0002441980 [em3 taskq] 100028 D - 0xffffff0002440d80 [em2 taskq] 100027 D - 0xffffff0002436200 [em1 taskq] 100026 D - 0xffffff000242fb80 [em0 taskq] 100020 D - 0xffffff0002358600 [thread = taskq] 100017 D - 0xffffff0002316b80 = [acpi_task_2] 100016 D - 0xffffff0002316b80 = [acpi_task_1] 100015 D - 0xffffff0002316b80 = [acpi_task_0] 100014 D - 0xffffff0002316c00 [kqueue = taskq] 100010 D - 0xffffff00021e0180 [firmware = taskq] 100000 D sched 0xffffffff80c49740 [swapper] db> show alllocks Process 1821 (sysctl) thread 0xffffff00025b0ae0 (100050) Process 1820 (zfs) thread 0xffffff0002ca4000 (100105) Process 1699 (bsdtar) thread 0xffffff000262dae0 (100093) Process 1377 (sshd) thread 0xffffff0002642740 (100090) Process 18 (syncer) thread 0xffffff000250d3a0 (100043) Process 17 (vnlru) thread 0xffffff000250d740 (100042) db> show lockedvnods Locked vnodes db> show allchains db> alltrace Tracing command sysctl pid 1821 tid 100050 td 0xffffff00025b0ae0 kdb_enter() at kdb_enter+0x3d kdb_sysctl_enter() at kdb_sysctl_enter+0x89 sysctl_root() at sysctl_root+0x113 userland_sysctl() at userland_sysctl+0x158 __sysctl() at __sysctl+0xaa syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (202, FreeBSD ELF64, __sysctl), rip =3D 0x80073cfac, rsp =3D = 0x7fffffffe298, rbp =3D 0x7fffffffe36c --- Tracing command zfs pid 1820 tid 100105 td 0xffffff0002ca4000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a txg_wait_synced() at txg_wait_synced+0x98 zfsvfs_teardown() at zfsvfs_teardown+0x1f6 zfs_suspend_fs() at zfs_suspend_fs+0x2b zfs_ioc_recv() at zfs_ioc_recv+0x28b zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffff8e98, rbp =3D 0x7fffffff9bd0 --- Tracing command csh pid 1818 tid 100102 td 0xffffff0002ca4ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x80094d19c, rsp =3D = 0x7fffffffe578, rbp =3D 0x800c0f500 --- Tracing command sshd pid 1816 tid 100098 td 0xffffff0002d2bae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x8013d7cfc, rsp =3D = 0x7fffffffdcb8, rbp =3D 0x7fffffffdd40 --- Tracing command bsdtar pid 1699 tid 100093 td 0xffffff000262dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sx_slock_hard() at _sx_slock_hard+0x1b7 _sx_slock() at _sx_slock+0xc1 zfs_freebsd_reclaim() at zfs_freebsd_reclaim+0x63 VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0xb5 vgonel() at vgonel+0x119 vnlru_free() at vnlru_free+0x345 getnewvnode() at getnewvnode+0x24f zfs_znode_cache_constructor() at zfs_znode_cache_constructor+0x43 zfs_znode_alloc() at zfs_znode_alloc+0x38 zfs_mknode() at zfs_mknode+0x259 zfs_freebsd_create() at zfs_freebsd_create+0x661 VOP_CREATE_APV() at VOP_CREATE_APV+0xb3 vn_open_cred() at vn_open_cred+0x473 kern_openat() at kern_openat+0x179 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (5, FreeBSD ELF64, open), rip =3D 0x800e35b4c, rsp =3D = 0x7fffffffe748, rbp =3D 0x1a4 --- Tracing command bsdtar pid 1698 tid 100107 td 0xffffff0002c90740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_write() at pipe_write+0x13fc dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x800e44d5c, rsp =3D = 0x7fffffffe818, rbp =3D 0x801004140 --- Tracing command sh pid 1697 tid 100091 td 0xffffff00026423a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip =3D 0x80093554c, rsp =3D = 0x7fffffffe8a8, rbp =3D 0x6a1 --- Tracing command csh pid 1385 tid 100104 td 0xffffff0002ca43a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a fork1() at fork1+0xfe2 vfork() at vfork+0x1c syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (66, FreeBSD ELF64, vfork), rip =3D 0x80094f31d, rsp =3D = 0x7fffffffe6c0, rbp =3D 0 --- Tracing command su pid 1384 tid 100103 td 0xffffff0002ca4740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip =3D 0x8009e854c, rsp =3D = 0x7fffffffe598, rbp =3D 0x569 --- Tracing command csh pid 1381 tid 100100 td 0xffffff0002d2b3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x80094d19c, rsp =3D = 0x7fffffffe558, rbp =3D 0x800c0f900 --- Tracing command sshd pid 1380 tid 100099 td 0xffffff0002d2b740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x8013d7cfc, rsp =3D = 0x7fffffffdcb8, rbp =3D 0x7fffffffdd40 --- Tracing command sshd pid 1377 tid 100090 td 0xffffff0002642740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x8013d7d7c, rsp =3D = 0x7fffffffdcd8, rbp =3D 0 --- Tracing command getty pid 1376 tid 100046 td 0xffffff00025b1ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1375 tid 100095 td 0xffffff000262d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1374 tid 100096 td 0xffffff0002cd43a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1373 tid 100097 td 0xffffff0002cd4000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1372 tid 100092 td 0xffffff0002642000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1371 tid 100094 td 0xffffff000262d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1370 tid 100048 td 0xffffff00025b13a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command getty pid 1369 tid 100054 td 0xffffff0002510ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip =3D 0x80084ed7c, rsp =3D = 0x7fffffffecc8, rbp =3D 0 --- Tracing command cron pid 1305 tid 100052 td 0xffffff00025b03a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip =3D 0x80093efac, rsp =3D = 0x7fffffffeb28, rbp =3D 0x3c --- Tracing command sendmail pid 1298 tid 100055 td 0xffffff0002510740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip =3D 0x800d3a19c, rsp =3D = 0x7fffffffcd68, rbp =3D 0x1 --- Tracing command sendmail pid 1292 tid 100053 td 0xffffff00025b0000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x800dd8cfc, rsp =3D = 0x7fffffffc1e8, rbp =3D 0x7fffffffc280 --- Tracing command sshd pid 1284 tid 100047 td 0xffffff00025b1740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x8013d7cfc, rsp =3D = 0x7fffffffddd8, rbp =3D 0x2 --- Tracing command syslogd pid 1005 tid 100051 td 0xffffff00025b0740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x800851cfc, rsp =3D = 0x7fffffffe118, rbp =3D 0x800a310c8 --- Tracing command devd pid 827 tid 100049 td 0xffffff00025b1000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip =3D 0x440ecc, rsp =3D = 0x7fffffffe898, rbp =3D 0x7fffffffe8b0 --- Tracing command flowcleaner pid 20 tid 100045 td 0xffffff0002485ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c flowtable_cleaner() at flowtable_cleaner+0xde fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800018cd30, rbp =3D 0 --- Tracing command softdepflush pid 19 tid 100044 td 0xffffff000250d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 softdep_flush() at softdep_flush+0x259 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000187d30, rbp =3D 0 --- Tracing command syncer pid 18 tid 100043 td 0xffffff000250d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a rrw_enter() at rrw_enter+0x135 zfs_sync() at zfs_sync+0x63 sync_fsync() at sync_fsync+0x13a VOP_FSYNC_APV() at VOP_FSYNC_APV+0xb5 sync_vnode() at sync_vnode+0x157 sched_sync() at sched_sync+0x1cc fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000182d30, rbp =3D 0 --- Tracing command vnlru pid 17 tid 100042 td 0xffffff000250d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sx_slock_hard() at _sx_slock_hard+0x1b7 _sx_slock() at _sx_slock+0xc1 zfs_freebsd_reclaim() at zfs_freebsd_reclaim+0x63 VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0xb5 vgonel() at vgonel+0x119 vnlru_free() at vnlru_free+0x345 vnlru_proc() at vnlru_proc+0x58e fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800017dd30, rbp =3D 0 --- Tracing command bufdaemon pid 16 tid 100041 td 0xffffff000250dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 buf_daemon() at buf_daemon+0x14a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000178d30, rbp =3D 0 --- Tracing command pagezero pid 15 tid 100040 td 0xffffff000250e000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vm_pagezero() at vm_pagezero+0x73 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000173d30, rbp =3D 0 --- Tracing command vmdaemon pid 14 tid 100039 td 0xffffff000250e3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 vm_daemon() at vm_daemon+0x4d fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800016ed30, rbp =3D 0 --- Tracing command pagedaemon pid 9 tid 100038 td 0xffffff000250e740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vm_pageout() at vm_pageout+0x855 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000169d30, rbp =3D 0 --- Tracing command xpt_thrd pid 8 tid 100037 td 0xffffff000250eae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 xpt_scanner_thread() at xpt_scanner_thread+0xd5 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000164d30, rbp =3D 0 --- Tracing command zfskern pid 7 tid 100085 td 0xffffff0002643ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a txg_thread_wait() at txg_thread_wait+0x5b txg_sync_thread() at txg_sync_thread+0x1d0 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- Tracing command zfskern pid 7 tid 100084 td 0xffffff0002645000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a txg_quiesce_thread() at txg_quiesce_thread+0x22b fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012413d30, rbp =3D 0 --- Tracing command zfskern pid 7 tid 100082 td 0xffffff000263dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 vdev_geom_worker() at vdev_geom_worker+0xf3 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123ffd30, rbp =3D 0 --- Tracing command zfskern pid 7 tid 100036 td 0xffffff0002510000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c l2arc_feed_thread() at l2arc_feed_thread+0x162 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800015fd30, rbp =3D 0 --- Tracing command zfskern pid 7 tid 100035 td 0xffffff0002356740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c arc_reclaim_thread() at arc_reclaim_thread+0x2b7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800015ad30, rbp =3D 0 --- Tracing command fdc0 pid 6 tid 100034 td 0xffffff0002356ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 fdc_thread() at fdc_thread+0x774 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000155d30, rbp =3D 0 --- Tracing command mpt_recovery0 pid 5 tid 100025 td 0xffffff0002353ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 mpt_recovery_thread() at mpt_recovery_thread+0x251 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800011ed30, rbp =3D 0 --- Tracing command yarrow pid 13 tid 100011 td 0xffffff00021e53a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 random_kthread() at random_kthread+0x1ad fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800004cd30, rbp =3D 0 --- Tracing command g_down pid 4 tid 100009 td 0xffffff00021e5ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 g_io_schedule_down() at g_io_schedule_down+0x236 g_down_procbody() at g_down_procbody+0x6f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000042d30, rbp =3D 0 --- Tracing command g_up pid 3 tid 100008 td 0xffffff00021f3000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 g_io_schedule_up() at g_io_schedule_up+0x133 g_up_procbody() at g_up_procbody+0x6f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800003dd30, rbp =3D 0 --- Tracing command g_event pid 2 tid 100007 td 0xffffff00021f33a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 g_event_procbody() at g_event_procbody+0xa1 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000038d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100033 td 0xffffff0002483000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100032 td 0xffffff00024833a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100031 td 0xffffff0002483740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100030 td 0xffffff0002483ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000137d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100024 td 0xffffff0002354000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80000d7d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100023 td 0xffffff00023543a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80000d2d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100022 td 0xffffff0002354740 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100021 td 0xffffff0002354ae0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100019 td 0xffffff00023563a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000078d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100018 td 0xffffff00021f3ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000073d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100013 td 0xffffff0002353000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800005ad30, rbp =3D 0 --- Tracing command intr pid 12 tid 100012 td 0xffffff00023533a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100006 td 0xffffff00021f3740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000033d30, rbp =3D 0 --- Tracing command intr pid 12 tid 100005 td 0xffffff00021e4000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800002ed30, rbp =3D 0 --- Tracing command intr pid 12 tid 100004 td 0xffffff00021e43a0 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xffffff00021e4740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d critical_exit() at critical_exit+0xa3 intr_event_handle() at intr_event_handle+0xd7 intr_execute_handlers() at intr_execute_handlers+0x5f lapic_handle_intr() at lapic_handle_intr+0x37 Xapic_isr1() at Xapic_isr1+0xa4 --- interrupt, rip =3D 0xffffffff808545f6, rsp =3D 0xffffff8000024b50, = rbp =3D 0xffffff8000024b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000024d30, rbp =3D 0 --- Tracing command init pid 1 tid 100002 td 0xffffff00021e4ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip =3D 0x40c86c, rsp =3D = 0x7fffffffe808, rbp =3D 0x401d40 --- Tracing command audit pid 10 tid 100001 td 0xffffff00021e5000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a audit_worker() at audit_worker+0x77 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800001ad30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100087 td 0xffffff00026433a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012421d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100086 td 0xffffff0002643740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff801241cd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100083 td 0xffffff000263d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012404d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100081 td 0xffffff000263e000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123fad30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100080 td 0xffffff000263e3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123f5d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100079 td 0xffffff000263e740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123f0d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100078 td 0xffffff000263eae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123ebd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100077 td 0xffffff0002628000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123e6d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100076 td 0xffffff0002628740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123e1d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100075 td 0xffffff0002628ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123dcd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100074 td 0xffffff00025103a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123d7d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100073 td 0xffffff0002645740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123d2d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100072 td 0xffffff00026453a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123cdd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100071 td 0xffffff000262a000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80123c8d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100070 td 0xffffff000262a3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001ffd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100069 td 0xffffff000262a740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001fad30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100068 td 0xffffff000262aae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001d2d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100067 td 0xffffff000262b000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001f5d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100066 td 0xffffff000262b3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001f0d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100065 td 0xffffff000262b740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001ebd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100064 td 0xffffff000262bae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001e6d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100063 td 0xffffff000262c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001e1d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100062 td 0xffffff000262c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001dcd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100061 td 0xffffff000262c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001d7d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100060 td 0xffffff000262cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001cdd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100059 td 0xffffff000262d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001c8d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100058 td 0xffffff000263d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff80001c3d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100057 td 0xffffff000263d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012409d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100056 td 0xffffff00026283a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff801240ed30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100029 td 0xffffff0002485000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000132d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100028 td 0xffffff00024853a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800012dd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100027 td 0xffffff0002485740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000128d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100026 td 0xffffff0002353740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000123d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100020 td 0xffffff0002356000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800007dd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100017 td 0xffffff0002352000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800006ed30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100016 td 0xffffff00023523a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000069d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100015 td 0xffffff0002352740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000064d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100014 td 0xffffff0002352ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff800005fd30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100010 td 0xffffff00021e5740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8000047d30, rbp =3D 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff80c49ba0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 scheduler() at scheduler+0x29c mi_startup() at mi_startup+0x59 btext() at btext+0x2c db> textdump set textdump set db> help ahd_dump ahd_in ahd_out ahd_pause ahd_sunit ahd_unpause alltrace b break bt c call capture continue d delete dhwatch dwatch examine gdb halt hwatch kill match next p panic print ps reboot reset run s script scripts search set show step t textdump thread trace unscript until w watch watchdog where write x db> panic #=20 msgbuf.txt: #=20 #=20 # cat msgbuf.txt 2400-0x243f mem 0xd89a0000-0xd89bffff,0xd8930000-0xd893ffff irq 19 at = device 5.0 on pci2 em3: Memory Access and/or Bus Master bits were not set! em3: [FILTER] em3: Ethernet address: 00:0c:29:11:4f:e9 pcib3: at device 21.0 on pci0 pci3: on pcib3 pcib4: at device 21.1 on pci0 pci4: on pcib4 pcib5: at device 21.2 on pci0 pci5: on pcib5 pcib6: at device 21.3 on pci0 pci6: on pcib6 pcib7: at device 21.4 on pci0 pci7: on pcib7 pcib8: at device 21.5 on pci0 pci8: on pcib8 pcib9: at device 21.6 on pci0 pci9: on pcib9 pcib10: at device 21.7 on pci0 pci10: on pcib10 pcib11: at device 22.0 on pci0 pci11: on pcib11 pcib12: at device 22.1 on pci0 pci12: on pcib12 pcib13: at device 22.2 on pci0 pci13: on pcib13 pcib14: at device 22.3 on pci0 pci14: on pcib14 pcib15: at device 22.4 on pci0 pci15: on pcib15 pcib16: at device 22.5 on pci0 pci16: on pcib16 pcib17: at device 22.6 on pci0 pci17: on pcib17 pcib18: at device 22.7 on pci0 pci18: on pcib18 pcib19: at device 23.0 on pci0 pci19: on pcib19 pcib20: at device 23.1 on pci0 pci20: on pcib20 pcib21: at device 23.2 on pci0 pci21: on pcib21 pcib22: at device 23.3 on pci0 pci22: on pcib22 pcib23: at device 23.4 on pci0 pci23: on pcib23 pcib24: at device 23.5 on pci0 pci24: on pcib24 pcib25: at device 23.6 on pci0 pci25: on pcib25 pcib26: at device 23.7 on pci0 pci26: on pcib26 pcib27: at device 24.0 on pci0 pci27: on pcib27 pcib28: at device 24.1 on pci0 pci28: on pcib28 pcib29: at device 24.2 on pci0 pci29: on pcib29 pcib30: at device 24.3 on pci0 pci30: on pcib30 pcib31: at device 24.4 on pci0 pci31: on pcib31 pcib32: at device 24.5 on pci0 pci32: on pcib32 pcib33: at device 24.6 on pci0 pci33: on pcib33 pcib34: at device 24.7 on pci0 pci34: on pcib34 acpi_acad0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse, device ID 3 ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppc0: [ITHREAD] ppbus0: on ppc0 plip0: on ppbus0 plip0: [ITHREAD] lpt0: on ppbus0 lpt0: [ITHREAD] lpt0: Interrupt-driven port ppi0: on ppbus0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: [FILTER] fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on = acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 cpu0: on acpi0 acpi_throttle0: on cpu0 orm0: at iomem = 0xc0000-0xc7fff,0xca000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff,0xcd000-0x= cdfff,0xdc000-0xdffff,0xe0000-0xe3fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on = isa0 ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is = present; to enable, add "vfs.zfs.prefetch_disable=3D0" to = /boot/loader.conf. ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable = behavior. Consider tuning vm.kmem_size and vm.kmem_size_max in /boot/loader.conf. ZFS filesystem version 3 ZFS storage pool version 14 Timecounter "TSC" frequency 2116630847 Hz quality 800 Timecounters tick every 10.000 msec Expensive timeout(9) function: 0xffffffff805ed840(0) 0.002412596 s acd0: DVDR at ata1-master = UDMA33=20 da0 at mpt0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device=20 da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da0: Command Queueing enabled da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) da1 at mpt0 bus 0 scbus0 target 1 lun 0 da1: Fixed Direct Access SCSI-2 device=20 da1: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da1: Command Queueing enabled da1: 4096MB (8388608 512 byte sectors: 255H 63S/T 522C) da2 at mpt0 bus 0 scbus0 target 2 lun 0 da2: Fixed Direct Access SCSI-2 device=20 da2: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da2: Command Queueing enabled da2: 4096MB (8388608 512 byte sectors: 255H 63S/T 522C) da3 at mpt0 bus 0 scbus0 target 3 lun 0 da3: Fixed Direct Access SCSI-2 device=20 da3: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da3: Command Queueing enabled da3: 12288MB (25165824 512 byte sectors: 255H 63S/T 1566C) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Trying to mount root from ufs:/dev/da0s1a WARNING: / was not properly dismounted /: mount pending error: blocks 4 files 1 <118>Setting hostuuid: 564d8c47-d7e5-fea4-92ef-f17760cfe4d2. <118>Setting hostid: 0x4d345a2c. lock order reversal: 1st 0xffffff000286d2e8 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:549 2nd 0xffffff000286c0d8 dn->dn_mtx (dn->dn_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:1173 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_block_freed() at dnode_block_freed+0x8e dbuf_read() at dbuf_read+0x155 dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0x12a dmu_read() at dmu_read+0x80 load_nvlist() at load_nvlist+0x85 spa_load() at spa_load+0x49a spa_open_common() at spa_open_common+0x12d spa_get_stats() at spa_get_stats+0x42 zfs_ioc_pool_stats() at zfs_ioc_pool_stats+0x2c zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffffd808, rbp =3D 0x801224140 --- Expensive timeout(9) function: 0xffffffff80892b50(0xffffff000244fe00) = 0.005049803 s lock order reversal: 1st 0xffffff0002868e70 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode_sync.c:381 2nd 0xffffff00026e7940 osi->os_lock (osi->os_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:323 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_destroy() at dnode_destroy+0xa6 dnode_buf_pageout() at dnode_buf_pageout+0xb2 dbuf_evict_user() at dbuf_evict_user+0x55 dbuf_clear() at dbuf_clear+0x5e dnode_evict_dbufs() at dnode_evict_dbufs+0x98 dmu_objset_evict_dbufs() at dmu_objset_evict_dbufs+0x11c dmu_objset_evict() at dmu_objset_evict+0xbf dsl_pool_close() at dsl_pool_close+0x52 spa_unload() at spa_unload+0xb2 spa_load() at spa_load+0x4da spa_open_common() at spa_open_common+0x12d spa_get_stats() at spa_get_stats+0x42 zfs_ioc_pool_stats() at zfs_ioc_pool_stats+0x2c zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffffd808, rbp =3D 0x801224140 --- lock order reversal: 1st 0xffffff0002864910 buf->b_lock (buf->b_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/arc.c:2509 2nd 0xffffff00028681a0 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:421 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_read_done() at dbuf_read_done+0x3b arc_read_done() at arc_read_done+0x1d2 zio_done() at zio_done+0x308 zio_execute() at zio_execute+0xb1 arc_read_nolock() at arc_read_nolock+0x3d0 arc_read() at arc_read+0xaf dbuf_read() at dbuf_read+0x62b dmu_buf_hold() at dmu_buf_hold+0xcc zap_lockdir() at zap_lockdir+0x6e zap_lookup_norm() at zap_lookup_norm+0x45 zap_lookup() at zap_lookup+0x2e dsl_prop_get_dd() at dsl_prop_get_dd+0xfc dsl_dataset_get_ref() at dsl_dataset_get_ref+0x4a7 dsl_dataset_hold() at dsl_dataset_hold+0xf2 dmu_objset_open() at dmu_objset_open+0xcf zil_check_log_chain() at zil_check_log_chain+0x2a dmu_objset_find_spa() at dmu_objset_find_spa+0x23f dmu_objset_find_spa() at dmu_objset_find_spa+0x324 dmu_objset_find_spa() at dmu_objset_find_spa+0x324 dmu_objset_find() at dmu_objset_find+0x2a spa_check_logs() at spa_check_logs+0x2a spa_load() at spa_load+0x6ca spa_load() at spa_load+0x4fe spa_open_common() at spa_open_common+0x12d spa_get_stats() at spa_get_stats+0x42 zfs_ioc_pool_stats() at zfs_ioc_pool_stats+0x2c zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffffd808, rbp =3D 0x801224140 --- lock order reversal: 1st 0xffffff000286d058 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1116 2nd 0xffffff000259ac38 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1120 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_dirty() at dbuf_dirty+0x892 dnode_setdirty() at dnode_setdirty+0x1a9 dbuf_dirty() at dbuf_dirty+0xa53 bplist_vacate() at bplist_vacate+0x4d spa_sync() at spa_sync+0x297 txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff000292d538 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 2nd 0xffffff000250c2f0 spa->spa_sync_bplist.bpl_lock = (spa->spa_sync_bplist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:235 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 bplist_enqueue_deferred() at bplist_enqueue_deferred+0x47 zio_free() at zio_free+0x105 arc_free() at arc_free+0x11c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x483 dbuf_write() at dbuf_write+0x24c dbuf_sync_list() at dbuf_sync_list+0x3eb dbuf_sync_list() at dbuf_sync_list+0x17f dnode_sync() at dnode_sync+0xc12 dmu_objset_sync() at dmu_objset_sync+0x134 dsl_pool_sync() at dsl_pool_sync+0x200 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002948c38 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 2nd 0xffffff00028a7000 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1859 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 dbuf_check_blkptr() at dbuf_check_blkptr+0x11e dbuf_sync_list() at dbuf_sync_list+0x34a dbuf_sync_list() at dbuf_sync_list+0x17f dnode_sync() at dnode_sync+0xc12 dmu_objset_sync_dnodes() at dmu_objset_sync_dnodes+0x60 dmu_objset_sync() at dmu_objset_sync+0x162 dsl_pool_sync() at dsl_pool_sync+0x200 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- <118>Entropy harvesting: <118> interrupts <118> ethernet <118> point_to_point <118> kickstart <118>. <118>Starting file system checks: <118>/dev/da0s1a: FREE BLK COUNT(S) WRONG IN SUPERBLK (SALVAGED) <118>/dev/da0s1a: SUMMARY INFORMATION BAD (SALVAGED) <118>/dev/da0s1a: BLK(S) MISSING IN BIT MAPS (SALVAGED) <118>/dev/da0s1a: 28961 files, 517324 used, 1004493 free (7877 frags, = 124577 blocks, 0.5% fragmentation) <118>Mounting local file systems: <118>. lock order reversal: 1st 0xffffff000263acc8 zfs (zfs) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/gfs.c:437 2nd 0xffffff000256e310 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zget() at zfs_zget+0x23c zfs_root() at zfs_root+0x50 zfsctl_create() at zfsctl_create+0x82 zfs_mount() at zfs_mount+0x7ef vfs_donmount() at vfs_donmount+0xcde nmount() at nmount+0x63 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (378, FreeBSD ELF64, nmount), rip =3D 0x800f48f1c, rsp =3D = 0x7fffffffced8, rbp =3D 0x7fffffffcef8 --- lock order reversal: 1st 0xffffff0002996078 zp->z_name_lock (zp->z_name_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_dir.c:212 2nd 0xffffff000256e330 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zget() at zfs_zget+0x23c zfs_dirent_lock() at zfs_dirent_lock+0x4a0 zfs_dirlook() at zfs_dirlook+0x90 zfs_lookup() at zfs_lookup+0x256 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x8d VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xaf vfs_cache_lookup() at vfs_cache_lookup+0xf0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x4a9 kern_statat_vnhook() at kern_statat_vnhook+0x8f kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x800fd8acc, rsp =3D = 0x7fffffffcf38, rbp =3D 0x7fffffffd3d0 --- lock order reversal: 1st 0xffffff000256e210 zfsvfs->z_teardown_inactive_lock = (zfsvfs->z_teardown_inactive_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vnops.c:3724 2nd 0xffffff000256e330 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:1027 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zinactive() at zfs_zinactive+0x95 zfs_inactive() at zfs_inactive+0x7e zfs_freebsd_inactive() at zfs_freebsd_inactive+0x1a VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xb5 vinactive() at vinactive+0x90 vputx() at vputx+0x2fc kern_statat_vnhook() at kern_statat_vnhook+0xfa kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x800fd8acc, rsp =3D = 0x7fffffffcf38, rbp =3D 0x7fffffffd3d0 --- <118>/etc/rc: WARNING: $hostname is not set -- see rc.conf(5). <118>Starting Network: lo0 em0 em1 em2 em3. <118>lo0: flags=3D8049 metric 0 mtu 16384 <118> options=3D3 <118> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6=20 <118> inet6 ::1 prefixlen 128=20 <118> inet 127.0.0.1 netmask 0xff000000=20 <118> nd6 options=3D3 <118>em0: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:d5 <118> inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>em1: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:cb <118> inet 10.0.1.2 netmask 0xffffff00 broadcast 10.0.1.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>em2: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:df <118> inet 10.0.2.2 netmask 0xffffff00 broadcast 10.0.2.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>em3: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:e9 <118> inet 172.16.2.118 netmask 0xffff0000 broadcast 172.16.255.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>add net default: gateway 172.16.0.2 Expensive timeout(9) function: 0xffffffff80329580(0xffffff8000272000) = 0.006700013 s <118>Starting devd. ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based = forwarding disabled, default to deny, logging disabled <118>Flushed all rules. <118>00100 allow ip from any to any via lo0 <118>00200 deny ip from any to 127.0.0.0/8 <118>00300 deny ip from 127.0.0.0/8 to any <118>65000 allow ip from any to any <118>Firewall rules loaded. <118>Creating and/or trimming log files <118>. <118>Starting syslogd. <118>No core dumps found. <118>ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib <118>32-bit compatibility ldconfig path: /usr/lib32 lock order reversal: 1st 0xffffff800a2f62e8 bufwait (bufwait) @ = /pool/newsrc/src/sys/kern/vfs_bio.c:2559 2nd 0xffffff0002c71400 dirhash (dirhash) @ = /pool/newsrc/src/sys/ufs/ufs/ufs_dirhash.c:285 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 ufsdirhash_acquire() at ufsdirhash_acquire+0x44 ufsdirhash_add() at ufsdirhash_add+0x19 ufs_direnter() at ufs_direnter+0x88b ufs_mkdir() at ufs_mkdir+0x623 VOP_MKDIR_APV() at VOP_MKDIR_APV+0xb9 kern_mkdirat() at kern_mkdirat+0x264 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (136, FreeBSD ELF64, mkdir), rip =3D 0x80072bb0c, rsp =3D = 0x7fffffffec88, rbp =3D 0x7fffffffef66 --- <118>Clearing /tmp (X related). <118>Updating motd: <118>. <118>Configuring syscons: <118> blanktime <118>. <118>Starting sshd. <118>Starting cron. <118>Starting background file system checks in 60 seconds. <118> <118>Wed Mar 10 17:29:09 CET 2010 <118>Mar 10 17:29:21 su: BAD SU borjam to root on /dev/pts/0 <118>Mar 10 17:29:24 su: BAD SU borjam to root on /dev/pts/0 <118>Mar 10 17:29:26 su: borjam to root on /dev/pts/0 <118>Mar 10 17:30:36 su: borjam to root on /dev/pts/1 lock order reversal: 1st 0xffffff00027e2438 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 2nd 0xffffff000286bb88 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:130 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 dnode_verify() at dnode_verify+0x70 dnode_hold_impl() at dnode_hold_impl+0x73 dmu_bonus_hold() at dmu_bonus_hold+0x31 bplist_hold() at bplist_hold+0x48 bplist_enqueue() at bplist_enqueue+0x4c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dbuf_write() at dbuf_write+0x24c dbuf_sync_list() at dbuf_sync_list+0x3eb dbuf_sync_list() at dbuf_sync_list+0x17f Fatal double fault rip =3D 0xffffffff805d322d rsp =3D 0xffffff8012414fe0 rbp =3D 0xffffff8012415090 cpuid =3D 0; apic id =3D 00 panic: double fault cpuid =3D 0 KDB: enter: panic shared sx zfsvfs->z_teardown_inactive_lock = (zfsvfs->z_teardown_inactive_lock) r =3D 0 (0xffffff000256d210) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vnops.c:3724 exclusive lockmgr zfs (zfs) r =3D 0 (0xffffff00055b3a58) locked @ = /pool/newsrc/src/sys/kern/vfs_subr.c:2091 exclusive lockmgr zfs (zfs) r =3D 0 (0xffffff0005618308) locked @ = /pool/newsrc/src/sys/kern/vfs_syscalls.c:3174 shared lockmgr zfs (zfs) r =3D 0 (0xffffff00057bda58) locked @ = /pool/newsrc/src/sys/kern/vfs_subr.c:2091 exclusive sx zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) r =3D 0 = (0xffffff000256d310) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 shared sx zp->z_name_lock (zp->z_name_lock) r =3D 0 (0xffffff00057c7c38) = locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_dir.c:212 exclusive lockmgr zfs (zfs) r =3D 0 (0xffffff00057ac578) locked @ = /pool/newsrc/src/sys/kern/vfs_subr.c:2091 exclusive sx so_rcv_sx (so_rcv_sx) r =3D 0 (0xffffff0002f63648) locked @ = /pool/newsrc/src/sys/kern/uipc_sockbuf.c:148 exclusive sx so_rcv_sx (so_rcv_sx) r =3D 0 (0xffffff0002cff3a0) locked @ = /pool/newsrc/src/sys/kern/uipc_sockbuf.c:148 exclusive sx ds->ds_deadlist.bpl_lock (ds->ds_deadlist.bpl_lock) r =3D 0 = (0xffffff00027e2438) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002948238) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002963538) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002949438) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002948b38) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002966238) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 exclusive sx dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) r =3D 0 = (0xffffff0002963438) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 0xffffff00055b39c0: tag zfs, type VDIR usecount 0, writecount 0, refcount 1 mountedhere 0 flags (VI_DOINGINACT) lock type zfs: EXCL by thread 0xffffff000528b3a0 (pid 1425) #0 0xffffffff80578cef at __lockmgr_args+0x4ef #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff8062d957 at _vn_lock+0x57 #4 0xffffffff806210bb at vget+0x7b #5 0xffffffff8060e570 at cache_lookup+0x4e0 #6 0xffffffff8060e890 at vfs_cache_lookup+0xc0 #7 0xffffffff808cbdb7 at VOP_LOOKUP_APV+0xb7 #8 0xffffffff80615513 at lookup+0x3d3 #9 0xffffffff806164e9 at namei+0x4a9 #10 0xffffffff806255cf at kern_statat_vnhook+0x8f #11 0xffffffff80625775 at kern_statat+0x15 #12 0xffffffff8062583a at lstat+0x2a #13 0xffffffff80876b48 at syscall+0x118 #14 0xffffffff8085cf91 at Xfast_syscall+0xe1 0xffffff0005618270: tag zfs, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags () lock type zfs: EXCL by thread 0xffffff000528c000 (pid 1422) #0 0xffffffff80578f77 at __lockmgr_args+0x777 #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff8062d957 at _vn_lock+0x57 #4 0xffffffff806299ba at setutimes+0x7a #5 0xffffffff80629cb2 at kern_lutimes+0xf2 #6 0xffffffff80876b48 at syscall+0x118 #7 0xffffffff8085cf91 at Xfast_syscall+0xe1 0xffffff00057bd9c0: tag zfs, type VDIR usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type zfs: SHARED (count 1) #0 0xffffffff80578cef at __lockmgr_args+0x4ef #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff8062d957 at _vn_lock+0x57 #4 0xffffffff806210bb at vget+0x7b #5 0xffffffff8060e570 at cache_lookup+0x4e0 #6 0xffffffff8060e890 at vfs_cache_lookup+0xc0 #7 0xffffffff808cbdb7 at VOP_LOOKUP_APV+0xb7 #8 0xffffffff80615513 at lookup+0x3d3 #9 0xffffffff806164e9 at namei+0x4a9 #10 0xffffffff806255cf at kern_statat_vnhook+0x8f #11 0xffffffff80625775 at kern_statat+0x15 #12 0xffffffff8062583a at lstat+0x2a #13 0xffffffff80876b48 at syscall+0x118 #14 0xffffffff8085cf91 at Xfast_syscall+0xe1 0xffffff00057ac4e0: tag zfs, type VDIR usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type zfs: EXCL by thread 0xffffff000262a3a0 (pid 1416) with shared waiters pending #0 0xffffffff80578cef at __lockmgr_args+0x4ef #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff8062d957 at _vn_lock+0x57 #4 0xffffffff806210bb at vget+0x7b #5 0xffffffff8060e570 at cache_lookup+0x4e0 #6 0xffffffff8060e890 at vfs_cache_lookup+0xc0 #7 0xffffffff808cbdb7 at VOP_LOOKUP_APV+0xb7 #8 0xffffffff80615513 at lookup+0x3d3 #9 0xffffffff806164e9 at namei+0x4a9 #10 0xffffffff80624df3 at kern_mkdirat+0xc3 #11 0xffffffff80876b48 at syscall+0x118 #12 0xffffffff8085cf91 at Xfast_syscall+0xe1 Uptime: 2m16s Textdump complete. Automatic reboot in 15 seconds - press a key on the console to abort --> Press a key on the console to reboot, --> or switch off the system now. Rebooting... Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights = reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-STABLE #3: Wed Mar 10 14:41:54 CET 2010 = root@pruebazfs1.sarenet.es:/pool/newsrc/obj/pool/newsrc/src/sys/DEBUG = amd64 WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM)2 Duo CPU T8100 @ 2.10GHz (2116.63-MHz = K8-class CPU) Origin =3D "GenuineIntel" Id =3D 0x10676 Stepping =3D 6 = Features=3D0xfebfbff Features2=3D0x80082201> AMD Features=3D0x20100800 AMD Features2=3D0x1 TSC: P-state invariant real memory =3D 536870912 (512 MB) avail memory =3D 495988736 (473 MB) ACPI APIC Table: MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port = 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x10c0-0x10cf at device 7.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] pci0: at device 7.3 (no driver attached) pci0: at device 7.7 (no driver attached) vgapci0: port 0x10d0-0x10df mem = 0xd0000000-0xd7ffffff,0xd8000000-0xd87fffff irq 16 at device 15.0 on = pci0 mpt0: port 0x1400-0x14ff mem = 0xd8820000-0xd883ffff,0xd8800000-0xd881ffff irq 17 at device 16.0 on = pci0 mpt0: [ITHREAD] mpt0: MPI Version=3D1.2.0.0 pcib2: at device 17.0 on pci0 pci2: on pcib2 em0: port 0x2000-0x203f = mem 0xd8940000-0xd895ffff,0xd8900000-0xd890ffff irq 18 at device 0.0 on = pci2 em0: Memory Access and/or Bus Master bits were not set! em0: [FILTER] em0: Ethernet address: 00:0c:29:11:4f:d5 em1: port 0x2040-0x207f = mem 0xd8960000-0xd897ffff,0xd8910000-0xd891ffff irq 19 at device 1.0 on = pci2 em1: Memory Access and/or Bus Master bits were not set! em1: [FILTER] em1: Ethernet address: 00:0c:29:11:4f:cb pci2: at device 2.0 (no driver attached) em2: port 0x20c0-0x20ff = mem 0xd8980000-0xd899ffff,0xd8920000-0xd892ffff irq 17 at device 3.0 on = pci2 em2: Memory Access and/or Bus Master bits were not set! em2: [FILTER] em2: Ethernet address: 00:0c:29:11:4f:df em3: port 0x2400-0x243f = mem 0xd89a0000-0xd89bffff,0xd8930000-0xd893ffff irq 19 at device 5.0 on = pci2 em3: Memory Access and/or Bus Master bits were not set! em3: [FILTER] em3: Ethernet address: 00:0c:29:11:4f:e9 pcib3: at device 21.0 on pci0 pci3: on pcib3 pcib4: at device 21.1 on pci0 pci4: on pcib4 pcib5: at device 21.2 on pci0 pci5: on pcib5 pcib6: at device 21.3 on pci0 pci6: on pcib6 pcib7: at device 21.4 on pci0 pci7: on pcib7 pcib8: at device 21.5 on pci0 pci8: on pcib8 pcib9: at device 21.6 on pci0 pci9: on pcib9 pcib10: at device 21.7 on pci0 pci10: on pcib10 pcib11: at device 22.0 on pci0 pci11: on pcib11 pcib12: at device 22.1 on pci0 pci12: on pcib12 pcib13: at device 22.2 on pci0 pci13: on pcib13 pcib14: at device 22.3 on pci0 pci14: on pcib14 pcib15: at device 22.4 on pci0 pci15: on pcib15 pcib16: at device 22.5 on pci0 pci16: on pcib16 pcib17: at device 22.6 on pci0 pci17: on pcib17 pcib18: at device 22.7 on pci0 pci18: on pcib18 pcib19: at device 23.0 on pci0 pci19: on pcib19 pcib20: at device 23.1 on pci0 pci20: on pcib20 pcib21: at device 23.2 on pci0 pci21: on pcib21 pcib22: at device 23.3 on pci0 pci22: on pcib22 pcib23: at device 23.4 on pci0 pci23: on pcib23 pcib24: at device 23.5 on pci0 pci24: on pcib24 pcib25: at device 23.6 on pci0 pci25: on pcib25 pcib26: at device 23.7 on pci0 pci26: on pcib26 pcib27: at device 24.0 on pci0 pci27: on pcib27 pcib28: at device 24.1 on pci0 pci28: on pcib28 pcib29: at device 24.2 on pci0 pci29: on pcib29 pcib30: at device 24.3 on pci0 pci30: on pcib30 pcib31: at device 24.4 on pci0 pci31: on pcib31 pcib32: at device 24.5 on pci0 pci32: on pcib32 pcib33: at device 24.6 on pci0 pci33: on pcib33 pcib34: at device 24.7 on pci0 pci34: on pcib34 acpi_acad0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse, device ID 3 ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppc0: [ITHREAD] ppbus0: on ppc0 plip0: on ppbus0 plip0: [ITHREAD] lpt0: on ppbus0 lpt0: [ITHREAD] lpt0: Interrupt-driven port ppi0: on ppbus0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: [FILTER] fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on = acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 cpu0: on acpi0 acpi_throttle0: on cpu0 orm0: at iomem = 0xc0000-0xc7fff,0xca000-0xcafff,0xcb000-0xcbfff,0xcc000-0xccfff,0xcd000-0x= cdfff,0xdc000-0xdffff,0xe0000-0xe3fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on = isa0 ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is = present; to enable, add "vfs.zfs.prefetch_disable=3D0" to = /boot/loader.conf. ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable = behavior. Consider tuning vm.kmem_size and vm.kmem_size_max in /boot/loader.conf. ZFS filesystem version 3 ZFS storage pool version 14 Timecounter "TSC" frequency 2116629050 Hz quality 800 Timecounters tick every 10.000 msec acd0: DVDR at ata1-master = UDMA33=20 Expensive timeout(9) function: 0xffffffff805ed840(0) 0.002413714 s Expensive timeout(9) function: 0xffffffff80885bf0(0xffffffff80e68900) = 0.003080000 s da0 at mpt0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device=20 da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da0: Command Queueing enabled da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) da1 at mpt0 bus 0 scbus0 target 1 lun 0 da1: Fixed Direct Access SCSI-2 device=20 da1: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da1: Command Queueing enabled da1: 4096MB (8388608 512 byte sectors: 255H 63S/T 522C) da2 at mpt0 bus 0 scbus0 target 2 lun 0 da2: Fixed Direct Access SCSI-2 device=20 da2: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da2: Command Queueing enabled da2: 4096MB (8388608 512 byte sectors: 255H 63S/T 522C) da3 at mpt0 bus 0 scbus0 target 3 lun 0 da3: Fixed Direct Access SCSI-2 device=20 da3: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da3: Command Queueing enabled da3: 12288MB (25165824 512 byte sectors: 255H 63S/T 1566C) WARNING: WITNESS option enabled, expect reduced performance. WARNING: DIAGNOSTIC option enabled, expect reduced performance. Trying to mount root from ufs:/dev/da0s1a WARNING: / was not properly dismounted <118>Setting hostuuid: 564d8c47-d7e5-fea4-92ef-f17760cfe4d2. <118>Setting hostid: 0x4d345a2c. lock order reversal: 1st 0xffffff00028732e8 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:549 2nd 0xffffff00028720d8 dn->dn_mtx (dn->dn_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:1173 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_block_freed() at dnode_block_freed+0x8e dbuf_read() at dbuf_read+0x155 dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0x12a dmu_read() at dmu_read+0x80 load_nvlist() at load_nvlist+0x85 spa_load() at spa_load+0x49a spa_open_common() at spa_open_common+0x12d spa_get_stats() at spa_get_stats+0x42 zfs_ioc_pool_stats() at zfs_ioc_pool_stats+0x2c zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 Expensive timeout(9) function: 0xffffffff80892b50(0xffffff000244fe00) = 0.003086146 s kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffffd808, rbp =3D 0x801224140 --- lock order reversal: 1st 0xffffff0002868e70 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode_sync.c:381 2nd 0xffffff0002767940 osi->os_lock (osi->os_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:323 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_destroy() at dnode_destroy+0xa6 dnode_buf_pageout() at dnode_buf_pageout+0xb2 dbuf_evict_user() at dbuf_evict_user+0x55 dbuf_clear() at dbuf_clear+0x5e dnode_evict_dbufs() at dnode_evict_dbufs+0x98 dmu_objset_evict_dbufs() at dmu_objset_evict_dbufs+0x11c dmu_objset_evict() at dmu_objset_evict+0xbf dsl_pool_close() at dsl_pool_close+0x52 spa_unload() at spa_unload+0xb2 spa_load() at spa_load+0x4da spa_open_common() at spa_open_common+0x12d spa_get_stats() at spa_get_stats+0x42 zfs_ioc_pool_stats() at zfs_ioc_pool_stats+0x2c zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffffd808, rbp =3D 0x801224140 --- lock order reversal: 1st 0xffffff0002873058 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1116 2nd 0xffffff000259ac38 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1120 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_sExpensive timeout(9) function: = 0xffffffff805ed8c0(0) 0.003133917 s elf_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_dirty() at dbuf_dirty+0x892 dnode_setdirty() at dnode_setdirty+0x1a9 dbuf_dirty() at dbuf_dirty+0xa53 bplist_vacate() at bplist_vacate+0x4d spa_sync() at spa_sync+0x297 txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002926838 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 2nd 0xffffff000250c2f0 spa->spa_sync_bplist.bpl_lock = (spa->spa_sync_bplist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:235 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 bplist_enqueue_deferred() at bplist_enqueue_deferred+0x47 zio_free() at zio_free+0x105 arc_free() at arc_free+0x11c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x483 dbuf_write() at dbuf_write+0x24c dbuf_sync_list() at dbuf_sync_list+0x3eb dbuf_sync_list() at dbuf_sync_list+0x17f dnode_sync() at dnode_sync+0xc12 dmu_objset_sync() at dmu_objset_sync+0x134 dsl_pool_sync() at dsl_pool_sync+0x200 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at Expensive timeout(9) function: = 0xffffffff80885bf0(0xffffffff80e68900) 0.003364952 s fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- <118>Entropy harvesting: <118> interrupts <118> ethernet <118> point_to_point <118> kickstart <118>. <118>Starting file system checks: <118>/dev/da0s1a: INCORRECT BLOCK COUNT I=3D259208 (16 should be 12) = (CORRECTED) <118>/dev/da0s1a: LINK COUNT DIR I=3D235521 OWNER=3Droot MODE=3D41777 <118>/dev/da0s1a: SIZE=3D3584 MTIME=3DMar 10 17:30 2010 COUNT 120 = SHOULD BE 119 (ADJUSTED) <118>/dev/da0s1a: SUMMARY INFORMATION BAD (SALVAGED) <118>/dev/da0s1a: BLK(S) MISSING IN BIT MAPS (SALVAGED) <118>/dev/da0s1a: 28967 files, 517344 used, 1004473 free (7889 frags, = 124573 blocks, 0.5% fragmentation) Expensive timeout(9) function: 0xffffffff80475420(0xffffffff80e77f60) = 0.004632152 s <118>Mounting local file systems: <118>. lock order reversal: 1st 0xffffff0002620cc8 zfs (zfs) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/gfs.c:437 2nd 0xffffff000256f310 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zget() at zfs_zget+0x23c zfs_root() at zfs_root+0x50 zfsctl_create() at zfsctl_create+0x82 zfs_mount() at zfs_mount+0x7ef vfs_donmount() at vfs_donmount+0xcde nmount() at nmount+0x63 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (378, FreeBSD ELF64, nmount), rip =3D 0x800f48f1c, rsp =3D = 0x7fffffffced8, rbp =3D 0x7fffffffcef8 --- lock order reversal: 1st 0xffffff0002655078 zp->z_name_lock (zp->z_name_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_dir.c:212 2nd 0xffffff000256f330 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zget() at zfs_zget+0x23c zfs_dirent_lock() at zfs_dirent_lock+0x4a0 zfs_dirlook() at zfs_dirlook+0x90 zfs_lookup() at zfs_lookup+0x256 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x8d VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xaf vfs_cache_lookup() at vfs_cache_lookup+0xf0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x4a9 kern_statat_vnhook() at kern_statat_vnhook+0x8f kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x800fd8acc, rsp =3D = 0x7fffffffcf38, rbp =3D 0x7fffffffd3d0 --- lock order reversal: 1st 0xffffff000256f210 zfsvfs->z_teardown_inactive_lock = (zfsvfs->z_teardown_inactive_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vnops.c:3724 2nd 0xffffff000256f330 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:1027 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zinactive() at zfs_zinactive+0x95 zfs_inactive() at zfs_inactive+0x7e zfs_freebsd_inactive() at zfs_freebsd_inactive+0x1a VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xb5 vinactive() at vinactive+0x90 vputx() at vputx+0x2fc kern_statat_vnhook() at kern_statat_vnhook+0xfa kern_statat() at kern_statat+0x15 lstat() at lstat+0x2a syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (190, FreeBSD ELF64, lstat), rip =3D 0x800fd8acc, rsp =3D = 0x7fffffffcf38, rbp =3D 0x7fffffffd3d0 --- <118>/etc/rc: WARNING: $hostname is not set -- see rc.conf(5). <118>Starting Network: lo0 em0 em1 em2 em3. <118>lo0: flags=3D8049 metric 0 mtu 16384 <118> options=3D3 <118> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6=20 <118> inet6 ::1 prefixlen 128=20 <118> inet 127.0.0.1 netmask 0xff000000=20 <118> nd6 options=3D3 <118>em0: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:d5 <118> inet 10.0.0.2 netmask 0xffffff00 broadcast 10.0.0.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>em1: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:cb <118> inet 10.0.1.2 netmask 0xffffff00 broadcast 10.0.1.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>em2: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:df <118> inet 10.0.2.2 netmask 0xffffff00 broadcast 10.0.2.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>em3: flags=3D8843 metric 0 = mtu 1500 <118> options=3D9b <118> ether 00:0c:29:11:4f:e9 <118> inet 172.16.2.118 netmask 0xffff0000 broadcast 172.16.255.255 <118> media: Ethernet autoselect (1000baseT ) <118> status: active <118>add net default: gateway 172.16.0.2 <118>Starting devd. Expensive timeout(9) function: 0xffffffff80329580(0xffffff8000284000) = 0.010275607 s ipfw2 (+ipv6) initialized, divert loadable, nat loadable, rule-based = forwarding disabled, default to deny, logging disabled <118>Flushed all rules. <118>00100 allow ip from any to any via lo0 <118>00200 deny ip from any to 127.0.0.0/8 <118>00300 deny ip from 127.0.0.0/8 to any <118>65000 allow ip from any to any <118>Firewall rules loaded. <118>Creating and/or trimming log files <118>. <118>Starting syslogd. <118>savecore: reboot after panic: double fault <118>Mar 10 17:33:17 savecore: reboot after panic: double fault <118>savecore: writing core to textdump.tar.1 <118>/var/crash/vmcore.1 not found <118>ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib <118>32-bit compatibility ldconfig path: /usr/lib32 lock order reversal: 1st 0xffffff800a2f8338 bufwait (bufwait) @ = /pool/newsrc/src/sys/kern/vfs_bio.c:2559 2nd 0xffffff0002ca5c00 dirhash (dirhash) @ = /pool/newsrc/src/sys/ufs/ufs/ufs_dirhash.c:285 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 ufsdirhash_acquire() at ufsdirhash_acquire+0x44 ufsdirhash_add() at ufsdirhash_add+0x19 ufs_direnter() at ufs_direnter+0x88b ufs_mkdir() at ufs_mkdir+0x623 VOP_MKDIR_APV() at VOP_MKDIR_APV+0xb9 kern_mkdirat() at kern_mkdirat+0x264 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (136, FreeBSD ELF64, mkdir), rip =3D 0x80072bb0c, rsp =3D = 0x7fffffffec88, rbp =3D 0x7fffffffef66 --- <118>Clearing /tmp (X related). <118>Updating motd: <118>. <118>Configuring syscons: <118> blanktime <118>. <118>Starting sshd. lock order reversal: 1st 0xffffff0002864b08 buf->b_lock (buf->b_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/arc.c:2509 2nd 0xffffff00025e9430 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:421 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_read_done() at dbuf_read_done+0x3b arc_read_nolock() at arc_read_nolock+0x526 arc_read() at arc_read+0xaf dbuf_read() at dbuf_read+0x62b dmu_buf_hold() at dmu_buf_hold+0xcc bplist_cache() at bplist_cache+0x4c bplist_iterate() at bplist_iterate+0xb3 spa_sync() at spa_sync+0x262 txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- <118>Starting cron. <118>Starting background file system checks in 60 seconds. <118> <118>Wed Mar 10 17:33:21 CET 2010 <118>Mar 10 17:33:30 su: borjam to root on /dev/pts/0 lock order reversal: 1st 0xffffff0002c6d238 dr->dt.di.dr_mtx (dr->dt.di.dr_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1905 2nd 0xffffff0002866000 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:543 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 dbuf_read() at dbuf_read+0x2ad dbuf_will_dirty() at dbuf_will_dirty+0x53 dsl_dataset_block_kill() at dsl_dataset_block_kill+0xe9 dbuf_write() at dbuf_write+0x24c dbuf_sync_list() at dbuf_sync_list+0x159 dbuf_sync_list() at dbuf_sync_list+0x17f dnode_sync() at dnode_sync+0xc12 dmu_objset_sync() at dmu_objset_sync+0x134 dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff000248d038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 2nd 0xffffff0002871b88 dn->dn_struct_rwlock (dn->dn_struct_rwlock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:130 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_slock() at _sx_slock+0x55 dnode_verify() at dnode_verify+0x70 dnode_hold_impl() at dnode_hold_impl+0x73 dmu_bonus_hold() at dmu_bonus_hold+0x31 bplist_hold() at bplist_hold+0x48 bplist_enqueue() at bplist_enqueue+0x4c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dmu_objset_sync() at dmu_objset_sync+0x1fe dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff000248d038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 2nd 0xffffff0002f340d8 dn->dn_mtx (dn->dn_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:606 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_hold_impl() at dnode_hold_impl+0x184 dmu_bonus_hold() at dmu_bonus_hold+0x31 bplist_hold() at bplist_hold+0x48 bplist_enqueue() at bplist_enqueue+0x4c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dmu_objset_sync() at dmu_objset_sync+0x1fe dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff000248d038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 2nd 0xffffff00028686c0 db->db_mtx (db->db_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1724 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_rele() at dbuf_rele+0x2d dnode_hold_impl() at dnode_hold_impl+0x20f dmu_bonus_hold() at dmu_bonus_hold+0x31 bplist_hold() at bplist_hold+0x48 bplist_enqueue() at bplist_enqueue+0x4c dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dmu_objset_sync() at dmu_objset_sync+0x1fe dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff000248d038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:189 2nd 0xffffff00027d9540 osi->os_lock (osi->os_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dnode.c:687 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dnode_setdirty() at dnode_setdirty+0xbc dbuf_dirty() at dbuf_dirty+0x516 bplist_enqueue() at bplist_enqueue+0xbd dsl_dataset_block_kill() at dsl_dataset_block_kill+0x119 dmu_objset_sync() at dmu_objset_sync+0x1fe dsl_pool_sync() at dsl_pool_sync+0x88 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002893210 zfsvfs->z_teardown_inactive_lock = (zfsvfs->z_teardown_inactive_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vfsops.c:917 2nd 0xffffff000248d0f8 ds->ds_rwlock (ds->ds_rwlock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dsl_dataset.c:2864 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dsl_dataset_clone_swap() at dsl_dataset_clone_swap+0x5a dmu_recv_end() at dmu_recv_end+0x94 zfs_ioc_recv() at zfs_ioc_recv+0x29d zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffff8e98, rbp =3D 0x7fffffff9bd0 --- lock order reversal: 1st 0xffffff000248d038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:152 2nd 0xffffff0002f34330 dn->dn_dbufs_mtx (dn->dn_dbufs_mtx) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:1518 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_destroy() at dbuf_destroy+0x58 bplist_cache() at bplist_cache+0x2e bplist_iterate() at bplist_iterate+0xb3 bplist_space_birthrange() at bplist_space_birthrange+0x60 dsl_dataset_clone_swap_sync() at dsl_dataset_clone_swap_sync+0xee dsl_sync_task_group_sync() at dsl_sync_task_group_sync+0x173 dsl_pool_sync() at dsl_pool_sync+0x122 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff000248d038 ds->ds_deadlist.bpl_lock = (ds->ds_deadlist.bpl_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/bplist.c:152 2nd 0xffffffff81150db0 h->hash_mutexes[i] (h->hash_mutexes[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dbuf.c:191 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 dbuf_destroy() at dbuf_destroy+0x111 bplist_cache() at bplist_cache+0x2e bplist_iterate() at bplist_iterate+0xb3 bplist_space_birthrange() at bplist_space_birthrange+0x60 dsl_dataset_clone_swap_sync() at dsl_dataset_clone_swap_sync+0xee dsl_sync_task_group_sync() at dsl_sync_task_group_sync+0x173 dsl_pool_sync() at dsl_pool_sync+0x122 spa_sync() at spa_sync+0x35e txg_sync_thread() at txg_sync_thread+0x2d7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip =3D 0, rsp =3D 0xffffff8012417d30, rbp =3D 0 --- lock order reversal: 1st 0xffffff0002893250 zfsvfs->z_znodes_lock (zfsvfs->z_znodes_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vfsops.c:1317 2nd 0xffffff0002893310 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:966 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_rezget() at zfs_rezget+0x4a zfs_resume_fs() at zfs_resume_fs+0x158 zfs_ioc_recv() at zfs_ioc_recv+0x2b4 zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip =3D 0x800fe7d1c, rsp =3D = 0x7fffffff8e98, rbp =3D 0x7fffffff9bd0 --- lock order reversal: 1st 0xffffff0004e911d0 zp->z_parent_lock (zp->z_parent_lock) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_dir.c:379 2nd 0xffffff0002893350 zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:866 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2a _witness_debugger() at _witness_debugger+0x2e witness_checkorder() at witness_checkorder+0x81e _sx_xlock() at _sx_xlock+0x55 zfs_zget() at zfs_zget+0x23c zfs_dirlook() at zfs_dirlook+0x1fc zfs_lookup() at zfs_lookup+0x256 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x8d VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xaf vfs_cache_lookup() at vfs_cache_lookup+0xf0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x4a9 kern_chdir() at kern_chdir+0x78 syscall() at syscall+0x118 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (12, FreeBSD ELF64, chdir), rip =3D 0x800da74dc, rsp =3D = 0x7fffffffe898, rbp =3D 0x801046f10 --- KDB: enter: sysctl debug.kdb.enter exclusive sleep mutex Giant (Giant) r =3D 0 (0xffffffff80c4c540) locked = @ /pool/newsrc/src/sys/kern/kern_sysctl.c:1417 shared sx sysctl lock (sysctl lock) r =3D 0 (0xffffffff80c4cec0) locked = @ /pool/newsrc/src/sys/kern/kern_sysctl.c:1521 exclusive sx zfsvfs->z_teardown_inactive_lock = (zfsvfs->z_teardown_inactive_lock) r =3D 0 (0xffffff0002893210) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_vfsops.c:917 shared sx ds->ds_rwlock (ds->ds_rwlock) r =3D 0 (0xffffff00190f50f8) = locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/dsl_dataset.c:519 exclusive sx zfsvfs->z_online_recv_lock (zfsvfs->z_online_recv_lock) r =3D= 0 (0xffffff0002893288) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_ioctl.c:2492 exclusive lockmgr zfs (zfs) r =3D 0 (0xffffff0007d43308) locked @ = /pool/newsrc/src/sys/kern/vfs_subr.c:888 exclusive sx zfsvfs->z_hold_mtx[i] (zfsvfs->z_hold_mtx[i]) r =3D 0 = (0xffffff00028963f0) locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_znode.c:768 shared sx zp->z_name_lock (zp->z_name_lock) r =3D 0 (0xffffff0007cf31f0) = locked @ = /pool/newsrc/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common= /fs/zfs/zfs_dir.c:212 exclusive lockmgr zfs (zfs) r =3D 0 (0xffffff0007d11098) locked @ = /pool/newsrc/src/sys/kern/vfs_subr.c:2091 exclusive sx so_rcv_sx (so_rcv_sx) r =3D 0 (0xffffff0002c763a0) locked @ = /pool/newsrc/src/sys/kern/uipc_sockbuf.c:148 exclusive lockmgr syncer (syncer) r =3D 0 (0xffffff0002947cc8) locked @ = /pool/newsrc/src/sys/kern/vfs_subr.c:1701 exclusive lockmgr zfs (zfs) r =3D 0 (0xffffff0007d61cc8) locked @ = /pool/newsrc/src/sys/kern/vfs_subr.c:888 0xffffff0007d11000: tag zfs, type VDIR usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type zfs: EXCL by thread 0xffffff000262dae0 (pid 1699) #0 0xffffffff80578cef at __lockmgr_args+0x4ef #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff8062d957 at _vn_lock+0x57 #4 0xffffffff806210bb at vget+0x7b #5 0xffffffff8060e570 at cache_lookup+0x4e0 #6 0xffffffff8060e890 at vfs_cache_lookup+0xc0 #7 0xffffffff808cbdb7 at VOP_LOOKUP_APV+0xb7 #8 0xffffffff80615513 at lookup+0x3d3 #9 0xffffffff806164e9 at namei+0x4a9 #10 0xffffffff8062d1a2 at vn_open_cred+0xc2 #11 0xffffffff8062b569 at kern_openat+0x179 #12 0xffffffff80876b48 at syscall+0x118 #13 0xffffffff8085cf91 at Xfast_syscall+0xe1 0xffffff0002947c30: tag syncer, type VNON usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type syncer: EXCL by thread 0xffffff000250d3a0 (pid 18) #0 0xffffffff80578f77 at __lockmgr_args+0x777 #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff8062d957 at _vn_lock+0x57 #4 0xffffffff806229a0 at sync_vnode+0x130 #5 0xffffffff80622c2c at sched_sync+0x1cc #6 0xffffffff80566aca at fork_exit+0x12a #7 0xffffffff8085d18e at fork_trampoline+0xe 0xffffff0007d43270: tag zfs, type VREG usecount 0, writecount 0, refcount 1 mountedhere 0 flags (VI_DOOMED) v_object 0xffffff0007d2e870 ref 0 pages 0 lock type zfs: EXCL by thread 0xffffff000262dae0 (pid 1699) #0 0xffffffff80578f77 at __lockmgr_args+0x777 #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff806231db at vnlru_free+0x20b #4 0xffffffff806235df at getnewvnode+0x24f #5 0xffffffff810a7533 at zfs_znode_cache_constructor+0x43 #6 0xffffffff810a9248 at zfs_znode_alloc+0x38 #7 0xffffffff810a96d9 at zfs_mknode+0x259 #8 0xffffffff810cda21 at zfs_freebsd_create+0x661 #9 0xffffffff808cbc43 at VOP_CREATE_APV+0xb3 #10 0xffffffff8062d553 at vn_open_cred+0x473 #11 0xffffffff8062b569 at kern_openat+0x179 #12 0xffffffff80876b48 at syscall+0x118 #13 0xffffffff8085cf91 at Xfast_syscall+0xe1 0xffffff0007d61c30: tag zfs, type VREG usecount 0, writecount 0, refcount 1 mountedhere 0 flags (VI_DOOMED) v_object 0xffffff0007d2e6c0 ref 0 pages 0 lock type zfs: EXCL by thread 0xffffff000250d740 (pid 17) #0 0xffffffff80578f77 at __lockmgr_args+0x777 #1 0xffffffff80610a59 at vop_stdlock+0x39 #2 0xffffffff808caedb at VOP_LOCK1_APV+0x9b #3 0xffffffff806231db at vnlru_free+0x20b #4 0xffffffff80623bce at vnlru_proc+0x58e #5 0xffffffff80566aca at fork_exit+0x12a #6 0xffffffff8085d18e at fork_trampoline+0xe panic: from debugger cpuid =3D 0 Uptime: 33m28s #=20 From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 15:21:19 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5EB001065676; Wed, 10 Mar 2010 15:21:19 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id D9BCE8FC14; Wed, 10 Mar 2010 15:21:18 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2DEEF.dip.t-dialin.net [217.226.222.239]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id DC7CA844588; Wed, 10 Mar 2010 16:21:12 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id E68F4536B; Wed, 10 Mar 2010 16:21:09 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1268234470; bh=WtBQx5HuCs6SupMMkwiVt0sh/K5jRd+PFN/CMJtH0x8=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=CjemjAI1Xwi25CHwWm97K4GELcHsHvvff4ere+V4VPFKVVfLbOr7IlnU/PzN/hBtv J/ctb2jXkdC9gDmABn5aUcEzZgaUYFGk7YbjXvkN8B6xQEHP4YVDyZqV+5DdItZClB ML4DipDQmjwilYIT+PQfCJwIBSKRwyODO/o4NK6/W2JmncfBqjvg/QgxTjKL6Y3x8x 3i/XbQxMjSEt/8HYg6RU+C/S2tc4/S0MkaWG/iQrWh2aS1mAcuCzJKjtYehKEusMk8 7eg0jbCFnlJkImEZHoD5580OWzDnRH5/vDLCNdbSv52ldvmSTHdVx7FtSkwbXiZtoo mJchWoNCnoKKA== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2AFL9OR027864; Wed, 10 Mar 2010 16:21:09 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Wed, 10 Mar 2010 16:21:09 +0100 Message-ID: <20100310162109.55405ymb9hpqks08@webmail.leidinger.net> Date: Wed, 10 Mar 2010 16:21:09 +0100 From: Alexander Leidinger To: John Baldwin References: <20100310113422.95932h10tv1qh2o0@webmail.leidinger.net> <201003100812.29749.jhb@freebsd.org> In-Reply-To: <201003100812.29749.jhb@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: DC7CA844588.4DA93 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-0.84, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, J_CHICKENPOX_73 0.60) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1268839274.81552@hM9xLNph6IbvblTCTWx+yQ X-EBL-Spam-Status: No Cc: freebsd-stable@freebsd.org, "Robert N. M. Watson" Subject: Re: is dtrace usable? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 15:21:19 -0000 Quoting John Baldwin (from Wed, 10 Mar 2010 08:12:29 -0500): > On Wednesday 10 March 2010 5:34:22 am Alexander Leidinger wrote: >> Quoting "Robert N. M. Watson" (from Tue, 9 Mar >> 2010 16:39:09 +0000): >> >> > >> > On Mar 9, 2010, at 2:16 PM, Alexander Leidinger wrote: >> > >> >>> From this you can see that sys.mk is included and parsed before > 'Makefile', >> >>> so the WITH_CTF=yes is not set until after sys.mk has been parsed. >> >> >> >> I think we need to find a different solution for this. The need to >> >> specify WITH_CTF at the command line is very error prone. :( >> > >> > You are neither the first person to have made this observation, nor >> > the first person to have failed to propose a solution in the form of >> > a patch :-). >> >> It is not a problem to provide a patch, the problem is something else. >> >> Is it correct that the result of the ctfmerge/cftconvert stuff is not >> covered by the CDDL? >> >> If yes, why not use it by default if the programs are available (I've >> read the comment for the NO_CTF part, but IMO we have a chicken&egg >> situation here, dtrace will not become popular if it is not easy to >> use it)? This default can be made only for the kernel (by making a >> copy of the definition of CTFCONVERT into bsd.prog.mk and bsd.lib.mk, >> or by undefining it there), or for kernel+userland (removing the >> !WITH_CTF -> NO_CTF part from sys.mk). > > Unfortunately the ctf stuff breaks static binaries. I think that if > that were > fixed we would simply enable it by default and be done. So it should work by enabling it by default for the kernel, and not for the userland (as we do not have a working PID provider, this shouldn't be a big problem anyway). And this is something which my patch is supposed to do, but doesn't because of the strange behavior described in the previous mail. Any ideas? Bye, Alexander. -- ... [concerning quotation marks] even if we *_d_i_d* quote anybody in this business, it probably would be gibberish. -- Thom McLeod http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 15:44:23 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1993F106564A for ; Wed, 10 Mar 2010 15:44:23 +0000 (UTC) (envelope-from paulo@nlink.com.br) Received: from smtp.nlink.com.br (smtp.nlink.com.br [201.12.59.3]) by mx1.freebsd.org (Postfix) with SMTP id 2B82C8FC17 for ; Wed, 10 Mar 2010 15:44:21 +0000 (UTC) Received: (qmail 62432 invoked from network); 10 Mar 2010 15:17:39 -0000 Received: from j1.nlink.com.br (paulo@intra.nlink.com.br@201.12.59.126) by smtp.nlink.com.br with SMTP; 10 Mar 2010 15:17:39 -0000 Message-ID: <4B97B813.50102@nlink.com.br> Date: Wed, 10 Mar 2010 12:17:39 -0300 From: Paulo Fragoso User-Agent: Thunderbird 2.0.0.17 (X11/20081030) MIME-Version: 1.0 To: Zavam, =?ISO-8859-1?Q?Vin=EDcius?= References: <8b5ad0e11001270926o19d1701fid30ba5a49cbb39b5@mail.gmail.com> <4B61295F.1030802@incunabulum.net> <8b5ad0e11001280245h703990a3t2085588e141b0ab6@mail.gmail.com> <8b5ad0e11002050329n7fe0173aw735211d7eaea731f@mail.gmail.com> In-Reply-To: <8b5ad0e11002050329n7fe0173aw735211d7eaea731f@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org Subject: Re: freebsd 8.0 stable amd64/x86 needs ~9min to bootup X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 15:44:23 -0000 This problem happened with me too. I don't know why boot loader makes intensive reads when installed on partition created by another fdisk (windows, linux, etc) and started by GRUB2. There is a workaround for this, you have to create all partitions with freebsd sysinstall included windows and linux partitions. After this do install for all others OSes using those partitions created by sysinstall. In my case there are 03 OSes installed on same HD: 1: Windows boot loader: 1023MB 2: Windows 7: 50GB 3: PCBSD 8.0: 70GB * -> 5: Linux 108GB 6: Linux swap This problem can be tested before install if you run sysinstall and choses partiotion menu if you receive alerts like this: "Partition X does not end on cylinder boundary" you will have problem with GRUB2. I'm using GRUB2 from Ubuntu 9.10 with this entry in file /etc/grub.d/40_custom: menuentry "FreeBSD/PCBSD 8.0 AMD64" { set root=(hd0,3) chainloader +1 } All works fine! Paulo. Em 05/02/2010 08:29, Zavam escreveu: > 2010/1/28 Zavam, Vinícius : > >> 2010/1/28 Bruce Simpson : >> >>> Try GRUB4DOS. I use this so on boxes where I have Windows installed, I can >>> keep GRUB in the NTFS partition. >>> >>> I haven't seen this issue and am tracking -STABLE on an ASUS V-series >>> machine. >>> >>> >> Simpson, >> I forgot to mention... but I tested it using boot0 (freebsd's >> bootmanager) with no success ;< >> >> had no shots with grub4dos, gag, lilo or grub2; I assume it may not be >> a bootmanager issue, but freebsd's btx bootstrap loader. >> my gentoo and windows o.s. can be loaded using grub or boot0. >> >> >> -- >> Zavam, Vinícius >> > > gentlemen, > morning. > > I just did new fresh installs using 80-STABLE/amd64 and > 90-CURRENT/amd64 snapshots. > unfortunately, no success. > > when tryied the 90-CURRENT I've created a slice to /boot (d) at the > beginning of the partition (ad4s4) and grub returned error code 18 > [1]. > strange. even installing in another slice, at the beginning of the > disk (ad4s1), the bootup process was slow too. > > [1] http://wiki.linuxquestions.org/wiki/GRUB#Error_18 > > > > From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 15:54:43 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 62868106566C for ; Wed, 10 Mar 2010 15:54:43 +0000 (UTC) (envelope-from stylinae@mail.uc.edu) Received: from bay0-omc1-s9.bay0.hotmail.com (bay0-omc1-s9.bay0.hotmail.com [65.54.190.20]) by mx1.freebsd.org (Postfix) with ESMTP id 491F08FC13 for ; Wed, 10 Mar 2010 15:54:43 +0000 (UTC) Received: from BL2PRD0102HT006.prod.exchangelabs.com ([65.54.190.61]) by bay0-omc1-s9.bay0.hotmail.com with Microsoft SMTPSVC(6.0.3790.3959); Wed, 10 Mar 2010 07:54:43 -0800 Received: from mail-pz0-f199.google.com (209.85.222.199) by pod51000.outlook.com (10.6.5.43) with Microsoft SMTP Server (TLS) id 14.0.482.37; Wed, 10 Mar 2010 15:54:41 +0000 Received: by pzk37 with SMTP id 37so4898661pzk.7 for ; Wed, 10 Mar 2010 07:54:39 -0800 (PST) MIME-Version: 1.0 Received: by 10.142.208.13 with SMTP id f13mr491040wfg.310.1268236479451; Wed, 10 Mar 2010 07:54:39 -0800 (PST) Date: Wed, 10 Mar 2010 10:54:39 -0500 Message-ID: <96af083b1003100754i3fbdc474n575f5a8cd217a000@mail.gmail.com> From: Adam Stylinski To: X-OriginalArrivalTime: 10 Mar 2010 15:54:43.0087 (UTC) FILETIME=[FFE669F0:01CAC069] Content-Type: text/plain; charset="ISO-8859-1" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: ZFS question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 15:54:43 -0000 I wasn't sure what the most appropriate mailing list to address this problem would be, so I'll go ahead and ask it here. This is probably most directed toward PJD, as he did directed the porting effort of zfs to freebsd. There are many read only sysctl handles to L2ARC and L1ARC caches, as well as performance stats. Some of these are self explanatory based on their naming conventions, however some are more vague than others. Could I perhaps get a detailed explanation of what those sysctls related to zfs actually are describing? I'm hoping to build a decent web interface with RRDTool that has accurate and useful information in regards to ZFS performance monitoring. Storage analytics are becoming an increasingly popular trend and are EMC's advantage over something like FreeNAS. My efforts are to make a nanoBSD based distro for NAS systems, but aim at different goals than FreeNAS (for example, including built in HAST support). I'm not sure if I'm going to use m0n0wall's framework or start from scratch, still evaluating which endeavor will be best to maintain and fit the job. Anyway, any explanation would be appreciated, even if you're not the great PJD :-p. From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 17:31:53 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7FB8E1065674; Wed, 10 Mar 2010 17:31:53 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id B3B198FC27; Wed, 10 Mar 2010 17:31:51 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 614F045CDD; Wed, 10 Mar 2010 18:31:49 +0100 (CET) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 7FAEE45C9B; Wed, 10 Mar 2010 18:31:43 +0100 (CET) Date: Wed, 10 Mar 2010 18:31:43 +0100 From: Pawel Jakub Dawidek To: Borja Marcos Message-ID: <20100310173143.GD1715@garage.freebsd.pl> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="BI5RvnYi6R4T2M87" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@freebsd.org, FreeBSD Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 17:31:53 -0000 --BI5RvnYi6R4T2M87 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Mar 10, 2010 at 04:12:36PM +0100, Borja Marcos wrote: > =09 > On Mar 10, 2010, at 12:02 PM, Pawel Jakub Dawidek wrote: >=20 > > Once the deadlock occur, enter DDB and send me the output of: > >=20 > > ps > > show alllocks > > show lockedvnods > > show allchains > > alltrace >=20 > (Again, crossposted to -fs, ZFS related) >=20 >=20 > Previous one was a panic when performing the test with several tar jobs r= unning in parallel. >=20 > Now this is a capture of the deadlock itself, instead of a panic. (I call= ed panic from the debugger to generate a dump) [...] Hmm, interesting. Especially those two traces: Tracing command zfs pid 1820 tid 100105 td 0xffffff0002ca4000 [...] _cv_wait() at _cv_wait+0x17a txg_wait_synced() at txg_wait_synced+0x98 zfsvfs_teardown() at zfsvfs_teardown+0x1f6 zfs_suspend_fs() at zfs_suspend_fs+0x2b zfs_ioc_recv() at zfs_ioc_recv+0x28b zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd [...] Tracing command bsdtar pid 1699 tid 100093 td 0xffffff000262dae0 [...] _sx_slock_hard() at _sx_slock_hard+0x1b7 _sx_slock() at _sx_slock+0xc1=20 zfs_freebsd_reclaim() at zfs_freebsd_reclaim+0x63 VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0xb5 vgonel() at vgonel+0x119 vnlru_free() at vnlru_free+0x345 getnewvnode() at getnewvnode+0x24f zfs_znode_cache_constructor() at zfs_znode_cache_constructor+0x43 zfs_znode_alloc() at zfs_znode_alloc+0x38 zfs_mknode() at zfs_mknode+0x259 zfs_freebsd_create() at zfs_freebsd_create+0x661 VOP_CREATE_APV() at VOP_CREATE_APV+0xb3 vn_open_cred() at vn_open_cred+0x473 kern_openat() at kern_openat+0x179 [...] This should be impossible. If we are that deep in zfsvfs_teardown(), it mea= ns that we hold the z_teardown_lock exclusively. And we do as 'show alllocks' output confirms. But if we are holding this lock exclusively we shouldn't be that deep in create code path, because we need hold this lock as reader. It isn't visible in 'show alllocks' output, because this lock is special (rrwlock.c). I see three possibilities: 1. We are looking at different file systems here. But where is deadlock coming from then? 2. There is a bug in rrwlock.c. Highly unlikely I think. 3. My thinking is incorrect somewhere. Let me do some more thinking and I'll get back to you (possibly with a patch that will help us to find right possibility). --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --BI5RvnYi6R4T2M87 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuX134ACgkQForvXbEpPzRsuACgzsjOtg3CjoVm65QoYNmS6GKg LasAoN0poZ4eavwo2Pl/LCiRUCGb67Vm =LFmy -----END PGP SIGNATURE----- --BI5RvnYi6R4T2M87-- From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 17:42:47 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6E781065670; Wed, 10 Mar 2010 17:42:47 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id ACC468FC20; Wed, 10 Mar 2010 17:42:46 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id TAA18018; Wed, 10 Mar 2010 19:42:44 +0200 (EET) (envelope-from avg@icyb.net.ua) Message-ID: <4B97DA13.1040900@icyb.net.ua> Date: Wed, 10 Mar 2010 19:42:43 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.23 (X11/20100211) MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> <20100310173143.GD1715@garage.freebsd.pl> In-Reply-To: <20100310173143.GD1715@garage.freebsd.pl> X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=KOI8-U Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, FreeBSD Stable , Borja Marcos Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 17:42:47 -0000 on 10/03/2010 19:31 Pawel Jakub Dawidek said the following: > This should be impossible. If we are that deep in zfsvfs_teardown(), it means > that we hold the z_teardown_lock exclusively. And we do as 'show alllocks' > output confirms. But if we are holding this lock exclusively we shouldn't be > that deep in create code path, because we need hold this lock as reader. > It isn't visible in 'show alllocks' output, because this lock is special > (rrwlock.c). BTW, it seems that our 'stock' rwlock implements exactly the same thing as rrwlock.c - recursive readers, etc. -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 18:04:09 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C48C106564A; Wed, 10 Mar 2010 18:04:09 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 6E4938FC15; Wed, 10 Mar 2010 18:04:07 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 8D74C45CD8; Wed, 10 Mar 2010 19:04:06 +0100 (CET) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id A994C45683; Wed, 10 Mar 2010 19:04:01 +0100 (CET) Date: Wed, 10 Mar 2010 19:04:01 +0100 From: Pawel Jakub Dawidek To: Andriy Gapon Message-ID: <20100310180401.GG1715@garage.freebsd.pl> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> <20100310173143.GD1715@garage.freebsd.pl> <4B97DA13.1040900@icyb.net.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="UTZ8bGhNySVQ9LYl" Content-Disposition: inline In-Reply-To: <4B97DA13.1040900@icyb.net.ua> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, FreeBSD Stable , Borja Marcos Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 18:04:09 -0000 --UTZ8bGhNySVQ9LYl Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Mar 10, 2010 at 07:42:43PM +0200, Andriy Gapon wrote: > on 10/03/2010 19:31 Pawel Jakub Dawidek said the following: > > This should be impossible. If we are that deep in zfsvfs_teardown(), it= means > > that we hold the z_teardown_lock exclusively. And we do as 'show allloc= ks' > > output confirms. But if we are holding this lock exclusively we shouldn= 't be > > that deep in create code path, because we need hold this lock as reader. > > It isn't visible in 'show alllocks' output, because this lock is special > > (rrwlock.c). >=20 > BTW, it seems that our 'stock' rwlock implements exactly the same thing as > rrwlock.c - recursive readers, etc. But you cannot sleep while holding our rwlock(9). --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --UTZ8bGhNySVQ9LYl Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuX3xAACgkQForvXbEpPzTK3wCghF57qjGHuo2cJWJ8ySWpSNr3 bXQAnj0etk/l9wDe7BFzglE0vs3igsOw =pLu5 -----END PGP SIGNATURE----- --UTZ8bGhNySVQ9LYl-- From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 18:51:39 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC51C1065670 for ; Wed, 10 Mar 2010 18:51:39 +0000 (UTC) (envelope-from google@vink.pl) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.155]) by mx1.freebsd.org (Postfix) with ESMTP id 4417E8FC23 for ; Wed, 10 Mar 2010 18:51:38 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id d23so848131fga.13 for ; Wed, 10 Mar 2010 10:51:38 -0800 (PST) Received: by 10.87.38.38 with SMTP id q38mr2437632fgj.66.1268247097913; Wed, 10 Mar 2010 10:51:37 -0800 (PST) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx.google.com with ESMTPS id 14sm1269906fxm.1.2010.03.10.10.51.33 (version=SSLv3 cipher=RC4-MD5); Wed, 10 Mar 2010 10:51:37 -0800 (PST) Received: by fxm23 with SMTP id 23so7482835fxm.3 for ; Wed, 10 Mar 2010 10:51:33 -0800 (PST) MIME-Version: 1.0 Received: by 10.223.15.147 with SMTP id k19mr2044359faa.87.1268247093459; Wed, 10 Mar 2010 10:51:33 -0800 (PST) In-Reply-To: <96af083b1003100754i3fbdc474n575f5a8cd217a000@mail.gmail.com> References: <96af083b1003100754i3fbdc474n575f5a8cd217a000@mail.gmail.com> Date: Wed, 10 Mar 2010 19:51:33 +0100 Message-ID: <2ae8edf31003101051i2c8920abqfc58757d899c1785@mail.gmail.com> From: Wiktor Niesiobedzki To: Adam Stylinski Content-Type: multipart/mixed; boundary=0015174736727e3eb4048176c9db X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-stable Subject: Re: ZFS question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 18:51:40 -0000 --0015174736727e3eb4048176c9db Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi, You may find some good information in arc_summary.pl script for FreebBSD by jhell: http://jhell.googlecode.com/files/arc_summary.pl I've also did some plugins for munin, where based on that, and looking on the code, I tried to provide some interpretation of L2ARC/ARC statistics (see attached scripts). I'm still not sure, if I have all the important stuff visible on the graphs. Cheers, Wiktor Niesiobedzki 2010/3/10 Adam Stylinski : > I wasn't sure what the most appropriate mailing list to address this prob= lem > would be, so I'll go ahead and ask it here. =C2=A0This is probably most d= irected > toward PJD, as he did directed the porting effort of zfs to freebsd. =C2= =A0There > are many read only sysctl handles to L2ARC and L1ARC caches, as well as > performance stats. =C2=A0Some of these are self explanatory based on thei= r naming > conventions, however some are more vague than others. =C2=A0Could I perha= ps get a > detailed explanation of what those sysctls related to zfs actually are > describing? =C2=A0I'm hoping to build a decent web interface with RRDTool= that > has accurate and useful information in regards to ZFS performance > monitoring. =C2=A0Storage analytics are becoming an increasingly popular = trend > and are EMC's advantage over something like FreeNAS. =C2=A0My efforts are= to make > a nanoBSD based distro for NAS systems, but aim at different goals than > FreeNAS (for example, including built in HAST support). =C2=A0I'm not sur= e if I'm > going to use m0n0wall's framework or start from scratch, still evaluating > which endeavor will be best to maintain and fit the job. =C2=A0Anyway, an= y > explanation would be appreciated, even if you're not the great PJD :-p. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > --0015174736727e3eb4048176c9db-- From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 20:52:45 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [IPv6:2001:4f8:fff6::35]) by hub.freebsd.org (Postfix) with ESMTP id C41A51065673 for ; Wed, 10 Mar 2010 20:52:45 +0000 (UTC) (envelope-from cperciva@freebsd.org) Received: from xps.daemonology.net (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx2.freebsd.org (Postfix) with SMTP id 6D7B21505AC for ; Wed, 10 Mar 2010 20:52:45 +0000 (UTC) Received: (qmail 84902 invoked from network); 10 Mar 2010 20:52:45 -0000 Received: from unknown (HELO xps.daemonology.net) (127.0.0.1) by localhost with SMTP; 10 Mar 2010 20:52:45 -0000 Message-ID: <4B98069C.6050104@freebsd.org> Date: Wed, 10 Mar 2010 12:52:44 -0800 From: FreeBSD Security Officer Organization: FreeBSD Project User-Agent: Thunderbird 2.0.0.23 (X11/20091215) MIME-Version: 1.0 To: freebsd security , FreeBSD Stable X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: FreeBSD 7.2-RELEASE EoL delayed to end of June 2010 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: security-officer@freebsd.org List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 20:52:45 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Everyone, In keeping with the FreeBSD Security Team policy concerning the EoL dates for "Normal" support releases, "a minimum of 12 months after the release, and for sufficient additional time (if needed) to ensure that there is a newer release for at least 3 months before the older Normal release expires" the EoL date for FreeBSD 7.2-RELEASE has been adjusted from the end of May 2010 to the end of June 2010. Due to an unfortunate limitation in the freebsd-update(8) utility, it will warn about the upcoming EoL based on the original end-of-May date until the next time a security update is pushed out for 7.2-RELEASE. Please note that this is only a one month reprieve; we expect 7.3-RELEASE to be announced later this month, and users of 7.2-RELEASE are advised to utilize the months of April, May, and June to ensure that their systems are upgraded before 7.2-RELEASE ceases to be supported. Once they are released, 7.3-RELEASE and 8.1-RELEASE will both receive "Extended" (i.e., 24 month) support from the FreeBSD Security Team. - -- Colin Percival Security Officer, FreeBSD | freebsd.org | The power to serve Founder / author, Tarsnap | tarsnap.com | Online backups for the truly paranoid -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (FreeBSD) iEYEARECAAYFAkuYBpwACgkQFdaIBMps37KQnwCdGOnAcchaMeN0B/Ayo3MHqNPM zq4AnRyDMMmayIDr27RmL+KF+n/0Kzae =drwk -----END PGP SIGNATURE----- From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 21:22:40 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92F4D106567C for ; Wed, 10 Mar 2010 21:22:40 +0000 (UTC) (envelope-from stb@lassitu.de) Received: from gilb.zs64.net (gilb.zs64.net [212.12.50.234]) by mx1.freebsd.org (Postfix) with ESMTP id 5CCED8FC2B for ; Wed, 10 Mar 2010 21:22:40 +0000 (UTC) Received: by gilb.zs64.net (Postfix, from stb@lassitu.de) id 367B553860; Wed, 10 Mar 2010 21:22:39 +0000 (UTC) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Stefan Bethke In-Reply-To: <2ae8edf31003101051i2c8920abqfc58757d899c1785@mail.gmail.com> Date: Wed, 10 Mar 2010 22:22:38 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <8EC9AEE3-726C-4329-BD22-32C617104DAE@lassitu.de> References: <96af083b1003100754i3fbdc474n575f5a8cd217a000@mail.gmail.com> <2ae8edf31003101051i2c8920abqfc58757d899c1785@mail.gmail.com> To: Wiktor Niesiobedzki X-Mailer: Apple Mail (2.1077) Cc: freebsd-stable Subject: Re: ZFS question X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 21:22:40 -0000 Am 10.03.2010 um 19:51 schrieb Wiktor Niesiobedzki: > I've also did some plugins for munin, where based on that, and looking > on the code, I tried to provide some interpretation of L2ARC/ARC > statistics (see attached scripts). I'm still not sure, if I have all > the important stuff visible on the graphs. The FreeBSD lists strip attachments. Would you mind posting a download = link, or are they listed in Munin Exchange? TIA, Stefan --=20 Stefan Bethke Fon +49 151 14070811 From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 22:19:28 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 528DE1065674 for ; Wed, 10 Mar 2010 22:19:28 +0000 (UTC) (envelope-from dmagda@ee.ryerson.ca) Received: from tomts5-srv.bellnexxia.net (tomts5.bellnexxia.net [209.226.175.25]) by mx1.freebsd.org (Postfix) with ESMTP id DC6828FC13 for ; Wed, 10 Mar 2010 22:19:27 +0000 (UTC) Received: from toip5.srvr.bell.ca ([209.226.175.88]) by tomts5-srv.bellnexxia.net (InterMail vM.5.01.06.13 201-253-122-130-113-20050324) with ESMTP id <20100310221926.NQCY2554.tomts5-srv.bellnexxia.net@toip5.srvr.bell.ca> for ; Wed, 10 Mar 2010 17:19:26 -0500 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApEBAA+nl0tMRCRU/2dsb2JhbAAH1xiEeQSDFw Received: from bas1-toronto09-1279534164.dsl.bell.ca (HELO [192.168.1.103]) ([76.68.36.84]) by toip5.srvr.bell.ca with ESMTP; 10 Mar 2010 17:18:36 -0500 Message-Id: <6B11D99B-AD2E-4B4E-A35F-9938D11992D3@ee.ryerson.ca> From: David Magda To: Dominic Fandrey In-Reply-To: <4B975765.1080907@bsdforen.de> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v936) Date: Wed, 10 Mar 2010 17:19:26 -0500 References: <4B9601D9.5030101@bsdforen.de> <20100309210949.F85436@sola.nimnet.asn.au> <4B964A7C.6010709@bsdforen.de> <20100309133045.GA7232@icarus.home.lan> <20100309133322.GA7362@icarus.home.lan> <4B975765.1080907@bsdforen.de> X-Mailer: Apple Mail (2.936) Cc: FreeBSD Stable Subject: Re: ntpd does not re-query servers, when a new interface appears X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 22:19:28 -0000 On Mar 10, 2010, at 03:25, Dominic Fandrey wrote: > In the meantime, your comments made me realize, that I can circumvent > this problem by adding the ntp pools to my /etc/hosts file. Up to a point: using DNS, the results round-robin--which helps the server operators--and dead servers are also removed from the pool automatically (AFAIK). You'll lose the latter with a static host table, which may affect things if things break upstream. From owner-freebsd-stable@FreeBSD.ORG Wed Mar 10 22:27:31 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C087D1065674 for ; Wed, 10 Mar 2010 22:27:31 +0000 (UTC) (envelope-from kamikaze@bsdforen.de) Received: from mail.bsdforen.de (bsdforen.de [212.204.60.79]) by mx1.freebsd.org (Postfix) with ESMTP id 7E4918FC19 for ; Wed, 10 Mar 2010 22:27:31 +0000 (UTC) Received: from mobileKamikaze.norad (unknown [88.130.200.31]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.bsdforen.de (Postfix) with ESMTP id 3CD9A8A17F9; Wed, 10 Mar 2010 23:27:30 +0100 (CET) Message-ID: <4B981CD1.2000202@bsdforen.de> Date: Wed, 10 Mar 2010 23:27:29 +0100 From: Dominic Fandrey User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-GB; rv:1.9.1.8) Gecko/20100302 Thunderbird/3.0.3 MIME-Version: 1.0 To: David Magda References: <4B9601D9.5030101@bsdforen.de> <20100309210949.F85436@sola.nimnet.asn.au> <4B964A7C.6010709@bsdforen.de> <20100309133045.GA7232@icarus.home.lan> <20100309133322.GA7362@icarus.home.lan> <4B975765.1080907@bsdforen.de> <6B11D99B-AD2E-4B4E-A35F-9938D11992D3@ee.ryerson.ca> In-Reply-To: <6B11D99B-AD2E-4B4E-A35F-9938D11992D3@ee.ryerson.ca> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: FreeBSD Stable Subject: Re: ntpd does not re-query servers, when a new interface appears X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Mar 2010 22:27:31 -0000 On 10/03/2010 23:19, David Magda wrote: > On Mar 10, 2010, at 03:25, Dominic Fandrey wrote: > >> In the meantime, your comments made me realize, that I can circumvent >> this problem by adding the ntp pools to my /etc/hosts file. > > Up to a point: using DNS, the results round-robin--which helps the > server operators--and dead servers are also removed from the pool > automatically (AFAIK). > > You'll lose the latter with a static host table, which may affect things > if things break upstream. I checked that. I just added the IPs of the pools. As soon as I get online, the pools still serve me different IPs every time. -- A: Because it fouls the order in which people normally read text. Q: Why is top-posting such a bad thing? A: Top-posting. Q: What is the most annoying thing on usenet and in e-mail? From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 07:45:38 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A985E1065672; Thu, 11 Mar 2010 07:45:38 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 367388FC27; Thu, 11 Mar 2010 07:45:37 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2DD69.dip.t-dialin.net [217.226.221.105]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id D27708444D8; Thu, 11 Mar 2010 08:45:30 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 8333F5402; Thu, 11 Mar 2010 08:45:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1268293527; bh=CULMdTKHhWLorRPNyx8Tmm4zgUkFlilXiK9S8maKbvc=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=0OUAAca2VtQKrzprenZ3TcjifLVYWsF4i1eiuT9Qb58aMZhnOuMECIbIHmbQa7hJq LSInvSodmUywg0PwEslXAl2Or2DrDyjqoiUfThzj8r84Fc20e2xLTM2jmYEwRIw4W/ uWh/mzAmZdh+Wm2+RyX8/WgWlgeBiroKCxzmi8VOUENYtjfkF4Fo6YYk7fBhf3bVf/ FZVYsyvSqYbPR0qctCaXkwlEhbGBUBb4bMStUFIYsptzo0Uve4w4bUVTkHKrW7ZDEM 2PR7l+I43IINwwq96knryZSVaTs7WwmAFJVmMowKDep4dJa41JQdhRNQLiZZ6W6oFN oPfSNibwSKwcA== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2B7jRka056645; Thu, 11 Mar 2010 08:45:27 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Thu, 11 Mar 2010 08:45:27 +0100 Message-ID: <20100311084527.2934034895hvgxaw@webmail.leidinger.net> Date: Thu, 11 Mar 2010 08:45:27 +0100 From: Alexander Leidinger To: Pawel Jakub Dawidek References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> <20100310173143.GD1715@garage.freebsd.pl> In-Reply-To: <20100310173143.GD1715@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: D27708444D8.58B03 X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.286, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, TW_DV 0.08, TW_ZF 0.08) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1268898333.40525@YCCAFIHqqWQfp0/PGkOy+Q X-EBL-Spam-Status: No Cc: freebsd-fs@FreeBSD.org, Stable , FreeBSD, Borja Marcos Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 07:45:38 -0000 Quoting Pawel Jakub Dawidek (from Wed, 10 Mar 2010 18:31:43 +0100): > On Wed, Mar 10, 2010 at 04:12:36PM +0100, Borja Marcos wrote: >> >> On Mar 10, 2010, at 12:02 PM, Pawel Jakub Dawidek wrote: >> >> > Once the deadlock occur, enter DDB and send me the output of: >> > >> > ps >> > show alllocks >> > show lockedvnods >> > show allchains >> > alltrace >> >> (Again, crossposted to -fs, ZFS related) >> >> >> Previous one was a panic when performing the test with several tar >> jobs running in parallel. >> >> Now this is a capture of the deadlock itself, instead of a panic. >> (I called panic from the debugger to generate a dump) > [...] > > Hmm, interesting. Especially those two traces: > > Tracing command zfs pid 1820 tid 100105 td 0xffffff0002ca4000 > [...] > _cv_wait() at _cv_wait+0x17a > txg_wait_synced() at txg_wait_synced+0x98 > zfsvfs_teardown() at zfsvfs_teardown+0x1f6 > zfs_suspend_fs() at zfs_suspend_fs+0x2b > zfs_ioc_recv() at zfs_ioc_recv+0x28b > zfsdev_ioctl() at zfsdev_ioctl+0x8d > devfs_ioctl_f() at devfs_ioctl_f+0x76 > kern_ioctl() at kern_ioctl+0xc5 > ioctl() at ioctl+0xfd > [...] > > Tracing command bsdtar pid 1699 tid 100093 td 0xffffff000262dae0 > [...] > _sx_slock_hard() at _sx_slock_hard+0x1b7 > _sx_slock() at _sx_slock+0xc1 > zfs_freebsd_reclaim() at zfs_freebsd_reclaim+0x63 > VOP_RECLAIM_APV() at VOP_RECLAIM_APV+0xb5 > vgonel() at vgonel+0x119 > vnlru_free() at vnlru_free+0x345 > getnewvnode() at getnewvnode+0x24f > zfs_znode_cache_constructor() at zfs_znode_cache_constructor+0x43 > zfs_znode_alloc() at zfs_znode_alloc+0x38 > zfs_mknode() at zfs_mknode+0x259 > zfs_freebsd_create() at zfs_freebsd_create+0x661 > VOP_CREATE_APV() at VOP_CREATE_APV+0xb3 > vn_open_cred() at vn_open_cred+0x473 > kern_openat() at kern_openat+0x179 > [...] > > This should be impossible. If we are that deep in zfsvfs_teardown(), it means > that we hold the z_teardown_lock exclusively. And we do as 'show alllocks' > output confirms. But if we are holding this lock exclusively we shouldn't be > that deep in create code path, because we need hold this lock as reader. > It isn't visible in 'show alllocks' output, because this lock is special > (rrwlock.c). > > I see three possibilities: > 1. We are looking at different file systems here. But where is deadlock > coming from then? > 2. There is a bug in rrwlock.c. Highly unlikely I think. > 3. My thinking is incorrect somewhere. There is a 4th possibility, if you can rule out everything else: bugs in the CPU. I stumbled upon this with ZFS (but UFS was exposing the problem much faster). The problem in my case was that the BIOS was not recognizing the CPU and as such was not uploading microcode updates. Borja, can you confirm that the CPU is correctly announced in FreeBSD (just look at "dmesg | grep CPU:" output, if it tells you it is a AMD or Intel XXX CPU it is correctly detected by the BIOS)? Bye, Alexander. -- Kissing a fish is like smoking a bicycle. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 08:54:49 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 88CA7106564A; Thu, 11 Mar 2010 08:54:49 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from proxypop1.sarenet.es (proxypop1.sarenet.es [194.30.0.99]) by mx1.freebsd.org (Postfix) with ESMTP id D19048FC0A; Thu, 11 Mar 2010 08:54:48 +0000 (UTC) Received: from [172.16.1.204] (unknown [192.148.167.2]) by proxypop1.sarenet.es (Postfix) with ESMTP id 8F1635CDE; Thu, 11 Mar 2010 09:54:47 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <20100311084527.2934034895hvgxaw@webmail.leidinger.net> Date: Thu, 11 Mar 2010 09:54:47 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <764BD545-B86C-47DC-9004-964EB2216AF0@sarenet.es> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> <20100310173143.GD1715@garage.freebsd.pl> <20100311084527.2934034895hvgxaw@webmail.leidinger.net> To: Alexander Leidinger X-Mailer: Apple Mail (2.1077) Cc: freebsd-fs@FreeBSD.org, Stable , FreeBSD@FreeBSD.ORG, Pawel Jakub Dawidek Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 08:54:49 -0000 On Mar 11, 2010, at 8:45 AM, Alexander Leidinger wrote: > Quoting Pawel Jakub Dawidek (from Wed, 10 Mar 2010 = 18:31:43 +0100): >=20 > There is a 4th possibility, if you can rule out everything else: bugs = in the CPU. I stumbled upon this with ZFS (but UFS was exposing the = problem much faster). The problem in my case was that the BIOS was not = recognizing the CPU and as such was not uploading microcode updates. >=20 > Borja, can you confirm that the CPU is correctly announced in FreeBSD = (just look at "dmesg | grep CPU:" output, if it tells you it is a AMD or = Intel XXX CPU it is correctly detected by the BIOS)? A CPU bug? Weird. Very. Let me explain the whole history of this. We are using ZFS to maintain a couple of servers in an active/passive = arrangement. At 30 second intervals we create a snapshot on the master = server and send it to the slave. Actually I prefer this scheme to = drbd-style arrangements, but that's another story ;) We started our tests and soon ran into problems: deadlocked filesystem. = At one point I remember that the deadlock affected UFS as well, not only = ZFS. I mean, having both ZFS and UFS, the system also lost access to the = UFS filesystems when this happened. Looking at the hours when it happened, it turned out to be one or two of = these events: periodic scripts running (which, among other things, = traverse the whole filesystem) and/or a backup being made with Bacula. = Either way, there seemed to be a problem: read activity on a dataset on = which I was receiving a snapshot at the same time could lead to a = deadlock. I am sure I have never tried to receive two snapshots = simultaneously, etc. The replicating program guaratees it. As the servers had to be rolled into production, and such tests with = real servers can be quite time consuming, I set up a couple of FreeBSD = virtual machines, using VMWare Fusion (version 2 then, now version 3) on = a Macbook (Macbook 4,1 Intel Core2Duo, 2.1 GHz) and tried to reproduce = it. To reproduce it, I set up a "master" machine, with /usr/src and /usr/obj = on a dataset (pool/src), replicating it at 30 second intervals to = another virtual machine, the slave. On the slave, I launch "tar" in an = infinite loop, so that the contents of the replicated dataser (pool/src) = is copied to another dataset (pool/thecopy). With that running, and, remember, there are replications at 30 second = intervals (longer if a replication takes a long time, of course) I run a = make buildworld on the master machine. The destination soon gets = deadlocked. I have tried to fiddle with the virtual machine, for example, trying to = offer a single or dual core CPU, and there's no difference. With dual = cores it *seems* to deadlock earlier, but I'm not sure. For the latest = test results I've posted, I was using a single core CPU.=20 The original machines on which I detected the problem (problem I have = subsequently reproduced successfully on virtual machines running on = VMWare Fusion) are Dell PowerEdge 2950, and this is the CPU description: Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Xeon(R) CPU L5420 @ 2.50GHz (2496.25-MHz = K8-class CPU) Origin =3D "GenuineIntel" Id =3D 0x1067a Stepping =3D 10 = Features=3D0xbfebfbff = Features2=3D0x40ce3bd AMD Features=3D0x20100800 AMD Features2=3D0x1 TSC: P-state invariant real memory =3D 8589934592 (8192 MB) avail memory =3D 8250003456 (7867 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 1 package(s) x 8 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0: Changing APIC ID to 8 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on = acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 The virtual machine (VMWare Fusion 3.0.0, Macbook, Mac OS X 10.6.2) = reports this: Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Intel(R) Core(TM)2 Duo CPU T8100 @ 2.10GHz (2116.62-MHz = K8-class CPU) Origin =3D "GenuineIntel" Id =3D 0x10676 Stepping =3D 6 = Features=3D0xfebfbff Features2=3D0x80082201> AMD Features=3D0x20100800 AMD Features2=3D0x1 TSC: P-state invariant real memory =3D 1153433600 (1100 MB) avail memory =3D 1090441216 (1039 MB) ACPI APIC Table: MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 In order to compare to Solaris, I installed a virtual machine running = Solaris 10 as well, and used it as a target for the replication. The = same test didn't deadlock and it seemed to work like a charm. Sometimes I've tried to run more than one "tar" job in parallel instead = of just one. It just makes it deadlock earlier, no other difference. Any more tests I can do? Borja. From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 13:06:16 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4C7B6106566B for ; Thu, 11 Mar 2010 13:06:16 +0000 (UTC) (envelope-from gallasch@free.de) Received: from smtp.free.de (smtp.free.de [91.204.6.103]) by mx1.freebsd.org (Postfix) with ESMTP id 44D1F8FC14 for ; Thu, 11 Mar 2010 13:06:13 +0000 (UTC) Received: (qmail 14165 invoked from network); 11 Mar 2010 13:39:28 +0100 Received: from smtp.free.de (HELO orwell.free.de) (gallasch@free.de@[91.204.4.103]) (envelope-sender ) by smtp.free.de (qmail-ldap-1.03) with AES128-SHA encrypted SMTP for ; 11 Mar 2010 13:39:28 +0100 Date: Thu, 11 Mar 2010 13:39:16 +0100 From: Kai Gallasch To: freebsd-stable@FreeBSD.org Message-ID: <20100311133916.42ba69b0@orwell.free.de> X-Mailer: Claws Mail 3.7.3 (GTK+ 2.18.6; powerpc-apple-darwin9.8.0) X-Face: 7"x0zA5=*cXGZw-xjU<">'+!3(KXTUXZVLD42KVN{'go[UQr"Mc.e(XW92N8plZ(9x.{x; I<|95e+b&GH-36\15F~L$YD*Y +u}o&KV?6.%"mJIkaY3G>BKNt`1|Y+%K1P4t; 47D65&(Y7h5Ll-[ltkhamx.-; ,jggK'}oMpUgEHFG YQ"9oXKAl>!d,J}T{)@uxvfu?YFWC*\~h+,^f Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Mailman-Approved-At: Thu, 11 Mar 2010 13:23:37 +0000 Cc: freebsd-fs@FreeBSD.org Subject: proliant server lockups with freebsd-amd64-stable (2010-03-10) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 13:06:16 -0000 Hi. I have some trouble with an opteron server locking up spontaneously. It looses all networks connectivity and even through console I can get no shell. Lockups occur mostly under disk load (periodic daily, bacula backup running, make buildworld/buildkernel) and I can provoke them easily. The following debug options are currently set in kernel.conf: options KDB options DDB options GDB options INVARIANTS options INVARIANT_SUPPORT options WITNESS options WITNESS_SKIPSPIN options SW_WATCHDOG # not available with ULE scheduler #options MP_WATCHDOG options DEBUG_LOCKS options DEBUG_VFS_LOCKS There is no special tuning in sysctl.conf done. Yesterday the server locked up again and through "sysctl debug.kdb.panic=1" I was able to enter the debugger, close before the console locked up completely. Below (below dmesg output) are my debug findings: In the end I did a "call doadump" but writing the dump failed, so it really must be a low level problem. Could this be ciss-related (show allchains), or is the ciss device just another victim? Sorry for the lengthy debug output, but I am glad I had the chance to collect the data. Regards, Kai. BTW: CC'ing fs@ as it might be fs-related. -------- dmesg: -------- FreeBSD 8.0-STABLE #10: Wed Mar 10 10:52:19 CET 2010 root@sonnenkraft.free.de:/usr/obj/usr/src/sys/DL385-G2 amd64 WARNING: WITNESS option enabled, expect reduced performance. Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Quad-Core AMD Opteron(tm) Processor 2352 (2100.08-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Stepping = 3 Features=0x178bfbff Features2=0x802009 AMD Features=0xee500800 AMD Features2=0x7ff TSC: P-state invariant real memory = 21474836480 (20480 MB) avail memory = 20685017088 (19726 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 cpu4 (AP): APIC ID: 4 cpu5 (AP): APIC ID: 5 cpu6 (AP): APIC ID: 6 cpu7 (AP): APIC ID: 7 ioapic0 irqs 0-15 on motherboard ioapic1 irqs 16-31 on motherboard ioapic2 irqs 32-47 on motherboard kbd1 at kbdmux0 cryptosoft0: on motherboard acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <32-bit timer at 3.579545MHz> port 0x920-0x923 on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 900 pcib0: on acpi0 pci0: on pcib0 vgapci0: port 0x1000-0x10ff mem 0xe8000000-0xefffffff,0xf7ff0000-0xf7ffffff irq 44 at device 3.0 on pci0 pci0: at device 4.0 (no driver attached) pci0: at device 4.2 (no driver attached) uhci0: port 0x1800-0x181f irq 45 at device 4.4 on pci0 uhci0: [ITHREAD] usbus0: on uhci0 pci0: at device 4.6 (no driver attached) pcib1: at device 5.0 on pci0 pci1: on pcib1 pcib2: at device 13.0 on pci1 pci2: on pcib2 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x500-0x50f at device 6.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] isab0: at device 6.2 on pci0 isa0: on isab0 pcib3: irq 42 at device 15.0 on pci0 pci5: on pcib3 pcib4: irq 38 at device 16.0 on pci0 pci8: on pcib4 pcib5: irq 39 at device 17.0 on pci0 pci14: on pcib5 pcib6: irq 40 at device 18.0 on pci0 pci11: on pcib6 pcib7: irq 41 at device 19.0 on pci0 pci3: on pcib7 pcib8: at device 0.0 on pci3 pci4: on pcib8 bce0: mem 0xf8000000-0xf9ffffff irq 41 at device 0.0 on pci4 miibus0: on bce0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bce0: Ethernet address: 00:1b:78:38:dd:02 bce0: [ITHREAD] bce0: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C (1.9.6); Flags (SPLT |MSI|MFW); MFW () pcib9: on acpi0 pci64: on pcib9 pcib10: irq 36 at device 15.0 on pci64 pci67: on pcib10 pcib11: irq 32 at device 16.0 on pci64 pci70: on pcib11 ciss0: port 0x4000-0x40ff mem 0xfdf00000-0xfdffffff,0xfdef0000-0xfdef0fff irq 32 at device 0.0 on pci70 ciss0: PERFORMANT Transport ciss0: [ITHREAD] pcib12: irq 33 at device 17.0 on pci64 pci73: on pcib12 pcib13: irq 34 at device 18.0 on pci64 pci65: on pcib13 pcib14: at device 0.0 on pci65 pci66: on pcib14 bce1: mem 0xfa000000-0xfbffffff irq 34 at device 0.0 on pci66 miibus1: on bce1 brgphy1: PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bce1: Ethernet address: 00:1b:78:38:dd:00 bce1: [ITHREAD] bce1: ASIC (0x57081020); Rev (B2); Bus (PCI-X, 64-bit, 133MHz); B/C (1.9.6); Flags (SPLT |MSI|MFW); MFW () pcib15: irq 35 at device 19.0 on pci64 pci74: on pcib15 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse, device ID 3 atrtc0: port 0x70-0x71 on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart0: console (9600,n,8,1) cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 orm0: at iomem 0xc0000-0xcafff,0xcb000-0xcefff,0xe5000-0xe6fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 ppc0: cannot reserve I/O port range uart1: at port 0x2f8-0x2ff irq 3 on isa0 uart1: [FILTER] Timecounters tick every 1.000 msec IPsec: Initialized Security Association Processing. usbus0: 12Mbps Full Speed USB v1.0 acd0: CDRW at ata0-master UDMA33 ugen0.1: <(0x103c)> at usbus0 uhub0: <(0x103c) UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0 uhub0: 2 ports with 2 removable, self powered ugen0.2: at usbus0 ukbd0: on usbus0 kbd2 at ukbd0 ums0: on usbus0 ums0: 3 buttons and [XY] coordinates ID=0 da0 at ciss0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SCSI-5 device da0: 135.168MB/s transfers da0: Command Queueing enabled da0: 36863MB (75496320 512 byte sectors: 255H 32S/T 9252C) da1 at ciss0 bus 0 scbus0 target 1 lun 0 da1: Fixed Direct Access SCSI-5 device da1: 135.168MB/s transfers da1: Command Queueing enabled da1: 243098MB (497866080 512 byte sectors: 255H 32S/T 61013C) da2 at ciss0 bus 0 scbus0 target 2 lun 0 da2: Fixed Direct Access SCSI-5 device da2: 135.168MB/s transfers da2: Command Queueing enabled da2: 139979MB (286677120 512 byte sectors: 255H 32S/T 35132C) da3 at ciss0 bus 0 scbus0 target 3 lun 0 da3: Fixed Direct Access SCSI-5 device da3: 135.168MB/s transfers da3: Command Queueing enabled da3: 139979MB (286677120 512 byte sectors: 255H 32S/T 35132C) da4 at ciss0 bus 0 scbus0 target 4 lun 0 da4: Fixed Direct Access SCSI-5 device da4: 135.168MB/s transfers da4: Command Queueing enabled da4: 139979MB (286677120 512 byte sectors: 255H 32S/T 35132C) SMP: AP CPU #3 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #4 Launched! SMP: AP CPU #5 Launched! WARNING: WITNESS option enabled, expect reduced performance. Trying to mount root from ufs:/dev/da0s1a WARNING: / was not properly dismounted ZFS filesystem version 3 ZFS storage pool version 14 -------- dmesg: END -------- ----- (0) "praeludium" ----- sysctl debug.kdb.panic=1 panic: kdb_sysctl_panic cpuid = 4 KDB: enter: panic [thread pid 68597 tid 100536 ] Stopped at kdb_enter+0x3d: movq $0,0x5246b0(%rip) ----- (1) "db> ps" ----- pid ppid pgrp uid state wmesg wchan cmd 68597 49133 68597 0 R+ CPU 4 sysctl 68596 68595 68595 22 SJ nanslp 0xffffffff808ca5c8 sshd 68595 1774 68595 0 SsJ sbwait 0xffffff0080fcce8c sshd 68593 6292 68593 65534 Ss ufs 0xffffff0243dc3098 inetd 68588 6292 68588 65534 Ss ufs 0xffffff0243dc3098 inetd 68587 6292 68587 65534 Ss ufs 0xffffff0243dc3098 inetd 68579 4427 4403 8 SJ zio->io_ 0xffffff02626da2b8 sh 68566 49896 68565 0 SE+ getblk 0xffffff819da189d8 grep 68564 49284 68564 0 SV+ getblk 0xffffff819da38be8 csh 68560 6292 68560 65534 Ss ufs 0xffffff0243dc3098 inetd 68559 68558 68558 22 SJ nanslp 0xffffffff808ca5c8 sshd 68558 5931 68558 0 SsJ sbwait 0xffffff0015b62694 sshd 68553 6292 68553 65534 Ss ufs 0xffffff0243dc3098 inetd 68552 6292 68552 65534 Ss ufs 0xffffff0243dc3098 inetd 68534 6292 68534 65534 Ss ufs 0xffffff0243dc3098 inetd 68532 68531 2558 82 SJ zio->io_ 0xffffff01ee145588 qmail-queue 68531 2583 2558 82 SJ wait 0xffffff01188398c0 qmail-smtpd 68527 68526 68526 22 SJ nanslp 0xffffffff808ca5c8 sshd 68526 5931 68526 0 SsJ sbwait 0xffffff03222e893c sshd 68525 6292 68525 65534 Ss ufs 0xffffff0243dc3098 inetd 68522 6292 68522 65534 Ss ufs 0xffffff0243dc3098 inetd 68521 6292 68521 65534 Ss getblk 0xffffff819da23ff8 python 68511 3778 3778 125 SJ kqread 0xffffff0118b26b00 proxymap 68510 3778 3778 0 SJ nanslp 0xffffffff808ca5c8 smtpd 68497 68495 49299 0 T+ biowr 0xffffff819da35f40 mktemp 68496 68494 49299 0 TE+ getblk 0xffffff819da38be8 tsort 68495 68494 49299 0 T+ piperd 0xffffff03b2025888 sh 68494 68493 49299 0 T+ sh 68493 68140 49299 0 T+ piperd 0xffffff01d7200b60 sh 68140 68139 49299 0 T+ make 68139 68138 49299 0 T+ sh 68138 68066 49299 0 T+ make 68066 66665 49299 0 T+ sh 67917 67915 67915 0 S zio->io_ 0xffffff0243466df8 zfs 67915 67913 67915 0 Ss pause 0xffffff014c8e7500 csh 67913 1399 67913 0 Ss select 0xffffff0226d7d4c0 sshd 67519 2051 2051 80 SJ select 0xffffff0080b0cdc0 httpd 66665 49405 49299 0 T+ make 49896 49816 49896 0 S+ pause 0xffffff011883e0a0 csh 49816 49531 49816 666 S+ wait 0xffffff011832e8c0 su 49531 49530 49531 666 Ss+ pause 0xffffff00696df0a0 tcsh 49530 49511 49511 666 S select 0xffffff0226f1b0c0 sshd 49511 1399 49511 0 Ss sbwait 0xffffff00a1106be4 sshd 49405 49299 49299 0 T+ make 49299 49284 49299 0 T+ make 49284 49283 49284 0 S+ ppwait 0xffffff04ed859450 csh 49283 49280 49283 666 S+ wait 0xffffff014cbac000 su 49280 49279 49280 666 Ss+ pause 0xffffff0007c6a0a0 tcsh 49279 49277 49277 666 S select 0xffffff0080990b40 sshd 49277 1399 49277 0 Ss sbwait 0xffffff01184e53ec sshd 49133 6335 49133 0 S+ pause 0xffffff03222870a0 csh 48183 2051 2051 80 SJ select 0xffffff04495163c0 httpd 47840 2051 2051 80 SJ select 0xffffff011813a7c0 httpd 47422 47420 47417 1001 SJ sbwait 0xffffff0118d9a3ec initial thread 47420 47419 47417 1001 SJ wait 0xffffff008083f000 bash 47419 47418 47417 1001 SJ wait 0xffffff04edcfc8c0 sh 47418 1 47417 0 SJ wait 0xffffff0322283000 atrun 47399 2051 2051 80 SJ select 0xffffff0449ab3440 httpd 46967 2051 2051 80 SJ select 0xffffff00a13a4240 httpd 46370 3778 3778 125 SJ kqread 0xffffff0322660700 pickup 46325 46323 46325 19420 S+J pause 0xffffff011883d0a0 screen 46323 46322 46323 19420 Ss+J wait 0xffffff014caf5460 bash 46322 46319 46319 19420 SJ select 0xffffff00a1184cc0 sshd 46319 3904 46319 0 SsJ sbwait 0xffffff014c9eb144 sshd 46245 2051 2051 80 SJ zio->io_ 0xffffff0007e0adf8 httpd 45112 2051 2051 80 SJ select 0xffffff0080b1c940 httpd 45014 2051 2051 80 SJ select 0xffffff008000e2c0 httpd 43851 4776 4776 80 SJ accept 0xffffff03b218685e httpd 43056 1752 1752 80 SJ select 0xffffff0080970cc0 httpd 43051 1752 1752 80 SJ select 0xffffff0080ca21c0 httpd 42965 1752 1752 80 SJ select 0xffffff03b20a6340 httpd 42955 1752 1752 80 SJ select 0xffffff02165c1a40 httpd 42939 1752 1752 80 SJ select 0xffffff014c475a40 httpd 42933 1752 1752 80 SJ select 0xffffff04331705c0 httpd 42520 1752 1752 80 SJ select 0xffffff0080b7abc0 httpd 42519 1752 1752 80 SJ lockf 0xffffff0226cde000 httpd 42326 1752 1752 80 SJ select 0xffffff00a1214340 httpd 42072 1752 1752 80 SJ kqread 0xffffff02160ae900 httpd 38848 2051 2051 80 SJ zfs 0xffffff021649e098 httpd 29104 3778 3778 125 SJ kqread 0xffffff0118d9dd00 qmgr 23197 5314 5314 80 SJ accept 0xffffff0449b73dae httpd 17921 17919 17921 19420 S+J select 0xffffff0253ada840 irssi 17919 17872 17919 19420 Ss+J wait 0xffffff0118839000 bash 17874 17872 17874 19420 Ss+J select 0xffffff0080ae80c0 initial thread 17872 1 17872 19420 SsJ select 0xffffff00a157acc0 screen 11037 11035 11037 19421 S+J select 0xffffff0080b7d8c0 irssi 11035 10729 11035 19421 Ss+J wait 0xffffff00696de460 bash 10731 10729 10731 19421 Ss+J select 0xffffff0071c5d140 initial thread 10729 1 10729 19421 SsJ select 0xffffff0423e9c9c0 screen 6589 2832 2832 80 SJ kqread 0xffffff01c0c0ae00 httpd 6335 1 6335 0 Ss+ wait 0xffffff0322284000 login 6334 1 6334 0 Ss+ ttyin 0xffffff00078d6ca8 getty 6333 1 6333 0 Ss+ ttyin 0xffffff00078d54a8 getty 6332 1 6332 0 Ss+ ttyin 0xffffff00078c4ca8 getty 6331 1 6331 0 Ss+ ttyin 0xffffff00078c48a8 getty 6330 1 6330 0 Ss+ ttyin 0xffffff00078d70a8 getty 6329 1 6329 0 Ss+ ttyin 0xffffff00078d78a8 getty 6328 1 6328 0 Ss+ ttyin 0xffffff00078d58a8 getty 6327 1 6327 0 Ss+ ttyin 0xffffff00078d84a8 getty 6314 1 6314 0 Ss select 0xffffff04edcac0c0 bsnmpd 6292 1 6292 0 Ss select 0xffffff04490cddc0 inetd 6258 1 6258 0 Ss nanslp 0xffffffff808ca5c8 cron 6251 1 6251 25 Ss pause 0xffffff04ed858500 sendmail 6247 1 6247 0 Ss select 0xffffff04eda4e9c0 sendmail 6221 1 6221 0 Ss (threaded) bacula-fd 100681 S ucond 0xffffff0449b6c100 bacula-fd 100154 S select 0xffffff04eda4ecc0 bacula-fd 6191 1 6191 0 SsJ nanslp 0xffffffff808ca5c8 cron 6182 1 6182 0 SsJ select 0xffffff04eda4f040 sshd 6072 1 6072 0 SsJ select 0xffffff04ed9987c0 syslogd 5948 5918 5918 80 SJ (threaded) httpd 100665 S lockf 0xffffff014c630000 httpd 100664 S ucond 0xffffff04ed656480 httpd 100663 S ucond 0xffffff04ed656400 httpd 100662 S ucond 0xffffff04ed656300 httpd 100661 S select 0xffffff00808aac40 httpd 100660 S ucond 0xffffff04ed656380 httpd 100659 S ucond 0xffffff04ed656280 httpd 100658 S ucond 0xffffff04ed656100 httpd 100657 S ucond 0xffffff04ed656000 httpd 100613 S ucond 0xffffff04ed656180 httpd 100656 S select 0xffffff00808ab3c0 httpd 100654 S ucond 0xffffff04ed672d80 httpd 100652 S ucond 0xffffff04ed672e00 httpd 100650 S ucond 0xffffff04ed672d00 httpd 100648 S ucond 0xffffff04ed672c80 httpd 100646 S ucond 0xffffff04ed672b80 httpd 100644 S ucond 0xffffff04ed672c00 httpd 100641 S ucond 0xffffff027de26680 httpd 100640 S ucond 0xffffff027de26600 httpd 100638 S ucond 0xffffff027de26580 httpd 100636 S ucond 0xffffff02a3988680 httpd 100635 S ucond 0xffffff02a3988700 httpd 100632 S ucond 0xffffff02a3988780 httpd 100630 S ucond 0xffffff0449b6ba00 httpd 100628 S ucond 0xffffff0449b6b900 httpd 100626 S ucond 0xffffff0449b6c000 httpd 100069 S piperd 0xffffff03b23d1b60 httpd 5947 5918 5918 80 SJ (threaded) httpd 100655 S kqread 0xffffff0007f75c00 httpd 100653 S ucond 0xffffff0449514700 httpd 100651 S ucond 0xffffff0449514680 httpd 100649 S ucond 0xffffff0449514480 httpd 100647 S ucond 0xffffff0449514500 httpd 100645 S ucond 0xffffff0449514900 httpd 100643 S ucond 0xffffff0449514880 httpd 100642 S select 0xffffff00800115c0 httpd 100639 S ucond 0xffffff00154ae100 httpd 100637 S ucond 0xffffff01eeccdd00 httpd 100634 S ucond 0xffffff020a053880 httpd 100633 S ucond 0xffffff0015f5f680 httpd 100631 S ucond 0xffffff020a00e680 httpd 100629 S ucond 0xffffff01eed29100 httpd 100627 S ucond 0xffffff01eed2aa80 httpd 100625 S ucond 0xffffff02a3988800 httpd 100624 S ucond 0xffffff02a3988880 httpd 100623 S ucond 0xffffff02a3988900 httpd 100622 S ucond 0xffffff02a3988980 httpd 100620 S ucond 0xffffff02a3988a00 httpd 100619 S ucond 0xffffff02a3988a80 httpd 100618 S ucond 0xffffff02a3988b00 httpd 100617 S ucond 0xffffff02a3988b80 httpd 100616 S ucond 0xffffff0449b6b800 httpd 100615 S ucond 0xffffff0449b6bd80 httpd 100614 S ucond 0xffffff0449b6bb80 httpd 100458 S piperd 0xffffff03b23d1b60 httpd 5940 1 5940 0 SsJ nanslp 0xffffffff808ca5c8 cron 5931 1 5931 0 SsJ select 0xffffff044961b740 sshd 5918 1 5918 0 SsJ select 0xffffff043390c6c0 httpd 5868 5867 5867 5121 SJ (threaded) python2.4 100701 S ucond 0xffffff02a304c280 python2.4 100700 S ucond 0xffffff02a304c300 python2.4 100699 S ucond 0xffffff02a304c400 python2.4 100698 S ucond 0xffffff026c4afa00 python2.4 100168 S select 0xffffff01c048c0c0 initial thread 5867 1 5867 0 SsJ select 0xffffff04ed045dc0 python2.4 5801 1 5801 0 SsJ select 0xffffff04ed0a0cc0 syslogd 5674 1 5674 0 SsJ nanslp 0xffffffff808ca5c8 cron 5665 1 5665 0 SsJ select 0xffffff0449e871c0 sshd 5639 5573 5570 88 S+J (threaded) mysqld 100612 S sigwait 0xffffff8243587a68 mysqld 100609 S ucond 0xffffff03b21af800 mysqld 100611 S select 0xffffff04494e22c0 mysqld 100610 S select 0xffffff03b2a623c0 mysqld 100608 S ucond 0xffffff03b2036580 mysqld 100607 S ucond 0xffffff03b21af580 mysqld 100606 S ucond 0xffffff03b21af600 mysqld 100605 S ucond 0xffffff03b21af880 mysqld 100577 S select 0xffffff0449e882c0 initial thread 5573 1 5570 88 S+J wait 0xffffff03fe7d08c0 sh 5470 1 5470 0 SsJ select 0xffffff0449d1e240 syslogd 5469 5314 5314 80 SJ accept 0xffffff0449b73dae httpd 5468 5314 5314 80 SJ accept 0xffffff0449b73dae httpd 5390 5389 5314 0 SJ select 0xffffff0449c0a040 ruby18 5389 5314 5314 0 SJ (threaded) ApplicationPoolServ 100783 S sbwait 0xffffff00a129fe8c ApplicationPoolServ 100782 S sbwait 0xffffff00718b9144 ApplicationPoolServ 100604 S fifoow 0xffffff0433f6b358 ApplicationPoolServ 100603 S ucond 0xffffff03fe5ddd00 ApplicationPoolServ 100225 S sbwait 0xffffff03b20873ec initial thread 5336 1 5336 0 SsJ nanslp 0xffffffff808ca5c8 cron 5327 1 5327 0 SsJ select 0xffffff0449b6b240 sshd 5314 1 5314 0 SsJ select 0xffffff0449b88640 httpd 5245 1 5245 0 SsJ accept 0xffffff03b230185e vsftpd 5176 1 5176 0 SsJ select 0xffffff0449ab2ac0 syslogd 5065 1 5065 0 SsJ select 0xffffff0423062a40 inetd 5051 1 5051 0 SsJ nanslp 0xffffffff808ca5c8 cron 5042 1 5042 0 SsJ select 0xffffff0423dc9440 sshd 4932 1 4932 0 SsJ select 0xffffff04495ff940 syslogd 4915 4776 4776 80 SJ accept 0xffffff03b218685e httpd 4914 4776 4776 80 SJ accept 0xffffff03b218685e httpd 4913 4776 4776 80 SJ accept 0xffffff03b218685e httpd 4912 4776 4776 80 SJ accept 0xffffff03b218685e httpd 4911 4776 4776 80 SJ accept 0xffffff03b218685e httpd 4807 1 4807 0 SsJ nanslp 0xffffffff808ca5c8 cron 4800 1 4800 25 SsJ pause 0xffffff03fe7cc500 sendmail 4796 1 4796 0 SsJ select 0xffffff044951d240 sendmail 4789 1 4789 0 SsJ select 0xffffff0423580440 sshd 4776 1 4776 0 SsJ select 0xffffff032295a1c0 httpd 4726 4725 4725 0 SJ (threaded) varnishd 100600 S ucond 0xffffff02a3988c80 cache-worker 100599 S ucond 0xffffff02a3988d00 cache-worker 100598 S ucond 0xffffff02a3988d80 cache-worker 100597 S ucond 0xffffff02a3988e00 cache-worker 100596 S ucond 0xffffff02a3989000 cache-worker 100595 S ucond 0xffffff02a3989080 cache-worker 100594 S ucond 0xffffff02a3989100 cache-worker 100593 S ucond 0xffffff02a3989180 cache-worker 100592 S ucond 0xffffff02a3989200 cache-worker 100591 S select 0xffffff0433697bc0 cache-acceptor 100590 S kqread 0xffffff0216dd2600 cache-kqueue 100589 S ucond 0xffffff02a3989280 cache-worker 100588 S nanslp 0xffffffff808ca5c8 cache-timeout 100587 S ucond 0xffffff03fe6be180 wrk_herder 100586 S nanslp 0xffffffff808ca5c8 wrk_herdtimer 100407 S select 0xffffff044951d7c0 cache-main 4725 1 4725 0 SsJ select 0xffffff0449124940 varnishd 4709 4708 4708 7001 SJ (threaded) python 100849 S ucond 0xffffff01185bb580 python 100848 S ucond 0xffffff00808bea00 python 100541 S select 0xffffff04331708c0 initial thread 4708 1 4708 0 SsJ select 0xffffff04493a9240 python 4628 1 4628 0 SsJ select 0xffffff03feda70c0 syslogd 4524 4426 4426 8 SJ select 0xffffff03b295f3c0 innfeed 4523 4426 4426 8 SJ sbwait 0xffffff03b2bbf694 perl 4512 1 4512 0 SsJ select 0xffffff03b2f8fbc0 inetd 4498 1 4498 0 SsJ nanslp 0xffffffff808ca5c8 cron 4492 1 4492 25 SsJ pause 0xffffff0322be4500 sendmail 4488 1 4488 0 SsJ select 0xffffff03fe6be6c0 sendmail 4481 1 4481 0 SsJ select 0xffffff03fe6a12c0 sshd 4428 1 4403 8 S+J nanslp 0xffffffff808ca5c8 perl 4427 1 4403 8 S+J wait 0xffffff03b2aee000 sh 4426 1 4426 8 SsJ zio->io_ 0xffffff01ee1832b8 innd 4340 1 4340 0 SsJ select 0xffffff03b29f4640 syslogd 4207 1 4207 0 SsJ nanslp 0xffffffff808ca5c8 cron 4201 1 4201 25 SsJ pause 0xffffff03b2af6500 sendmail 4197 1 4197 0 SsJ select 0xffffff03b2d0fbc0 sendmail 4185 1 4185 0 SsJ select 0xffffff03b2a3a240 sshd 4175 1 4175 0 SsJ select 0xffffff03b2b7b0c0 perl 4103 1 4103 53 SsJ (threaded) named 100529 S kqread 0xffffff01ee4afd00 named 100528 S ucond 0xffffff02a3057500 named 100527 S ucond 0xffffff02a36d8280 named 100526 S ucond 0xffffff027dd68a80 named 100525 S ucond 0xffffff02a36d8700 named 100524 S ucond 0xffffff02a36d8500 named 100523 S ucond 0xffffff02a36d8300 named 100522 S ucond 0xffffff02a36d8400 named 100521 S ucond 0xffffff02a36d8b80 named 100520 S ucond 0xffffff02a36d8900 named 100406 S sigwait 0xffffff8243181a68 named 4098 1 4098 0 SsJ select 0xffffff03b27f3640 syslogd 3947 3937 24 85 S+J piperd 0xffffff0015427888 qmail-clean 3946 3937 24 86 S+J select 0xffffff03b25846c0 qmail-rspawn 3945 3937 24 0 S+J select 0xffffff03229248c0 qmail-lspawn 3944 3937 24 83 S+J piperd 0xffffff02e77b2b60 splogger 3937 1 24 87 S+J select 0xffffff032278fa40 qmail-send 3930 1 3930 0 SsJ nanslp 0xffffffff808ca5c8 cron 3904 1 3904 0 SsJ select 0xffffff03b22f9440 sshd 3855 1 3855 0 SsJ select 0xffffff02a3a81cc0 iscsi-target 3843 1 3843 0 SsJ select 0xffffff0322f483c0 perl 3835 3806 3805 88 S+J (threaded) mysqld 100487 S sigwait 0xffffff8243316a68 mysqld 100484 S ucond 0xffffff0322cea900 mysqld 100486 S select 0xffffff03b20ae7c0 mysqld 100485 S select 0xffffff03b2236a40 mysqld 100483 S ucond 0xffffff02a36d8680 mysqld 100482 S ucond 0xffffff02a36d9880 mysqld 100481 S ucond 0xffffff03b21af500 mysqld 100480 S ucond 0xffffff03b203f480 mysqld 100397 S select 0xffffff03b2036a40 initial thread 3806 1 3805 88 S+J wait 0xffffff0322be5460 sh 3792 3791 3791 65534 SJ kqread 0xffffff0216eec000 nginx 3791 1 3791 0 SsJ pause 0xffffff027dfe80a0 nginx 3778 1 3778 0 SsJ kqread 0xffffff02e7f14300 master 3702 1 3702 0 SsJ select 0xffffff0322cebdc0 perl 3552 1 3552 0 SsJ select 0xffffff032283bd40 syslogd 3406 1 3406 0 SsJ nanslp 0xffffffff808ca5c8 cron 3397 1 3397 0 SsJ select 0xffffff03220466c0 sshd 3396 3391 3371 85 SJ piperd 0xffffff02e77b22d8 qmail-clean 3395 3391 3371 86 SJ select 0xffffff03222b7dc0 qmail-rspawn 3394 3391 3371 0 SJ select 0xffffff032225fbc0 qmail-lspawn 3393 3386 3371 83 SJ piperd 0xffffff01d720eb60 multilog 3392 3385 3371 82 SJ accept 0xffffff02a3cb4dae tcpserver 3391 3383 3371 87 SJ select 0xffffff02e768cdc0 qmail-send 3390 3384 3371 83 SJ piperd 0xffffff026ce4e888 multilog 3386 3373 3371 0 SJ select 0xffffff032225dac0 supervise 3385 3373 3371 0 SJ select 0xffffff02e77cb640 supervise 3384 3373 3371 0 SJ select 0xffffff03222b4440 supervise 3383 3373 3371 0 SJ select 0xffffff02a3fb79c0 supervise 3374 1 3371 0 SJ piperd 0xffffff02e77b05b0 readproctitle 3373 1 3371 0 SJ nanslp 0xffffffff808ca5c8 svscan 3274 1 3274 0 SsJ select 0xffffff02a35a67c0 syslogd 3250 3116 3116 80 SJ (threaded) httpd 100448 S accept 0xffffff02a3afc5b6 httpd 100447 S ucond 0xffffff02e70df400 httpd 100446 S ucond 0xffffff02e7d3e500 httpd 100445 S ucond 0xffffff02e7ecfa00 httpd 100444 S ucond 0xffffff02e7ecf980 httpd 100443 S ucond 0xffffff02e7ecf900 httpd 100442 S ucond 0xffffff02e7ecf880 httpd 100441 S ucond 0xffffff02e7ecf800 httpd 100440 S ucond 0xffffff02e7ecf780 httpd 100439 S ucond 0xffffff02e7ecf700 httpd 100438 S ucond 0xffffff02e7ecf680 httpd 100437 S ucond 0xffffff02e7ecf600 httpd 100436 S ucond 0xffffff02e7ecf580 httpd 100435 S ucond 0xffffff02e7ecf500 httpd 100434 S ucond 0xffffff02e7ecf480 httpd 100433 S ucond 0xffffff02e7ecf400 httpd 100432 S ucond 0xffffff02a30e4780 httpd 100431 S ucond 0xffffff02a317dc00 httpd 100430 S ucond 0xffffff02a317db80 httpd 100429 S ucond 0xffffff02a317db00 httpd 100428 S ucond 0xffffff02a317da80 httpd 100427 S ucond 0xffffff02a317da00 httpd 100426 S ucond 0xffffff02a317d980 httpd 100425 S ucond 0xffffff02a317d900 httpd 100424 S ucond 0xffffff02a317d880 httpd 100423 S ucond 0xffffff02a317d800 httpd 100404 S piperd 0xffffff02e77b0888 httpd 3146 1 3146 0 SsJ nanslp 0xffffffff808ca5c8 cron 3140 1 3140 25 SsJ pause 0xffffff0015bb30a0 sendmail 3136 1 3136 0 SsJ select 0xffffff02e76ab7c0 sendmail 3129 1 3129 0 SsJ select 0xffffff02e7993840 sshd 3116 1 3116 0 SsJ select 0xffffff02e7b68640 httpd 3061 1 3061 0 SsJ select 0xffffff02e769c8c0 proftpd 2998 1 2998 0 SsJ select 0xffffff02e7517740 syslogd 2864 1 2864 0 SsJ nanslp 0xffffffff808ca5c8 cron 2858 1 2858 25 SsJ pause 0xffffff027dfea0a0 sendmail 2854 1 2854 0 SsJ select 0xffffff02a3c6d2c0 sendmail 2853 2832 2832 80 SJ lockf 0xffffff014c4a8180 httpd 2852 2832 2832 80 SJ lockf 0xffffff00a18c6200 httpd 2845 1 2845 0 SsJ select 0xffffff02a3cd5e40 sshd 2832 1 2832 0 SsJ select 0xffffff02a3a8eb40 httpd 2804 1 2804 0 SsJ (threaded) nscd 100396 S kqread 0xffffff02a3928700 nscd 100395 S kqread 0xffffff02a3928700 nscd 100394 S kqread 0xffffff02a3928700 nscd 100393 S kqread 0xffffff02a3928700 nscd 100392 S kqread 0xffffff02a3928700 nscd 100391 S kqread 0xffffff02a3928700 nscd 100390 S kqread 0xffffff02a3928700 nscd 100389 S kqread 0xffffff02a3928700 nscd 100195 S uwait 0xffffff0007d50080 nscd 2722 1 2722 0 SsJ select 0xffffff02a3921440 syslogd 2593 1 2593 0 SsJ nanslp 0xffffffff808ca5c8 cron 2592 2587 2558 85 SJ piperd 0xffffff01d720e000 qmail-clean 2591 2587 2558 86 SJ select 0xffffff02a3284ac0 qmail-rspawn 2590 2587 2558 0 SJ select 0xffffff02a317a840 qmail-lspawn 2589 2576 2558 83 SJ piperd 0xffffff0007cbbb60 multilog 2587 2575 2558 87 SJ select 0xffffff02a3436c40 qmail-send 2586 2574 2558 83 SJ piperd 0xffffff0007cbb2d8 multilog 2583 2573 2558 82 SJ accept 0xffffff020ac94b06 tcpserver 2577 1 2577 0 SsJ select 0xffffff02a31d64c0 sshd 2576 2560 2558 0 SJ select 0xffffff025375fd40 supervise 2575 2560 2558 0 SJ select 0xffffff01c0abacc0 supervise 2574 2560 2558 0 SJ select 0xffffff02a30a1d40 supervise 2573 2560 2558 0 SJ select 0xffffff02a31d6540 supervise 2561 1 2558 0 SJ piperd 0xffffff0015b81000 readproctitle 2560 1 2558 0 SJ nanslp 0xffffffff808ca5c8 svscan 2478 2316 2316 80 SJ (threaded) httpd 100380 S lockf 0xffffff00a1214280 httpd 100379 S ucond 0xffffff027de26980 httpd 100378 S ucond 0xffffff027de26900 httpd 100377 S ucond 0xffffff027de26880 httpd 100376 S ucond 0xffffff027de26800 httpd 100375 S ucond 0xffffff027de26780 httpd 100374 S ucond 0xffffff027de26700 httpd 100373 S ucond 0xffffff0007c12e00 httpd 100372 S ucond 0xffffff01c0dfc400 httpd 100371 S ucond 0xffffff01c0dfc380 httpd 100370 S ucond 0xffffff020af7ad00 httpd 100328 S ucond 0xffffff020af7ad80 httpd 100369 S ucond 0xffffff020af7ae00 httpd 100367 S ucond 0xffffff02a30e4b80 httpd 100365 S ucond 0xffffff02a30e4b00 httpd 100362 S ucond 0xffffff02a30e4a80 httpd 100361 S select 0xffffff008064b9c0 httpd 100359 S ucond 0xffffff02a30e4980 httpd 100357 S select 0xffffff0036219e40 httpd 100355 S select 0xffffff0253d004c0 httpd 100353 S select 0xffffff007169e1c0 httpd 100350 S select 0xffffff007176a740 httpd 100347 S ucond 0xffffff020af7b000 httpd 100344 S select 0xffffff04492e2dc0 httpd 100341 S ucond 0xffffff0007d4f000 httpd 100339 S select 0xffffff0449517bc0 httpd 100327 S piperd 0xffffff0007cc7888 httpd 2477 2316 2316 80 SJ (threaded) httpd 100368 S kqread 0xffffff021697e800 httpd 100366 S ucond 0xffffff02a317d600 httpd 100364 S ucond 0xffffff02a317d580 httpd 100363 S ucond 0xffffff02a317d500 httpd 100360 S ucond 0xffffff00153bb380 httpd 100358 S ucond 0xffffff027dee0b80 httpd 100356 S ucond 0xffffff027dee2980 httpd 100354 S ucond 0xffffff027dee1500 httpd 100352 S ucond 0xffffff027dee1180 httpd 100351 S ucond 0xffffff027dee1100 httpd 100349 S ucond 0xffffff027dee0c00 httpd 100348 S ucond 0xffffff027dee3000 httpd 100346 S ucond 0xffffff027d416280 httpd 100345 S ucond 0xffffff027dee3100 httpd 100343 S ucond 0xffffff027d5bd900 httpd 100342 S select 0xffffff008064fb40 httpd 100340 S ucond 0xffffff027dee2b80 httpd 100338 S ucond 0xffffff027dee2e00 httpd 100337 S ucond 0xffffff027ded3200 httpd 100336 S ucond 0xffffff027ded0680 httpd 100334 S ucond 0xffffff027dcfd080 httpd 100333 S select 0xffffff00413a5840 httpd 100332 S ucond 0xffffff027dee1680 httpd 100331 S ucond 0xffffff027dee1980 httpd 100330 S ucond 0xffffff027dee1580 httpd 100329 S ucond 0xffffff027dee1a00 httpd 100326 S piperd 0xffffff0007cc7888 httpd 2459 1 2459 0 SsJ select 0xffffff027ded02c0 syslogd 2338 1 2338 0 SsJ nanslp 0xffffffff808ca5c8 cron 2329 1 2329 0 SsJ select 0xffffff027daa36c0 sshd 2316 1 2316 0 SsJ select 0xffffff027dbb8740 httpd 2267 2266 2266 5201 SJ (threaded) python2.4 100583 S ucond 0xffffff027de26c80 python2.4 100582 S ucond 0xffffff027de26c00 python2.4 100581 S ucond 0xffffff027de26b80 python2.4 100580 S ucond 0xffffff027de26b00 python2.4 100183 S select 0xffffff0423bd6840 initial thread 2266 1 2266 0 SsJ select 0xffffff026ccaa940 python2.4 2205 2051 2051 80 SJ zfs 0xffffff021649e098 httpd 2200 1 2200 0 SsJ select 0xffffff026ce6b5c0 syslogd 2073 1 2073 0 SsJ nanslp 0xffffffff808ca5c8 cron 2064 1 2064 0 SsJ select 0xffffff026c82a9c0 sshd 2051 1 2051 0 SsJ select 0xffffff026caf0cc0 httpd 2001 2000 2000 5351 SJ (threaded) python 100585 S ucond 0xffffff03fe6be200 python 100584 S ucond 0xffffff03b2f8f700 python 100243 S select 0xffffff044910f640 initial thread 2000 1 2000 0 SsJ select 0xffffff00153423c0 python 1986 1985 1985 5301 SJ (threaded) python 100567 D zio->io_ 0xffffff0007dc8588 python 100566 S ucond 0xffffff02a3989380 python 100249 S select 0xffffff03b2f8f640 initial thread 1985 1 1985 0 SsJ select 0xffffff0243fbc540 python 1921 1 1921 0 SsJ select 0xffffff0015cd5d40 syslogd 1785 1 1785 0 SsJ nanslp 0xffffffff808ca5c8 cron 1774 1 1774 0 SsJ select 0xffffff020aadd7c0 sshd 1773 1752 1752 0 SJ piperd 0xffffff0007cb72d8 initial thread 1772 1752 1752 0 SJ piperd 0xffffff01c05fa5b0 initial thread 1771 1752 1752 0 SJ piperd 0xffffff01c05fa2d8 initial thread 1769 1752 1752 0 SJ piperd 0xffffff01c05fa000 initial thread 1768 1752 1752 0 SJ piperd 0xffffff01c05f9b60 initial thread 1767 1752 1752 0 SJ piperd 0xffffff01c05f9888 initial thread 1752 1 1752 0 SsJ select 0xffffff020aa0e3c0 httpd 1688 1687 1687 5241 SJ (threaded) python 100261 S ucond 0xffffff0007d4f200 python 100260 S ucond 0xffffff0007d4f180 python 100181 S select 0xffffff026c2aa7c0 initial thread 1687 1 1687 0 SsJ select 0xffffff01d7db7640 python 1679 1678 1678 5221 SJ (threaded) python2.4 100253 S ucond 0xffffff0015479180 python2.4 100252 S ucond 0xffffff00154ae400 python2.4 100177 S select 0xffffff01eef34b40 initial thread 1678 1 1678 0 SsJ select 0xffffff01c0625840 python2.4 1674 1673 1673 5141 SJ (threaded) python2.4 100259 S ucond 0xffffff020af7b280 python2.4 100258 S ucond 0xffffff020af7b300 python2.4 100169 S select 0xffffff0216c249c0 initial thread 1673 1 1673 0 SsJ select 0xffffff01c0ab9a40 python2.4 1669 1668 1668 5161 SJ (threaded) python2.4 100255 S ucond 0xffffff00154ae700 python2.4 100254 S ucond 0xffffff00154ae380 python2.4 100166 S select 0xffffff020ab9a340 initial thread 1668 1 1668 0 SsJ select 0xffffff01c06231c0 python2.4 1664 1663 1663 5181 SJ (threaded) python2.4 100257 S ucond 0xffffff020af7c100 python2.4 100256 S ucond 0xffffff020af7c000 python2.4 100171 S select 0xffffff01d7a133c0 initial thread 1663 1 1663 0 SsJ select 0xffffff01c048bdc0 python2.4 1592 1 1592 0 SsJ select 0xffffff0015f57140 syslogd 1399 1 1399 0 Ss select 0xffffff0015465dc0 sshd 1380 1 1380 1 Ss sbwait 0xffffff0015c0e694 rwhod 1344 1 1344 0 Ss select 0xffffff0015340dc0 ntpd 1255 1254 1254 0 S (threaded) nfsd 100165 S rpcsvc 0xffffff0007e99b20 nfsd: service 100164 S rpcsvc 0xffffff0007e99ba0 nfsd: service 100163 S rpcsvc 0xffffff0007e99c20 nfsd: service 100066 S rpcsvc 0xffffff001530ce20 nfsd: master 1254 1 1254 0 Ss accept 0xffffff0015b875b6 nfsd 1252 1 1252 0 Ss select 0xffffff0015340040 mountd 1132 1 1132 0 Ss select 0xffffff0007c10c40 rpcbind 1109 1 1109 0 Ss select 0xffffff0007f79040 syslogd 936 1 936 0 Ss select 0xffffff0015340a40 devd 903 1 903 0 Ss select 0xffffff0015465840 moused 632 1 632 0 Ss select 0xffffff0015465c40 ospfd 626 1 626 0 Ss select 0xffffff0007e9b3c0 zebra 43 0 0 0 SL (threaded) zfskern 100112 D zio->io_ 0xffffff024351e2b8 [txg_thread_enter] 100111 D tx->tx_q 0xffffff0007e2aa60 [txg_thread_enter] 100109 D vgeom:io 0xffffff0007f60a10 [vdev da3] 100108 D vgeom:io 0xffffff0007f60b90 [vdev da2] 100081 D l2arc_fe 0xffffffff80f11640 [l2arc_feed_thread] 100070 D arc_recl 0xffffffff80f09060 [arc_reclaim_thread] 23 0 0 0 RL [flowcleaner] 22 0 0 0 SL sdflush 0xffffffff80aa5518 [softdepflush] 21 0 0 0 SL vlruwt 0xffffff0007b91460 [vnlru] 20 0 0 0 SL zio->io_ 0xffffff026271fb28 [syncer] 19 0 0 0 SL psleep 0xffffffff80a8b648 [bufdaemon] 18 0 0 0 SL pgzero 0xffffffff80aa6fac [pagezero] 17 0 0 0 SL psleep 0xffffffff80aa6348 [vmdaemon] 16 0 0 0 SL psleep 0xffffffff80aa630c [pagedaemon] 15 0 0 0 SL jsw:wait 0xffffffff808c6a90 [g_journal switcher] 9 0 0 0 SL ccb_scan 0xffffffff808aa160 [xpt_thrd] 8 0 0 0 SL pftm 0xffffffff801d43a0 [pfpurge] 7 0 0 0 SL idle 0xffffff80005ec0f0 [ciss_notify0] 14 0 0 0 SL (threaded) usb 100042 D - 0xffffff80005d6ef0 [usbus0] 100041 D - 0xffffff80005d6e98 [usbus0] 100040 D - 0xffffff80005d6e40 [usbus0] 100039 D - 0xffffff80005d6de8 [usbus0] 6 0 0 0 SL crypto_r 0xffffffff80aa4340 [crypto returns] 5 0 0 0 SL crypto_w 0xffffffff80aa4300 [crypto] 13 0 0 0 SL - 0xffffffff808ca2a4 [yarrow] 4 0 0 0 LL *cissmtx 0xffffff04ed820c00 [g_down] 3 0 0 0 SL - 0xffffffff808c6a00 [g_up] 2 0 0 0 SL - 0xffffffff808c69f0 [g_event] 12 0 0 0 RL (threaded) intr 100051 Run CPU 5 [swi0: uart uart] 100050 I [irq12: psm0] 100049 I [irq1: atkbd0] 100048 I [irq258: bce1] 100046 L *cissmtx 0xffffff04ed820c00 [irq257: ciss0] 100045 I [irq256: bce0] 100044 I [irq15: ata1] 100043 I [irq14: ata0] 100038 I [irq45: uhci0] 100037 I [irq9: acpi0] 100035 I [swi6: task queue] 100034 I [swi6: Giant taskq] 100032 I [swi5: +] 100028 Run CPU 2 [swi2: cambio] 100020 I [swi4: clock] 100019 I [swi4: clock] 100018 I [swi4: clock] 100017 I [swi4: clock] 100016 I [swi4: clock] 100015 I [swi4: clock] 100014 I [swi4: clock] 100013 L *cissmtx 0xffffff04ed820c00 [swi4: clock] 100012 I [swi3: vm] 100011 I [swi1: netisr 0] 11 0 0 0 RL (threaded) idle 100010 Run CPU 0 [idle: cpu0] 100009 Run CPU 1 [idle: cpu1] 100008 CanRun [idle: cpu2] 100007 Run CPU 3 [idle: cpu3] 100006 CanRun [idle: cpu4] 100005 CanRun [idle: cpu5] 100004 Run CPU 6 [idle: cpu6] 100003 Run CPU 7 [idle: cpu7] 1 0 1 0 SLs wait 0xffffff00073468c0 [init] 10 0 0 0 SL audit_wo 0xffffffff80aa4870 [audit] 0 0 0 0 SLs (threaded) kernel 100131 D - 0xffffff00153f5500 [zil_clean] 100130 D - 0xffffff00153f5c80 [zil_clean] 100129 D - 0xffffff001538db80 [zil_clean] 100128 D - 0xffffff00153ba300 [zil_clean] 100127 D - 0xffffff00153bab80 [zil_clean] 100126 D - 0xffffff00153bb480 [zil_clean] 100125 D - 0xffffff00153bbb00 [zil_clean] 100124 D - 0xffffff00153bc400 [zil_clean] 100123 D - 0xffffff00153bd480 [zil_clean] 100122 D - 0xffffff001538c400 [zil_clean] 100121 D - 0xffffff0015343c00 [zil_clean] 100120 D - 0xffffff001538a500 [zil_clean] 100119 D - 0xffffff001538ad80 [zil_clean] 100118 D - 0xffffff001538b880 [zil_clean] 100117 D - 0xffffff001538c280 [zil_clean] 100116 D - 0xffffff001538ca80 [zil_clean] 100115 D - 0xffffff0015082600 [zil_clean] 100114 D - 0xffffff00150e9280 [zil_clean] 100110 D - 0xffffff0007f60080 [zfs_vn_rele_taskq] 100107 D - 0xffffff0007f60d00 [spa_zio] 100106 D - 0xffffff0007f60d80 [spa_zio] 100105 D - 0xffffff0007f60e00 [spa_zio] 100104 D - 0xffffff0007f61000 [spa_zio] 100103 D - 0xffffff0007f61080 [spa_zio] 100102 D - 0xffffff0007f61100 [spa_zio] 100101 D - 0xffffff0007f61180 [spa_zio] 100100 D - 0xffffff0007f61200 [spa_zio_7] 100099 D - 0xffffff0007f61200 [spa_zio_6] 100098 D - 0xffffff0007f61200 [spa_zio_5] 100097 D - 0xffffff0007f61200 [spa_zio_4] 100096 D - 0xffffff0007f61200 [spa_zio_3] 100095 D - 0xffffff0007f61200 [spa_zio_2] 100094 D - 0xffffff0007f61200 [spa_zio_1] 100093 D - 0xffffff0007f61200 [spa_zio_0] 100092 D - 0xffffff0007f61280 [spa_zio_7] 100091 D - 0xffffff0007f61280 [spa_zio_6] 100090 D - 0xffffff0007f61280 [spa_zio_5] 100089 D - 0xffffff0007f61280 [spa_zio_4] 100088 D - 0xffffff0007f61280 [spa_zio_3] 100087 D - 0xffffff0007f61280 [spa_zio_2] 100086 D - 0xffffff0007f61280 [spa_zio_1] 100085 D - 0xffffff0007f61280 [spa_zio_0] 100084 D - 0xffffff0007f61300 [spa_zio] 100083 D - 0xffffff0007f61380 [spa_zio] 100082 D - 0xffffff0007f61e00 [spa_zio] 100036 D - 0xffffff000742ac00 [kqueue taskq] 100033 D - 0xffffff000742b000 [thread taskq] 100031 D - 0xffffff000742b180 [acpi_task_2] 100030 D - 0xffffff000742b180 [acpi_task_1] 100029 D - 0xffffff000742b180 [acpi_task_0] 100024 D - 0xffffff0007343400 [firmware taskq] 100000 D sched 0xffffffff808c6ba0 [swapper] ----- (2) "db> bt" ----- Tracing pid 68597 tid 100536 td 0xffffff02a31a63a0 kdb_enter() at kdb_enter+0x3d panic() at panic+0x17b kdb_trap() at kdb_trap sysctl_root() at sysctl_root+0x113 userland_sysctl() at userland_sysctl+0x158 __sysctl() at __sysctl+0xaa syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (202, FreeBSD ELF64, __sysctl), rip = 0x80073cb6c, rsp = 0x7fffffffe248, rbp = 0x7fffffffe31c --- ----- (3) "db> show alllocks" ----- Process 68597 (sysctl) thread 0xffffff02a31a63a0 (100536) exclusive sleep mutex Giant (Giant) r = 0 (0xffffffff808c99a0) locked @ /usr/src/sys/kern/kern_sysctl.c:1417 shared sx sysctl lock (sysctl lock) r = 0 (0xffffffff808ca320) locked @ /usr/src/sys/kern/kern_sysctl.c:1521 Process 68595 (sshd) thread 0xffffff021610c740 (100207) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff0080fcce40) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 68593 (inetd) thread 0xffffff0015b753a0 (100213) shared lockmgr ufs (ufs) r = 0 (0xffffff00153f77e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68588 (inetd) thread 0xffffff02269333a0 (100878) shared lockmgr ufs (ufs) r = 0 (0xffffff00153f77e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68587 (inetd) thread 0xffffff04ed9733a0 (100758) shared lockmgr ufs (ufs) r = 0 (0xffffff00153f77e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68579 (sh) thread 0xffffff04ed96aae0 (100764) exclusive lockmgr zfs (zfs) r = 0 (0xffffff01d79247e8) locked @ /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1212 Process 68566 (grep) thread 0xffffff0015bb73a0 (100217) exclusive lockmgr ufs (ufs) r = 0 (0xffffff00153f7578) locked @ /usr/src/sys/kern/vfs_subr.c:2204 Process 68564 (csh) thread 0xffffff04ed974740 (100672) exclusive lockmgr ufs (ufs) r = 0 (0xffffff01ee7a6578) locked @ /usr/src/sys/ufs/ffs/ffs_vfsops.c:1465 shared lockmgr ufs (ufs) r = 0 (0xffffff00153f77e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68560 (inetd) thread 0xffffff0041398000 (100709) shared lockmgr ufs (ufs) r = 0 (0xffffff00153f77e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68558 (sshd) thread 0xffffff0007c6f3a0 (100141) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff0015b62648) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 68553 (inetd) thread 0xffffff02fb700000 (100891) shared lockmgr ufs (ufs) r = 0 (0xffffff00153f77e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68552 (inetd) thread 0xffffff020ab44ae0 (100250) shared lockmgr ufs (ufs) r = 0 (0xffffff00153f77e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68534 (inetd) thread 0xffffff0015ebc000 (100186) shared lockmgr ufs (ufs) r = 0 (0xffffff00153f77e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68532 (qmail-queue) thread 0xffffff0080909ae0 (100816) shared lockmgr zfs (zfs) r = 0 (0xffffff02260eb308) locked @ /usr/src/sys/kern/vfs_syscalls.c:3508 Process 68526 (sshd) thread 0xffffff0449bcf000 (100687) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff03222e88f0) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 68525 (inetd) thread 0xffffff01180d1ae0 (100882) shared lockmgr ufs (ufs) r = 0 (0xffffff00153f77e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68522 (inetd) thread 0xffffff00a11db740 (100853) shared lockmgr ufs (ufs) r = 0 (0xffffff00153f77e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68521 (python) thread 0xffffff02fb58b740 (100897) exclusive lockmgr ufs (ufs) r = 0 (0xffffff0243dc3098) locked @ /usr/src/sys/kern/vfs_subr.c:2204 Process 68497 (mktemp) thread 0xffffff026ce6f3a0 (100289) exclusive lockmgr bufwait (bufwait) r = 0 (0xffffff819da35fd8) locked @ /usr/src/sys/kern/vfs_bio.c:2559 exclusive lockmgr ufs (ufs) r = 0 (0xffffff04334ed098) locked @ /usr/src/sys/ufs/ffs/ffs_vfsops.c:1465 exclusive lockmgr ufs (ufs) r = 0 (0xffffff0015b467e8) locked @ /usr/src/sys/kern/vfs_subr.c:2091 Process 68496 (tsort) thread 0xffffff03fe6cc000 (100715) exclusive lockmgr ufs (ufs) r = 0 (0xffffff024d04b578) locked @ /usr/src/sys/kern/vfs_subr.c:2204 Process 49511 (sshd) thread 0xffffff008090f000 (100789) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff00a1106b98) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 49277 (sshd) thread 0xffffff0080907ae0 (100824) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff01184e53a0) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 47422 (mplayer) thread 0xffffff020ab103a0 (100235) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff0118d9a3a0) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 46319 (sshd) thread 0xffffff03fe6cdae0 (100559) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff014c9eb0f8) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 46245 (httpd) thread 0xffffff008090d740 (100798) exclusive lockmgr zfs (zfs) r = 0 (0xffffff021649e098) locked @ /usr/src/sys/kern/vfs_vnops.c:607 Process 5389 (ApplicationPoolServ) thread 0xffffff0041391740 (100783) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff00a129fe40) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 5389 (ApplicationPoolServ) thread 0xffffff0041391ae0 (100782) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff00718b90f8) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 5389 (ApplicationPoolServ) thread 0xffffff0007d6c3a0 (100225) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff03b20873a0) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 4523 (perl) thread 0xffffff03b2dcb000 (100570) exclusive sx so_rcv_sx (so_rcv_sx) r = 0 (0xffffff03b2bbf648) locked @ /usr/src/sys/kern/uipc_sockbuf.c:148 Process 4426 (innd) thread 0xffffff03229c9ae0 (100517) exclusive lockmgr zfs (zfs) r = 0 (0xffffff024dfaacc8) locked @ /usr/src/sys/modules/zfs/../../cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c:1212 Process 1986 (python) thread 0xffffff02a39fd740 (100567) shared lockmgr zfs (zfs) r = 0 (0xffffff0216f0da58) locked @ /usr/src/sys/kern/vfs_vnops.c:536 Process 20 (syncer) thread 0xffffff0007b993a0 (100059) exclusive lockmgr syncer (syncer) r = 0 (0xffffff00153b97e8) locked @ /usr/src/sys/kern/vfs_subr.c:1701 Process 12 (intr) thread 0xffffff000739c3a0 (100028) exclusive sleep mutex cissmtx (cissmtx) r = 0 (0xffffff80005ec148) locked @ /usr/src/sys/cam/cam_xpt.c:4732 ----- (4) "db> show allchains" ----- chain 1: thread 100023 (pid 4, g_down) blocked on lock 0xffffff80005ec148 (sleep mutex) "cissmtx" thread 100028 (pid 12, swi2: cambio) running on CPU 2 chain 2: thread 100046 (pid 12, irq257: ciss0) blocked on lock 0xffffff80005ec148 (sleep mutex) "cissmtx" thread 100028 (pid 12, swi2: cambio) running on CPU 2 chain 3: thread 100013 (pid 12, swi4: clock) blocked on lock 0xffffff80005ec148 (sleep mutex) "cissmtx" thread 100028 (pid 12, swi2: cambio) running on CPU 2 ----- (5) "db> show lockedvnods" ----- Locked vnodes 0xffffff0015b46750: tag ufs, type VDIR usecount 1, writecount 0, refcount 3 mountedhere 0 flags () v_object 0xffffff0015483a20 ref 0 pages 1 lock type ufs: EXCL by thread 0xffffff026ce6f3a0 (pid 68497) #0 0xffffffff8037fb3f at __lockmgr_args+0x4ef #1 0xffffffff805ae38c at ffs_lock+0x9c #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff8042816b at vget+0x7b #5 0xffffffff80415690 at cache_lookup+0x4e0 #6 0xffffffff804159b0 at vfs_cache_lookup+0xc0 #7 0xffffffff806414a7 at VOP_LOOKUP_APV+0xb7 #8 0xffffffff8041c633 at lookup+0x3d3 #9 0xffffffff8041d5d9 at namei+0x479 #10 0xffffffff804342c2 at vn_open_cred+0xc2 #11 0xffffffff80432679 at kern_openat+0x179 #12 0xffffffff80604ea2 at syscall+0x102 #13 0xffffffff805eade1 at Xfast_syscall+0xe1 ino 47109, on dev da0s1a 0xffffff04334ed000: tag ufs, type VREG usecount 1, writecount 0, refcount 1 mountedhere 0 flags () lock type ufs: EXCL by thread 0xffffff026ce6f3a0 (pid 68497) #0 0xffffffff8037fdc7 at __lockmgr_args+0x777 #1 0xffffffff805a987f at ffs_vgetf+0x1bf #2 0xffffffff8058f474 at ffs_valloc+0x174 #3 0xffffffff805b950c at ufs_makeinode+0x8c #4 0xffffffff80641333 at VOP_CREATE_APV+0xb3 #5 0xffffffff80434682 at vn_open_cred+0x482 #6 0xffffffff80432679 at kern_openat+0x179 #7 0xffffffff80604ea2 at syscall+0x102 #8 0xffffffff805eade1 at Xfast_syscall+0xe1 ino 49547, on dev da0s1a 0xffffff00153f7750: tag ufs, type VDIR usecount 10, writecount 0, refcount 13 mountedhere 0 flags () v_object 0xffffff0007cff6c0 ref 0 pages 2 lock type ufs: SHARED (count 10) #0 0xffffffff8037fb3f at __lockmgr_args+0x4ef #1 0xffffffff805ae38c at ffs_lock+0x9c #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff80435665 at vn_close+0xc5 #5 0xffffffff804357aa at vn_closefile+0x5a #6 0xffffffff8035f113 at _fdrop+0x23 #7 0xffffffff80360b7b at closef+0x5b #8 0xffffffff80361160 at kern_close+0x110 #9 0xffffffff80604ea2 at syscall+0x102 #10 0xffffffff805eade1 at Xfast_syscall+0xe1 ino 32897, on dev da0s1e 0xffffff00153f74e0: tag ufs, type VREG usecount 0, writecount 0, refcount 2 mountedhere 0 flags (VI_DOINGINACT) v_object 0xffffff001535bbd0 ref 0 pages 21 lock type ufs: EXCL by thread 0xffffff0015bb73a0 (pid 68566) #0 0xffffffff8037fdc7 at __lockmgr_args+0x777 #1 0xffffffff805ae38c at ffs_lock+0x9c #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff8042942e at vputx+0x2fe #5 0xffffffff8036be85 at exit1+0x415 #6 0xffffffff8036caae at sys_exit+0xe #7 0xffffffff80604ea2 at syscall+0x102 #8 0xffffffff805eade1 at Xfast_syscall+0xe1 ino 33428, on dev da0s1e 0xffffff0243dc3000: tag ufs, type VREG usecount 0, writecount 0, refcount 11 mountedhere 0 flags (VI_DOINGINACT) v_object 0xffffff00413156c0 ref 0 pages 3 lock type ufs: EXCL by thread 0xffffff02fb58b740 (pid 68521) with exclusive waiters pending #0 0xffffffff8037fdc7 at __lockmgr_args+0x777 #1 0xffffffff805ae38c at ffs_lock+0x9c #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff8042942e at vputx+0x2fe #5 0xffffffff8036a4bf at kern_execve+0xbbf #6 0xffffffff8036aded at execve+0x3d #7 0xffffffff80604ea2 at syscall+0x102 #8 0xffffffff805eade1 at Xfast_syscall+0xe1 ino 35212, on dev da0s1e 0xffffff024d04b4e0: tag ufs, type VREG usecount 0, writecount 0, refcount 2 mountedhere 0 flags (VI_DOINGINACT) v_object 0xffffff024d370a20 ref 0 pages 3 lock type ufs: EXCL by thread 0xffffff03fe6cc000 (pid 68496) #0 0xffffffff8037fdc7 at __lockmgr_args+0x777 #1 0xffffffff805ae38c at ffs_lock+0x9c #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff8042942e at vputx+0x2fe #5 0xffffffff8036be85 at exit1+0x415 #6 0xffffffff8039758f at sigexit+0x8f #7 0xffffffff80399660 at cursig+0 #8 0xffffffff803d3019 at ast+0x1a9 #9 0xffffffff805eba19 at doreti_ast+0x1f ino 35382, on dev da0s1e 0xffffff01ee7a64e0: tag ufs, type VNON usecount 1, writecount 0, refcount 1 mountedhere 0 flags () lock type ufs: EXCL by thread 0xffffff04ed974740 (pid 68564) #0 0xffffffff8037fdc7 at __lockmgr_args+0x777 #1 0xffffffff805a987f at ffs_vgetf+0x1bf #2 0xffffffff805b41a6 at ufs_lookup_+0xb46 #3 0xffffffff8063f24f at VOP_CACHEDLOOKUP_APV+0xaf #4 0xffffffff804159e0 at vfs_cache_lookup+0xf0 #5 0xffffffff806414a7 at VOP_LOOKUP_APV+0xb7 #6 0xffffffff8041c633 at lookup+0x3d3 #7 0xffffffff8041d5d9 at namei+0x479 #8 0xffffffff80369ba3 at kern_execve+0x2a3 #9 0xffffffff8036aded at execve+0x3d #10 0xffffffff80604ea2 at syscall+0x102 #11 0xffffffff805eade1 at Xfast_syscall+0xe1 ino 35352, on dev da0s1e 0xffffff00153b9750: tag syncer, type VNON usecount 1, writecount 0, refcount 2 mountedhere 0 flags () lock type syncer: EXCL by thread 0xffffff0007b993a0 (pid 20) #0 0xffffffff8037fdc7 at __lockmgr_args+0x777 #1 0xffffffff80417b79 at vop_stdlock+0x39 #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff80429a30 at sync_vnode+0x130 #5 0xffffffff80429cbc at sched_sync+0x1cc #6 0xffffffff8036d91a at fork_exit+0x12a #7 0xffffffff805eafde at fork_trampoline+0xe 0xffffff024dfaac30: tag zfs, type VDIR usecount 1, writecount 0, refcount 1 mountedhere 0 flags () lock type zfs: EXCL by thread 0xffffff03229c9ae0 (pid 4426) #0 0xffffffff8037fb3f at __lockmgr_args+0x4ef #1 0xffffffff80417b79 at vop_stdlock+0x39 #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff80ea2e22 at zfs_lookup+0x2c2 #5 0xffffffff80ea3c81 at zfs_freebsd_lookup+0x81 #6 0xffffffff8063f24f at VOP_CACHEDLOOKUP_APV+0xaf #7 0xffffffff804159e0 at vfs_cache_lookup+0xf0 #8 0xffffffff806414a7 at VOP_LOOKUP_APV+0xb7 #9 0xffffffff8041c633 at lookup+0x3d3 #10 0xffffffff8041d5d9 at namei+0x479 #11 0xffffffff804342c2 at vn_open_cred+0xc2 #12 0xffffffff80432679 at kern_openat+0x179 #13 0xffffffff80604ea2 at syscall+0x102 #14 0xffffffff805eade1 at Xfast_syscall+0xe1 0xffffff01d7924750: tag zfs, type VREG usecount 2, writecount 0, refcount 3 mountedhere 0 flags (VV_TEXT) v_object 0xffffff026c5d0e58 ref 1 pages 1 lock type zfs: EXCL by thread 0xffffff04ed96aae0 (pid 68579) #0 0xffffffff8037fdc7 at __lockmgr_args+0x777 #1 0xffffffff80417b79 at vop_stdlock+0x39 #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff80ea2e22 at zfs_lookup+0x2c2 #5 0xffffffff80ea3c81 at zfs_freebsd_lookup+0x81 #6 0xffffffff8063f24f at VOP_CACHEDLOOKUP_APV+0xaf #7 0xffffffff804159e0 at vfs_cache_lookup+0xf0 #8 0xffffffff806414a7 at VOP_LOOKUP_APV+0xb7 #9 0xffffffff8041c633 at lookup+0x3d3 #10 0xffffffff8041d5d9 at namei+0x479 #11 0xffffffff80369ba3 at kern_execve+0x2a3 #12 0xffffffff8036aded at execve+0x3d #13 0xffffffff80604ea2 at syscall+0x102 #14 0xffffffff805eade1 at Xfast_syscall+0xe1 0xffffff02260eb270: tag zfs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags () v_object 0xffffff026ca3bbd0 ref 0 pages 0 lock type zfs: SHARED (count 1) #0 0xffffffff8037fb3f at __lockmgr_args+0x4ef #1 0xffffffff80417b79 at vop_stdlock+0x39 #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff80433880 at fsync+0xc0 #5 0xffffffff80604ea2 at syscall+0x102 #6 0xffffffff805eade1 at Xfast_syscall+0xe1 0xffffff021649e000: tag zfs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags () v_object 0xffffff026cabe870 ref 0 pages 0 lock type zfs: EXCL by thread 0xffffff008090d740 (pid 46245) with exclusive waiters pending #0 0xffffffff8037fdc7 at __lockmgr_args+0x777 #1 0xffffffff80417b79 at vop_stdlock+0x39 #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff80436098 at vn_write+0x218 #5 0xffffffff803dbea5 at dofilewrite+0x85 #6 0xffffffff803dd6a0 at kern_writev+0x60 #7 0xffffffff803dd7a5 at write+0x55 #8 0xffffffff80604ea2 at syscall+0x102 #9 0xffffffff805eade1 at Xfast_syscall+0xe1 0xffffff0216f0d9c0: tag zfs, type VREG usecount 1, writecount 1, refcount 1 mountedhere 0 flags () v_object 0xffffff01d7910a20 ref 0 pages 0 lock type zfs: SHARED (count 1) #0 0xffffffff8037fb3f at __lockmgr_args+0x4ef #1 0xffffffff80417b79 at vop_stdlock+0x39 #2 0xffffffff806405cb at VOP_LOCK1_APV+0x9b #3 0xffffffff80434a87 at _vn_lock+0x57 #4 0xffffffff80436327 at vn_read+0x1c7 #5 0xffffffff803dc051 at dofileread+0xa1 #6 0xffffffff803dc380 at kern_readv+0x60 #7 0xffffffff803dc485 at read+0x55 #8 0xffffffff80604ea2 at syscall+0x102 #9 0xffffffff805eade1 at Xfast_syscall+0xe1 ----- (6) "db> alltrace" ----- (very long output, for the sake of completeness!) Tracing command sysctl pid 68597 tid 100536 td 0xffffff02a31a63a0 kdb_enter() at kdb_enter+0x3d panic() at panic+0x17b kdb_trap() at kdb_trap sysctl_root() at sysctl_root+0x113 userland_sysctl() at userland_sysctl+0x158 __sysctl() at __sysctl+0xaa syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (202, FreeBSD ELF64, __sysctl), rip = 0x80073cb6c, rsp = 0x7fffffffe248, rbp = 0x7fffffffe31c --- Tracing command sshd pid 68596 tid 100684 td 0xffffff0449bcfae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x8013bbb6c, rsp = 0x7fffffffa4d8, rbp = 0x26 --- Tracing command sshd pid 68595 tid 100207 td 0xffffff021610c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x8013d693c, rsp = 0x7fffffffdbd8, rbp = 0 --- Tracing command inetd pid 68593 tid 100213 td 0xffffff0015b753a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e ffs_lock() at ffs_lock+0x9c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b cache_lookup() at cache_lookup+0x4e0 vfs_cache_lookup() at vfs_cache_lookup+0xc0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 kern_execve() at kern_execve+0x2a3 execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x8009dc58c, rsp = 0x7fffffffdd78, rbp = 0x800c46200 --- Tracing command inetd pid 68588 tid 100878 td 0xffffff02269333a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e ffs_lock() at ffs_lock+0x9c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b cache_lookup() at cache_lookup+0x4e0 vfs_cache_lookup() at vfs_cache_lookup+0xc0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 kern_execve() at kern_execve+0x2a3 execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x8009dc58c, rsp = 0x7fffffffdd78, rbp = 0x800c46200 --- Tracing command inetd pid 68587 tid 100758 td 0xffffff04ed9733a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e ffs_lock() at ffs_lock+0x9c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b cache_lookup() at cache_lookup+0x4e0 vfs_cache_lookup() at vfs_cache_lookup+0xc0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 kern_execve() at kern_execve+0x2a3 execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x8009dc58c, rsp = 0x7fffffffdd78, rbp = 0x800c46200 --- Tracing command sh pid 68579 tid 100764 td 0xffffff04ed96aae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a zio_wait() at zio_wait+0x61 dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0x22a dmu_buf_hold_array() at dmu_buf_hold_array+0x62 dmu_read_uio() at dmu_read_uio+0x3f zfs_freebsd_read() at zfs_freebsd_read+0x5c1 VOP_READ_APV() at VOP_READ_APV+0xaf vnode_pager_generic_getpages() at vnode_pager_generic_getpages+0x3f3 VOP_GETPAGES_APV() at VOP_GETPAGES_APV+0xb5 vnode_pager_getpages() at vnode_pager_getpages+0x8c exec_map_first_page() at exec_map_first_page+0x13c kern_execve() at kern_execve+0x32e execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x80093558c, rsp = 0x7fffffffccc8, rbp = 0x800c20430 --- Tracing command grep pid 68566 tid 100217 td 0xffffff0015bb73a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e getblk() at getblk+0xf4 breadn() at breadn+0x3f bread() at bread+0x1e ffs_update() at ffs_update+0x122 ufs_inactive() at ufs_inactive+0x15e VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xb5 vinactive() at vinactive+0x90 vputx() at vputx+0x2de exit1() at exit1+0x415 sys_exit() at sys_exit+0xe syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (1, FreeBSD ELF64, sys_exit), rip = 0x800a32b2c, rsp = 0x7fffffffe9e8, rbp = 0 --- Tracing command csh pid 68564 tid 100672 td 0xffffff04ed974740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e getblk() at getblk+0xf4 breadn() at breadn+0x3f bread() at bread+0x1e ffs_vgetf() at ffs_vgetf+0x2d4 ufs_lookup_() at ufs_lookup_+0xb46 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xaf vfs_cache_lookup() at vfs_cache_lookup+0xf0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 kern_execve() at kern_execve+0x2a3 execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x80096458c, rsp = 0x7fffffffe5c8, rbp = 0x800c0d578 --- Tracing command inetd pid 68560 tid 100709 td 0xffffff0041398000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e ffs_lock() at ffs_lock+0x9c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b cache_lookup() at cache_lookup+0x4e0 vfs_cache_lookup() at vfs_cache_lookup+0xc0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 kern_execve() at kern_execve+0x2a3 execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x8009dc58c, rsp = 0x7fffffffdd78, rbp = 0x800c46200 --- Tracing command sshd pid 68559 tid 100741 td 0xffffff0041392000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x8013bbb6c, rsp = 0x7fffffffa4d8, rbp = 0x26 --- Tracing command sshd pid 68558 tid 100141 td 0xffffff0007c6f3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x8013d693c, rsp = 0x7fffffffdbd8, rbp = 0 --- Tracing command inetd pid 68553 tid 100891 td 0xffffff02fb700000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e ffs_lock() at ffs_lock+0x9c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b cache_lookup() at cache_lookup+0x4e0 vfs_cache_lookup() at vfs_cache_lookup+0xc0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 kern_execve() at kern_execve+0x2a3 execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x8009dc58c, rsp = 0x7fffffffdd78, rbp = 0x800c46200 --- Tracing command inetd pid 68552 tid 100250 td 0xffffff020ab44ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e ffs_lock() at ffs_lock+0x9c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b cache_lookup() at cache_lookup+0x4e0 vfs_cache_lookup() at vfs_cache_lookup+0xc0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 kern_execve() at kern_execve+0x2a3 execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x8009dc58c, rsp = 0x7fffffffdd78, rbp = 0x800c46200 --- Tracing command inetd pid 68534 tid 100186 td 0xffffff0015ebc000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e ffs_lock() at ffs_lock+0x9c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b cache_lookup() at cache_lookup+0x4e0 vfs_cache_lookup() at vfs_cache_lookup+0xc0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 kern_execve() at kern_execve+0x2a3 execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x8009dc58c, rsp = 0x7fffffffdd78, rbp = 0x800c46200 --- Tracing command qmail-queue pid 68532 tid 100816 td 0xffffff0080909ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a zio_wait() at zio_wait+0x61 zil_commit() at zil_commit+0x3e1 zfs_freebsd_fsync() at zfs_freebsd_fsync+0xd7 VOP_FSYNC_APV() at VOP_FSYNC_APV+0xb5 fsync() at fsync+0x148 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (95, FreeBSD ELF64, fsync), rip = 0x8007396bc, rsp = 0x7fffffffeb48, rbp = 0x7fffffffeba0 --- Tracing command qmail-smtpd pid 68531 tid 100777 td 0xffffff0118850000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x800aa55cc, rsp = 0x7fffffffea98, rbp = 0x7fffffffead4 --- Tracing command sshd pid 68527 tid 100621 td 0xffffff03b21f4000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x8013bbb6c, rsp = 0x7fffffffa4d8, rbp = 0x26 --- Tracing command sshd pid 68526 tid 100687 td 0xffffff0449bcf000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x8013d693c, rsp = 0x7fffffffdbd8, rbp = 0 --- Tracing command inetd pid 68525 tid 100882 td 0xffffff01180d1ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e ffs_lock() at ffs_lock+0x9c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b cache_lookup() at cache_lookup+0x4e0 vfs_cache_lookup() at vfs_cache_lookup+0xc0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 kern_execve() at kern_execve+0x2a3 execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x8009dc58c, rsp = 0x7fffffffdd78, rbp = 0x800c46200 --- Tracing command inetd pid 68522 tid 100853 td 0xffffff00a11db740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e ffs_lock() at ffs_lock+0x9c VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vget() at vget+0x7b cache_lookup() at cache_lookup+0x4e0 vfs_cache_lookup() at vfs_cache_lookup+0xc0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 kern_execve() at kern_execve+0x2a3 execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (59, FreeBSD ELF64, execve), rip = 0x8009dc58c, rsp = 0x7fffffffdd78, rbp = 0x800c46200 --- Tracing command python pid 68521 tid 100897 td 0xffffff02fb58b740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e getblk() at getblk+0xf4 breadn() at breadn+0x3f bread() at bread+0x1e ffs_update() at ffs_update+0x122 ufs_inactive() at ufs_inactive+0x15e VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xb5 vinactive() at vinactive+0x90 vputx() at vputx+0x2de kern_execve() at kern_execve+0xbbf execve() at execve+0x3d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x800605770, rsp = 0x7fffffffebf8, rbp = 0 --- Tracing command proxymap pid 68511 tid 100865 td 0xffffff01180d1740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800c49b9c, rsp = 0x7fffffffde38, rbp = 0x7fffffffde40 --- Tracing command smtpd pid 68510 tid 100890 td 0xffffff02fb7003a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x800c82b6c, rsp = 0x7fffffffe688, rbp = 0 --- Tracing command mktemp pid 68497 tid 100289 td 0xffffff026ce6f3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 bwait() at bwait+0x64 bufwait() at bufwait+0x23 bufwrite() at bufwrite+0x158 ffs_update() at ffs_update+0x1cc ufs_makeinode() at ufs_makeinode+0x1f4 VOP_CREATE_APV() at VOP_CREATE_APV+0xb3 vn_open_cred() at vn_open_cred+0x482 kern_openat() at kern_openat+0x179 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (5, FreeBSD ELF64, open), rip = 0x80072b70c, rsp = 0x7fffffffe228, rbp = 0x800a04092 --- Tracing command tsort pid 68496 tid 100715 td 0xffffff03fe6cc000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e getblk() at getblk+0xf4 breadn() at breadn+0x3f bread() at bread+0x1e ffs_update() at ffs_update+0x122 ufs_inactive() at ufs_inactive+0x15e VOP_INACTIVE_APV() at VOP_INACTIVE_APV+0xb5 vinactive() at vinactive+0x90 vputx() at vputx+0x2de exit1() at exit1+0x415 sigexit() at sigexit+0x8f cursig() at cursig ast() at ast+0x1a9 doreti_ast() at doreti_ast+0x1f Tracing command sh pid 68495 tid 100822 td 0xffffff00809083a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x8009bc93c, rsp = 0x7fffffffe0c8, rbp = 0x800c072b3 --- Tracing command sh pid 68494 tid 100730 td 0xffffff0080912740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2c0 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009355cc, rsp = 0x7fffffffe058, rbp = 0xffffffff --- Tracing command sh pid 68493 tid 100838 td 0xffffff00a11d9ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x8009bc93c, rsp = 0x7fffffffe1b8, rbp = 0x800c08891 --- Tracing command make pid 68140 tid 100245 td 0xffffff0216844ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2c0 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x41f89c, rsp = 0x7fffffffd888, rbp = 0x7fffffffd8d0 --- Tracing command sh pid 68139 tid 100138 td 0xffffff0007c70000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2c0 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009355cc, rsp = 0x7fffffffe1d8, rbp = 0xffffffff --- Tracing command make pid 68138 tid 100210 td 0xffffff01c04df000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2c0 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x41f89c, rsp = 0x7fffffffd988, rbp = 0x7fffffffd9d0 --- Tracing command sh pid 68066 tid 100831 td 0xffffff0080906000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2c0 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009355cc, rsp = 0x7fffffffe248, rbp = 0xffffffff --- Tracing command zfs pid 67917 tid 100843 td 0xffffff01180d0740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a zio_wait() at zio_wait+0x61 traverse_read() at traverse_read+0x1b3 find_block() at find_block+0x1c9 traverse_more() at traverse_more+0x22f traverse_dsl_dataset() at traverse_dsl_dataset+0x50 dmu_sendbackup() at dmu_sendbackup+0x1ea zfs_ioc_send() at zfs_ioc_send+0x171 zfsdev_ioctl() at zfsdev_ioctl+0x8d devfs_ioctl_f() at devfs_ioctl_f+0x76 kern_ioctl() at kern_ioctl+0xc5 ioctl() at ioctl+0xfd syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (54, FreeBSD ELF64, ioctl), rip = 0x800fe78dc, rsp = 0x7fffffffba18, rbp = 0x80121d600 --- Tracing command csh pid 67915 tid 100790 td 0xffffff008090eae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x80094d21c, rsp = 0x7fffffffe578, rbp = 0x800c5b400 --- Tracing command sshd pid 67913 tid 100881 td 0xffffff01186aa740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdc48, rbp = 0x7fffffffdcd0 --- Tracing command httpd pid 67519 tid 100736 td 0xffffff00413483a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa128, rbp = 0x7fffffffa238 --- Tracing command make pid 66665 tid 100519 td 0xffffff03229c93a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2c0 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x41f89c, rsp = 0x7fffffffd998, rbp = 0x7fffffffd9e0 --- Tracing command csh pid 49896 tid 100767 td 0xffffff04ed96a000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x80094d21c, rsp = 0x7fffffffe4c8, rbp = 0x800c5c600 --- Tracing command su pid 49816 tid 100759 td 0xffffff04ed973000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009e85cc, rsp = 0x7fffffffe5c8, rbp = 0xc2e8 --- Tracing command tcsh pid 49531 tid 100707 td 0xffffff04ed95aae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x80094d21c, rsp = 0x7fffffffe598, rbp = 0x800c5c200 --- Tracing command sshd pid 49530 tid 100868 td 0xffffff0226962ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdc48, rbp = 0x7fffffffdcd0 --- Tracing command sshd pid 49511 tid 100789 td 0xffffff008090f000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x8013d693c, rsp = 0x7fffffffdc68, rbp = 0 --- Tracing command make pid 49405 tid 100247 td 0xffffff02168443a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2c0 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x41f89c, rsp = 0x7fffffffdc28, rbp = 0x7fffffffdc70 --- Tracing command make pid 49299 tid 100858 td 0xffffff00a11d9000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2c0 sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x41f89c, rsp = 0x7fffffffdca8, rbp = 0x7fffffffdcf0 --- Tracing command csh pid 49284 tid 100502 td 0xffffff02a30e6740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a fork1() at fork1+0xfe2 vfork() at vfork+0x1c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (66, FreeBSD ELF64, vfork), rip = 0x80094f39d, rsp = 0x7fffffffe6c0, rbp = 0 --- Tracing command su pid 49283 tid 100829 td 0xffffff0080906740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009e85cc, rsp = 0x7fffffffe5c8, rbp = 0xc084 --- Tracing command tcsh pid 49280 tid 100139 td 0xffffff0007c6fae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x80094d21c, rsp = 0x7fffffffe598, rbp = 0x800c5c600 --- Tracing command sshd pid 49279 tid 100806 td 0xffffff008090a000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdc48, rbp = 0x7fffffffdcd0 --- Tracing command sshd pid 49277 tid 100824 td 0xffffff0080907ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x8013d693c, rsp = 0x7fffffffdc68, rbp = 0 --- Tracing command csh pid 49133 tid 100422 td 0xffffff02a3a27740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x80094d21c, rsp = 0x7fffffffe628, rbp = 0x800c5c300 --- Tracing command httpd pid 48183 tid 100494 td 0xffffff0322949000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa128, rbp = 0x7fffffffa238 --- Tracing command httpd pid 47840 tid 100875 td 0xffffff0226960000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa128, rbp = 0x7fffffffa238 --- Tracing command mplayer pid 47422 tid 100235 td 0xffffff020ab103a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f soreceive_generic() at soreceive_generic+0xf99 kern_recvit() at kern_recvit+0x1c1 recvit() at recvit+0x21 recvfrom() at recvfrom+0x82 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (29, FreeBSD ELF64, recvfrom), rip = 0x8019777ac, rsp = 0x7fffffffd908, rbp = 0x400 --- Tracing command bash pid 47420 tid 100719 td 0xffffff03fe6c93a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x800bb85cc, rsp = 0x7fffffffe468, rbp = 0xb93e --- Tracing command sh pid 47419 tid 100688 td 0xffffff02a31a2740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009355cc, rsp = 0x7fffffffea88, rbp = 0xb939 --- Tracing command atrun pid 47418 tid 100460 td 0xffffff02a3a20740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8008cd5cc, rsp = 0x7fffffffe9e8, rbp = 0x4 --- Tracing command httpd pid 47399 tid 100161 td 0xffffff0007ca4ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa128, rbp = 0x7fffffffa238 --- Tracing command httpd pid 46967 tid 100761 td 0xffffff04ed96e740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa128, rbp = 0x7fffffffa238 --- Tracing command pickup pid 46370 tid 100727 td 0xffffff00809133a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800c4eb9c, rsp = 0x7fffffffda18, rbp = 0x7fffffffda20 --- Tracing command screen pid 46325 tid 100773 td 0xffffff0118852000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x800a5721c, rsp = 0x7fffffffe3a8, rbp = 0x12 --- Tracing command bash pid 46323 tid 100836 td 0xffffff02a319d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x800b9b5cc, rsp = 0x7fffffffe8e8, rbp = 0x800e453c0 --- Tracing command sshd pid 46322 tid 100762 td 0xffffff04ed96e3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdc38, rbp = 0x7fffffffdcc0 --- Tracing command sshd pid 46319 tid 100559 td 0xffffff03fe6cdae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x8013d693c, rsp = 0x7fffffffdc58, rbp = 0 --- Tracing command httpd pid 46245 tid 100798 td 0xffffff008090d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a zio_wait() at zio_wait+0x61 dbuf_read() at dbuf_read+0x3e4 dmu_tx_check_ioerr() at dmu_tx_check_ioerr+0x9a dmu_tx_count_write() at dmu_tx_count_write+0x4c dmu_tx_hold_write() at dmu_tx_hold_write+0x4a zfs_freebsd_write() at zfs_freebsd_write+0x3ea VOP_WRITE_APV() at VOP_WRITE_APV+0x129 vn_write() at vn_write+0x2a2 dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x80117191c, rsp = 0x7fffffff61a8, rbp = 0x7fffffff82a0 --- Tracing command httpd pid 45112 tid 100546 td 0xffffff03229c9000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa128, rbp = 0x7fffffffa238 --- Tracing command httpd pid 45014 tid 100477 td 0xffffff032292d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa128, rbp = 0x7fffffffa238 --- Tracing command httpd pid 43851 tid 100230 td 0xffffff01c0ef0740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x800fd6c0c, rsp = 0x7fffffffe998, rbp = 0x7fffffffe9d8 --- Tracing command httpd pid 43056 tid 100667 td 0xffffff03b21f3740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa208, rbp = 0x7fffffffa308 --- Tracing command httpd pid 43051 tid 100731 td 0xffffff00809123a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa208, rbp = 0x7fffffffa308 --- Tracing command httpd pid 42965 tid 100788 td 0xffffff008090f3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa208, rbp = 0x7fffffffa308 --- Tracing command httpd pid 42955 tid 100785 td 0xffffff0118841000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa208, rbp = 0x7fffffffa308 --- Tracing command httpd pid 42939 tid 100805 td 0xffffff008090a3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa208, rbp = 0x7fffffffa308 --- Tracing command httpd pid 42933 tid 100280 td 0xffffff0015c2b740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa208, rbp = 0x7fffffffa308 --- Tracing command httpd pid 42520 tid 100718 td 0xffffff04ed9693a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa208, rbp = 0x7fffffffa308 --- Tracing command httpd pid 42519 tid 100703 td 0xffffff04ed95bae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 lf_advlockasync() at lf_advlockasync+0xddd lf_advlock() at lf_advlock+0x47 vop_stdadvlock() at vop_stdadvlock+0xb3 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0xb5 flock() at flock+0x133 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (131, FreeBSD ELF64, flock), rip = 0x801101fac, rsp = 0x7fffffffe978, rbp = 0x7fffffffe9c8 --- Tracing command httpd pid 42326 tid 100835 td 0xffffff02a318dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111cefc, rsp = 0x7fffffffa208, rbp = 0x7fffffffa308 --- Tracing command httpd pid 42072 tid 100724 td 0xffffff04ed968740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x801157b9c, rsp = 0x7fffffffe938, rbp = 0x7fffffffe9c8 --- Tracing command httpd pid 38848 tid 100839 td 0xffffff0041399ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vn_write() at vn_write+0x218 dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x80117191c, rsp = 0x7fffffff61a8, rbp = 0x7fffffff82a0 --- Tracing command qmgr pid 29104 tid 100191 td 0xffffff01c0f7d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800c5cb9c, rsp = 0x7fffffffd9e8, rbp = 0x7fffffffd9f0 --- Tracing command httpd pid 23197 tid 100456 td 0xffffff02a3a21740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x8012c4c0c, rsp = 0x7fffffffe858, rbp = 0x801613be0 --- Tracing command irssi pid 17921 tid 100244 td 0xffffff0216845000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80191fefc, rsp = 0x7fffffffe5d8, rbp = 0x2 --- Tracing command bash pid 17919 tid 100779 td 0xffffff011884f740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x800b9b5cc, rsp = 0x7fffffffe428, rbp = 0x800e55800 --- Tracing command centerim pid 17874 tid 100694 td 0xffffff0041398ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x801ccd8bc, rsp = 0x7fffffffd098, rbp = 0x5 --- Tracing command screen pid 17872 tid 100514 td 0xffffff0322918740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800af58bc, rsp = 0x7fffffffe288, rbp = 0x7fffffffe310 --- Tracing command irssi pid 11037 tid 100714 td 0xffffff04ed969740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80191fefc, rsp = 0x7fffffffe5f8, rbp = 0x2 --- Tracing command bash pid 11035 tid 100710 td 0xffffff04ed95a3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x800b9b5cc, rsp = 0x7fffffffe448, rbp = 0x800e5d0c0 --- Tracing command centerim pid 10731 tid 100199 td 0xffffff01c0ee53a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x801ccd8bc, rsp = 0x7fffffffd208, rbp = 0x4 --- Tracing command screen pid 10729 tid 100473 td 0xffffff0322d26000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800af58bc, rsp = 0x7fffffffe2b8, rbp = 0x7fffffffe340 --- Tracing command httpd pid 6589 tid 100691 td 0xffffff0041399740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x801170b9c, rsp = 0x7fffffffe958, rbp = 0x7fffffffe9fc --- Tracing command login pid 6335 tid 100457 td 0xffffff02a3a213a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009ea5cc, rsp = 0x7fffffffec48, rbp = 0 --- Tracing command getty pid 6334 tid 100556 td 0xffffff03fe6ce740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80084e93c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 6333 tid 100530 td 0xffffff02a3a0c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80084e93c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 6332 tid 100281 td 0xffffff0015c2b3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80084e93c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 6331 tid 100459 td 0xffffff02a3a20ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80084e93c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 6330 tid 100489 td 0xffffff02a30eb3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80084e93c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 6329 tid 100495 td 0xffffff0322901ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80084e93c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 6328 tid 100474 td 0xffffff032292dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80084e93c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command getty pid 6327 tid 100516 td 0xffffff0322918000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e tty_wait() at tty_wait+0x48 ttydisc_read() at ttydisc_read+0x2f1 ttydev_read() at ttydev_read+0xab devfs_read_f() at devfs_read_f+0x86 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80084e93c, rsp = 0x7fffffffecc8, rbp = 0 --- Tracing command bsnmpd pid 6314 tid 100167 td 0xffffff00153c6000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800a6b8bc, rsp = 0x7fffffffc3d8, rbp = 0x7fffffffc5cc --- Tracing command inetd pid 6292 tid 100506 td 0xffffff02a30e0ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800a638bc, rsp = 0x7fffffffdd88, rbp = 0 --- Tracing command cron pid 6258 tid 100174 td 0xffffff0015c053a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffeb28, rbp = 0x3c --- Tracing command sendmail pid 6251 tid 100674 td 0xffffff04ed974000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x800d3921c, rsp = 0x7fffffffcd78, rbp = 0x1 --- Tracing command sendmail pid 6247 tid 100668 td 0xffffff03b21f33a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800dd78bc, rsp = 0x7fffffffc1e8, rbp = 0x7fffffffc280 --- Tracing command bacula-fd pid 6221 tid 100681 td 0xffffff0449bb8740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f do_cv_wait() at do_cv_wait+0x57a __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800ef52ac, rsp = 0x7fffffbfee78, rbp = 0x801c741c0 --- Tracing command bacula-fd pid 6221 tid 100154 td 0xffffff0007cc4740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x801a158bc, rsp = 0x7fffffffdb98, rbp = 0x4 --- Tracing command cron pid 6191 tid 100679 td 0xffffff04ed97cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sshd pid 6182 tid 100680 td 0xffffff04ed97c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0 --- Tracing command syslogd pid 6072 tid 100286 td 0xffffff026cdfe000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command httpd pid 5948 tid 100665 td 0xffffff04ed821ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 lf_advlockasync() at lf_advlockasync+0xddd lf_advlock() at lf_advlock+0x47 vop_stdadvlock() at vop_stdadvlock+0xb3 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0xb5 flock() at flock+0x133 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (131, FreeBSD ELF64, flock), rip = 0x801104fac, rsp = 0x7ffffc7e4f08, rbp = 0x7ffffc7e4f7c --- Tracing command httpd pid 5948 tid 100664 td 0xffffff04ed823000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffc9e5e68, rbp = 0x804c14280 --- Tracing command httpd pid 5948 tid 100663 td 0xffffff04ed8233a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcbe6e68, rbp = 0x804c14440 --- Tracing command httpd pid 5948 tid 100662 td 0xffffff04ed823740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcde7e68, rbp = 0x804c14600 --- Tracing command httpd pid 5948 tid 100661 td 0xffffff04ed823ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7ffffcfe47b8, rbp = 0x7ffffcfe48b8 --- Tracing command httpd pid 5948 tid 100660 td 0xffffff04ed824000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd1e9e68, rbp = 0x804c14980 --- Tracing command httpd pid 5948 tid 100659 td 0xffffff04ed8243a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd3eae68, rbp = 0x804c14b40 --- Tracing command httpd pid 5948 tid 100658 td 0xffffff04ed824740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd5ebe68, rbp = 0x804c14d00 --- Tracing command httpd pid 5948 tid 100657 td 0xffffff04ed824ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd7ece68, rbp = 0x804c14ec0 --- Tracing command httpd pid 5948 tid 100613 td 0xffffff04ed825000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd9ede68, rbp = 0x804c15080 --- Tracing command httpd pid 5948 tid 100656 td 0xffffff04ed8253a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7ffffdbea7b8, rbp = 0x7ffffdbea8b8 --- Tracing command httpd pid 5948 tid 100654 td 0xffffff04ed825740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffddefe68, rbp = 0x804c15400 --- Tracing command httpd pid 5948 tid 100652 td 0xffffff04ed825ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffdff0e68, rbp = 0x804c155c0 --- Tracing command httpd pid 5948 tid 100650 td 0xffffff04ed826000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe1f1e68, rbp = 0x804c15780 --- Tracing command httpd pid 5948 tid 100648 td 0xffffff04ed8263a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe3f2e68, rbp = 0x804c15940 --- Tracing command httpd pid 5948 tid 100646 td 0xffffff04ed826740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe5f3e68, rbp = 0x804c15b00 --- Tracing command httpd pid 5948 tid 100644 td 0xffffff04ed826ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe7f4e68, rbp = 0x804c15cc0 --- Tracing command httpd pid 5948 tid 100641 td 0xffffff02a31a83a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe9f5e68, rbp = 0x804c15e80 --- Tracing command httpd pid 5948 tid 100640 td 0xffffff02a31a8740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffebf6e68, rbp = 0x804c16040 --- Tracing command httpd pid 5948 tid 100638 td 0xffffff02a31a8ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffedf7e68, rbp = 0x804c16200 --- Tracing command httpd pid 5948 tid 100636 td 0xffffff02a3856000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffeff8e68, rbp = 0x804c163c0 --- Tracing command httpd pid 5948 tid 100635 td 0xffffff02a38563a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff1f9e68, rbp = 0x804c16580 --- Tracing command httpd pid 5948 tid 100632 td 0xffffff02a3856740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff3fae68, rbp = 0x804c16740 --- Tracing command httpd pid 5948 tid 100630 td 0xffffff0449bb8ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff5fbe68, rbp = 0x804c16900 --- Tracing command httpd pid 5948 tid 100628 td 0xffffff0449bb9000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff7fce68, rbp = 0x804c16ac0 --- Tracing command httpd pid 5948 tid 100626 td 0xffffff0449bb93a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff9fde68, rbp = 0x804c16c80 --- Tracing command httpd pid 5948 tid 100069 td 0xffffff0007ca33a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80117493c, rsp = 0x7fffffffe948, rbp = 0 --- Tracing command httpd pid 5947 tid 100655 td 0xffffff04ed81a000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x80115ab9c, rsp = 0x7ffffc7e4ec8, rbp = 0x7ffffc7e4f7c --- Tracing command httpd pid 5947 tid 100653 td 0xffffff04ed81a3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffc9e5e68, rbp = 0x804c14280 --- Tracing command httpd pid 5947 tid 100651 td 0xffffff04ed81a740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcbe6e68, rbp = 0x804c14440 --- Tracing command httpd pid 5947 tid 100649 td 0xffffff04ed81aae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcde7e68, rbp = 0x804c14600 --- Tracing command httpd pid 5947 tid 100647 td 0xffffff04ed81c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcfe8e68, rbp = 0x804c147c0 --- Tracing command httpd pid 5947 tid 100645 td 0xffffff04ed81c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd1e9e68, rbp = 0x804c14980 --- Tracing command httpd pid 5947 tid 100643 td 0xffffff04ed81c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd3eae68, rbp = 0x804c14b40 --- Tracing command httpd pid 5947 tid 100642 td 0xffffff04ed81cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7ffffd5e77b8, rbp = 0x7ffffd5e78b8 --- Tracing command httpd pid 5947 tid 100639 td 0xffffff0015c05ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd7ece68, rbp = 0x804c14ec0 --- Tracing command httpd pid 5947 tid 100637 td 0xffffff01d7013000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd9ede68, rbp = 0x804c15080 --- Tracing command httpd pid 5947 tid 100634 td 0xffffff01d70133a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffdbeee68, rbp = 0x804c15240 --- Tracing command httpd pid 5947 tid 100633 td 0xffffff01d7013740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffddefe68, rbp = 0x804c15400 --- Tracing command httpd pid 5947 tid 100631 td 0xffffff01d7013ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffdff0e68, rbp = 0x804c155c0 --- Tracing command httpd pid 5947 tid 100629 td 0xffffff01c0cba000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe1f1e68, rbp = 0x804c15780 --- Tracing command httpd pid 5947 tid 100627 td 0xffffff01c0cba3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe3f2e68, rbp = 0x804c15940 --- Tracing command httpd pid 5947 tid 100625 td 0xffffff02a3856ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe5f3e68, rbp = 0x804c15b00 --- Tracing command httpd pid 5947 tid 100624 td 0xffffff02a397b000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe7f4e68, rbp = 0x804c15cc0 --- Tracing command httpd pid 5947 tid 100623 td 0xffffff02a397b3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe9f5e68, rbp = 0x804c15e80 --- Tracing command httpd pid 5947 tid 100622 td 0xffffff02a397b740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffebf6e68, rbp = 0x804c16040 --- Tracing command httpd pid 5947 tid 100620 td 0xffffff02a397bae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffedf7e68, rbp = 0x804c16200 --- Tracing command httpd pid 5947 tid 100619 td 0xffffff02a388e000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffeff8e68, rbp = 0x804c163c0 --- Tracing command httpd pid 5947 tid 100618 td 0xffffff02a388e3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff1f9e68, rbp = 0x804c16580 --- Tracing command httpd pid 5947 tid 100617 td 0xffffff02a388e740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff3fae68, rbp = 0x804c16740 --- Tracing command httpd pid 5947 tid 100616 td 0xffffff0449bb9740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff5fbe68, rbp = 0x804c16900 --- Tracing command httpd pid 5947 tid 100615 td 0xffffff0449bb9ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff7fce68, rbp = 0x804c16ac0 --- Tracing command httpd pid 5947 tid 100614 td 0xffffff0449bba000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff9fde68, rbp = 0x804c16c80 --- Tracing command httpd pid 5947 tid 100458 td 0xffffff02a3a21000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80117493c, rsp = 0x7fffffffe948, rbp = 0 --- Tracing command cron pid 5940 tid 100305 td 0xffffff02162ff3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sshd pid 5931 tid 100409 td 0xffffff026cd12000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0 --- Tracing command httpd pid 5918 tid 100198 td 0xffffff0007d18ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8011748bc, rsp = 0x7fffffffe9a8, rbp = 0 --- Tracing command python2.4 pid 5868 tid 100701 td 0xffffff02a30d6ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5c2ac, rsp = 0x7fffff8fb658, rbp = 0x8106cce40 --- Tracing command python2.4 pid 5868 tid 100700 td 0xffffff02a30d7000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5c2ac, rsp = 0x7fffff9fc658, rbp = 0x8108261c0 --- Tracing command python2.4 pid 5868 tid 100699 td 0xffffff02a30d73a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5c2ac, rsp = 0x7fffffafd658, rbp = 0x810826380 --- Tracing command python2.4 pid 5868 tid 100698 td 0xffffff02a30d7740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5c2ac, rsp = 0x7fffffbfe658, rbp = 0x810826540 --- Tracing command python2.4 pid 5868 tid 100168 td 0xffffff00153c5ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e558bc, rsp = 0x7fffffff1838, rbp = 0x12 --- Tracing command python2.4 pid 5867 tid 100152 td 0xffffff0007cc5000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e558bc, rsp = 0x7fffffff18d8, rbp = 0x4 --- Tracing command syslogd pid 5801 tid 100218 td 0xffffff0015bb7000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command cron pid 5674 tid 100500 td 0xffffff0322901000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sshd pid 5665 tid 100531 td 0xffffff02a3a0c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0x4 --- Tracing command mysqld pid 5639 tid 100612 td 0xffffff03b21f43a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigtimedwait() at kern_sigtimedwait+0x55d sigwait() at sigwait+0x74 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (429, FreeBSD ELF64, sigwait), rip = 0x801475b3c, rsp = 0x7ffffebf6f08, rbp = 0x80180a200 --- Tracing command mysqld pid 5639 tid 100609 td 0xffffff03b21f4740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x80131b2ac, rsp = 0x7ffffedf7e88, rbp = 0x80180a3c0 --- Tracing command mysqld pid 5639 tid 100611 td 0xffffff03b21f4ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8015148bc, rsp = 0x7ffffeff8f18, rbp = 0 --- Tracing command mysqld pid 5639 tid 100610 td 0xffffff03b21f5000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8015148bc, rsp = 0x7fffff1f9f08, rbp = 0 --- Tracing command mysqld pid 5639 tid 100608 td 0xffffff03b21f5740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x80131b2ac, rsp = 0x7fffff5fbbc8, rbp = 0x80180aac0 --- Tracing command mysqld pid 5639 tid 100607 td 0xffffff03b21f5ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x80131b2ac, rsp = 0x7fffff7fcbc8, rbp = 0x80180ac80 --- Tracing command mysqld pid 5639 tid 100606 td 0xffffff03b21f6000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x80131b2ac, rsp = 0x7fffff9fdbc8, rbp = 0x80180ae40 --- Tracing command mysqld pid 5639 tid 100605 td 0xffffff03b21f63a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x80131b2ac, rsp = 0x7fffffbfebc8, rbp = 0x802d161c0 --- Tracing command mysqld pid 5639 tid 100577 td 0xffffff03fe84f3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8015148bc, rsp = 0x7fffffffe3b8, rbp = 0xd --- Tracing command sh pid 5573 tid 100563 td 0xffffff03b2dd33a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009355cc, rsp = 0x7fffffffdf08, rbp = 0x15c2 --- Tracing command syslogd pid 5470 tid 100384 td 0xffffff027d9a8740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command httpd pid 5469 tid 100159 td 0xffffff0007ccb3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x8012c4c0c, rsp = 0x7fffffffe838, rbp = 0x801613be0 --- Tracing command httpd pid 5468 tid 100562 td 0xffffff03b2dd3740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x8012c4c0c, rsp = 0x7fffffffe838, rbp = 0x801613be0 --- Tracing command ruby18 pid 5390 tid 100229 td 0xffffff01c0ef0ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800c6c8bc, rsp = 0x7fffffff7598, rbp = 0x7fffffff7620 --- Tracing command ApplicationPoolServ pid 5389 tid 100783 td 0xffffff0041391740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x800f1893c, rsp = 0x7fffffb78628, rbp = 0x7fffffb786b0 --- Tracing command ApplicationPoolServ pid 5389 tid 100782 td 0xffffff0041391ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x800f1893c, rsp = 0x7fffffb99628, rbp = 0x7fffffb996b0 --- Tracing command ApplicationPoolServ pid 5389 tid 100604 td 0xffffff03fe6ccae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 fifo_open() at fifo_open+0x2f6 VOP_OPEN_APV() at VOP_OPEN_APV+0xaf vn_open_cred() at vn_open_cred+0x316 kern_openat() at kern_openat+0x179 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (5, FreeBSD ELF64, open), rip = 0x800f0970c, rsp = 0x7fffffbdd8b8, rbp = 0x8 --- Tracing command ApplicationPoolServ pid 5389 tid 100603 td 0xffffff03fe6cd000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f do_cv_wait() at do_cv_wait+0x57a __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8008e82ac, rsp = 0x7fffffbfe9b8, rbp = 0x80160ae40 --- Tracing command ApplicationPoolServ pid 5389 tid 100225 td 0xffffff0007d6c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x800f1893c, rsp = 0x7fffffffdeb8, rbp = 0x7fffffffdf40 --- Tracing command cron pid 5336 tid 100450 td 0xffffff02a3a27000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sshd pid 5327 tid 100557 td 0xffffff03fe6ce3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0x4 --- Tracing command httpd pid 5314 tid 100421 td 0xffffff02e7e5d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013288bc, rsp = 0x7fffffffe9d8, rbp = 0 --- Tracing command vsftpd pid 5245 tid 100503 td 0xffffff02a30e63a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x800df5c0c, rsp = 0x7fffffffe8e8, rbp = 0x7fffffffe980 --- Tracing command syslogd pid 5176 tid 100228 td 0xffffff0007c6c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command inetd pid 5065 tid 100560 td 0xffffff03fe6cd740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800a638bc, rsp = 0x7fffffffdd38, rbp = 0 --- Tracing command cron pid 5051 tid 100576 td 0xffffff03fe84f740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sshd pid 5042 tid 100385 td 0xffffff027d9a83a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0x4 --- Tracing command syslogd pid 4932 tid 100572 td 0xffffff03fe818740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command httpd pid 4915 tid 100303 td 0xffffff02162ffae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x800fd6c0c, rsp = 0x7fffffffe998, rbp = 0x7fffffffe9d8 --- Tracing command httpd pid 4914 tid 100420 td 0xffffff02e7e5dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x800fd6c0c, rsp = 0x7fffffffe998, rbp = 0x7fffffffe9d8 --- Tracing command httpd pid 4913 tid 100548 td 0xffffff0322be0740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x800fd6c0c, rsp = 0x7fffffffe998, rbp = 0x7fffffffe9d8 --- Tracing command httpd pid 4912 tid 100335 td 0xffffff02a318c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x800fd6c0c, rsp = 0x7fffffffe998, rbp = 0x7fffffffe9d8 --- Tracing command httpd pid 4911 tid 100547 td 0xffffff0322be0ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x800fd6c0c, rsp = 0x7fffffffe998, rbp = 0x7fffffffe9d8 --- Tracing command cron pid 4807 tid 100508 td 0xffffff0322bda000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3b --- Tracing command sendmail pid 4800 tid 100578 td 0xffffff03fe84f000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x800d3921c, rsp = 0x7fffffffccf8, rbp = 0x1 --- Tracing command sendmail pid 4796 tid 100551 td 0xffffff032200cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800dd78bc, rsp = 0x7fffffffc168, rbp = 0x7fffffffc200 --- Tracing command sshd pid 4789 tid 100569 td 0xffffff03b2dcb3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0x4 --- Tracing command httpd pid 4776 tid 100192 td 0xffffff01c0f7d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80103a8bc, rsp = 0x7fffffffea48, rbp = 0 --- Tracing command varnishd pid 4726 tid 100600 td 0xffffff02a39e6000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7ffffdfee968, rbp = 0x806413cc0 --- Tracing command varnishd pid 4726 tid 100599 td 0xffffff02a39e63a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7ffffe1ef968, rbp = 0x806413e80 --- Tracing command varnishd pid 4726 tid 100598 td 0xffffff02a39e6740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7ffffe3f0968, rbp = 0x806414040 --- Tracing command varnishd pid 4726 tid 100597 td 0xffffff02a39e6ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7ffffe5f1968, rbp = 0x806414200 --- Tracing command varnishd pid 4726 tid 100596 td 0xffffff02a39e7000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7ffffe7f2968, rbp = 0x8064143c0 --- Tracing command varnishd pid 4726 tid 100595 td 0xffffff02a39e73a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7ffffe9f3968, rbp = 0x806414580 --- Tracing command varnishd pid 4726 tid 100594 td 0xffffff02a39e7740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7ffffebf4968, rbp = 0x806414740 --- Tracing command varnishd pid 4726 tid 100593 td 0xffffff02a39e7ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7ffffedf5968, rbp = 0x806414900 --- Tracing command varnishd pid 4726 tid 100592 td 0xffffff02a39fd000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7ffffeff6968, rbp = 0x806414ac0 --- Tracing command varnishd pid 4726 tid 100591 td 0xffffff03fe786000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x800d8eefc, rsp = 0x7fffff1f9ea8, rbp = 0 --- Tracing command varnishd pid 4726 tid 100590 td 0xffffff03fe7863a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800dc9b9c, rsp = 0x7fffff3f9fa8, rbp = 0x80100ac88 --- Tracing command varnishd pid 4726 tid 100589 td 0xffffff02a39fd3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7fffff5f9968, rbp = 0x806414c80 --- Tracing command varnishd pid 4726 tid 100588 td 0xffffff03fe786740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x800dc8b6c, rsp = 0x7fffff7fc668, rbp = 0 --- Tracing command varnishd pid 4726 tid 100587 td 0xffffff03fe786ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800acb2ac, rsp = 0x7fffff9fdec8, rbp = 0x8011181c0 --- Tracing command varnishd pid 4726 tid 100586 td 0xffffff03fe7a4000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x800dc8b6c, rsp = 0x7fffffbfec78, rbp = 0x801118388 --- Tracing command varnishd pid 4726 tid 100407 td 0xffffff026cd12740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x800d8eefc, rsp = 0x7fffffffe5d8, rbp = 0x801017c40 --- Tracing command varnishd pid 4725 tid 100555 td 0xffffff03fe6ceae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x800d8eefc, rsp = 0x7fffffffe5e8, rbp = 0x801033080 --- Tracing command python pid 4709 tid 100849 td 0xffffff0118841ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c602ac, rsp = 0x7fffffafd668, rbp = 0x8146ebd40 --- Tracing command python pid 4709 tid 100848 td 0xffffff01188453a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c602ac, rsp = 0x7fffffbfe668, rbp = 0x8146ebf00 --- Tracing command python pid 4709 tid 100541 td 0xffffff03b2dceae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e598bc, rsp = 0x7ffffffeb158, rbp = 0x9 --- Tracing command python pid 4708 tid 100292 td 0xffffff026cdc5740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e598bc, rsp = 0x7ffffffee288, rbp = 0x4 --- Tracing command syslogd pid 4628 tid 100574 td 0xffffff03fe818000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command innfeed pid 4524 tid 100515 td 0xffffff03229183a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80077f8bc, rsp = 0x7fffffffd5e8, rbp = 0x800a28800 --- Tracing command perl pid 4523 tid 100570 td 0xffffff03b2dcb000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_generic() at soreceive_generic+0xf99 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x800cc393c, rsp = 0x7fffffffb7b8, rbp = 0 --- Tracing command inetd pid 4512 tid 100571 td 0xffffff03fe818ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800a638bc, rsp = 0x7fffffffdd18, rbp = 0 --- Tracing command cron pid 4498 tid 100282 td 0xffffff0015c2b000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sendmail pid 4492 tid 100467 td 0xffffff02a3a28740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x800d3921c, rsp = 0x7fffffffccf8, rbp = 0x1 --- Tracing command sendmail pid 4488 tid 100552 td 0xffffff032200c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800dd78bc, rsp = 0x7fffffffc168, rbp = 0x7fffffffc200 --- Tracing command sshd pid 4481 tid 100553 td 0xffffff032200c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0 --- Tracing command perl pid 4428 tid 100451 td 0xffffff02a3a24ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x800ca8b6c, rsp = 0x7fffffffd8f8, rbp = 0xcfb978 --- Tracing command sh pid 4427 tid 100543 td 0xffffff03b2dce3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009355cc, rsp = 0x7fffffffccc8, rbp = 0x1133 --- Tracing command innd pid 4426 tid 100517 td 0xffffff03229c9ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a zio_wait() at zio_wait+0x61 dbuf_read() at dbuf_read+0x3e4 dmu_buf_hold() at dmu_buf_hold+0xcc zap_lockdir() at zap_lockdir+0x55 zap_lookup_norm() at zap_lookup_norm+0x45 zap_lookup() at zap_lookup+0x2e zfs_dirent_lock() at zfs_dirent_lock+0x503 zfs_dirlook() at zfs_dirlook+0x69 zfs_lookup() at zfs_lookup+0x1f0 zfs_freebsd_lookup() at zfs_freebsd_lookup+0x81 VOP_CACHEDLOOKUP_APV() at VOP_CACHEDLOOKUP_APV+0xaf vfs_cache_lookup() at vfs_cache_lookup+0xf0 VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xb7 lookup() at lookup+0x3d3 namei() at namei+0x479 vn_open_cred() at vn_open_cred+0xc2 kern_openat() at kern_openat+0x179 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (5, FreeBSD ELF64, open), rip = 0x800d1370c, rsp = 0x7fffffffcce8, rbp = 0 --- Tracing command syslogd pid 4340 tid 100542 td 0xffffff03b2dce740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command cron pid 4207 tid 100476 td 0xffffff032292d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sendmail pid 4201 tid 100533 td 0xffffff02a39ffae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x800d3921c, rsp = 0x7fffffffccf8, rbp = 0x1 --- Tracing command sendmail pid 4197 tid 100532 td 0xffffff02a3a0c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800dd78bc, rsp = 0x7fffffffc168, rbp = 0x7fffffffc200 --- Tracing command sshd pid 4185 tid 100507 td 0xffffff0322bda3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd48, rbp = 0x4 --- Tracing command perl pid 4175 tid 100475 td 0xffffff032292d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800cb48bc, rsp = 0x7fffffffea98, rbp = 0x3 --- Tracing command named pid 4103 tid 100529 td 0xffffff02a3a0cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800c82b9c, rsp = 0x7ffffe9f5e08, rbp = 0x201 --- Tracing command named pid 4103 tid 100528 td 0xffffff02a3a0d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f do_cv_wait() at do_cv_wait+0x57a __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800aa32ac, rsp = 0x7ffffebf6db8, rbp = 0x800e0a200 --- Tracing command named pid 4103 tid 100527 td 0xffffff02a3a0d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800aa32ac, rsp = 0x7ffffedf7eb8, rbp = 0x800e0a3c0 --- Tracing command named pid 4103 tid 100526 td 0xffffff02a3a0d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800aa32ac, rsp = 0x7ffffeff8eb8, rbp = 0x800e0a580 --- Tracing command named pid 4103 tid 100525 td 0xffffff02a3a0dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800aa32ac, rsp = 0x7fffff1f9eb8, rbp = 0x800e0a740 --- Tracing command named pid 4103 tid 100524 td 0xffffff02a3a0e000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800aa32ac, rsp = 0x7fffff3faeb8, rbp = 0x800e0a900 --- Tracing command named pid 4103 tid 100523 td 0xffffff02a3a0e3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800aa32ac, rsp = 0x7fffff5fbeb8, rbp = 0x800e0aac0 --- Tracing command named pid 4103 tid 100522 td 0xffffff02a3a0e740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800aa32ac, rsp = 0x7fffff7fceb8, rbp = 0x800e0ac80 --- Tracing command named pid 4103 tid 100521 td 0xffffff02a3a0eae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800aa32ac, rsp = 0x7fffff9fdeb8, rbp = 0x800e0ae40 --- Tracing command named pid 4103 tid 100520 td 0xffffff02a3a0f000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800aa32ac, rsp = 0x7fffffbfeeb8, rbp = 0x800e9d1c0 --- Tracing command named pid 4103 tid 100406 td 0xffffff026cd12ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigtimedwait() at kern_sigtimedwait+0x55d sigwait() at sigwait+0x74 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (429, FreeBSD ELF64, sigwait), rip = 0x800bfdb3c, rsp = 0x7fffffffeae8, rbp = 0x800e041c0 --- Tracing command syslogd pid 4098 tid 100386 td 0xffffff027d9a8000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe058, rbp = 0x800a330c8 --- Tracing command qmail-clean pid 3947 tid 100512 td 0xffffff02a30d7ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80073b93c, rsp = 0x7fffffffec18, rbp = 0x503aa0 --- Tracing command qmail-rspawn pid 3946 tid 100511 td 0xffffff02a30e0000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80073c8bc, rsp = 0x7fffffffec48, rbp = 0x1 --- Tracing command qmail-lspawn pid 3945 tid 100510 td 0xffffff02a30e03a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80073c8bc, rsp = 0x7fffffffec28, rbp = 0x1 --- Tracing command splogger pid 3944 tid 100509 td 0xffffff02a30e0740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80073a93c, rsp = 0x7fffffffec98, rbp = 0x501dc0 --- Tracing command qmail-send pid 3937 tid 100505 td 0xffffff02a30e6000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8007438bc, rsp = 0x7fffffffebd8, rbp = 0x7fffffffebe0 --- Tracing command cron pid 3930 tid 100306 td 0xffffff02162ff000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea88, rbp = 0x3c --- Tracing command sshd pid 3904 tid 100295 td 0xffffff02161083a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd58, rbp = 0 --- Tracing command iscsi-target pid 3855 tid 100324 td 0xffffff02a303d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x800818efc, rsp = 0x7fffffffde18, rbp = 0x7fffffffde50 --- Tracing command perl pid 3843 tid 100283 td 0xffffff0015b80ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800ca68bc, rsp = 0x7fffffffeb08, rbp = 0x3 --- Tracing command mysqld pid 3835 tid 100487 td 0xffffff03229493a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigtimedwait() at kern_sigtimedwait+0x55d sigwait() at sigwait+0x74 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (429, FreeBSD ELF64, sigwait), rip = 0x8013e9b3c, rsp = 0x7ffffebf6f08, rbp = 0x80160a200 --- Tracing command mysqld pid 3835 tid 100484 td 0xffffff0322949740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x80128f2ac, rsp = 0x7ffffedf7e88, rbp = 0x80160a3c0 --- Tracing command mysqld pid 3835 tid 100486 td 0xffffff0322949ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8014888bc, rsp = 0x7ffffeff8f18, rbp = 0 --- Tracing command mysqld pid 3835 tid 100485 td 0xffffff0322cbc000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8014888bc, rsp = 0x7fffff1f9f08, rbp = 0 --- Tracing command mysqld pid 3835 tid 100483 td 0xffffff02a3a0f3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x80128f2ac, rsp = 0x7fffff5fbbb8, rbp = 0x80160aac0 --- Tracing command mysqld pid 3835 tid 100482 td 0xffffff02a3a0f740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x80128f2ac, rsp = 0x7fffff7fcbb8, rbp = 0x80160ac80 --- Tracing command mysqld pid 3835 tid 100481 td 0xffffff03b21f6740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x80128f2ac, rsp = 0x7fffff9fdbb8, rbp = 0x80160ae40 --- Tracing command mysqld pid 3835 tid 100480 td 0xffffff03b21f6ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x80128f2ac, rsp = 0x7fffffbfebb8, rbp = 0x8021bb1c0 --- Tracing command mysqld pid 3835 tid 100397 td 0xffffff026cc9d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8014888bc, rsp = 0x7fffffffe4d8, rbp = 0xd --- Tracing command sh pid 3806 tid 100464 td 0xffffff02e7e5d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x8009355cc, rsp = 0x7fffffffe258, rbp = 0xedd --- Tracing command nginx pid 3792 tid 100470 td 0xffffff0322d26ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800beab9c, rsp = 0x7fffffffe888, rbp = 0 --- Tracing command nginx pid 3791 tid 100381 td 0xffffff02a318c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x800b6621c, rsp = 0x7fffffffe9d8, rbp = 0x1 --- Tracing command master pid 3778 tid 100472 td 0xffffff0322d263a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800c46b9c, rsp = 0x7fffffffdce8, rbp = 0x7fffffffdcf0 --- Tracing command perl pid 3702 tid 100215 td 0xffffff0015bb7ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800ca68bc, rsp = 0x7fffffffeaa8, rbp = 0x3 --- Tracing command syslogd pid 3552 tid 100463 td 0xffffff02a3a0fae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command cron pid 3406 tid 100151 td 0xffffff0007cc53a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sshd pid 3397 tid 100455 td 0xffffff02a3a21ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdce8, rbp = 0x4 --- Tracing command qmail-clean pid 3396 tid 100319 td 0xffffff027dfee3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80073b93c, rsp = 0x7fffffffebf8, rbp = 0x503aa0 --- Tracing command qmail-rspawn pid 3395 tid 100248 td 0xffffff0216844000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80073c8bc, rsp = 0x7fffffffec28, rbp = 0x1 --- Tracing command qmail-lspawn pid 3394 tid 100316 td 0xffffff0216843000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80073c8bc, rsp = 0x7fffffffec08, rbp = 0x1 --- Tracing command multilog pid 3393 tid 100410 td 0xffffff0216847ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80073f93c, rsp = 0x7fffffffeb38, rbp = 0x1 --- Tracing command tcpserver pid 3392 tid 100077 td 0xffffff0007b963a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x80082ec0c, rsp = 0x7fffffffeae8, rbp = 0xffffffff --- Tracing command qmail-send pid 3391 tid 100417 td 0xffffff02e7e3b740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8007438bc, rsp = 0x7fffffffebb8, rbp = 0x7fffffffebc0 --- Tracing command multilog pid 3390 tid 100454 td 0xffffff02a3a24000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80073f93c, rsp = 0x7fffffffeb38, rbp = 0x1 --- Tracing command supervise pid 3386 tid 100411 td 0xffffff02e7118000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x8006e8efc, rsp = 0x7fffffffea98, rbp = 0x2 --- Tracing command supervise pid 3385 tid 100388 td 0xffffff026cc9d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x8006e8efc, rsp = 0x7fffffffea88, rbp = 0x2 --- Tracing command supervise pid 3384 tid 100401 td 0xffffff027dd5e3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x8006e8efc, rsp = 0x7fffffffea98, rbp = 0x2 --- Tracing command supervise pid 3383 tid 100317 td 0xffffff0216301ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x8006e8efc, rsp = 0x7fffffffea98, rbp = 0x2 --- Tracing command readproctitle pid 3374 tid 100399 td 0xffffff027dd5eae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80073993c, rsp = 0x7fffffffea48, rbp = 0x7fffffffed5d --- Tracing command svscan pid 3373 tid 100419 td 0xffffff02e7e3b000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x800721b6c, rsp = 0x7fffffffec28, rbp = 0x7fffffffeca0 --- Tracing command syslogd pid 3274 tid 100416 td 0xffffff02e7e3bae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command httpd pid 3250 tid 100448 td 0xffffff02e71183a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x801128c0c, rsp = 0x7ffffc7e4dc8, rbp = 0x7ffffc7e4f08 --- Tracing command httpd pid 3250 tid 100447 td 0xffffff02e7118740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffc9e5e58, rbp = 0x806414280 --- Tracing command httpd pid 3250 tid 100446 td 0xffffff02e7118ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffcbe6e58, rbp = 0x806414440 --- Tracing command httpd pid 3250 tid 100445 td 0xffffff02e7ff4000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffcde7e58, rbp = 0x806414600 --- Tracing command httpd pid 3250 tid 100444 td 0xffffff02e7ff43a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffcfe8e58, rbp = 0x8064147c0 --- Tracing command httpd pid 3250 tid 100443 td 0xffffff02e7ff4740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffd1e9e58, rbp = 0x806414980 --- Tracing command httpd pid 3250 tid 100442 td 0xffffff02e7ff4ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffd3eae58, rbp = 0x806414b40 --- Tracing command httpd pid 3250 tid 100441 td 0xffffff02e7ff5000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffd5ebe58, rbp = 0x806414d00 --- Tracing command httpd pid 3250 tid 100440 td 0xffffff02e7ff53a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffd7ece58, rbp = 0x806414ec0 --- Tracing command httpd pid 3250 tid 100439 td 0xffffff02e7ff5740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffd9ede58, rbp = 0x806415080 --- Tracing command httpd pid 3250 tid 100438 td 0xffffff02e7ff5ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffdbeee58, rbp = 0x806415240 --- Tracing command httpd pid 3250 tid 100437 td 0xffffff02e7ff6000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffddefe58, rbp = 0x806415400 --- Tracing command httpd pid 3250 tid 100436 td 0xffffff02e7ff63a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffdff0e58, rbp = 0x8064155c0 --- Tracing command httpd pid 3250 tid 100435 td 0xffffff02e7ff6740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffe1f1e58, rbp = 0x806415780 --- Tracing command httpd pid 3250 tid 100434 td 0xffffff02e7ff6ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffe3f2e58, rbp = 0x806415940 --- Tracing command httpd pid 3250 tid 100433 td 0xffffff032200c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffe5f3e58, rbp = 0x806415b00 --- Tracing command httpd pid 3250 tid 100432 td 0xffffff02a30efae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffe7f4e58, rbp = 0x806415cc0 --- Tracing command httpd pid 3250 tid 100431 td 0xffffff02a3188000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffe9f5e58, rbp = 0x806415e80 --- Tracing command httpd pid 3250 tid 100430 td 0xffffff02a31883a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffebf6e58, rbp = 0x806416040 --- Tracing command httpd pid 3250 tid 100429 td 0xffffff02a3188740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffedf7e58, rbp = 0x806416200 --- Tracing command httpd pid 3250 tid 100428 td 0xffffff02a3188ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7ffffeff8e58, rbp = 0x8064163c0 --- Tracing command httpd pid 3250 tid 100427 td 0xffffff02a318a000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7fffff1f9e58, rbp = 0x806416580 --- Tracing command httpd pid 3250 tid 100426 td 0xffffff02a318a3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7fffff3fae58, rbp = 0x806416740 --- Tracing command httpd pid 3250 tid 100425 td 0xffffff02a318a740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7fffff5fbe58, rbp = 0x806416900 --- Tracing command httpd pid 3250 tid 100424 td 0xffffff02a318aae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7fffff7fce58, rbp = 0x806416ac0 --- Tracing command httpd pid 3250 tid 100423 td 0xffffff02a318c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f932ac, rsp = 0x7fffff9fde58, rbp = 0x806416c80 --- Tracing command httpd pid 3250 tid 100404 td 0xffffff0007d6b740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80118c93c, rsp = 0x7fffffffe958, rbp = 0 --- Tracing command cron pid 3146 tid 100383 td 0xffffff027d9a8ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sendmail pid 3140 tid 100162 td 0xffffff0007d633a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x800d3921c, rsp = 0x7fffffffccf8, rbp = 0x1 --- Tracing command sendmail pid 3136 tid 100400 td 0xffffff027dd5e740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800dd78bc, rsp = 0x7fffffffc168, rbp = 0x7fffffffc200 --- Tracing command sshd pid 3129 tid 100300 td 0xffffff0216300740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0x4 --- Tracing command httpd pid 3116 tid 100293 td 0xffffff026cdc53a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80118c8bc, rsp = 0x7fffffffe9a8, rbp = 0 --- Tracing command proftpd pid 3061 tid 100325 td 0xffffff02a3034ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800dfa8bc, rsp = 0x7fffffffeb48, rbp = 0x2 --- Tracing command syslogd pid 2998 tid 100398 td 0xffffff026cc9d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command cron pid 2864 tid 100403 td 0xffffff026cdfeae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sendmail pid 2858 tid 100323 td 0xffffff02a303d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_sigsuspend() at kern_sigsuspend+0x9a sigsuspend() at sigsuspend+0x34 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x800d3921c, rsp = 0x7fffffffccf8, rbp = 0x1 --- Tracing command sendmail pid 2854 tid 100322 td 0xffffff02a303d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800dd78bc, rsp = 0x7fffffffc168, rbp = 0x7fffffffc200 --- Tracing command httpd pid 2853 tid 100221 td 0xffffff02163013a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 lf_advlockasync() at lf_advlockasync+0xddd lf_advlock() at lf_advlock+0x47 vop_stdadvlock() at vop_stdadvlock+0xb3 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0xb5 flock() at flock+0x133 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (131, FreeBSD ELF64, flock), rip = 0x80111afac, rsp = 0x7fffffffe978, rbp = 0x7fffffffe9fc --- Tracing command httpd pid 2852 tid 100311 td 0xffffff021610cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 lf_advlockasync() at lf_advlockasync+0xddd lf_advlock() at lf_advlock+0x47 vop_stdadvlock() at vop_stdadvlock+0xb3 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0xb5 flock() at flock+0x133 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (131, FreeBSD ELF64, flock), rip = 0x80111afac, rsp = 0x7fffffffe978, rbp = 0x7fffffffe9fc --- Tracing command sshd pid 2845 tid 100149 td 0xffffff0007d6e740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0x4 --- Tracing command httpd pid 2832 tid 100287 td 0xffffff026ce6fae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80118a8bc, rsp = 0x7fffffffe9b8, rbp = 0 --- Tracing command nscd pid 2804 tid 100396 td 0xffffff02a39fe000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800a76b9c, rsp = 0x7ffffedf7e98, rbp = 0x7ffffedf7f80 --- Tracing command nscd pid 2804 tid 100395 td 0xffffff02a39fe3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800a76b9c, rsp = 0x7ffffeff8e98, rbp = 0x7ffffeff8f80 --- Tracing command nscd pid 2804 tid 100394 td 0xffffff02a39fe740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800a76b9c, rsp = 0x7fffff1f9e98, rbp = 0x7fffff1f9f80 --- Tracing command nscd pid 2804 tid 100393 td 0xffffff02a39feae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800a76b9c, rsp = 0x7fffff3fae98, rbp = 0x7fffff3faf80 --- Tracing command nscd pid 2804 tid 100392 td 0xffffff02a39ff000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800a76b9c, rsp = 0x7fffff5fbe98, rbp = 0x7fffff5fbf80 --- Tracing command nscd pid 2804 tid 100391 td 0xffffff0007d69ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800a76b9c, rsp = 0x7fffff7fce98, rbp = 0x7fffff7fcf80 --- Tracing command nscd pid 2804 tid 100390 td 0xffffff0007d6b000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800a76b9c, rsp = 0x7fffff9fde98, rbp = 0x7fffff9fdf80 --- Tracing command nscd pid 2804 tid 100389 td 0xffffff0007d6b3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x800a76b9c, rsp = 0x7fffffbfee98, rbp = 0x7fffffbfef80 --- Tracing command nscd pid 2804 tid 100195 td 0xffffff0007d61ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_wait() at do_wait+0x6bb __umtx_op_wait() at __umtx_op_wait+0x5e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x8007872ac, rsp = 0x7fffffffe9f8, rbp = 0x800c0ae40 --- Tracing command syslogd pid 2722 tid 100147 td 0xffffff0007d71000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command cron pid 2593 tid 100307 td 0xffffff02162feae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffeab8, rbp = 0x3c --- Tracing command qmail-clean pid 2592 tid 100136 td 0xffffff0007c70740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80073b93c, rsp = 0x7fffffffec38, rbp = 0x502480 --- Tracing command qmail-rspawn pid 2591 tid 100231 td 0xffffff01c0ef03a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80073c8bc, rsp = 0x7fffffffec58, rbp = 0x12c0 --- Tracing command qmail-lspawn pid 2590 tid 100309 td 0xffffff02162fe3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x80073d8bc, rsp = 0x7fffffffec38, rbp = 0x12c0 --- Tracing command multilog pid 2589 tid 100219 td 0xffffff00153c9ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80073f93c, rsp = 0x7fffffffeb28, rbp = 0x1 --- Tracing command qmail-send pid 2587 tid 100318 td 0xffffff027dfee740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8007448bc, rsp = 0x7fffffffeba8, rbp = 0x7fffffffec30 --- Tracing command multilog pid 2586 tid 100313 td 0xffffff027ddee3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80073f93c, rsp = 0x7fffffffeb28, rbp = 0x1 --- Tracing command tcpserver pid 2583 tid 100156 td 0xffffff0007cc4000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x80082ec0c, rsp = 0x7fffffffea78, rbp = 0xffffffff --- Tracing command sshd pid 2577 tid 100314 td 0xffffff0216843740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0 --- Tracing command supervise pid 2576 tid 100113 td 0xffffff0007c70ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x8006e8efc, rsp = 0x7fffffffea98, rbp = 0x2 --- Tracing command supervise pid 2575 tid 100080 td 0xffffff0007d183a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x8006e8efc, rsp = 0x7fffffffea88, rbp = 0x2 --- Tracing command supervise pid 2574 tid 100304 td 0xffffff02162ff740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x8006e8efc, rsp = 0x7fffffffea98, rbp = 0x2 --- Tracing command supervise pid 2573 tid 100308 td 0xffffff02162fe740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x8006e8efc, rsp = 0x7fffffffea88, rbp = 0x2 --- Tracing command readproctitle pid 2561 tid 100241 td 0xffffff0216845ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80073993c, rsp = 0x7fffffffea48, rbp = 0x7fffffffed5d --- Tracing command svscan pid 2560 tid 100132 td 0xffffff0007d73000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x800721b6c, rsp = 0x7fffffffec28, rbp = 0x7fffffffeca0 --- Tracing command httpd pid 2478 tid 100380 td 0xffffff02a31a6740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 lf_advlockasync() at lf_advlockasync+0xddd lf_advlock() at lf_advlock+0x47 vop_stdadvlock() at vop_stdadvlock+0xb3 VOP_ADVLOCK_APV() at VOP_ADVLOCK_APV+0xb5 flock() at flock+0x133 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (131, FreeBSD ELF64, flock), rip = 0x801104fac, rsp = 0x7ffffc7e4f08, rbp = 0x7ffffc7e4f7c --- Tracing command httpd pid 2478 tid 100379 td 0xffffff02a31a6ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffc9e5e68, rbp = 0x80524e440 --- Tracing command httpd pid 2478 tid 100378 td 0xffffff02a31a7000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcbe6e68, rbp = 0x80524e600 --- Tracing command httpd pid 2478 tid 100377 td 0xffffff02a31a73a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcde7e68, rbp = 0x80524e7c0 --- Tracing command httpd pid 2478 tid 100376 td 0xffffff02a31a7740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcfe8e68, rbp = 0x80524e980 --- Tracing command httpd pid 2478 tid 100375 td 0xffffff02a31a7ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd1e9e68, rbp = 0x80524eb40 --- Tracing command httpd pid 2478 tid 100374 td 0xffffff02a31a8000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd3eae68, rbp = 0x80524ed00 --- Tracing command httpd pid 2478 tid 100373 td 0xffffff0007d6e3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd5ebe68, rbp = 0x80524eec0 --- Tracing command httpd pid 2478 tid 100372 td 0xffffff0015ebc740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd7ece68, rbp = 0x80524f080 --- Tracing command httpd pid 2478 tid 100371 td 0xffffff0015ebcae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd9ede68, rbp = 0x80524f240 --- Tracing command httpd pid 2478 tid 100370 td 0xffffff020a9b6000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffdbeee68, rbp = 0x80524f400 --- Tracing command httpd pid 2478 tid 100328 td 0xffffff020a9b63a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffddefe68, rbp = 0x80524f5c0 --- Tracing command httpd pid 2478 tid 100369 td 0xffffff020a9b6740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffdff0e68, rbp = 0x80524f780 --- Tracing command httpd pid 2478 tid 100367 td 0xffffff02a30ebae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe1f1e68, rbp = 0x80524f940 --- Tracing command httpd pid 2478 tid 100365 td 0xffffff02a30ee000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe3f2e68, rbp = 0x80524fb00 --- Tracing command httpd pid 2478 tid 100362 td 0xffffff02a30ee3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe5f3e68, rbp = 0x80524fcc0 --- Tracing command httpd pid 2478 tid 100361 td 0xffffff02a30ee740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7ffffe7f07b8, rbp = 0x7ffffe7f08b8 --- Tracing command httpd pid 2478 tid 100359 td 0xffffff02a30eeae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe9f5e68, rbp = 0x805250040 --- Tracing command httpd pid 2478 tid 100357 td 0xffffff02a30ef000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7ffffebf27b8, rbp = 0x7ffffebf28b8 --- Tracing command httpd pid 2478 tid 100355 td 0xffffff02a30ef3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7ffffedf37b8, rbp = 0x7ffffedf38b8 --- Tracing command httpd pid 2478 tid 100353 td 0xffffff02a30ef740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7ffffeff47b8, rbp = 0x7ffffeff48b8 --- Tracing command httpd pid 2478 tid 100350 td 0xffffff0007d6c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7fffff1f57b8, rbp = 0x7fffff1f58b8 --- Tracing command httpd pid 2478 tid 100347 td 0xffffff020a9b6ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff3fae68, rbp = 0x805250900 --- Tracing command httpd pid 2478 tid 100344 td 0xffffff020ab21000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7fffff5f77b8, rbp = 0x7fffff5f78b8 --- Tracing command httpd pid 2478 tid 100341 td 0xffffff0007d6cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff7fce68, rbp = 0x805250c80 --- Tracing command httpd pid 2478 tid 100339 td 0xffffff0007d6e000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7fffff9f97b8, rbp = 0x7fffff9f98b8 --- Tracing command httpd pid 2478 tid 100327 td 0xffffff02a30343a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80117493c, rsp = 0x7fffffffe948, rbp = 0 --- Tracing command httpd pid 2477 tid 100368 td 0xffffff02a318cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_kevent() at kern_kevent+0x32b kevent() at kevent+0x90 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (363, FreeBSD ELF64, kevent), rip = 0x80115ab9c, rsp = 0x7ffffc7e4ec8, rbp = 0x7ffffc7e4f7c --- Tracing command httpd pid 2477 tid 100366 td 0xffffff02a318d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffc9e5e68, rbp = 0x80524e440 --- Tracing command httpd pid 2477 tid 100364 td 0xffffff02a318d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcbe6e68, rbp = 0x80524e600 --- Tracing command httpd pid 2477 tid 100363 td 0xffffff02a318d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcde7e68, rbp = 0x80524e7c0 --- Tracing command httpd pid 2477 tid 100360 td 0xffffff027dfeeae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffcfe8e68, rbp = 0x80524e980 --- Tracing command httpd pid 2477 tid 100358 td 0xffffff02a30cc000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd1e9e68, rbp = 0x80524eb40 --- Tracing command httpd pid 2477 tid 100356 td 0xffffff02a30cc3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd3eae68, rbp = 0x80524ed00 --- Tracing command httpd pid 2477 tid 100354 td 0xffffff02a30cc740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd5ebe68, rbp = 0x80524eec0 --- Tracing command httpd pid 2477 tid 100352 td 0xffffff02a30ccae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd7ece68, rbp = 0x80524f080 --- Tracing command httpd pid 2477 tid 100351 td 0xffffff02a30cd000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffd9ede68, rbp = 0x80524f240 --- Tracing command httpd pid 2477 tid 100349 td 0xffffff02a30cd3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffdbeee68, rbp = 0x80524f400 --- Tracing command httpd pid 2477 tid 100348 td 0xffffff02a30cd740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffddefe68, rbp = 0x80524f5c0 --- Tracing command httpd pid 2477 tid 100346 td 0xffffff02a30cdae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffdff0e68, rbp = 0x80524f780 --- Tracing command httpd pid 2477 tid 100345 td 0xffffff02a30ce000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe1f1e68, rbp = 0x80524f940 --- Tracing command httpd pid 2477 tid 100343 td 0xffffff02a30ce3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe3f2e68, rbp = 0x80524fb00 --- Tracing command httpd pid 2477 tid 100342 td 0xffffff02a30ce740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7ffffe5ef7b8, rbp = 0x7ffffe5ef8b8 --- Tracing command httpd pid 2477 tid 100340 td 0xffffff02a30ceae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe7f4e68, rbp = 0x80524fe80 --- Tracing command httpd pid 2477 tid 100338 td 0xffffff02a3178000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffe9f5e68, rbp = 0x805250040 --- Tracing command httpd pid 2477 tid 100337 td 0xffffff02a31783a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffebf6e68, rbp = 0x805250200 --- Tracing command httpd pid 2477 tid 100336 td 0xffffff02a3178740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffedf7e68, rbp = 0x8052503c0 --- Tracing command httpd pid 2477 tid 100334 td 0xffffff027ddeeae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7ffffeff8e68, rbp = 0x805250580 --- Tracing command httpd pid 2477 tid 100333 td 0xffffff02a3010000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x80111fefc, rsp = 0x7fffff1f57b8, rbp = 0x7fffff1f58b8 --- Tracing command httpd pid 2477 tid 100332 td 0xffffff02a30103a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff3fae68, rbp = 0x805250900 --- Tracing command httpd pid 2477 tid 100331 td 0xffffff02a3010740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff5fbe68, rbp = 0x805250ac0 --- Tracing command httpd pid 2477 tid 100330 td 0xffffff02a3010ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff7fce68, rbp = 0x805250c80 --- Tracing command httpd pid 2477 tid 100329 td 0xffffff02a3034000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800f7b2ac, rsp = 0x7fffff9fde68, rbp = 0x805250e40 --- Tracing command httpd pid 2477 tid 100326 td 0xffffff02a3034740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80117493c, rsp = 0x7fffffffe948, rbp = 0 --- Tracing command syslogd pid 2459 tid 100312 td 0xffffff027ddee740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command cron pid 2338 tid 100301 td 0xffffff02163003a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sshd pid 2329 tid 100150 td 0xffffff0007cc5740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0 --- Tracing command httpd pid 2316 tid 100226 td 0xffffff0007c6cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8011748bc, rsp = 0x7fffffffe9a8, rbp = 0 --- Tracing command python2.4 pid 2267 tid 100583 td 0xffffff02a31a53a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5c2ac, rsp = 0x7fffff8fb658, rbp = 0x80d4bc2c0 --- Tracing command python2.4 pid 2267 tid 100582 td 0xffffff02a31a5740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5c2ac, rsp = 0x7fffff9fc658, rbp = 0x80d4bc480 --- Tracing command python2.4 pid 2267 tid 100581 td 0xffffff02a31a5ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5c2ac, rsp = 0x7fffffafd658, rbp = 0x80d4bc640 --- Tracing command python2.4 pid 2267 tid 100580 td 0xffffff02a31a6000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5c2ac, rsp = 0x7fffffbfe658, rbp = 0x80d4bc800 --- Tracing command python2.4 pid 2267 tid 100183 td 0xffffff0015ebc3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e558bc, rsp = 0x7fffffff1838, rbp = 0x18 --- Tracing command python2.4 pid 2266 tid 100296 td 0xffffff0216301000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e558bc, rsp = 0x7fffffff18b8, rbp = 0x4 --- Tracing command httpd pid 2205 tid 100279 td 0xffffff0015c2bae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d __lockmgr_args() at __lockmgr_args+0x78e vop_stdlock() at vop_stdlock+0x39 VOP_LOCK1_APV() at VOP_LOCK1_APV+0x9b _vn_lock() at _vn_lock+0x57 vn_write() at vn_write+0x218 dofilewrite() at dofilewrite+0x85 kern_writev() at kern_writev+0x60 write() at write+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (4, FreeBSD ELF64, write), rip = 0x80117191c, rsp = 0x7fffffff6188, rbp = 0x7fffffff8280 --- Tracing command syslogd pid 2200 tid 100285 td 0xffffff026cdfe3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command cron pid 2073 tid 100251 td 0xffffff020ab44740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sshd pid 2064 tid 100227 td 0xffffff0007c6c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0 --- Tracing command httpd pid 2051 tid 100175 td 0xffffff0015c05000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8011718bc, rsp = 0x7fffffffe9b8, rbp = 0 --- Tracing command python pid 2001 tid 100585 td 0xffffff03fe7a43a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffafd658, rbp = 0x811169080 --- Tracing command python pid 2001 tid 100584 td 0xffffff03fe7a4740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffbfe658, rbp = 0x811169240 --- Tracing command python pid 2001 tid 100243 td 0xffffff02168453a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7ffffffebc48, rbp = 0x9 --- Tracing command python pid 2000 tid 100065 td 0xffffff0007ca43a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7ffffffee6c8, rbp = 0x4 --- Tracing command python pid 1986 tid 100567 td 0xffffff02a39fd740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a zio_wait() at zio_wait+0x61 dmu_buf_hold_array_by_dnode() at dmu_buf_hold_array_by_dnode+0x22a dmu_buf_hold_array() at dmu_buf_hold_array+0x62 dmu_read_uio() at dmu_read_uio+0x3f zfs_freebsd_read() at zfs_freebsd_read+0x571 VOP_READ_APV() at VOP_READ_APV+0xaf vn_read() at vn_read+0x256 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x800e5493c, rsp = 0x7fffffaf0fd8, rbp = 0x800f99cb0 --- Tracing command python pid 1986 tid 100566 td 0xffffff02a39fdae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffbfcaa8, rbp = 0x810fe83c0 --- Tracing command python pid 1986 tid 100249 td 0xffffff0216843ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7ffffffec6c8, rbp = 0x17 --- Tracing command python pid 1985 tid 100237 td 0xffffff0216847740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7ffffffeecb8, rbp = 0x4 --- Tracing command syslogd pid 1921 tid 100239 td 0xffffff02168473a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command cron pid 1785 tid 100211 td 0xffffff0015b75ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _sleep() at _sleep+0x23f kern_nanosleep() at kern_nanosleep+0x118 nanosleep() at nanosleep+0x6e syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (240, FreeBSD ELF64, nanosleep), rip = 0x80093eb6c, rsp = 0x7fffffffea98, rbp = 0x3c --- Tracing command sshd pid 1774 tid 100208 td 0xffffff01c04df3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd18, rbp = 0x1 --- Tracing command rotatelogs pid 1773 tid 100194 td 0xffffff0007d613a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80112093c, rsp = 0x7ffffffee538, rbp = 0x801413110 --- Tracing command rotatelogs pid 1772 tid 100190 td 0xffffff0007d61740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80112093c, rsp = 0x7ffffffee538, rbp = 0x801413110 --- Tracing command rotatelogs pid 1771 tid 100189 td 0xffffff0007d18740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80112093c, rsp = 0x7ffffffee538, rbp = 0x801413110 --- Tracing command rotatelogs pid 1769 tid 100188 td 0xffffff0007d61000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80112093c, rsp = 0x7ffffffee538, rbp = 0x801413110 --- Tracing command rotatelogs pid 1768 tid 100185 td 0xffffff0007d62000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80112093c, rsp = 0x7ffffffee538, rbp = 0x801413110 --- Tracing command rotatelogs pid 1767 tid 100184 td 0xffffff0007d63740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 pipe_read() at pipe_read+0x4a3 dofileread() at dofileread+0xa1 kern_readv() at kern_readv+0x60 read() at read+0x55 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (3, FreeBSD ELF64, read), rip = 0x80112093c, rsp = 0x7ffffffee538, rbp = 0x801413110 --- Tracing command httpd pid 1752 tid 100182 td 0xffffff0007d62ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8011718bc, rsp = 0x7fffffffe9c8, rbp = 0 --- Tracing command python pid 1688 tid 100261 td 0xffffff0007d6bae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffafd658, rbp = 0x810d446c0 --- Tracing command python pid 1688 tid 100260 td 0xffffff0007d6c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffbfe658, rbp = 0x810d44880 --- Tracing command python pid 1688 tid 100181 td 0xffffff0007d62740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7ffffffeb8f8, rbp = 0xf --- Tracing command python pid 1687 tid 100180 td 0xffffff0015b8bae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7ffffffee578, rbp = 0x4 --- Tracing command python2.4 pid 1679 tid 100253 td 0xffffff0015b61740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffafd658, rbp = 0x8105d0b00 --- Tracing command python2.4 pid 1679 tid 100252 td 0xffffff0015b61ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffbfe658, rbp = 0x8105d0cc0 --- Tracing command python2.4 pid 1679 tid 100177 td 0xffffff0015c28740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7fffffff1818, rbp = 0x9 --- Tracing command python2.4 pid 1678 tid 100176 td 0xffffff0015c28ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7fffffff18a8, rbp = 0x4 --- Tracing command python2.4 pid 1674 tid 100259 td 0xffffff020ab44000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffafd658, rbp = 0x8106feb00 --- Tracing command python2.4 pid 1674 tid 100258 td 0xffffff020ab443a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffbfe658, rbp = 0x8106fecc0 --- Tracing command python2.4 pid 1674 tid 100169 td 0xffffff00153c5740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7fffffff1818, rbp = 0xf --- Tracing command python2.4 pid 1673 tid 100153 td 0xffffff0007cc4ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7fffffff18b8, rbp = 0x4 --- Tracing command python2.4 pid 1669 tid 100255 td 0xffffff0015b61000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffafd658, rbp = 0x8106d2b00 --- Tracing command python2.4 pid 1669 tid 100254 td 0xffffff0015b613a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffbfe658, rbp = 0x8106d2cc0 --- Tracing command python2.4 pid 1669 tid 100166 td 0xffffff00153c63a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7fffffff1828, rbp = 0xe --- Tracing command python2.4 pid 1668 tid 100170 td 0xffffff00153c53a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7fffffff18c8, rbp = 0x4 --- Tracing command python2.4 pid 1664 tid 100257 td 0xffffff021610c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffafd658, rbp = 0x8106beb00 --- Tracing command python2.4 pid 1664 tid 100256 td 0xffffff021610c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 do_cv_wait() at do_cv_wait+0x7d4 __umtx_op_cv_wait() at __umtx_op_cv_wait+0x5c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (454, FreeBSD ELF64, _umtx_op), rip = 0x800c5b2ac, rsp = 0x7fffffbfe658, rbp = 0x8106becc0 --- Tracing command python2.4 pid 1664 tid 100171 td 0xffffff00153c5000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7fffffff1848, rbp = 0xd --- Tracing command python2.4 pid 1663 tid 100172 td 0xffffff0007f6dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800e548bc, rsp = 0x7fffffff18f8, rbp = 0x4 --- Tracing command syslogd pid 1592 tid 100178 td 0xffffff0015c283a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe098, rbp = 0x800a10058 --- Tracing command sshd pid 1399 tid 100160 td 0xffffff0007ccb000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8013d68bc, rsp = 0x7fffffffdd68, rbp = 0 --- Tracing command rwhod pid 1380 tid 100155 td 0xffffff0007cc43a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 soreceive_dgram() at soreceive_dgram+0x180 kern_recvit() at kern_recvit+0x1c1 recvit() at recvit+0x21 recvfrom() at recvfrom+0x82 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (29, FreeBSD ELF64, recvfrom), rip = 0x8006f67ac, rsp = 0x7fffffffe7d8, rbp = 0x7fffffffe880 --- Tracing command ntpd pid 1344 tid 100158 td 0xffffff0007ccb740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800d508bc, rsp = 0x7fffffffec08, rbp = 0x7fffffffed30 --- Tracing command nfsd pid 1255 tid 100165 td 0xffffff0015b80000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c svc_run_internal() at svc_run_internal+0x7fe svc_thread_start() at svc_thread_start+0xb fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0xc, rip = 0x80069fdfc, rsp = 0x7fffffffe6b8, rbp = 0x7 --- Tracing command nfsd pid 1255 tid 100164 td 0xffffff0015b803a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c svc_run_internal() at svc_run_internal+0x7fe svc_thread_start() at svc_thread_start+0xb fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0xc, rip = 0x80069fdfc, rsp = 0x7fffffffe6b8, rbp = 0x7 --- Tracing command nfsd pid 1255 tid 100163 td 0xffffff0015b80740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c svc_run_internal() at svc_run_internal+0x7fe svc_thread_start() at svc_thread_start+0xb fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0xc, rip = 0x80069fdfc, rsp = 0x7fffffffe6b8, rbp = 0x7 --- Tracing command nfsd pid 1255 tid 100066 td 0xffffff0007ca4000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c svc_run_internal() at svc_run_internal+0x7fe svc_run() at svc_run+0x94 nfssvc_nfsd() at nfssvc_nfsd+0xa2 nfssvc_nfsserver() at nfssvc_nfsserver+0x5b nfssvc() at nfssvc+0x7c syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (155, FreeBSD ELF64, nfssvc), rip = 0x80069fdfc, rsp = 0x7fffffffe6b8, rbp = 0x7 --- Tracing command nfsd pid 1254 tid 100142 td 0xffffff0007c6f000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_accept() at kern_accept+0x1fc accept() at accept+0x75 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (30, FreeBSD ELF64, accept), rip = 0x8006d9c0c, rsp = 0x7fffffffe958, rbp = 0x800a04060 --- Tracing command mountd pid 1252 tid 100074 td 0xffffff0007c9fae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008528bc, rsp = 0x7fffffffec48, rbp = 0x7fffffffed40 --- Tracing command rpcbind pid 1132 tid 100078 td 0xffffff0007c9f3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 poll() at poll+0x2b6 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (209, FreeBSD ELF64, poll), rip = 0x800906efc, rsp = 0x7fffffffcab8, rbp = 0x800c130c0 --- Tracing command syslogd pid 1109 tid 100076 td 0xffffff0007b96740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8008518bc, rsp = 0x7fffffffe108, rbp = 0x800a310e0 --- Tracing command devd pid 936 tid 100071 td 0xffffff0007b973a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x440ecc, rsp = 0x7fffffffe898, rbp = 0x7fffffffe8b0 --- Tracing command moused pid 903 tid 100073 td 0xffffff0007b96ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8009708bc, rsp = 0x7fffffffe848, rbp = 0x7fffffffed10 --- Tracing command ospfd pid 632 tid 100072 td 0xffffff0007b97000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_timedwait_sig() at sleepq_timedwait_sig+0x19 _cv_timedwait_sig() at _cv_timedwait_sig+0x18c seltdwait() at seltdwait+0x56 kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x800c2c8bc, rsp = 0x7fffffffe988, rbp = 0 --- Tracing command zebra pid 626 tid 100068 td 0xffffff0007ca3740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _cv_wait_sig() at _cv_wait_sig+0x17e seltdwait() at seltdwait+0xac kern_select() at kern_select+0x620 select() at select+0x5d syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (93, FreeBSD ELF64, select), rip = 0x8016fc8bc, rsp = 0x7fffffffe978, rbp = 0 --- Tracing command zfskern pid 43 tid 100112 td 0xffffff0007cc5ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a zio_wait() at zio_wait+0x61 dsl_pool_sync() at dsl_pool_sync+0xee spa_sync() at spa_sync+0x355 txg_sync_thread() at txg_sync_thread+0x28f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bc3d30, rbp = 0 --- Tracing command zfskern pid 43 tid 100111 td 0xffffff0007d18000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a txg_thread_wait() at txg_thread_wait+0x79 txg_quiesce_thread() at txg_quiesce_thread+0xb5 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bbfd30, rbp = 0 --- Tracing command zfskern pid 43 tid 100109 td 0xffffff0007d63000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 vdev_geom_worker() at vdev_geom_worker+0xf3 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242ba6d30, rbp = 0 --- Tracing command zfskern pid 43 tid 100108 td 0xffffff0007d64000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 vdev_geom_worker() at vdev_geom_worker+0xf3 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242ba1d30, rbp = 0 --- Tracing command zfskern pid 43 tid 100081 td 0xffffff0007c9f000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c l2arc_feed_thread() at l2arc_feed_thread+0x144 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b29d30, rbp = 0 --- Tracing command zfskern pid 43 tid 100070 td 0xffffff0007ca3000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _cv_timedwait() at _cv_timedwait+0x18c arc_reclaim_thread() at arc_reclaim_thread+0x299 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242af2d30, rbp = 0 --- Tracing command flowcleaner pid 23 tid 100062 td 0xffffff0007b97740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sched_bind() at sched_bind+0x7f flowtable_cleaner() at flowtable_cleaner+0x177 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82407c7d30, rbp = 0 --- Tracing command softdepflush pid 22 tid 100061 td 0xffffff0007b97ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 softdep_flush() at softdep_flush+0x259 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82407c2d30, rbp = 0 --- Tracing command vnlru pid 21 tid 100060 td 0xffffff0007b99000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vnlru_proc() at vnlru_proc+0x5cf fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82407bdd30, rbp = 0 --- Tracing command syncer pid 20 tid 100059 td 0xffffff0007b993a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a zio_wait() at zio_wait+0x61 zil_commit() at zil_commit+0x3e1 zfs_sync() at zfs_sync+0xa6 sync_fsync() at sync_fsync+0x13a VOP_FSYNC_APV() at VOP_FSYNC_APV+0xb5 sync_vnode() at sync_vnode+0x157 sched_sync() at sched_sync+0x1cc fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82407b8d30, rbp = 0 --- Tracing command bufdaemon pid 19 tid 100058 td 0xffffff0007b99740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 buf_daemon() at buf_daemon+0x14a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82407b3d30, rbp = 0 --- Tracing command pagezero pid 18 tid 100057 td 0xffffff0007b99ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vm_pagezero() at vm_pagezero+0x73 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82407aed30, rbp = 0 --- Tracing command vmdaemon pid 17 tid 100056 td 0xffffff000742f3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 vm_daemon() at vm_daemon+0x4d fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82407a9d30, rbp = 0 --- Tracing command pagedaemon pid 16 tid 100055 td 0xffffff000742f740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 vm_pageout() at vm_pageout+0x855 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82407a4d30, rbp = 0 --- Tracing command g_journal switcher pid 15 tid 100054 td 0xffffff000742fae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 g_journal_switcher() at g_journal_switcher+0x4b fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82404f8d30, rbp = 0 --- Tracing command xpt_thrd pid 9 tid 100053 td 0xffffff0007880000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 xpt_scanner_thread() at xpt_scanner_thread+0xd5 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82404f3d30, rbp = 0 --- Tracing command pfpurge pid 8 tid 100052 td 0xffffff00078803a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 pf_purge_thread() at pf_purge_thread+0x31 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82404eed30, rbp = 0 --- Tracing command ciss_notify0 pid 7 tid 100047 td 0xffffff0007883740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 ciss_notify_thread() at ciss_notify_thread+0x276 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82404c9d30, rbp = 0 --- Tracing command usb pid 14 tid 100042 td 0xffffff000742c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8240417d30, rbp = 0 --- Tracing command usb pid 14 tid 100041 td 0xffffff000742cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8240412d30, rbp = 0 --- Tracing command usb pid 14 tid 100040 td 0xffffff000742d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff824040dd30, rbp = 0 --- Tracing command usb pid 14 tid 100039 td 0xffffff000742d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a usb_process() at usb_process+0x15f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8240408d30, rbp = 0 --- Tracing command crypto returns pid 6 tid 100027 td 0xffffff000739c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 crypto_ret_proc() at crypto_ret_proc+0x19a fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000108d30, rbp = 0 --- Tracing command crypto pid 5 tid 100026 td 0xffffff000735d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 crypto_proc() at crypto_proc+0x1d1 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000103d30, rbp = 0 --- Tracing command yarrow pid 13 tid 100025 td 0xffffff000735dae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 random_kthread() at random_kthread+0x1ad fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000fed30, rbp = 0 --- Tracing command g_down pid 4 tid 100023 td 0xffffff000735e3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d turnstile_wait() at turnstile_wait+0x243 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xe1 dastrategy() at dastrategy+0x46 g_disk_start() at g_disk_start+0xf4 g_io_schedule_down() at g_io_schedule_down+0x1e2 g_down_procbody() at g_down_procbody+0x6f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000f4d30, rbp = 0 --- Tracing command g_up pid 3 tid 100022 td 0xffffff000735e740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 g_io_schedule_up() at g_io_schedule_up+0x133 g_up_procbody() at g_up_procbody+0x6f fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000efd30, rbp = 0 --- Tracing command g_event pid 2 tid 100021 td 0xffffff000735eae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 g_event_procbody() at g_event_procbody+0xa1 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000ead30, rbp = 0 --- Tracing command intr pid 12 tid 100051 td 0xffffff0007880740 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff80387589, rsp = 0xffffff8000062fe0, rbp = 0xffffff82404e9b70 --- _mtx_lock_sleep() at _mtx_lock_sleep+0x119 _mtx_lock_flags() at _mtx_lock_flags+0xe1 intr_event_execute_handlers() at intr_event_execute_handlers+0x5f ithread_loop() at ithread_loop+0xb2 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82404e9d30, rbp = 0 --- Tracing command intr pid 12 tid 100050 td 0xffffff0007880ae0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100049 td 0xffffff0007883000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff82404dfd30, rbp = 0 --- Tracing command intr pid 12 tid 100048 td 0xffffff00078833a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100046 td 0xffffff0007883ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d turnstile_wait() at turnstile_wait+0x243 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xe1 ciss_perf_msi_intr() at ciss_perf_msi_intr+0x4c intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xb2 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff824047ed30, rbp = 0 --- Tracing command intr pid 12 tid 100045 td 0xffffff000739cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8240478d30, rbp = 0 --- Tracing command intr pid 12 tid 100044 td 0xffffff000742c000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100043 td 0xffffff000742c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff824043fd30, rbp = 0 --- Tracing command intr pid 12 tid 100038 td 0xffffff000742d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8240403d30, rbp = 0 --- Tracing command intr pid 12 tid 100037 td 0xffffff000742dae0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100035 td 0xffffff0007360740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000130d30, rbp = 0 --- Tracing command intr pid 12 tid 100034 td 0xffffff0007360ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800012bd30, rbp = 0 --- Tracing command intr pid 12 tid 100032 td 0xffffff000739a3a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100028 td 0xffffff000739c3a0 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff80253d86, rsp = 0xffffff800004dfe0, rbp = 0xffffff800010cef0 --- ciss_get_request() at ciss_get_request+0x16 ciss_cam_action() at ciss_cam_action+0x2e2 xpt_run_dev_sendq() at xpt_run_dev_sendq+0x180 cam_periph_error() at cam_periph_error+0x465 dadone() at dadone+0xb3 camisr_runqueue() at camisr_runqueue+0x113 camisr() at camisr+0xd3 intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xb2 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800010dd30, rbp = 0 --- Tracing command intr pid 12 tid 100020 td 0xffffff0007360000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000e5d30, rbp = 0 --- Tracing command intr pid 12 tid 100019 td 0xffffff00073603a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000e0d30, rbp = 0 --- Tracing command intr pid 12 tid 100018 td 0xffffff0007354ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000dbd30, rbp = 0 --- Tracing command intr pid 12 tid 100017 td 0xffffff000735c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000d6d30, rbp = 0 --- Tracing command intr pid 12 tid 100016 td 0xffffff000735c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000d1d30, rbp = 0 --- Tracing command intr pid 12 tid 100015 td 0xffffff000735c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000ccd30, rbp = 0 --- Tracing command intr pid 12 tid 100014 td 0xffffff000735cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000c7d30, rbp = 0 --- Tracing command intr pid 12 tid 100013 td 0xffffff000735d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d turnstile_wait() at turnstile_wait+0x243 _mtx_lock_sleep() at _mtx_lock_sleep+0xd6 _mtx_lock_flags() at _mtx_lock_flags+0xe1 softclock() at softclock+0x248 intr_event_execute_handlers() at intr_event_execute_handlers+0x66 ithread_loop() at ithread_loop+0xb2 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000c2d30, rbp = 0 --- Tracing command intr pid 12 tid 100012 td 0xffffff000735d3a0 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xffffff000734a3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d ithread_loop() at ithread_loop+0x246 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000b8d30, rbp = 0 --- Tracing command idle pid 11 tid 100010 td 0xffffff000734a740 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff805e2446, rsp = 0xffffffff80ae73c0, rbp = 0xffffff80000b3b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000b3d30, rbp = 0 --- Tracing command idle pid 11 tid 100009 td 0xffffff000734aae0 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff805e2446, rsp = 0xffffff8000046fe0, rbp = 0xffffff80000aeb60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000aed30, rbp = 0 --- Tracing command idle pid 11 tid 100008 td 0xffffff0007354000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sched_idletd() at sched_idletd+0x26c fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000a9d30, rbp = 0 --- Tracing command idle pid 11 tid 100007 td 0xffffff00073543a0 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff805e2446, rsp = 0xffffff8000054fe0, rbp = 0xffffff80000a4b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000a4d30, rbp = 0 --- Tracing command idle pid 11 tid 100006 td 0xffffff0007354740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sched_preempt() at sched_preempt+0xbc ipi_bitmap_handler() at ipi_bitmap_handler+0x70 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x9b --- interrupt, rip = 0xffffffff805e2446, rsp = 0xffffff800009fb50, rbp = 0xffffff800009fb60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800009fd30, rbp = 0 --- Tracing command idle pid 11 tid 100005 td 0xffffff0007349000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sched_preempt() at sched_preempt+0xbc ipi_bitmap_handler() at ipi_bitmap_handler+0x70 Xipi_intr_bitmap_handler() at Xipi_intr_bitmap_handler+0x9b --- interrupt, rip = 0xffffffff805e2446, rsp = 0xffffff800009ab50, rbp = 0xffffff800009ab60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800009ad30, rbp = 0 --- Tracing command idle pid 11 tid 100004 td 0xffffff00073493a0 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff805e2446, rsp = 0xffffff8000069fe0, rbp = 0xffffff8000095b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000095d30, rbp = 0 --- Tracing command idle pid 11 tid 100003 td 0xffffff0007349740 cpustop_handler() at cpustop_handler+0x40 ipi_nmi_handler() at ipi_nmi_handler+0x30 trap() at trap+0x19a nmi_calltrap() at nmi_calltrap+0x8 --- trap 0x13, rip = 0xffffffff805e2446, rsp = 0xffffff8000070fe0, rbp = 0xffffff8000090b60 --- acpi_cpu_c1() at acpi_cpu_c1+0x6 acpi_cpu_idle() at acpi_cpu_idle+0x1f6 sched_idletd() at sched_idletd+0x123 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000090d30, rbp = 0 --- Tracing command init pid 1 tid 100002 td 0xffffff0007349ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_catch_signals() at sleepq_catch_signals+0x2af sleepq_wait_sig() at sleepq_wait_sig+0x16 _sleep() at _sleep+0x309 kern_wait() at kern_wait+0x3f7 wait4() at wait4+0x35 syscall() at syscall+0x102 Xfast_syscall() at Xfast_syscall+0xe1 --- syscall (7, FreeBSD ELF64, wait4), rip = 0x40c49c, rsp = 0x7fffffffe808, rbp = 0x401d40 --- Tracing command audit pid 10 tid 100001 td 0xffffff000734a000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _cv_wait() at _cv_wait+0x17a audit_worker() at audit_worker+0x77 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000086d30, rbp = 0 --- Tracing command kernel pid 0 tid 100131 td 0xffffff00153c6740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242c22d30, rbp = 0 --- Tracing command kernel pid 0 tid 100130 td 0xffffff00153c6ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242c1dd30, rbp = 0 --- Tracing command kernel pid 0 tid 100129 td 0xffffff00153c7000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242c18d30, rbp = 0 --- Tracing command kernel pid 0 tid 100128 td 0xffffff00153c73a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242c13d30, rbp = 0 --- Tracing command kernel pid 0 tid 100127 td 0xffffff00153c7740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242c0ed30, rbp = 0 --- Tracing command kernel pid 0 tid 100126 td 0xffffff00153c7ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242c09d30, rbp = 0 --- Tracing command kernel pid 0 tid 100125 td 0xffffff00153c9000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242c04d30, rbp = 0 --- Tracing command kernel pid 0 tid 100124 td 0xffffff00153c93a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bffd30, rbp = 0 --- Tracing command kernel pid 0 tid 100123 td 0xffffff00153c9740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bfad30, rbp = 0 --- Tracing command kernel pid 0 tid 100122 td 0xffffff0007d64740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bf5d30, rbp = 0 --- Tracing command kernel pid 0 tid 100121 td 0xffffff0007d64ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bf0d30, rbp = 0 --- Tracing command kernel pid 0 tid 100120 td 0xffffff0007d68000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bebd30, rbp = 0 --- Tracing command kernel pid 0 tid 100119 td 0xffffff0007d683a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242be6d30, rbp = 0 --- Tracing command kernel pid 0 tid 100118 td 0xffffff0007d68740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242be1d30, rbp = 0 --- Tracing command kernel pid 0 tid 100117 td 0xffffff0007d68ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bdcd30, rbp = 0 --- Tracing command kernel pid 0 tid 100116 td 0xffffff0007d69000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bd7d30, rbp = 0 --- Tracing command kernel pid 0 tid 100115 td 0xffffff0007d693a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bd2d30, rbp = 0 --- Tracing command kernel pid 0 tid 100114 td 0xffffff0007d69740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bcdd30, rbp = 0 --- Tracing command kernel pid 0 tid 100110 td 0xffffff0007d643a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242babd30, rbp = 0 --- Tracing command kernel pid 0 tid 100107 td 0xffffff0007d63ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b9cd30, rbp = 0 --- Tracing command kernel pid 0 tid 100106 td 0xffffff0007c713a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b97d30, rbp = 0 --- Tracing command kernel pid 0 tid 100105 td 0xffffff0007c71740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b92d30, rbp = 0 --- Tracing command kernel pid 0 tid 100104 td 0xffffff0007c71ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b8dd30, rbp = 0 --- Tracing command kernel pid 0 tid 100103 td 0xffffff0007d73740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b65d30, rbp = 0 --- Tracing command kernel pid 0 tid 100102 td 0xffffff0007d73ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b83d30, rbp = 0 --- Tracing command kernel pid 0 tid 100101 td 0xffffff0007d733a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b6fd30, rbp = 0 --- Tracing command kernel pid 0 tid 100100 td 0xffffff0007f64000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b79d30, rbp = 0 --- Tracing command kernel pid 0 tid 100099 td 0xffffff0007f643a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b74d30, rbp = 0 --- Tracing command kernel pid 0 tid 100098 td 0xffffff0007f64740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b88d30, rbp = 0 --- Tracing command kernel pid 0 tid 100097 td 0xffffff0007f64ae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b7ed30, rbp = 0 --- Tracing command kernel pid 0 tid 100096 td 0xffffff0007f6a000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b6ad30, rbp = 0 --- Tracing command kernel pid 0 tid 100095 td 0xffffff0007f6a3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b4cd30, rbp = 0 --- Tracing command kernel pid 0 tid 100094 td 0xffffff0007f6a740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b5bd30, rbp = 0 --- Tracing command kernel pid 0 tid 100093 td 0xffffff0007f6aae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b60d30, rbp = 0 --- Tracing command kernel pid 0 tid 100092 td 0xffffff0007f6b000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b51d30, rbp = 0 --- Tracing command kernel pid 0 tid 100091 td 0xffffff0007f6b3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b3dd30, rbp = 0 --- Tracing command kernel pid 0 tid 100090 td 0xffffff0007f6b740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b56d30, rbp = 0 --- Tracing command kernel pid 0 tid 100089 td 0xffffff0007f6bae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b42d30, rbp = 0 --- Tracing command kernel pid 0 tid 100088 td 0xffffff0007f6c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b47d30, rbp = 0 --- Tracing command kernel pid 0 tid 100087 td 0xffffff0007f6c3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b38d30, rbp = 0 --- Tracing command kernel pid 0 tid 100086 td 0xffffff0007f6c740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b33d30, rbp = 0 --- Tracing command kernel pid 0 tid 100085 td 0xffffff0007f6cae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242b2ed30, rbp = 0 --- Tracing command kernel pid 0 tid 100084 td 0xffffff0007f6d000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bb5d30, rbp = 0 --- Tracing command kernel pid 0 tid 100083 td 0xffffff0007f6d3a0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bb0d30, rbp = 0 --- Tracing command kernel pid 0 tid 100082 td 0xffffff0007f6d740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8242bbad30, rbp = 0 --- Tracing command kernel pid 0 tid 100036 td 0xffffff000742f000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000135d30, rbp = 0 --- Tracing command kernel pid 0 tid 100033 td 0xffffff000739a000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000126d30, rbp = 0 --- Tracing command kernel pid 0 tid 100031 td 0xffffff000739a740 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff800011cd30, rbp = 0 --- Tracing command kernel pid 0 tid 100030 td 0xffffff000739aae0 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000117d30, rbp = 0 --- Tracing command kernel pid 0 tid 100029 td 0xffffff000739c000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d msleep_spin() at msleep_spin+0x209 taskqueue_thread_loop() at taskqueue_thread_loop+0x62 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff8000112d30, rbp = 0 --- Tracing command kernel pid 0 tid 100024 td 0xffffff000735e000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_wait() at sleepq_wait+0x4d _sleep() at _sleep+0x357 taskqueue_thread_loop() at taskqueue_thread_loop+0xb7 fork_exit() at fork_exit+0x12a fork_trampoline() at fork_trampoline+0xe --- trap 0, rip = 0, rsp = 0xffffff80000f9d30, rbp = 0 --- Tracing command kernel pid 0 tid 100000 td 0xffffffff808c7000 sched_switch() at sched_switch+0x154 mi_switch() at mi_switch+0x21d sleepq_switch() at sleepq_switch+0x123 sleepq_timedwait() at sleepq_timedwait+0x4d _sleep() at _sleep+0x341 scheduler() at scheduler+0x29c mi_startup() at mi_startup+0x59 btext() at btext+0x2c ----- (7) "db> call doadump" ----- (fails!) Physical memory: 20466 MB Dumping 5722 MB: From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 14:08:33 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1BD081065673; Thu, 11 Mar 2010 14:08:33 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 6613E8FC17; Thu, 11 Mar 2010 14:08:32 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2DD69.dip.t-dialin.net [217.226.221.105]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 11FA48444D8; Thu, 11 Mar 2010 15:08:26 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id E5BC5542A; Thu, 11 Mar 2010 15:08:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1268316503; bh=DMkCh8DidYAAaGwTEv1a018dtLXIZDuI6Jlt3phmOZo=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=NOL1fV8lCJXp0iH/JcnisWgYTL3AEd4kq9uBMeeb9u7pVDMpDrszTunnAtAYlMeYJ DpYhBieMEocA0337evOknyRmxm2APEuPef049aCegHuH9BYrG+SxicW6RQUZMUCn6v KsI+c3ilPBrDJQEe8+VTgCCInWZlWI4oeiBt0GY+bFyIO5RxDx1RHjvL/W4zTMNtsS 6Jyabcyl73wNX9aYOsBgX+eTerwfZTGjlbiGbyqAYDgMPtAvm4vxvLduq9FWotBR63 E4kDkOk0xG8vC6KgTxyH1KAziqxJDC+YI+b3YRbGtSqjQDNes3EAqeSMaNNAcLkDbj CgNU4/4slVQMA== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2BE8MTD042722; Thu, 11 Mar 2010 15:08:22 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Thu, 11 Mar 2010 15:08:22 +0100 Message-ID: <20100311150822.107231cvjvgs9gsg@webmail.leidinger.net> Date: Thu, 11 Mar 2010 15:08:22 +0100 From: Alexander Leidinger To: Borja Marcos References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> <20100310173143.GD1715@garage.freebsd.pl> <20100311084527.2934034895hvgxaw@webmail.leidinger.net> <764BD545-B86C-47DC-9004-964EB2216AF0@sarenet.es> In-Reply-To: <764BD545-B86C-47DC-9004-964EB2216AF0@sarenet.es> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 11FA48444D8.551BF X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.363, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, TW_ZF 0.08) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1268921306.69505@9ibt2Iy+sh0nBQqflFRYlw X-EBL-Spam-Status: No Cc: freebsd-fs@FreeBSD.org, Pawel Jakub Dawidek , FreeBSD@FreeBSD.org, Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 14:08:33 -0000 Quoting Borja Marcos (from Thu, 11 Mar 2010 09:54:47 +0100): > > On Mar 11, 2010, at 8:45 AM, Alexander Leidinger wrote: > >> Quoting Pawel Jakub Dawidek (from Wed, 10 Mar >> 2010 18:31:43 +0100): >> >> There is a 4th possibility, if you can rule out everything else: >> bugs in the CPU. I stumbled upon this with ZFS (but UFS was >> exposing the problem much faster). The problem in my case was that >> the BIOS was not recognizing the CPU and as such was not uploading >> microcode updates. >> >> Borja, can you confirm that the CPU is correctly announced in >> FreeBSD (just look at "dmesg | grep CPU:" output, if it tells you >> it is a AMD or Intel XXX CPU it is correctly detected by the BIOS)? > > A CPU bug? Weird. Very. It depends. CPUs have bugs. You do not want to run any modern CPU without an microcode update. The BIOS is doing it for you at system start. I do not want to say that this is the problem you have, I just want to point out that it may be possible (but see below). I got hit by this last december and I was finding the solution (replacing the complete system, as only replacing the CPU was not an option) in January. > As the servers had to be rolled into production, and such tests with > real servers can be quite time consuming, I set up a couple of > FreeBSD virtual machines, using VMWare Fusion (version 2 then, now > version 3) on a Macbook (Macbook 4,1 Intel Core2Duo, 2.1 GHz) and > tried to reproduce it. > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel(R) Xeon(R) CPU L5420 @ 2.50GHz (2496.25-MHz > K8-class CPU) > Origin = "GenuineIntel" Id = 0x1067a Stepping = 10 > The virtual machine (VMWare Fusion 3.0.0, Macbook, Mac OS X 10.6.2) > reports this: > Timecounter "i8254" frequency 1193182 Hz quality 0 > CPU: Intel(R) Core(TM)2 Duo CPU T8100 @ 2.10GHz (2116.62-MHz > K8-class CPU) > Origin = "GenuineIntel" Id = 0x10676 Stepping = 6 Summary: you confirmed the problem on a different kind of CPU. Because of this it makes it even more unlikely that it is a CPU problem. Bye, Alexander. -- A good plan today is better than a perfect plan tomorrow. -- Patton http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 14:09:41 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 755301065670; Thu, 11 Mar 2010 14:09:41 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 086AA8FC17; Thu, 11 Mar 2010 14:09:41 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2DD69.dip.t-dialin.net [217.226.221.105]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id C614C8444D9; Thu, 11 Mar 2010 15:09:36 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id 0216A542B; Thu, 11 Mar 2010 15:09:33 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1268316574; bh=T4U5PK74rC/89MAeNS2ju+5QO61PySgkmqG4/FBVMRU=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=VIw09OfpFPsHM0te/srzc4ONgn+hkdmXFVKc6kFrqXRk75aq7ooaPSVX55UDZ807G RZToslMqoEFZuj3SoPg7zpvtyJh4dtIjYHkor5sY/glUEsJt0jbF5U33M7lFY5v0oI XdNt7aJ6z6DDDY0YAQ412/kraRXScfHOBcBlQxkInxoadsI+PCJdPKyxKCIPJ+qPD5 vSbn6Ksf/H9NpaTQJKSUinQVUBdHxSs7s90Avx4EFbU6vQjTAsjjPBBfvXEpAHjTb4 fhmV98m9erz6yfOg1ilZ3jfmd062cOouP5oLSuJKifnO+Vnd5mNGsZnEEy6CsPaGPp M2aLz8nu7Yofw== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2BE9XUp042953; Thu, 11 Mar 2010 15:09:33 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Thu, 11 Mar 2010 15:09:33 +0100 Message-ID: <20100311150933.18276bypviae5mw4@webmail.leidinger.net> Date: Thu, 11 Mar 2010 15:09:33 +0100 From: Alexander Leidinger To: Ivan Voras References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> <20100310173143.GD1715@garage.freebsd.pl> <20100311084527.2934034895hvgxaw@webmail.leidinger.net> <764BD545-B86C-47DC-9004-964EB2216AF0@sarenet.es> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: C614C8444D9.78C0E X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.363, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, TW_ZF 0.08) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1268921377.22527@02IFKIAdn8Etlq6wt1Ru3g X-EBL-Spam-Status: No Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 14:09:41 -0000 Quoting Ivan Voras (from Thu, 11 Mar 2010 11:59:01 +0100): > On 03/11/10 09:54, Borja Marcos wrote: > > I don't know about the rest but this: > >> CPU: Intel(R) Xeon(R) CPU L5420 @ 2.50GHz (2496.25-MHz >> K8-class CPU) > > does not agree with this: > >> FreeBSD/SMP: 1 package(s) x 8 core(s) > > The Xeon 54xx series does not come in 8 core packages. Either it is > 2xquad-core or a Xeon 55xx. Can also be a problem in the layout detection logic... Bye, Alexander. -- If we can ever make red tape nutritional, we can feed the world. -- R. Schaeberle, "Management Accounting" http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 14:29:10 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D5329106566B for ; Thu, 11 Mar 2010 14:29:10 +0000 (UTC) (envelope-from freebsd-stable@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 53EBC8FC12 for ; Thu, 11 Mar 2010 14:29:09 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1NpjNr-0007cO-1F for freebsd-stable@freebsd.org; Thu, 11 Mar 2010 15:29:07 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 11 Mar 2010 15:29:07 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 11 Mar 2010 15:29:07 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-stable@freebsd.org From: Ivan Voras Date: Thu, 11 Mar 2010 15:28:29 +0100 Lines: 31 Message-ID: References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> <20100310173143.GD1715@garage.freebsd.pl> <20100311084527.2934034895hvgxaw@webmail.leidinger.net> <764BD545-B86C-47DC-9004-964EB2216AF0@sarenet.es> <20100311150933.18276bypviae5mw4@webmail.leidinger.net> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.5) Gecko/20100118 Thunderbird/3.0 In-Reply-To: <20100311150933.18276bypviae5mw4@webmail.leidinger.net> Cc: freebsd-fs@freebsd.org Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 14:29:10 -0000 On 03/11/10 15:09, Alexander Leidinger wrote: > > Quoting Ivan Voras (from Thu, 11 Mar 2010 11:59:01 > +0100): > >> On 03/11/10 09:54, Borja Marcos wrote: >> >> I don't know about the rest but this: >> >>> CPU: Intel(R) Xeon(R) CPU L5420 @ 2.50GHz (2496.25-MHz K8-class CPU) >> >> does not agree with this: >> >>> FreeBSD/SMP: 1 package(s) x 8 core(s) >> >> The Xeon 54xx series does not come in 8 core packages. Either it is >> 2xquad-core or a Xeon 55xx. > > Can also be a problem in the layout detection logic... Not likely, because the 54xx family is very wide spread and nothing special with regards to its topology. It also has the same topology as 53xx. These are systems limited to two physical sockets, each of which can have a single, dual or a quad core CPU and the 5xxx motherboards accept all CPUs from series 50xx, 51xx, 52xx, 53xx, 54xx. In short - these are very, very common systems. My guess would be that someone, somewhere is lying - I don't know if CPUID can be (wrongly) updated by microcode, for example, or if the mptable can be corrupted in a way to represent two physical (socketed) CPUs as one. From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 16:57:48 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D19CA106566B; Thu, 11 Mar 2010 16:57:48 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.freebsd.org (Postfix) with ESMTP id 97AF18FC1E; Thu, 11 Mar 2010 16:57:48 +0000 (UTC) Received: from smtp2.sentex.ca (smtp2c.sentex.ca [64.7.153.30]) by smarthost1.sentex.ca (8.14.3/8.14.3) with ESMTP id o2BGvkQc078449; Thu, 11 Mar 2010 11:57:46 -0500 (EST) (envelope-from tinderbox@freebsd.org) Received: from freebsd-legacy.sentex.ca (freebsd-legacy.sentex.ca [64.7.128.104]) by smtp2.sentex.ca (8.14.4/8.14.4) with ESMTP id o2BGvkQ5065053; Thu, 11 Mar 2010 11:57:46 -0500 (EST) (envelope-from tinderbox@freebsd.org) Received: by freebsd-legacy.sentex.ca (Postfix, from userid 666) id 74EC7241A2; Thu, 11 Mar 2010 11:57:46 -0500 (EST) Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Message-Id: <20100311165746.74EC7241A2@freebsd-legacy.sentex.ca> Date: Thu, 11 Mar 2010 11:57:46 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at smtp2.sentex.ca X-Virus-Status: Clean X-Scanned-By: MIMEDefang 2.64 on 64.7.153.18 Cc: Subject: [releng_6 tinderbox] failure on i386/i386 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 16:57:48 -0000 TB --- 2010-02-12 20:33:34 - tinderbox 2.6 running on freebsd-legacy.sentex.ca TB --- 2010-02-12 20:33:34 - starting RELENG_6 tinderbox run for i386/i386 TB --- 2010-02-12 20:33:34 - cleaning the object tree TB --- 2010-02-12 20:34:04 - cvsupping the source tree TB --- 2010-02-12 20:34:04 - /usr/bin/csup -z -r 3 -g -L 1 -h localhost -s /tinderbox/RELENG_6/i386/i386/supfile TB --- 2010-03-11 16:57:46 - WARNING: /usr/bin/csup returned exit code 1 TB --- 2010-03-11 16:57:46 - ERROR: unable to cvsup the source tree TB --- 2010-03-11 16:57:46 - 4.11 user 4.67 system 2319851.44 real http://tinderbox.freebsd.org/tinderbox-releng_6-RELENG_6-i386-i386.full From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 17:02:08 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 960321065687; Thu, 11 Mar 2010 17:02:08 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.freebsd.org (Postfix) with ESMTP id 5BF5D8FC2F; Thu, 11 Mar 2010 17:02:07 +0000 (UTC) Received: from smtp2.sentex.ca (smtp2c.sentex.ca [64.7.153.30]) by smarthost1.sentex.ca (8.14.3/8.14.3) with ESMTP id o2BH26bG079029; Thu, 11 Mar 2010 12:02:06 -0500 (EST) (envelope-from tinderbox@freebsd.org) Received: from freebsd-legacy.sentex.ca (freebsd-legacy.sentex.ca [64.7.128.104]) by smtp2.sentex.ca (8.14.4/8.14.4) with ESMTP id o2BH26Ws074599; Thu, 11 Mar 2010 12:02:06 -0500 (EST) (envelope-from tinderbox@freebsd.org) Received: by freebsd-legacy.sentex.ca (Postfix, from userid 666) id 67D1D241A2; Thu, 11 Mar 2010 12:02:06 -0500 (EST) Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Message-Id: <20100311170206.67D1D241A2@freebsd-legacy.sentex.ca> Date: Thu, 11 Mar 2010 12:02:06 -0500 (EST) X-Virus-Scanned: clamav-milter 0.95.1 at smtp2.sentex.ca X-Virus-Status: Clean X-Scanned-By: MIMEDefang 2.64 on 64.7.153.18 Cc: Subject: [releng_6 tinderbox] failure on amd64/amd64 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 17:02:08 -0000 TB --- 2010-02-12 20:14:26 - tinderbox 2.6 running on freebsd-legacy.sentex.ca TB --- 2010-02-12 20:14:26 - starting RELENG_6 tinderbox run for amd64/amd64 TB --- 2010-02-12 20:14:26 - cleaning the object tree TB --- 2010-02-12 20:15:02 - cvsupping the source tree TB --- 2010-02-12 20:15:02 - /usr/bin/csup -z -r 3 -g -L 1 -h localhost -s /tinderbox/RELENG_6/amd64/amd64/supfile TB --- 2010-03-11 17:02:06 - WARNING: /usr/bin/csup returned exit code 1 TB --- 2010-03-11 17:02:06 - ERROR: unable to cvsup the source tree TB --- 2010-03-11 17:02:06 - 3.93 user 4.78 system 2321259.84 real http://tinderbox.freebsd.org/tinderbox-releng_6-RELENG_6-amd64-amd64.full From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 17:26:12 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AACC5106564A; Thu, 11 Mar 2010 17:26:12 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from proxypop1.sarenet.es (proxypop1.sarenet.es [194.30.0.99]) by mx1.freebsd.org (Postfix) with ESMTP id 5EE118FC0C; Thu, 11 Mar 2010 17:26:11 +0000 (UTC) Received: from [172.16.1.204] (unknown [192.148.167.2]) by proxypop1.sarenet.es (Postfix) with ESMTP id 8EAA06158; Thu, 11 Mar 2010 18:26:10 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1077) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <20100311150822.107231cvjvgs9gsg@webmail.leidinger.net> Date: Thu, 11 Mar 2010 18:26:09 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <35ADB9B1-F571-4EE4-9089-5363ACEBE159@sarenet.es> References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> <20100310173143.GD1715@garage.freebsd.pl> <20100311084527.2934034895hvgxaw@webmail.leidinger.net> <764BD545-B86C-47DC-9004-964EB2216AF0@sarenet.es> <20100311150822.107231cvjvgs9gsg@webmail.leidinger.net> To: Alexander Leidinger X-Mailer: Apple Mail (2.1077) Cc: freebsd-fs@FreeBSD.org, Pawel Jakub Dawidek , FreeBSD@FreeBSD.org, Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 17:26:12 -0000 On Mar 11, 2010, at 3:08 PM, Alexander Leidinger wrote: >>> Borja, can you confirm that the CPU is correctly announced in = FreeBSD (just look at "dmesg | grep CPU:" output, if it tells you it is = a AMD or Intel XXX CPU it is correctly detected by the BIOS)? >>=20 >> A CPU bug? Weird. Very. >=20 > It depends. CPUs have bugs. You do not want to run any modern CPU = without an microcode update. The BIOS is doing it for you at system = start. >=20 > I do not want to say that this is the problem you have, I just want to = point out that it may be possible (but see below). I got hit by this = last december and I was finding the solution (replacing the complete = system, as only replacing the CPU was not an option) in January. Of course CPUs have bugs, I don't doubt it. I was just wondering how I = coud reproduce the problem with a different hardware :) That's why I = said it was unlikely. Besides, such a low level fault should produce many more problems than = such a well defined failure mode, as far as I know. >> As the servers had to be rolled into production, and such tests with = real servers can be quite time consuming, I set up a couple of FreeBSD = virtual machines, using VMWare Fusion (version 2 then, now version 3) on = a Macbook (Macbook 4,1 Intel Core2Duo, 2.1 GHz) and tried to reproduce = it. >=20 >> Timecounter "i8254" frequency 1193182 Hz quality 0 >> CPU: Intel(R) Xeon(R) CPU L5420 @ 2.50GHz (2496.25-MHz = K8-class CPU) >> Origin =3D "GenuineIntel" Id =3D 0x1067a Stepping =3D 10 >=20 >> The virtual machine (VMWare Fusion 3.0.0, Macbook, Mac OS X 10.6.2) = reports this: >> Timecounter "i8254" frequency 1193182 Hz quality 0 >> CPU: Intel(R) Core(TM)2 Duo CPU T8100 @ 2.10GHz (2116.62-MHz = K8-class CPU) >> Origin =3D "GenuineIntel" Id =3D 0x10676 Stepping =3D 6 >=20 > Summary: you confirmed the problem on a different kind of CPU. >=20 > Because of this it makes it even more unlikely that it is a CPU = problem. Indeed :) Borja. >=20 > Bye, > Alexander. >=20 > --=20 > A good plan today is better than a perfect plan tomorrow. > -- Patton >=20 > http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID =3D = B0063FE7 > http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID =3D = 72077137 From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 18:35:08 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 589AE1065673 for ; Thu, 11 Mar 2010 18:35:08 +0000 (UTC) (envelope-from freebsd-stable@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 107D58FC1D for ; Thu, 11 Mar 2010 18:35:07 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1NpnDs-0004UJ-CY for freebsd-stable@freebsd.org; Thu, 11 Mar 2010 19:35:04 +0100 Received: from 207.155.204.151.ptr.us.xo.net ([207.155.204.151]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 11 Mar 2010 19:35:04 +0100 Received: from atkin901 by 207.155.204.151.ptr.us.xo.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 11 Mar 2010 19:35:04 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-stable@freebsd.org From: Mark Atkinson Date: Thu, 11 Mar 2010 10:25:17 -0800 Lines: 20 Message-ID: References: <20100311133916.42ba69b0@orwell.free.de> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: 207.155.204.151.ptr.us.xo.net User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.1.8) Gecko/20100303 Thunderbird/3.0.3 In-Reply-To: <20100311133916.42ba69b0@orwell.free.de> Cc: freebsd-fs@freebsd.org Subject: Re: proliant server lockups with freebsd-amd64-stable (2010-03-10) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 18:35:08 -0000 On 03/11/10 04:39, Kai Gallasch wrote: > Hi. > > I have some trouble with an opteron server locking up spontaneously. It looses > all networks connectivity and even through console I can get no shell. > > Lockups occur mostly under disk load (periodic daily, bacula backup > running, make buildworld/buildkernel) and I can provoke them easily. try adding: vm.pmap.pg_ps_enabled="0" to /boot/loader.conf and reboot. Long thread reference: http://thread.gmane.org/gmane.os.freebsd.current/121035/focus=121457 possible AMD "errata 147" From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 18:57:35 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A7F711065680; Thu, 11 Mar 2010 18:57:35 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id B494B8FC19; Thu, 11 Mar 2010 18:57:34 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id UAA09304; Thu, 11 Mar 2010 20:57:30 +0200 (EET) (envelope-from avg@icyb.net.ua) Message-ID: <4B993D19.60504@icyb.net.ua> Date: Thu, 11 Mar 2010 20:57:29 +0200 From: Andriy Gapon User-Agent: Thunderbird 2.0.0.23 (X11/20100211) MIME-Version: 1.0 To: Mark Atkinson References: <20100311133916.42ba69b0@orwell.free.de> In-Reply-To: X-Enigmail-Version: 0.95.7 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: proliant server lockups with freebsd-amd64-stable (2010-03-10) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 18:57:35 -0000 on 11/03/2010 20:25 Mark Atkinson said the following: > On 03/11/10 04:39, Kai Gallasch wrote: >> Hi. >> >> I have some trouble with an opteron server locking up spontaneously. It looses >> all networks connectivity and even through console I can get no shell. >> >> Lockups occur mostly under disk load (periodic daily, bacula backup >> running, make buildworld/buildkernel) and I can provoke them easily. > > try adding: > > vm.pmap.pg_ps_enabled="0" The lockups happen with this setting too. This is likely a red herring, because Kai already has Alan's workaround from head. > to /boot/loader.conf and reboot. > > Long thread reference: > > http://thread.gmane.org/gmane.os.freebsd.current/121035/focus=121457 > > possible AMD "errata 147" 147? -- Andriy Gapon From owner-freebsd-stable@FreeBSD.ORG Thu Mar 11 22:05:41 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15D0B106568F for ; Thu, 11 Mar 2010 22:05:41 +0000 (UTC) (envelope-from kenneth.hatteland@kleppnett.no) Received: from asav4.lyse.net (asav4.lyse.net [81.167.36.150]) by mx1.freebsd.org (Postfix) with ESMTP id CA7088FC18 for ; Thu, 11 Mar 2010 22:05:40 +0000 (UTC) Received: from localhost (localhost.localdomain [127.0.0.1]) by asav4.lyse.net (Postfix) with ESMTP id 9EACC6C655 for ; Thu, 11 Mar 2010 22:41:29 +0100 (CET) X-Virus-Scanned: amavisd-new at lyse.net Received: from terra.hatteland1.org (129.81-166-80.customer.lyse.net [81.166.80.129]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: kenneth.hatteland@kleppnett.no) by asav4.lyse.net (Postfix) with ESMTP id 1D2666C65A for ; Thu, 11 Mar 2010 22:41:28 +0100 (CET) Message-ID: <4B996388.4040305@kleppnett.no> Date: Thu, 11 Mar 2010 22:41:28 +0100 From: Kenneth Hatteland User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; nb-NO; rv:1.9.1.8) Gecko/20100307 Thunderbird/3.0.3 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: make installworld problem X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2010 22:05:41 -0000 suddenly after a while my FreeBSD 8.0 stable machine always fails make installworld. it has rarely been a problem before and when I appeared I just rebooted into multi user, csup and build it all once again and then success. Now It always fails with an error saying this or that directory is empty.......and csup and a new buildworld does not fix this. maybe you need more input to have an opinion...just name it and I`ll provide it. My machine functions perfectly, but I can not finalize upgrades right now...buggerem Blessed be.. Kenneth, Norway From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 02:45:50 2010 Return-Path: Delivered-To: FreeBSD-Stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CFACB1065672 for ; Fri, 12 Mar 2010 02:45:50 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id 7101C8FC15 for ; Fri, 12 Mar 2010 02:45:50 +0000 (UTC) Received: (qmail 77836 invoked by uid 89); 12 Mar 2010 02:45:04 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 02:45:03 -0000 Message-ID: <4B99AAD3.9060606@ibctech.ca> Date: Thu, 11 Mar 2010 21:45:39 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: FreeBSD Stable X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: Subject: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 02:45:50 -0000 Hi all, please forgive the verbosity, but I wanted to include as much detail as possible (without including config files) up front. Summary: SSH works to the jail box host OS, but not to the jails themselves. On a box that is dedicated to jails, I source upgraded from 7.2 to: FreeBSD jail.eagle.ca 8.0-STABLE FreeBSD 8.0-STABLE #0: Thu Mar 11 13:22:59 EST 2010 root@jail.eagle.ca:/usr/obj/usr/src/sys/JAIL i386 I then proceeded to update each jail by `make installworld DESTDIR=$JAILDIR', followed by (for most, but not quite all jails), a `mergemaster -Uia'. During the upgrade of the host OS, I know I rebooted here: - after the `installkernel' phase of the host OS - after the `installworld' phase of the host OS ...in between the above stages, I performed a `mergemaster -Uia' Now, I can SSH into the host, but not the jails it contains. I have been Googling for hours trying to figure this out. I've honestly exhausted all of the possibilities I could find regarding the issue, even some that dated back to the good ol' 4.x days. It seemingly has something to do with tty/pts, but I just can't figure it out. When trying to SSH to a jail from the resident host OS: %ssh acct.eagle.ca Password: PTY allocation request failed on channel 0 Warning: no access to tty (Bad file descriptor). Thus no job control in this shell. ...and when I attempt SSH to a jail from a remote box, the connection drops immediately after I enter the password. In both cases, the following is logged into /var/log/messages within the jail host: Mar 11 21:35:40 acct sshd[8175]: error: openpty: No such file or directory Mar 11 21:35:40 acct sshd[8178]: error: session_pty_req: session 0 alloc failed A listing of the dev directory on the jail doesn't show pts (but it does on the host OS): %ll /usr/jails/acct/dev total 1 dr-xr-xr-x 2 root wheel 512 Mar 11 17:12 fd lrwxr-xr-x 1 root wheel 14 Mar 11 17:12 log -> ../var/run/log crw-rw-rw- 1 root wheel 0, 7 Mar 11 21:11 null crw-rw-rw- 1 root wheel 0, 12 Mar 11 12:10 random lrwxr-xr-x 1 root wheel 4 Mar 11 17:12 stderr -> fd/2 lrwxr-xr-x 1 root wheel 4 Mar 11 17:12 stdin -> fd/0 lrwxr-xr-x 1 root wheel 4 Mar 11 17:12 stdout -> fd/1 lrwxr-xr-x 1 root wheel 6 Mar 11 17:12 urandom -> random crw-rw-rw- 1 root wheel 0, 8 Mar 11 17:10 zero ...and I've tried to make Google-advised changes to the /etc/defaults/devfs.rules file to no avail. ;) All other services within the host OS and the jails are operating perfectly. I'm just missing my tty's. Can someone hit me with the cluebat, because I really feel that I've missed something. Thanks, Steve From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 02:49:50 2010 Return-Path: Delivered-To: FreeBSD-Stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E2E8106566B for ; Fri, 12 Mar 2010 02:49:50 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id 626D88FC08 for ; Fri, 12 Mar 2010 02:49:50 +0000 (UTC) Received: by pvg3 with SMTP id 3so315753pvg.13 for ; Thu, 11 Mar 2010 18:49:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=ayFht59q4DydE3ZXmDx8FR4PX9EEt2ZzSj5PGAFPa/Y=; b=Zbq/tw6cpUK37VRms5H8lR8QnntHsvVQqfrs8FKUTKM/82sfyOJspn38l4HqsR3JpP mIIl7RrHnYri2yWWo8PbR3p8u9r7jDT0j8r8MGWiQnvTvRd5HStDLJbuQqFHbhOP2aa/ gCv1bzRsk2B4AUqQKNVC1pByFW15mXq5shh5A= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=bZHGWijIy/gkPIXnEY/VH9vdMljrRtJCOmYGiYbsHbAJDEyd5+E0EOmB92EyFW9ZWW Edhr/qc6EnZYU8q2adBvsHvOavTkGa8ijjwBFI4Kedj0DG2hu1OmdkmZnsp63m68zMel SMKlHJimcCuYNgBaJWTuWlCb/mMTv3m+cf3CA= MIME-Version: 1.0 Received: by 10.115.114.32 with SMTP id r32mr1940724wam.189.1268362189742; Thu, 11 Mar 2010 18:49:49 -0800 (PST) In-Reply-To: <4B99AAD3.9060606@ibctech.ca> References: <4B99AAD3.9060606@ibctech.ca> Date: Thu, 11 Mar 2010 18:49:49 -0800 Message-ID: From: Xin LI To: Steve Bertrand Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD Stable Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 02:49:50 -0000 Sounds like you have something wrong with devfs.rules in jail section? On Mar 11, 2010 6:46 PM, "Steve Bertrand" wrote: Hi all, please forgive the verbosity, but I wanted to include as much detail as possible (without including config files) up front. Summary: SSH works to the jail box host OS, but not to the jails themselves. On a box that is dedicated to jails, I source upgraded from 7.2 to: FreeBSD jail.eagle.ca 8.0-STABLE FreeBSD 8.0-STABLE #0: Thu Mar 11 13:22:59 EST 2010 root@jail.eagle.ca:/usr/obj/usr/src/sys/JAIL i386 I then proceeded to update each jail by `make installworld DESTDIR=$JAILDIR', followed by (for most, but not quite all jails), a `mergemaster -Uia'. During the upgrade of the host OS, I know I rebooted here: - after the `installkernel' phase of the host OS - after the `installworld' phase of the host OS ...in between the above stages, I performed a `mergemaster -Uia' Now, I can SSH into the host, but not the jails it contains. I have been Googling for hours trying to figure this out. I've honestly exhausted all of the possibilities I could find regarding the issue, even some that dated back to the good ol' 4.x days. It seemingly has something to do with tty/pts, but I just can't figure it out. When trying to SSH to a jail from the resident host OS: %ssh acct.eagle.ca Password: PTY allocation request failed on channel 0 Warning: no access to tty (Bad file descriptor). Thus no job control in this shell. ...and when I attempt SSH to a jail from a remote box, the connection drops immediately after I enter the password. In both cases, the following is logged into /var/log/messages within the jail host: Mar 11 21:35:40 acct sshd[8175]: error: openpty: No such file or directory Mar 11 21:35:40 acct sshd[8178]: error: session_pty_req: session 0 alloc failed A listing of the dev directory on the jail doesn't show pts (but it does on the host OS): %ll /usr/jails/acct/dev total 1 dr-xr-xr-x 2 root wheel 512 Mar 11 17:12 fd lrwxr-xr-x 1 root wheel 14 Mar 11 17:12 log -> ../var/run/log crw-rw-rw- 1 root wheel 0, 7 Mar 11 21:11 null crw-rw-rw- 1 root wheel 0, 12 Mar 11 12:10 random lrwxr-xr-x 1 root wheel 4 Mar 11 17:12 stderr -> fd/2 lrwxr-xr-x 1 root wheel 4 Mar 11 17:12 stdin -> fd/0 lrwxr-xr-x 1 root wheel 4 Mar 11 17:12 stdout -> fd/1 lrwxr-xr-x 1 root wheel 6 Mar 11 17:12 urandom -> random crw-rw-rw- 1 root wheel 0, 8 Mar 11 17:10 zero ...and I've tried to make Google-advised changes to the /etc/defaults/devfs.rules file to no avail. ;) All other services within the host OS and the jails are operating perfectly. I'm just missing my tty's. Can someone hit me with the cluebat, because I really feel that I've missed something. Thanks, Steve _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 02:55:33 2010 Return-Path: Delivered-To: FreeBSD-Stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C1466106564A for ; Fri, 12 Mar 2010 02:55:33 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id 606498FC12 for ; Fri, 12 Mar 2010 02:55:33 +0000 (UTC) Received: (qmail 78206 invoked by uid 89); 12 Mar 2010 02:54:47 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 02:54:47 -0000 Message-ID: <4B99AD1B.3040409@ibctech.ca> Date: Thu, 11 Mar 2010 21:55:23 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: Xin LI References: <4B99AAD3.9060606@ibctech.ca> In-Reply-To: X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD Stable Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 02:55:33 -0000 On 2010.03.11 21:49, Xin LI wrote: > Sounds like you have something wrong with devfs.rules in jail section? Did I miss or fsck something up by not reading UPDATING properly? Here is what I have on the host OS: %cat /etc/defaults/devfs.rules | grep -v "^#" [devfsrules_hide_all=1] add hide [devfsrules_unhide_basic=2] add path null unhide add path zero unhide add path crypto unhide add path random unhide add path urandom unhide [devfsrules_unhide_login=3] add path 'ptyp*' unhide add path 'ptyq*' unhide add path 'ptyr*' unhide add path 'ptys*' unhide add path 'ptyP*' unhide add path 'ptyQ*' unhide add path 'ptyR*' unhide add path 'ptyS*' unhide add path 'ttyp*' unhide add path 'ttyq*' unhide add path 'ttyr*' unhide add path 'ttys*' unhide add path 'ttyP*' unhide add path 'ttyQ*' unhide add path 'ttyR*' unhide add path 'ttyS*' unhide add path 'pts/*' unhide add path 'pty/*' unhide add path fd unhide add path 'fd/*' unhide add path stdin unhide add path stdout unhide add path stderr unhide [devfsrules_jail=4] add include $devfsrules_hide_all add include $devfsrules_unhide_basic add include $devfsrules_unhide_login ...and on a jail OS: %cat /usr/jails/devel/etc/defaults/devfs.rules | grep -v "^#" [devfsrules_hide_all=1] add hide [devfsrules_unhide_basic=2] add path null unhide add path zero unhide add path crypto unhide add path random unhide add path urandom unhide [devfsrules_unhide_login=3] add path 'ptyp*' unhide add path 'ptyq*' unhide add path 'ptyr*' unhide add path 'ptys*' unhide add path 'ptyP*' unhide add path 'ptyQ*' unhide add path 'ptyR*' unhide add path 'ptyS*' unhide add path 'ttyp*' unhide add path 'ttyq*' unhide add path 'ttyr*' unhide add path 'ttys*' unhide add path 'ttyP*' unhide add path 'ttyQ*' unhide add path 'ttyR*' unhide add path 'ttyS*' unhide add path 'pts/*' unhide add path 'pty/*' unhide add path fd unhide add path 'fd/*' unhide add path stdin unhide add path stdout unhide add path stderr unhide add path pts unhide [devfsrules_jail=4] add include $devfsrules_hide_all add include $devfsrules_unhide_basic add include $devfsrules_unhide_login Thanks for such a quick response LI, Steve From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 03:00:54 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E927C1065670 for ; Fri, 12 Mar 2010 03:00:54 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id 6D32C8FC17 for ; Fri, 12 Mar 2010 03:00:54 +0000 (UTC) Received: (qmail 78544 invoked by uid 89); 12 Mar 2010 03:00:08 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 03:00:08 -0000 Message-ID: <4B99AE5C.30601@ibctech.ca> Date: Thu, 11 Mar 2010 22:00:44 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: Kenneth Hatteland References: <4B996388.4040305@kleppnett.no> In-Reply-To: <4B996388.4040305@kleppnett.no> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org Subject: Re: make installworld problem X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 03:00:55 -0000 On 2010.03.11 16:41, Kenneth Hatteland wrote: > suddenly after a while my FreeBSD 8.0 stable machine always fails make > installworld. it has rarely been a problem before and when I appeared I > just rebooted into multi user, csup and build it all once again and then > success. > > Now It always fails with an error saying this or that directory is > empty.......and csup and a new buildworld does not fix this. > > maybe you need more input to have an opinion...just name it and I`ll > provide it. - error codes - directory names - did you `rm -rf /usr/obj/*' prior to re-csup'ing > My machine functions perfectly, but I can not finalize > upgrades right now...buggerem Provide a bit more detail with where exactly it is failing ;) Steve From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 03:14:08 2010 Return-Path: Delivered-To: FreeBSD-Stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CCE7F1065673 for ; Fri, 12 Mar 2010 03:14:08 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-px0-f200.google.com (mail-px0-f200.google.com [209.85.216.200]) by mx1.freebsd.org (Postfix) with ESMTP id A3AAB8FC1B for ; Fri, 12 Mar 2010 03:14:08 +0000 (UTC) Received: by pxi38 with SMTP id 38so272526pxi.27 for ; Thu, 11 Mar 2010 19:14:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=/R4HsWPW/4wnvyQjlgNcYTJNKZFHbh6igenOZFNGpks=; b=gzkQ/TWr87rPcAQ6MSppowY1KwbN1fRJRttZiuJVDKO84L/jPzlgb2X5Ba8T62X3q5 1Q/monTptqly9Dj8WZO5CuOsuGxbZiGRoyu3aziKqayrCySjElN8Zrex2Nh2+UQqd9EG x0Rg3M2QAI3dg1+yK6ZWtnbJPODlZTo4UIDTQ= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=up5SrBfIf94DucK6YLq+S4KsFHpwQHaPafKr6N/snm8wAFeur4kyWxOwfviJGPMnIF jfyMh2cx5Vq/1zr/QdGGVHsuds43NA0a2Ha+NvbHnQDVqWqvezvOf3Ic5ZJP+mm5iryQ IxuF8qCHY0OTb0tGhBkRAuIgJsuxRythc+rcM= MIME-Version: 1.0 Received: by 10.115.39.9 with SMTP id r9mr970894waj.140.1268363647213; Thu, 11 Mar 2010 19:14:07 -0800 (PST) In-Reply-To: <4B99AD1B.3040409@ibctech.ca> References: <4B99AAD3.9060606@ibctech.ca> <4B99AD1B.3040409@ibctech.ca> Date: Thu, 11 Mar 2010 19:14:07 -0800 Message-ID: From: Xin LI To: Steve Bertrand Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD Stable Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 03:14:08 -0000 Looks like you have a stale configuration. DId you done mergemaster? Cheers, -- Xin LI http://www.delphij.net From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 03:14:42 2010 Return-Path: Delivered-To: FreeBSD-Stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6AB4B1065678 for ; Fri, 12 Mar 2010 03:14:42 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id E49018FC28 for ; Fri, 12 Mar 2010 03:14:41 +0000 (UTC) Received: (qmail 79229 invoked by uid 89); 12 Mar 2010 03:13:55 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 03:13:55 -0000 Message-ID: <4B99B197.3060504@ibctech.ca> Date: Thu, 11 Mar 2010 22:14:31 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: Xin LI References: <4B99AAD3.9060606@ibctech.ca> In-Reply-To: X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD Stable Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 03:14:42 -0000 On 2010.03.11 21:49, Xin LI wrote: > Sounds like you have something wrong with devfs.rules in jail section? Heh, Thanks Xin LI, I knew the cluebat would work ;) This line: "add path pts unhide" ...I had added to the jails, but NOT to the host /etc/defaults/devfs.rules. It took me sending a cat of each file to realize I was missing it in the host OS file. I had even used diff(8) earlier, but that must have been after much eye-strain. Although I have no idea what "add path pts unhide" actually does, after adding it into the proper location within the host OS filesystem, I'm all back up and rolling ;) Absolute kudos to Richard Bader who started me down the right path in this thread: http://www.mail-archive.com/freebsd-jail@freebsd.org/msg01061.html The only thing that I didn't get from it was that the "add path pts unhide" is required in /etc/defaults/devfs.rules file on the HOST OS. Thanks much! 7.2-8.0 source upgrade with jails is 100% successful! Good work dev team! Steve From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 03:18:27 2010 Return-Path: Delivered-To: FreeBSD-Stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 509B6106564A for ; Fri, 12 Mar 2010 03:18:27 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id C92568FC0A for ; Fri, 12 Mar 2010 03:18:26 +0000 (UTC) Received: (qmail 79299 invoked by uid 89); 12 Mar 2010 03:17:39 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 03:17:39 -0000 Message-ID: <4B99B277.506@ibctech.ca> Date: Thu, 11 Mar 2010 22:18:15 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: Xin LI References: <4B99AAD3.9060606@ibctech.ca> <4B99AD1B.3040409@ibctech.ca> In-Reply-To: X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD Stable Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 03:18:27 -0000 On 2010.03.11 22:14, Xin LI wrote: > Looks like you have a stale configuration. DId you done mergemaster? Yes. I performed it as such: # mergemaster -Uia under the host OS, and then subsequently within each jail. I'm not very familiar with mergemaster, so could my use of the args been incorrect? Steve From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 03:20:12 2010 Return-Path: Delivered-To: FreeBSD-Stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id ED16F10656AB for ; Fri, 12 Mar 2010 03:20:12 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id A686A8FC0A for ; Fri, 12 Mar 2010 03:20:12 +0000 (UTC) Received: (qmail 79338 invoked by uid 89); 12 Mar 2010 03:19:26 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 03:19:26 -0000 Message-ID: <4B99B2E2.6050705@ibctech.ca> Date: Thu, 11 Mar 2010 22:20:02 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: Xin LI References: <4B99AAD3.9060606@ibctech.ca> <4B99B197.3060504@ibctech.ca> In-Reply-To: <4B99B197.3060504@ibctech.ca> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: FreeBSD Stable Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 03:20:13 -0000 On 2010.03.11 22:14, Steve Bertrand wrote: > I had even used diff(8) earlier, err, diff(1) From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 03:53:54 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6C2D1065673 for ; Fri, 12 Mar 2010 03:53:54 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id 4FF168FC1F for ; Fri, 12 Mar 2010 03:53:54 +0000 (UTC) Received: (qmail 80130 invoked by uid 89); 12 Mar 2010 03:53:08 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 03:53:08 -0000 Message-ID: <4B99BAC7.8080509@ibctech.ca> Date: Thu, 11 Mar 2010 22:53:43 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: FreeBSD X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-jail@freebsd.org Subject: Jails & 8.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 03:53:54 -0000 Sorry for the cross-post, but this is a 'thank-you', not a request for help. I want to express my sincere appreciation for all of those who made FreeBSD jails a viable virtual server solution for us who required multiple IPs, particularly those who demand/require IPv6 support: %jls -v JID Hostname Path Name State CPUSetID IP Address(es) [..snip..] 8 v6only /usr/jails/onlyv6.com 8 ACTIVE 9 208.70.104.90 2607:f118:8c0:800::64 Not only that, FreeBSD 8 is just absolutely fantastic. Although I've only been using FreeBSD since 4.3, I never could have dreamt that the OS would ever have a release that is so close to its core values, but at the same time so feature rich and stable, particularly for those who like to use the OS as a network (L2/L3) platform in many cases. My hats off. Thanks all! What a tremendous job. Steve From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 03:55:07 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A9FD2106566B for ; Fri, 12 Mar 2010 03:55:07 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 47BAF8FC1A for ; Fri, 12 Mar 2010 03:55:07 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 3E14BA55114; Fri, 12 Mar 2010 11:55:06 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id 3WWtmb2uYs5h; Fri, 12 Mar 2010 11:55:00 +0800 (CST) Received: from delta.delphij.net (140a.hackerdojo.com [66.92.0.185]) (using SSLv3 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 6C9DDA5512C; Fri, 12 Mar 2010 11:54:58 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:subject:references:in-reply-to:x-enigmail-version:openpgp: content-type:content-transfer-encoding; b=K92fEPKnH4wIRWsd6CZFY3MiztS9APziwNa97XyX8dadpZtPmLjM5BVjrjsj+DNON smf1iYFZm2QchcsD3IucA== Message-ID: <4B99BB0E.7080000@delphij.net> Date: Thu, 11 Mar 2010 19:54:54 -0800 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100304 Thunderbird/3.0.3 ThunderBrowse/3.2.8.1 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4B99AAD3.9060606@ibctech.ca> <4B99B197.3060504@ibctech.ca> <4B99B2E2.6050705@ibctech.ca> In-Reply-To: <4B99B2E2.6050705@ibctech.ca> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 03:55:07 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Weird, it seems that RELENG_8_0 do contain the correct file... What does 'ident /etc/defaults/devfs.rules' show? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLmbsOAAoJEATO+BI/yjfBYWwH/jLink3OiDf1Rth8Id+BZL8B P8I2+0jgCny3DjDltRqy0s1+rzF0hjQjpUrUR/T/lPIBRqi2CgQ6RuKe4fHjlZ5c JR2p7AI0BnrU6gpJmVyhoROUZAEEMIrqxXji00N/o+ZuRvuVO+YfVYnuJrMcp78+ vll0bWrHp1xevXf6cH4FcTaurKsexZAZbBvHnrWL6v8WWK1+CtuRmbWjStaNg2tK 6SSOE3P2IEXVW3LNQ9iHV0GycgnbdnsYTTOjsQG2BQTpNRozMvQk3f9oiVMHDn2F a2vBewVkWkzfTX9IBX6o6aYbIInRQo0TYitj5nnP8vI+nF4WI/hnAebv9dO4ymQ= =6LSi -----END PGP SIGNATURE----- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 04:26:35 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 358A51065679 for ; Fri, 12 Mar 2010 04:26:35 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id AF4CE8FC20 for ; Fri, 12 Mar 2010 04:26:34 +0000 (UTC) Received: (qmail 81507 invoked by uid 89); 12 Mar 2010 04:25:49 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 04:25:49 -0000 Message-ID: <4B99C270.9040703@ibctech.ca> Date: Thu, 11 Mar 2010 23:26:24 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: d@delphij.net References: <4B99AAD3.9060606@ibctech.ca> <4B99B197.3060504@ibctech.ca> <4B99B2E2.6050705@ibctech.ca> <4B99BB0E.7080000@delphij.net> In-Reply-To: <4B99BB0E.7080000@delphij.net> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, Xin LI Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 04:26:35 -0000 On 2010.03.11 22:54, Xin LI wrote: > Weird, it seems that RELENG_8_0 do contain the correct file... What > does 'ident /etc/defaults/devfs.rules' show? ...not necessarily 8_0: %grep tag /etc/supfile *default tag=RELENG_8 fwiw: %ident /etc/defaults/devfs.rules /etc/defaults/devfs.rules: $FreeBSD: src/etc/defaults/devfs.rules,v 1.4.2.1.6.1 2009/04/15 03:14:26 kensmith Exp $ ...again... perhaps my arg list to mergemaster was the issue. If it was, it would be good to know ;) Otherwise, it was a standard src upgrade. Let me know if you need anything further. I can simulate the procedure on other hardware tomorrow to isolate the issue if you feel that this may affect others, or if it can prove that I was just sleeping on the job. Steve From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 04:30:44 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 64C06106566B for ; Fri, 12 Mar 2010 04:30:44 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 04AD88FC1A for ; Fri, 12 Mar 2010 04:30:44 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 4AED9A551AF; Fri, 12 Mar 2010 12:30:42 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id ym6mgeDyrCkq; Fri, 12 Mar 2010 12:30:36 +0800 (CST) Received: from delta.delphij.net (140a.hackerdojo.com [66.92.0.185]) (using SSLv3 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 260D6A55114; Fri, 12 Mar 2010 12:30:34 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=QsahSd0+13ua4SH1F9kPkBGeOqdb0QFuXJRLC8KLvDcInAwEVZ7MpwnpAwkQllyeY FI6PMUugPNiAFXDlY6tSg== Message-ID: <4B99C368.6080704@delphij.net> Date: Thu, 11 Mar 2010 20:30:32 -0800 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100304 Thunderbird/3.0.3 ThunderBrowse/3.2.8.1 MIME-Version: 1.0 To: Steve Bertrand References: <4B99AAD3.9060606@ibctech.ca> <4B99B197.3060504@ibctech.ca> <4B99B2E2.6050705@ibctech.ca> <4B99BB0E.7080000@delphij.net> <4B99C270.9040703@ibctech.ca> In-Reply-To: <4B99C270.9040703@ibctech.ca> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, d@delphij.net Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 04:30:44 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2010/03/11 20:26, Steve Bertrand wrote: > On 2010.03.11 22:54, Xin LI wrote: >> Weird, it seems that RELENG_8_0 do contain the correct file... What >> does 'ident /etc/defaults/devfs.rules' show? > > ...not necessarily 8_0: > > %grep tag /etc/supfile > > *default tag=RELENG_8 Oh that shouldn't a problem :) > fwiw: > > %ident /etc/defaults/devfs.rules > > /etc/defaults/devfs.rules: > $FreeBSD: src/etc/defaults/devfs.rules,v 1.4.2.1.6.1 2009/04/15 > 03:14:26 kensmith Exp $ This is a old version which comes from 7.2, so that's the problem... > ...again... perhaps my arg list to mergemaster was the issue. If it was, > it would be good to know ;) > > Otherwise, it was a standard src upgrade. > > Let me know if you need anything further. I can simulate the procedure > on other hardware tomorrow to isolate the issue if you feel that this > may affect others, or if it can prove that I was just sleeping on the job. Could you please check ident on /usr/src/etc/defaults/devfs.rules? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLmcNoAAoJEATO+BI/yjfBnQwH/jN//MTEruj3TSPOcQBqmEJd ii9wFhqQfZOHboAAkzNeARX+obGpDhJSEtwSpf4MpoK5jq9i5fEhHaE1uIGmlt+4 ZM9oZkN3elTnTrscU4v9CddPgPXrjSfZMavWQUk/gmoksRrXKAmsdSfAubZBqC9m BpcNaR5+V/DmMfLQqvG3Clfh7/yooyb/X+t6zgcEWNZXGXk/xEwglho7njdQF/LS IFNF9aaPqn+kSucQyVPBKGpQXN98dn4aKjDDu9/Vz4CvDVHNoAL7uNuiZgAC0JD9 Jj5UIYOz7xXzDqNx37jgoMPnqQlxAtsUAQDDxerBcKdIgRlSunpyc5i2W+2lA1o= =dr6j -----END PGP SIGNATURE----- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 04:33:30 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D5291065673 for ; Fri, 12 Mar 2010 04:33:30 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 21A6A8FC1C for ; Fri, 12 Mar 2010 04:33:29 +0000 (UTC) Received: by pwj4 with SMTP id 4so488345pwj.13 for ; Thu, 11 Mar 2010 20:33:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=u2EuJTlFdT4Yf4LfBs+h0JCtamC9W6RNMf9BDYTPgoM=; b=NUd3RBM4KV+ZXiWUo0QrQU4D8TBy60tBDuJZsTLR+cRhhRiOYkYKJDUQnF82WVF7na yTf3OuBflFo5kn6dz4HMhxDPqEQhqamY74LdLBmIGUo6+nZ4QPE2aY/r1B/kUJnisHuz n7Z/KtebRqY80kXgkp0Cj0oCMpnQ1aSS6joC0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=brW1bQLyZtmWosaOfoVKhErqxa8hjZskkVPmPw50bc5Ly51Pn2oCXH5g7+f8fXnqlC lWflzuZ92z9VaZgag7ATdEPJX/neb1VyOLThmRz44x3JeXLcs/DRmA5Q0nlvhrJ1vsHb 9r1tS/pRnsmJSltdE67XTQu8xEwZ+zCdJpVPM= MIME-Version: 1.0 Received: by 10.143.27.20 with SMTP id e20mr2033530wfj.256.1268368409498; Thu, 11 Mar 2010 20:33:29 -0800 (PST) In-Reply-To: <4B99C270.9040703@ibctech.ca> References: <4B99AAD3.9060606@ibctech.ca> <4B99B197.3060504@ibctech.ca> <4B99B2E2.6050705@ibctech.ca> <4B99BB0E.7080000@delphij.net> <4B99C270.9040703@ibctech.ca> Date: Thu, 11 Mar 2010 20:33:29 -0800 Message-ID: <7d6fde3d1003112033k3ce3b040y8edac8772747862f@mail.gmail.com> From: Garrett Cooper To: Steve Bertrand Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org, d@delphij.net, Xin LI Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 04:33:30 -0000 On Thu, Mar 11, 2010 at 8:26 PM, Steve Bertrand wrote: > On 2010.03.11 22:54, Xin LI wrote: >> Weird, it seems that RELENG_8_0 do contain the correct file... =A0What >> does 'ident /etc/defaults/devfs.rules' show? > > ...not necessarily 8_0: > > %grep tag /etc/supfile > > *default tag=3DRELENG_8 > > fwiw: > > %ident /etc/defaults/devfs.rules > > /etc/defaults/devfs.rules: > =A0 =A0 $FreeBSD: src/etc/defaults/devfs.rules,v 1.4.2.1.6.1 2009/04/15 > 03:14:26 kensmith Exp $ > > ...again... perhaps my arg list to mergemaster was the issue. If it was, > it would be good to know ;) > > Otherwise, it was a standard src upgrade. > > Let me know if you need anything further. I can simulate the procedure > on other hardware tomorrow to isolate the issue if you feel that this > may affect others, or if it can prove that I was just sleeping on the job= . I've done a few RELENG_8_0 to STABLE-8 to 9-CURRENT upgrades lately and mergemaster was goofing up the contents a bit based on the RCS versions. I had to hand-edit a crapload of stuff going from 8 to 9, and I still don't trust mergemaster's automatic merging logic because it goofs up on /etc/group // /etc/passwd still (doesn't merge anything, discards my info, etc) for starters. -a doesn't actually do any merging though, FWIW: -a Run automatically. This option will leave all the files t= hat differ from the installed versions in the temporary direct= ory to be dealt with by hand. If the temproot directory exist= s, it creates a new one in a previously non-existent director= y. This option unsets the verbose flag, but other than -U it = is compatible with all other options. Setting -a makes -w superfluous. Also, "add path pts unhide" unmasks all psuedo TTY dev nodes so that applications that use openpty(3) and friends (like sshd) can allocate them at login. Thanks, -Garrett From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 04:38:28 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 76D3B1065673 for ; Fri, 12 Mar 2010 04:38:28 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id 15F0F8FC12 for ; Fri, 12 Mar 2010 04:38:27 +0000 (UTC) Received: (qmail 81903 invoked by uid 89); 12 Mar 2010 04:37:42 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 04:37:42 -0000 Message-ID: <4B99C53A.4010009@ibctech.ca> Date: Thu, 11 Mar 2010 23:38:18 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: d@delphij.net References: <4B99AAD3.9060606@ibctech.ca> <4B99B197.3060504@ibctech.ca> <4B99B2E2.6050705@ibctech.ca> <4B99BB0E.7080000@delphij.net> <4B99C270.9040703@ibctech.ca> <4B99C368.6080704@delphij.net> In-Reply-To: <4B99C368.6080704@delphij.net> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, Xin LI Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 04:38:28 -0000 On 2010.03.11 23:30, Xin LI wrote: > On 2010/03/11 20:26, Steve Bertrand wrote: >> On 2010.03.11 22:54, Xin LI wrote: >>> Weird, it seems that RELENG_8_0 do contain the correct file... What >>> does 'ident /etc/defaults/devfs.rules' show? > >> ...not necessarily 8_0: > >> %grep tag /etc/supfile > >> *default tag=RELENG_8 > > Oh that shouldn't a problem :) > >> fwiw: > >> %ident /etc/defaults/devfs.rules > >> /etc/defaults/devfs.rules: >> $FreeBSD: src/etc/defaults/devfs.rules,v 1.4.2.1.6.1 2009/04/15 >> 03:14:26 kensmith Exp $ > > This is a old version which comes from 7.2, so that's the problem... > >> ...again... perhaps my arg list to mergemaster was the issue. If it was, >> it would be good to know ;) > >> Otherwise, it was a standard src upgrade. > >> Let me know if you need anything further. I can simulate the procedure >> on other hardware tomorrow to isolate the issue if you feel that this >> may affect others, or if it can prove that I was just sleeping on the job. > > Could you please check ident on /usr/src/etc/defaults/devfs.rules? %uname -a FreeBSD jail.eagle.ca 8.0-STABLE FreeBSD 8.0-STABLE #0: Thu Mar 11 13:22:59 EST 2010 root@jail.eagle.ca:/usr/obj/usr/src/sys/JAIL i386 %ident /usr/src/etc/defaults/devfs.rules /usr/src/etc/defaults/devfs.rules: $FreeBSD: src/etc/defaults/devfs.rules,v 1.6.2.2 2010/01/11 18:42:29 jhb Exp $ Steve ps. I can see that I did something wrong ;) From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 04:41:41 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6BB21065674 for ; Fri, 12 Mar 2010 04:41:41 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [IPv6:2001:470:a803::1]) by mx1.freebsd.org (Postfix) with ESMTP id 8ED0B8FC1A for ; Fri, 12 Mar 2010 04:41:40 +0000 (UTC) Received: from mail.geekcn.org (tarsier.geekcn.org [211.166.10.233]) by tarsier.geekcn.org (Postfix) with ESMTP id 8B0ACA56290; Fri, 12 Mar 2010 12:41:39 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([211.166.10.233]) by mail.geekcn.org (mail.geekcn.org [211.166.10.233]) (amavisd-new, port 10024) with LMTP id RrCBe0HUqv6c; Fri, 12 Mar 2010 12:41:33 +0800 (CST) Received: from delta.delphij.net (140a.hackerdojo.com [66.92.0.185]) (using SSLv3 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 86049A562A1; Fri, 12 Mar 2010 12:41:32 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:subject:references:in-reply-to:x-enigmail-version:openpgp: content-type:content-transfer-encoding; b=KI7etZj2mWhZcScaffENyxmgXNrVBrUCPlnbQYXKQ4hsFES+CaXk6z55qG0f1MBiP BlIAX/+NHr5SRyVHNSXYw== Message-ID: <4B99C5FA.4020801@delphij.net> Date: Thu, 11 Mar 2010 20:41:30 -0800 From: Xin LI Organization: The Geek China Organization User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.8) Gecko/20100304 Thunderbird/3.0.3 ThunderBrowse/3.2.8.1 MIME-Version: 1.0 To: freebsd-stable@freebsd.org References: <4B99AAD3.9060606@ibctech.ca> <4B99B197.3060504@ibctech.ca> <4B99B2E2.6050705@ibctech.ca> <4B99BB0E.7080000@delphij.net> <4B99C270.9040703@ibctech.ca> <7d6fde3d1003112033k3ce3b040y8edac8772747862f@mail.gmail.com> In-Reply-To: <7d6fde3d1003112033k3ce3b040y8edac8772747862f@mail.gmail.com> X-Enigmail-Version: 1.0.1 OpenPGP: id=3FCA37C1; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 04:41:42 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 2010/03/11 20:33, Garrett Cooper wrote: > I've done a few RELENG_8_0 to STABLE-8 to 9-CURRENT upgrades lately > and mergemaster was goofing up the contents a bit based on the RCS > versions. I had to hand-edit a crapload of stuff going from 8 to 9, > and I still don't trust mergemaster's automatic merging logic because > it goofs up on /etc/group // /etc/passwd still (doesn't merge > anything, discards my info, etc) for starters. > > -a doesn't actually do any merging though, FWIW: > > -a Run automatically. This option will leave all the files that > differ from the installed versions in the temporary directory > to be dealt with by hand. If the temproot directory exists, > it creates a new one in a previously non-existent directory. > This option unsets the verbose flag, but other than -U it is > compatible with all other options. Setting -a makes -w > superfluous. > > Also, "add path pts unhide" unmasks all psuedo TTY dev nodes so that > applications that use openpty(3) and friends (like sshd) can allocate > them at login. Yes that's right. Thanks for posting, perhaps we can document it in FAQ? Cheers, - -- Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! Live free or die -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iQEcBAEBAgAGBQJLmcX6AAoJEATO+BI/yjfB4lgIAKF4XMGFSU0VsvmfUD2WIWoE N5UQTKiew4cyytk80SS0ProGemMnWykAfZIDyhovCxp7WRAPJMmrqhulcYXHi+hW DBP4Dx8AsHu/pQ0edauIfnEXYy4gd9g9D6yRIS8407izYTPYs1FvfW0GeftLwEfn Ngm/bYggw1g/Nc5ercqc3AV1vXJJSZ6c3VL8r1ohlsiCRlybMaXBxQrU/KSjDFoz /xI4TOJ5+L+gCpfFB8Tvso8IueZ/YqVSPKnC95bOwGR16sERmACZ7rRUIg7zXyhA VhKttjqfLrHRn6yp905OvS3rkmtCWBfpqIM8Ia34owjSlZbTUK7i0phiDw3uADE= =IXRh -----END PGP SIGNATURE----- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 04:45:52 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 036DB1065670 for ; Fri, 12 Mar 2010 04:45:52 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id 9521D8FC1E for ; Fri, 12 Mar 2010 04:45:51 +0000 (UTC) Received: (qmail 82471 invoked by uid 89); 12 Mar 2010 04:45:06 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 04:45:06 -0000 Message-ID: <4B99C6FE.7040801@ibctech.ca> Date: Thu, 11 Mar 2010 23:45:50 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: Garrett Cooper References: <4B99AAD3.9060606@ibctech.ca> <4B99B197.3060504@ibctech.ca> <4B99B2E2.6050705@ibctech.ca> <4B99BB0E.7080000@delphij.net> <4B99C270.9040703@ibctech.ca> <7d6fde3d1003112033k3ce3b040y8edac8772747862f@mail.gmail.com> In-Reply-To: <7d6fde3d1003112033k3ce3b040y8edac8772747862f@mail.gmail.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, d@delphij.net, Xin LI Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 04:45:52 -0000 On 2010.03.11 23:33, Garrett Cooper wrote: > On Thu, Mar 11, 2010 at 8:26 PM, Steve Bertrand wrote: >> On 2010.03.11 22:54, Xin LI wrote: >>> Weird, it seems that RELENG_8_0 do contain the correct file... What >>> does 'ident /etc/defaults/devfs.rules' show? >> >> ...not necessarily 8_0: >> >> %grep tag /etc/supfile >> >> *default tag=RELENG_8 >> >> fwiw: >> >> %ident /etc/defaults/devfs.rules >> >> /etc/defaults/devfs.rules: >> $FreeBSD: src/etc/defaults/devfs.rules,v 1.4.2.1.6.1 2009/04/15 >> 03:14:26 kensmith Exp $ >> >> ...again... perhaps my arg list to mergemaster was the issue. If it was, >> it would be good to know ;) >> >> Otherwise, it was a standard src upgrade. >> >> Let me know if you need anything further. I can simulate the procedure >> on other hardware tomorrow to isolate the issue if you feel that this >> may affect others, or if it can prove that I was just sleeping on the job. > > I've done a few RELENG_8_0 to STABLE-8 to 9-CURRENT upgrades lately > and mergemaster was goofing up the contents a bit based on the RCS > versions. I had to hand-edit a crapload of stuff going from 8 to 9, > and I still don't trust mergemaster's automatic merging logic because > it goofs up on /etc/group // /etc/passwd still (doesn't merge > anything, discards my info, etc) for starters. > > -a doesn't actually do any merging though, FWIW: heh. Then this explains it. ...at least my own idiocy (or perhaps lack thereof for trying too hard) keeps me at the OS troubleshooting level ;) So long as the OS I'm troubleshooting doesn't have a gui, I'm ok with it. I'm obviously going to get a better understanding of mergemaster. More importantly, I have a colleague who wanted to help this afternoon, but I didn't know how to describe the problem. This thread will help. Thanks! Steve From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 04:59:13 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 563A01065670 for ; Fri, 12 Mar 2010 04:59:13 +0000 (UTC) (envelope-from steve@ibctech.ca) Received: from smtp.ibctech.ca (v6.ibctech.ca [IPv6:2607:f118::b6]) by mx1.freebsd.org (Postfix) with SMTP id E8EED8FC08 for ; Fri, 12 Mar 2010 04:59:12 +0000 (UTC) Received: (qmail 82980 invoked by uid 89); 12 Mar 2010 04:58:25 -0000 Received: from unknown (HELO ?192.168.1.114?) (steve@ibctech.ca@::ffff:208.70.104.100) by ::ffff:208.70.104.210 with ESMTPA; 12 Mar 2010 04:58:25 -0000 Message-ID: <4B99CA1D.4070905@ibctech.ca> Date: Thu, 11 Mar 2010 23:59:09 -0500 From: Steve Bertrand User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: d@delphij.net References: <4B99AAD3.9060606@ibctech.ca> <4B99B197.3060504@ibctech.ca> <4B99B2E2.6050705@ibctech.ca> <4B99BB0E.7080000@delphij.net> <4B99C270.9040703@ibctech.ca> <7d6fde3d1003112033k3ce3b040y8edac8772747862f@mail.gmail.com> <4B99C5FA.4020801@delphij.net> In-Reply-To: <4B99C5FA.4020801@delphij.net> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, Xin LI Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 04:59:13 -0000 On 2010.03.11 23:41, Xin LI wrote: > On 2010/03/11 20:33, Garrett Cooper wrote: >> I've done a few RELENG_8_0 to STABLE-8 to 9-CURRENT upgrades lately >> and mergemaster was goofing up the contents a bit based on the RCS >> versions. I had to hand-edit a crapload of stuff going from 8 to 9, >> and I still don't trust mergemaster's automatic merging logic because >> it goofs up on /etc/group // /etc/passwd still (doesn't merge >> anything, discards my info, etc) for starters. > >> -a doesn't actually do any merging though, FWIW: > >> -a Run automatically. This option will leave all the files that >> differ from the installed versions in the temporary directory >> to be dealt with by hand. If the temproot directory exists, >> it creates a new one in a previously non-existent directory. >> This option unsets the verbose flag, but other than -U it is >> compatible with all other options. Setting -a makes -w >> superfluous. > >> Also, "add path pts unhide" unmasks all psuedo TTY dev nodes so that >> applications that use openpty(3) and friends (like sshd) can allocate >> them at login. > > Yes that's right. Thanks for posting, perhaps we can document it in FAQ? Well, if anything, the FAQ should simply say "read the damn handbook and follow it". If I had of followed the same procedure that I've been using for the last 7 years (at least) instead of trying to 'automate' (-a) prior to reading the man page thoroughly, I would have been ok. Sorry for the noise, but thanks for the help ;) Cheers, Steve From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 07:30:04 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB5EA1065677 for ; Fri, 12 Mar 2010 07:30:04 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-pz0-f199.google.com (mail-pz0-f199.google.com [209.85.222.199]) by mx1.freebsd.org (Postfix) with ESMTP id 9F2DA8FC26 for ; Fri, 12 Mar 2010 07:30:04 +0000 (UTC) Received: by pzk37 with SMTP id 37so254072pzk.7 for ; Thu, 11 Mar 2010 23:30:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=4ecNZ1nadSVzYXryhRjpilSRZCyHg+AgJAJuZ+N+TnI=; b=rjCrU4PDVXSZRs7CRoF6j9nGY7+WKwHcd6Eud2g8e8ET0SsPk8+3otorRMHHEFLdBl ajfwjL2NlPgMxFnGyWnnsgUhNAGhyv7ofgIX69SchGycL/udzFqAuEQXzeMFmuXSnwLl A0FmzjKBIBic3Hfw3bexeHR1rJ2NdcpQrqO14= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=bP+jAxgIuBPwtaQkD+Vd3BPrwWFhDWe3cQvkDp1Pjfsg7P253rIs5vVFLHmjgmqjGC WjTl8CRBxhOds85GobExKyw5Qyv65aE3X3dA7ONYCMa3RvMShCoWiyfjODOI5ypvlN2U yfJ8figGdqdfzBIuuryGsNE0aPspr7Zz70GOM= MIME-Version: 1.0 Received: by 10.143.27.11 with SMTP id e11mr177894wfj.280.1268379004255; Thu, 11 Mar 2010 23:30:04 -0800 (PST) In-Reply-To: <4B99C5FA.4020801@delphij.net> References: <4B99AAD3.9060606@ibctech.ca> <4B99B197.3060504@ibctech.ca> <4B99B2E2.6050705@ibctech.ca> <4B99BB0E.7080000@delphij.net> <4B99C270.9040703@ibctech.ca> <7d6fde3d1003112033k3ce3b040y8edac8772747862f@mail.gmail.com> <4B99C5FA.4020801@delphij.net> Date: Thu, 11 Mar 2010 23:30:04 -0800 Message-ID: <7d6fde3d1003112330s15b0d1edjdcd885d29c4680d0@mail.gmail.com> From: Garrett Cooper To: d@delphij.net Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable@freebsd.org Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 07:30:05 -0000 On Thu, Mar 11, 2010 at 8:41 PM, Xin LI wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 2010/03/11 20:33, Garrett Cooper wrote: >> I've done a few RELENG_8_0 to STABLE-8 to 9-CURRENT upgrades lately >> and mergemaster was goofing up the contents a bit based on the RCS >> versions. I had to hand-edit a crapload of stuff going from 8 to 9, >> and I still don't trust mergemaster's automatic merging logic because >> it goofs up on /etc/group // /etc/passwd still (doesn't merge >> anything, discards my info, etc) for starters. >> >> -a doesn't actually do any merging though, FWIW: >> >> =A0 =A0 =A0-a =A0 =A0 =A0 =A0 =A0Run automatically. =A0This option will = leave all the files that >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0differ from the installed versions in= the temporary directory >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0to be dealt with by hand. =A0If the t= emproot directory exists, >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0it creates a new one in a previously = non-existent directory. >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0This option unsets the verbose flag, = but other than -U it is >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0compatible with all other options. = =A0Setting -a makes -w >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0superfluous. >> >> Also, "add path pts unhide" unmasks all psuedo TTY dev nodes so that >> applications that use openpty(3) and friends (like sshd) can allocate >> them at login. > > Yes that's right. =A0Thanks for posting, perhaps we can document it in FA= Q? Unfortunately this doesn't really fit in a FAQ because it could be any of the failures caused by openpty failure: ERRORS The openpty() function may fail and set the global variable errno for = any of the errors specified for the grantpt(3), posix_openpt(3), ptsname(3= ), and unlockpt(3) functions and the revoke(2) system call. In addition to this, forkpty() may set it to any value as described fo= r fork(2). Note how openpty(3) isn't descriptive in terms of _what_ individual failures can be reported by it, but instead point to other manpages :)... I've run into whacky ass issues with /var/run permissions in the past, etc, which is why I don't suggest putting this into a FAQ -- the bucket is way too huge to deal with. Steve's right though -- if you're willing to use an option and the documentation clearly states the exact behavior and caveats, you are responsible for the results of any failure through misuse of the tool (exceptions being IMO NUL pointer exceptions, abort(3) calls not thrown by assert(3)s, etc). You're more than welcome to make suggestions on how things can be improved, but as I've discovered in some areas, some of the suggestions one makes aren't necessarily accepted with open minds. Excessive and/or redundant documentation (as I'm starting to understand after playing the opposite side for a while) is no better than little or no documentation; complete and concise documentation is what's needed more than anything else. Thanks, -Garrett From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 08:10:45 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BA2E2106567A; Fri, 12 Mar 2010 08:10:45 +0000 (UTC) (envelope-from alexander@leidinger.net) Received: from mail.ebusiness-leidinger.de (mail.ebusiness-leidinger.de [217.11.53.44]) by mx1.freebsd.org (Postfix) with ESMTP id 3E6EC8FC13; Fri, 12 Mar 2010 08:10:45 +0000 (UTC) Received: from outgoing.leidinger.net (pD9E2D538.dip.t-dialin.net [217.226.213.56]) by mail.ebusiness-leidinger.de (Postfix) with ESMTPSA id 434288453C9; Fri, 12 Mar 2010 09:10:39 +0100 (CET) Received: from webmail.leidinger.net (webmail.leidinger.net [192.168.1.102]) by outgoing.leidinger.net (Postfix) with ESMTP id AB6A654CE; Fri, 12 Mar 2010 09:10:35 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=Leidinger.net; s=outgoing-alex; t=1268381435; bh=ztVDoIz8rrYJYFhIrv+VOWSa+dq8hHAR593pDFwMG/Q=; h=Message-ID:Date:From:To:Cc:Subject:References:In-Reply-To: MIME-Version:Content-Type:Content-Transfer-Encoding; b=Rq8NA3leV7MKGmhZAC2eaOzyT8/RGzuq4X0JUrVxDYxVU7ia6ML/f+CDpwN35jGRX D2lkIMc8GwdgIH75ZsmChI+JOWU5Sjd1mFJTyMoKazdVOh7SCEuJR0ERhV7+kNGdDM qHRtXzM+vN+oz5UDxBytIHMTm3oMqRY2qcfAQnR2YcZW9ebSw+ia8w4qo4MYGlzAyd 7ZV1CgH/jNrYUoA5KRXG7w71PXfAM2LXKwOYKJtLCE5DiT2uP39YVuXCcuFsgvl8uL GJFHsS7Ha8ElcCS4dW5yzW+QlfIEK+5vCUz9OXUd3c6jiJQddfoKwndvcjTkprkjjk NzFPXLLcBaHQQ== Received: (from www@localhost) by webmail.leidinger.net (8.14.3/8.13.8/Submit) id o2C8AZZf093080; Fri, 12 Mar 2010 09:10:35 +0100 (CET) (envelope-from Alexander@Leidinger.net) Received: from pslux.cec.eu.int (pslux.cec.eu.int [158.169.9.14]) by webmail.leidinger.net (Horde Framework) with HTTP; Fri, 12 Mar 2010 09:10:35 +0100 Message-ID: <20100312091035.961823cwec0rxijo@webmail.leidinger.net> Date: Fri, 12 Mar 2010 09:10:35 +0100 From: Alexander Leidinger To: Borja Marcos References: <864468D4-DCE9-493B-9280-00E5FAB2A05C@lassitu.de> <20100309122954.GE3155@garage.freebsd.pl> <20100309125815.GF3155@garage.freebsd.pl> <20100310110202.GA1715@garage.freebsd.pl> <20100310173143.GD1715@garage.freebsd.pl> <20100311084527.2934034895hvgxaw@webmail.leidinger.net> <764BD545-B86C-47DC-9004-964EB2216AF0@sarenet.es> <20100311150822.107231cvjvgs9gsg@webmail.leidinger.net> <35ADB9B1-F571-4EE4-9089-5363ACEBE159@sarenet.es> In-Reply-To: <35ADB9B1-F571-4EE4-9089-5363ACEBE159@sarenet.es> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.4) X-EBL-MailScanner-Information: Please contact the ISP for more information X-EBL-MailScanner-ID: 434288453C9.E4D6B X-EBL-MailScanner: Found to be clean X-EBL-MailScanner-SpamCheck: not spam, spamhaus-ZEN, SpamAssassin (not cached, score=-1.363, required 6, autolearn=disabled, ALL_TRUSTED -1.44, DKIM_SIGNED 0.00, DKIM_VERIFIED -0.00, TW_ZF 0.08) X-EBL-MailScanner-From: alexander@leidinger.net X-EBL-MailScanner-Watermark: 1268986240.38003@u8CKQTup46wL395ZMBCtEw X-EBL-Spam-Status: No Cc: freebsd-fs@FreeBSD.org, Pawel Jakub Dawidek , FreeBSD@FreeBSD.org, Stable Subject: Re: Many processes stuck in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 08:10:45 -0000 Quoting Borja Marcos (from Thu, 11 Mar 2010 18:26:09 +0100): > Of course CPUs have bugs, I don't doubt it. I was just wondering how > I coud reproduce the problem with a different hardware :) That's why > I said it was unlikely. > > Besides, such a low level fault should produce many more problems > than such a well defined failure mode, as far as I know. In my case I had a 7.1 system which was running fine. After updating to 7.2 I got deadlocks after some minutes with UFS. Switching to ZFS for the main data partition extended the lifetime to 3-4 hours. After updating ZFS in 7-stable with the code from 8-stable this was extended to 6 hours (periodic daily triggered the problem faster), and after switching to exclusive locks instead of shared locks in ZFS the system survived a night with several jails running periodic daily (but I had to reboot in the morning because apache was not able to serve data anymore). Everything else was working correctly. I would say this was a very narrow problem case. Bye, Alexander. -- Adult, n.: One old enough to know better. http://www.Leidinger.net Alexander @ Leidinger.net: PGP ID = B0063FE7 http://www.FreeBSD.org netchild @ FreeBSD.org : PGP ID = 72077137 From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 10:13:09 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 462E11065676 for ; Fri, 12 Mar 2010 10:13:09 +0000 (UTC) (envelope-from byshenknet@byshenk.net) Received: from core.byshenk.net (core.byshenk.net [62.58.73.230]) by mx1.freebsd.org (Postfix) with ESMTP id CF67D8FC1A for ; Fri, 12 Mar 2010 10:13:08 +0000 (UTC) Received: from core.byshenk.net (localhost [127.0.0.1]) by core.byshenk.net (8.14.4/8.14.4) with ESMTP id o2CADpGZ029804; Fri, 12 Mar 2010 11:13:51 +0100 (CET) (envelope-from byshenknet@core.byshenk.net) Received: (from byshenknet@localhost) by core.byshenk.net (8.14.4/8.14.4/Submit) id o2CADouh029803; Fri, 12 Mar 2010 11:13:50 +0100 (CET) (envelope-from byshenknet) Date: Fri, 12 Mar 2010 11:13:50 +0100 From: Greg Byshenk To: freebsd-stable@freebsd.org Message-ID: <20100312101350.GH2303@core.byshenk.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7d6fde3d1003112033k3ce3b040y8edac8772747862f@mail.gmail.com> User-Agent: Mutt/1.4.2.3i X-Spam-Status: No, score=4.1 required=5.0 tests=ALL_TRUSTED,FAKE_REPLY_C, FH_DATE_PAST_20XX autolearn=no version=3.2.5 X-Spam-Level: **** X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on core.byshenk.net Cc: Garrett Cooper , d@delphij.net, Steve Bertrand , Xin LI Subject: Re: I broke my SSH to jails after 7.2-8.0 src upgrade X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 10:13:09 -0000 On Thu, Mar 11, 2010 at 08:33:29PM -0800, Garrett Cooper wrote: > I've done a few RELENG_8_0 to STABLE-8 to 9-CURRENT upgrades lately > and mergemaster was goofing up the contents a bit based on the RCS > versions. I had to hand-edit a crapload of stuff going from 8 to 9, > and I still don't trust mergemaster's automatic merging logic because > it goofs up on /etc/group // /etc/passwd still (doesn't merge > anything, discards my info, etc) for starters. > > -a doesn't actually do any merging though, FWIW: [...] I would put in a word for 'mergemaster -F' (or maybe '-iF') in such cases. It doesn't try to automate much, but it allows one to concentrate on actual differences by automating the handling of those files where only the VCS Id is different. -- greg byshenk - gbyshenk@byshenk.net - Leiden, NL From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 11:40:30 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DFD9E106566C; Fri, 12 Mar 2010 11:40:30 +0000 (UTC) (envelope-from andrew.hotlab@hotmail.com) Received: from blu0-omc1-s22.blu0.hotmail.com (blu0-omc1-s22.blu0.hotmail.com [65.55.116.33]) by mx1.freebsd.org (Postfix) with ESMTP id ADADC8FC12; Fri, 12 Mar 2010 11:40:30 +0000 (UTC) Received: from BLU138-W14 ([65.55.116.8]) by blu0-omc1-s22.blu0.hotmail.com with Microsoft SMTPSVC(6.0.3790.3959); Fri, 12 Mar 2010 03:40:29 -0800 Message-ID: X-Originating-IP: [81.174.54.98] From: Andrew Hotlab To: , Date: Fri, 12 Mar 2010 11:40:29 +0000 Importance: Normal Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginalArrivalTime: 12 Mar 2010 11:40:29.0755 (UTC) FILETIME=[D10828B0:01CAC1D8] Cc: freebsd-jail@freebsd.org Subject: RE: Jails & 8.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 11:40:31 -0000 ---------------------------------------- > Date: Thu=2C 11 Mar 2010 22:53:43 -0500 > From: steve@ibctech.ca > To: freebsd-stable@freebsd.org > CC: freebsd-jail@freebsd.org > Subject: Jails & 8.0 > > Sorry for the cross-post=2C but this is a 'thank-you'=2C not a request fo= r help. > > I want to express my sincere appreciation for all of those who made > FreeBSD jails a viable virtual server solution for us who required > multiple IPs=2C particularly those who demand/require IPv6 support: > > > Not only that=2C FreeBSD 8 is just absolutely fantastic. Although I've > only been using FreeBSD since 4.3=2C I never could have dreamt that the O= S > would ever have a release that is so close to its core values=2C but at > the same time so feature rich and stable=2C particularly for those who > like to use the OS as a network (L2/L3) platform in many cases. > > My hats off. Thanks all! What a tremendous job. > I couldn't agree more: FreeBSD today is really able to bring a tremendous v= alue to a lot of enterprise-grade environments. Coming from a deep Microsof= t experience as multi-certified MSFT specialist=2C I have been playing with= FreeBSD since the 6.0-RELEASE=2C and I must say that FreeBSD is perhaps th= e best OS when you need to effectively consolidate workloads and administra= tion efforts. The Jail system is simply amazing=2C and I'm really excited to see how the = VIMAGE feature=2C when it will be released as "producion-ready"=2C will inc= rease even more this value. To meet the always evolving business requirements today=2C IT pros need a s= implified architecture to keep TCP as low as possible=2C while sustaining m= ore and more workloads. Tools such ezjail=2C which allow to maintain a lot = of jails as they were almost only one=2C make businesses to obtain a true "= consolidation"=2C whom other proprietary and open-sourced OSes are far away= to reach. So... thank you very much=2C to anyone who contributed and still works to m= ake this great project to evolve! I guess to to help you a bit by advocati= ng FreeBSD as much as I can among customers=2C partners and institutions in= my country. =A0But a big work still remains to be done: to bring to BSD te= chnologies the visibility they deserve among IT managers and professionals = working in business environments. =A0I'll do my best to make this happen in= my country=2C I hope=2C also with the support of the FreeBSD Foundation an= d the BSD Certification Group=2C whom I'm in the process to make a few prop= osals to. Sincerely. Andrew =20 _________________________________________________________________ Your E-mail and More On-the-Go. Get Windows Live Hotmail Free. https://signup.live.com/signup.aspx?id=3D60969= From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 11:50:34 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 95066106564A; Fri, 12 Mar 2010 11:50:34 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (chello089077043238.chello.pl [89.77.43.238]) by mx1.freebsd.org (Postfix) with ESMTP id 2A1E48FC17; Fri, 12 Mar 2010 11:50:33 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id B847945EE5; Fri, 12 Mar 2010 12:50:31 +0100 (CET) Received: from localhost (pdawidek.wheel.pl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id EC56345EB2; Fri, 12 Mar 2010 12:50:27 +0100 (CET) Date: Fri, 12 Mar 2010 12:50:28 +0100 From: Pawel Jakub Dawidek To: Kai Gallasch Message-ID: <20100312115028.GG1819@garage.freebsd.pl> References: <20100311133916.42ba69b0@orwell.free.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="7gGkHNMELEOhSGF6" Content-Disposition: inline In-Reply-To: <20100311133916.42ba69b0@orwell.free.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT i386 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.org, mav@FreeBSD.org, freebsd-stable@FreeBSD.org Subject: Re: proliant server lockups with freebsd-amd64-stable (2010-03-10) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 11:50:34 -0000 --7gGkHNMELEOhSGF6 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 11, 2010 at 01:39:16PM +0100, Kai Gallasch wrote: > Hi. >=20 > I have some trouble with an opteron server locking up spontaneously. It l= ooses > all networks connectivity and even through console I can get no shell. >=20 > Lockups occur mostly under disk load (periodic daily, bacula backup > running, make buildworld/buildkernel) and I can provoke them easily. [...] > 4 0 0 0 LL *cissmtx 0xffffff04ed820c00 [g_down] [...] > 100046 L *cissmtx 0xffffff04ed820c00 [irq257: cis= s0] [...] I was analizing similar problem as potential ZFS bug. It turned out to be bug in ciss(4) and I believe mav@ (CCed) has fix for that. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --7gGkHNMELEOhSGF6 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkuaKoQACgkQForvXbEpPzQmpACfYv5IvtyEzVrdjxL7/dsS8QtI JNUAn065Ro15k663XOO3y8YFrMf15Owq =5M1X -----END PGP SIGNATURE----- --7gGkHNMELEOhSGF6-- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 12:14:14 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51949106566B; Fri, 12 Mar 2010 12:14:14 +0000 (UTC) (envelope-from pb@fasterix.frmug.org) Received: from fasterix.frmug.org (unknown [IPv6:2a01:e35:8b50:2c40::2]) by mx1.freebsd.org (Postfix) with ESMTP id 9CAEC8FC18; Fri, 12 Mar 2010 12:14:11 +0000 (UTC) Received: from fasterix.frmug.org (localhost [127.0.0.1]) by fasterix.frmug.org (8.14.3/8.14.3) with ESMTP id o2CCEAY8079536 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 12 Mar 2010 13:14:10 +0100 (CET) (envelope-from pb@fasterix.frmug.org) Received: (from pb@localhost) by fasterix.frmug.org (8.14.3/8.14.3/Submit) id o2CCE9ll079535; Fri, 12 Mar 2010 13:14:09 +0100 (CET) (envelope-from pb) Date: Fri, 12 Mar 2010 13:14:09 +0100 From: Pierre Beyssac To: freebsd-stable@freebsd.org Message-ID: <20100312121409.GA79294@fasterix.frmug.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="EeQfGwPcQSOJBaQU" Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-message-flag: Warning! Use of Microsoft Outlook is dangerous and makes your system susceptible to worms and viruses Cc: mav@freebsd.org Subject: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 12:14:14 -0000 --EeQfGwPcQSOJBaQU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Hello, I'm having "interrupt storm detected" messages on a Dell Inspiron 580 running up-to-date 8-STABLE (amd64 arch). The interrupts seem to come from one of the atapci controllers, apparently atapci0 (main controller, with a SATA disk and an ATAPI optical drive). ata_interrupt gets called at a variable rate, between 1000-150000 times per second, constantly, even when the disk is not used. >From adding debug sysctl code in ata-all.c:ata_interrupt_locked() I have been able to check that: ch->running is NULL (breaks loop in "do we have a running request") ch->state=0 ch->unit=0 ch->devices=1 (ATA_ATA_MASTER) most of the time. Here's attached dmesg output, pciconf -lv output, kernel configuration and vmstat -i output. A -current kernel exhibits the same behaviour. Any hint/idea how to debug this further would be really appreciated... -- Pierre Beyssac pb@fasterix.frmug.org --EeQfGwPcQSOJBaQU Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="dmesg.txt" Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-STABLE #14: Fri Mar 12 01:37:24 CET 2010 root@inspiron:/usr/src/sys/amd64/compile/INSP580 amd64 WARNING: WITNESS option enabled, expect reduced performance. Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff80a2a000. Preloaded elf obj module "/boot/kernel/ehci.ko" at 0xffffffff80a2a240. Preloaded elf obj module "/boot/kernel/usb.ko" at 0xffffffff80a2a868. Preloaded elf obj module "/boot/kernel/ukbd.ko" at 0xffffffff80a2af10. Timecounter "i8254" frequency 1193182 Hz quality 0 Calibrating TSC clock ... TSC clock: 2660007980 Hz CPU: Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz (2660.01-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x106e5 Stepping = 5 Features=0xbfebfbff Features2=0x98e3fd AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant real memory = 8589934592 (8192 MB) Physical memory chunk(s): 0x0000000000001000 - 0x000000000009bfff, 634880 bytes (155 pages) 0x0000000000a5e000 - 0x00000000bd77ffff, 3167887360 bytes (773410 pages) 0x0000000100000000 - 0x000000022f12ffff, 5084741632 bytes (1241392 pages) avail memory = 8210997248 (7830 MB) ACPI APIC Table: INTR: Adding local APIC 2 as a target INTR: Adding local APIC 4 as a target INTR: Adding local APIC 6 as a target FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 4 cpu3 (AP): APIC ID: 6 APIC: CPU 0 has ACPI ID 1 APIC: CPU 1 has ACPI ID 2 APIC: CPU 2 has ACPI ID 3 APIC: CPU 3 has ACPI ID 4 ULE: setup cpu 0 ULE: setup cpu 1 ULE: setup cpu 2 ULE: setup cpu 3 ACPI: RSDP 0xf9b00 00024 (v2 ACPIAM) ACPI: XSDT 0xbd780100 0006C (v1 DELL FX09 20091130 MSFT 00000097) ACPI: FACP 0xbd780290 000F4 (v4 DELL FX09 20091130 MSFT 00000097) ACPI: DSDT 0xbd780660 05B02 (v2 1AAAA 1AAAA000 00000000 INTL 20051117) ACPI: FACS 0xbd78e000 00040 ACPI: APIC 0xbd780390 0008C (v2 DELL FX09 20091130 MSFT 00000097) ACPI: MCFG 0xbd780420 0003C (v1 DELL OEMMCFG 20091130 MSFT 00000097) ACPI: SLIC 0xbd780460 00176 (v1 DELL FX09 20091130 MSFT 00000097) ACPI: OSFR 0xbd7805e0 00080 (v1 DELL FX09 20091130 MSFT 00000097) ACPI: OEMB 0xbd78e040 00072 (v1 DELL FX09 20091130 MSFT 00000097) ACPI: HPET 0xbd78a660 00038 (v1 DELL OEMHPET 20091130 MSFT 00000097) ACPI: ASF! 0xbd78a6a0 00099 (v32 LEGEND I865PASF 00000001 INTL 20051117) ACPI: SSDT 0xbd78f6f0 00363 (v1 DpgPmm CpuPm 00000012 INTL 20051117) MADT: Found IO APIC ID 7, Interrupt 0 at 0xfec00000 ioapic0: Changing APIC ID to 7 ioapic0: Routing external 8259A's -> intpin 0 MADT: Interrupt override: source 0, irq 2 ioapic0: Routing IRQ 0 -> intpin 2 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level ioapic0 irqs 0-23 on motherboard cpu0 BSP: ID: 0x00000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x0001000f pcm: 0x00010400 null: random: VESA: information block 0000 56 45 53 41 00 03 f0 01 00 c0 01 00 00 00 44 00 0010 00 01 00 01 0f 0c 29 01 00 c0 bb 00 00 c0 3a 4d 0020 00 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0040 00 00 00 00 00 01 01 01 03 01 05 01 07 01 10 01 0050 11 01 13 01 14 01 16 01 17 01 19 01 1a 01 0d 01 0060 0e 01 20 01 93 01 95 01 96 01 b3 01 b5 01 b6 01 0070 c3 01 c5 01 c6 01 33 01 35 01 36 01 53 01 55 01 0080 56 01 63 01 65 01 66 01 21 01 22 01 23 01 24 01 0090 43 01 45 01 46 01 73 01 75 01 76 01 83 01 85 01 00a0 86 01 d3 01 d5 01 d6 01 e3 01 e5 01 e6 01 ff ff 00b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 VESA: 53 mode(s) found VESA: v3.0, 16384k memory, flags:0x1, mode table:0xffffff8000099044 (1000044) VESA: ATI ATOMBIOS VESA: (C) 1988-2005, ATI Technologies Inc. CEDAR 01.00 io: kbd: new array size 4 kbd1 at kbdmux0 mem: acpi0: on motherboard PCIe: Memory Mapped configuration base @ 0xe0000000 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 acpi0: [MPSAFE] acpi0: [ITHREAD] ACPI: Executed 1 blocks of module-level executable AML code acpi0: Power Button (fixed) acpi0: wakeup code va 0xffffff800001b000 pa 0x4000 AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.IELK.RXA0 -> bus 0 dev 0 func 0 AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.FHR0 -> bus 0 dev 31 func 0 AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.PIX0 -> bus 0 dev 31 func 0 acpi0: reservation of fee00000, 1000 (3) failed acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, bdf00000 (3) failed ACPI timer: 1/1 0/223 1/1 1/1 1/1 1/1 1/1 1/1 1/1 1/1 -> 9 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: on acpi0 ACPI: SSDT 0xbd78e0c0 01130 (v1 DpgPmm P001Ist 00000011 INTL 20051117) ACPI: SSDT 0xbd78f1f0 004F4 (v1 PmRef P001Cst 00003001 INTL 20051117) cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 3 4 6 7 10 11 12 14 15 Validation 0 10 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 5 Validation 0 5 N 0 5 After Disable 0 255 N 0 5 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 3 N 0 3 4 6 7 10 11 12 14 15 Validation 0 3 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 3 4 6 7 10 11 12 14 15 Validation 0 11 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 6 7 10 11 12 14 15 Validation 0 255 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 6 7 10 11 12 14 15 Validation 0 255 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 15 N 0 3 4 6 7 10 11 12 14 15 Validation 0 15 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 14 N 0 3 4 6 7 10 11 12 14 15 Validation 0 14 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 acpi_hpet0: vend: 0x8086 rev: 0x1 num: 8 hz: 14318180 opts: legacy_route 64-bit Timecounter "HPET" frequency 14318180 Hz quality 900 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x8086, dev=0xd131, revid=0x11 domain=0, bus=0, slot=0, func=0 class=06-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd138, revid=0x11 domain=0, bus=0, slot=3, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0107, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x1a (6500 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 3 supports D0 D3 current D0 MSI supports 2 messages, vector masks pcib0: matched entry for 0.3.INTA pcib0: slot 3 INTA hardwired to IRQ 16 found-> vendor=0x8086, dev=0xd155, revid=0x11 domain=0, bus=0, slot=8, func=0 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd156, revid=0x11 domain=0, bus=0, slot=8, func=1 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd157, revid=0x11 domain=0, bus=0, slot=8, func=2 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd158, revid=0x11 domain=0, bus=0, slot=8, func=3 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd150, revid=0x11 domain=0, bus=0, slot=16, func=0 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd151, revid=0x11 domain=0, bus=0, slot=16, func=1 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3b64, revid=0x06 domain=0, bus=0, slot=22, func=0 class=07-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xfbdfe000, size 4, enabled pcib0: matched entry for 0.22.INTA pcib0: slot 22 INTA hardwired to IRQ 16 found-> vendor=0x8086, dev=0x3b3c, revid=0x06 domain=0, bus=0, slot=26, func=0 class=0c-03-20, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xfbdfc000, size 10, enabled pcib0: matched entry for 0.26.INTA pcib0: slot 26 INTA hardwired to IRQ 16 unknown: Reserved 0x400 bytes for rid 0x10 type 3 at 0xfbdfc000 found-> vendor=0x8086, dev=0x3b56, revid=0x06 domain=0, bus=0, slot=27, func=0 class=04-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=15 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xfbdf8000, size 14, enabled pcib0: matched entry for 0.27.INTA pcib0: slot 27 INTA hardwired to IRQ 22 found-> vendor=0x8086, dev=0x3b42, revid=0x06 domain=0, bus=0, slot=28, func=0 class=06-04-00, hdrtype=0x01, mfdev=1 cmdreg=0x0104, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) intpin=a, irq=5 powerspec 2 supports D0 D3 current D0 MSI supports 1 message pcib0: matched entry for 0.28.INTA pcib0: slot 28 INTA hardwired to IRQ 17 found-> vendor=0x8086, dev=0x3b4c, revid=0x06 domain=0, bus=0, slot=28, func=5 class=06-04-00, hdrtype=0x01, mfdev=1 cmdreg=0x0106, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) intpin=b, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 1 message pcib0: matched entry for 0.28.INTB pcib0: slot 28 INTB hardwired to IRQ 16 found-> vendor=0x8086, dev=0x3b34, revid=0x06 domain=0, bus=0, slot=29, func=0 class=0c-03-20, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=14 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xfbdf6000, size 10, enabled pcib0: matched entry for 0.29.INTA pcib0: slot 29 INTA hardwired to IRQ 23 unknown: Reserved 0x400 bytes for rid 0x10 type 3 at 0xfbdf6000 found-> vendor=0x8086, dev=0x244e, revid=0xa6 domain=0, bus=0, slot=30, func=0 class=06-04-01, hdrtype=0x01, mfdev=0 cmdreg=0x0104, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3b08, revid=0x06 domain=0, bus=0, slot=31, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3b20, revid=0x06 domain=0, bus=0, slot=31, func=2 class=01-01-8f, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x02b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=11 powerspec 3 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xdc00, size 3, enabled map[14]: type I/O Port, range 32, base 0xd880, size 2, enabled map[18]: type I/O Port, range 32, base 0xd800, size 3, enabled map[1c]: type I/O Port, range 32, base 0xd480, size 2, enabled map[20]: type I/O Port, range 32, base 0xd400, size 4, enabled map[24]: type I/O Port, range 32, base 0xd080, size 4, enabled pcib0: matched entry for 0.31.INTB pcib0: slot 31 INTB hardwired to IRQ 19 found-> vendor=0x8086, dev=0x3b30, revid=0x06 domain=0, bus=0, slot=31, func=3 class=0c-05-00, hdrtype=0x00, mfdev=0 cmdreg=0x0003, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=3 map[10]: type Memory, range 64, base 0xfbdf4000, size 8, enabled map[20]: type I/O Port, range 32, base 0x400, size 5, enabled pcib0: matched entry for 0.31.INTC pcib0: slot 31 INTC hardwired to IRQ 18 found-> vendor=0x8086, dev=0x3b26, revid=0x06 domain=0, bus=0, slot=31, func=5 class=01-01-85, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x02b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=11 powerspec 3 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xcc00, size 3, enabled map[14]: type I/O Port, range 32, base 0xc880, size 2, enabled map[18]: type I/O Port, range 32, base 0xc800, size 3, enabled map[1c]: type I/O Port, range 32, base 0xc480, size 2, enabled map[20]: type I/O Port, range 32, base 0xc400, size 4, enabled map[24]: type I/O Port, range 32, base 0xc080, size 4, enabled pcib0: matched entry for 0.31.INTB pcib0: slot 31 INTB hardwired to IRQ 19 pcib1: irq 16 at device 3.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: I/O decode 0xe000-0xefff pcib1: memory decode 0xfbe00000-0xfbefffff pcib1: prefetched decode 0xd0000000-0xdfffffff pcib1: could not get PCI interrupt routing table for \\_SB_.PCI0.P0P3 - AE_NOT_FOUND pci1: on pcib1 pci1: domain=0, physical bus=1 found-> vendor=0x1002, dev=0x68f9, revid=0x00 domain=0, bus=1, slot=0, func=0 class=03-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 3 supports D0 D1 D2 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Prefetchable Memory, range 64, base 0xd0000000, size 28, enabled pcib1: requested memory range 0xd0000000-0xdfffffff: good map[18]: type Memory, range 64, base 0xfbee0000, size 17, enabled pcib1: requested memory range 0xfbee0000-0xfbefffff: good map[20]: type I/O Port, range 32, base 0xe000, size 8, enabled pcib1: requested I/O range 0xe000-0xe0ff: in range pcib0: matched entry for 0.3.INTA pcib0: slot 3 INTA hardwired to IRQ 16 pcib1: slot 0 INTA is routed to irq 16 found-> vendor=0x1002, dev=0xaa68, revid=0x00 domain=0, bus=1, slot=0, func=1 class=04-03-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=5 powerspec 3 supports D0 D1 D2 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xfbebc000, size 14, enabled pcib1: requested memory range 0xfbebc000-0xfbebffff: good pcib0: matched entry for 0.3.INTB pcib0: slot 3 INTB hardwired to IRQ 17 pcib1: slot 0 INTB is routed to irq 17 vgapci0: port 0xe000-0xe0ff mem 0xd0000000-0xdfffffff,0xfbee0000-0xfbefffff irq 16 at device 0.0 on pci1 pci1: at device 0.1 (no driver attached) pci0: at device 8.0 (no driver attached) pci0: at device 8.1 (no driver attached) pci0: at device 8.2 (no driver attached) pci0: at device 8.3 (no driver attached) pci0: at device 16.0 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 22.0 (no driver attached) ehci0: mem 0xfbdfc000-0xfbdfc3ff irq 16 at device 26.0 on pci0 ioapic0: routing intpin 16 (PCI IRQ 16) to lapic 0 vector 49 ehci0: [MPSAFE] ehci0: [ITHREAD] usbus0: EHCI version 1.0 usbus0: on ehci0 pci0: at device 27.0 (no driver attached) pcib2: irq 17 at device 28.0 on pci0 pcib2: domain 0 pcib2: secondary bus 2 pcib2: subordinate bus 2 pcib2: I/O decode 0x0-0x0 pcib2: no prefetched decode pci2: on pcib2 pci2: domain=0, physical bus=2 pcib3: irq 16 at device 28.5 on pci0 pcib3: domain 0 pcib3: secondary bus 3 pcib3: subordinate bus 3 pcib3: I/O decode 0x0-0x0 pcib3: memory decode 0xfbf00000-0xfbffffff pcib3: no prefetched decode pci3: on pcib3 pci3: domain=0, physical bus=3 found-> vendor=0x14e4, dev=0x1691, revid=0x01 domain=0, bus=3, slot=0, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=5 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xfbff0000, size 16, enabled pcib3: requested memory range 0xfbff0000-0xfbffffff: good pcib3: matched entry for 3.0.INTA pcib3: slot 0 INTA hardwired to IRQ 17 bge0: mem 0xfbff0000-0xfbffffff irq 17 at device 0.0 on pci3 bge0: Reserved 0x10000 bytes for rid 0x10 type 3 at 0xfbff0000 bge0: adjust device control 0x2000 -> 0x5000 bge0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 256 to local APIC 0 vector 50 bge0: using IRQ 256 for MSI bge0: CHIP ID 0x57780001; ASIC REV 0x57780; CHIP REV 0x577800; PCI-E bge0: Disabling fastboot bge0: Disabling fastboot miibus0: on bge0 ukphy0: PHY 1 on miibus0 ukphy0: OUI 0x00d897, model 0x0019, rev. 1 ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bge0: bpf attached bge0: Ethernet address: 00:25:64:f4:27:26 bge0: [MPSAFE] bge0: [FILTER] ehci1: mem 0xfbdf6000-0xfbdf63ff irq 23 at device 29.0 on pci0 ioapic0: routing intpin 23 (PCI IRQ 23) to lapic 0 vector 51 ehci1: [MPSAFE] ehci1: [ITHREAD] usbus1: EHCI version 1.0 usbus1: on ehci1 pcib4: at device 30.0 on pci0 pcib4: domain 0 pcib4: secondary bus 4 pcib4: subordinate bus 4 pcib4: I/O decode 0x0-0x0 pcib4: no prefetched decode pcib4: Subtractively decoded bridge. pci4: on pcib4 pci4: domain=0, physical bus=4 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0xdc00-0xdc07,0xd880-0xd883,0xd800-0xd807,0xd480-0xd483,0xd400-0xd40f,0xd080-0xd08f irq 19 at device 31.2 on pci0 atapci0: Reserved 0x10 bytes for rid 0x20 type 4 at 0xd400 ioapic0: routing intpin 19 (PCI IRQ 19) to lapic 0 vector 52 atapci0: [MPSAFE] atapci0: [ITHREAD] ata2: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x10 type 4 at 0xdc00 atapci0: Reserved 0x4 bytes for rid 0x14 type 4 at 0xd880 ata2: [MPSAFE] ata2: [ITHREAD] ata3: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x18 type 4 at 0xd800 atapci0: Reserved 0x4 bytes for rid 0x1c type 4 at 0xd480 ata3: [MPSAFE] ata3: [ITHREAD] pci0: at device 31.3 (no driver attached) atapci1: port 0xcc00-0xcc07,0xc880-0xc883,0xc800-0xc807,0xc480-0xc483,0xc400-0xc40f,0xc080-0xc08f irq 19 at device 31.5 on pci0 atapci1: Reserved 0x10 bytes for rid 0x20 type 4 at 0xc400 atapci1: [MPSAFE] atapci1: [ITHREAD] ata4: on atapci1 atapci1: Reserved 0x8 bytes for rid 0x10 type 4 at 0xcc00 atapci1: Reserved 0x4 bytes for rid 0x14 type 4 at 0xc880 ata4: [MPSAFE] ata4: [ITHREAD] ata5: on atapci1 atapci1: Reserved 0x8 bytes for rid 0x18 type 4 at 0xc800 atapci1: Reserved 0x4 bytes for rid 0x1c type 4 at 0xc480 ata5: [MPSAFE] ata5: [ITHREAD] acpi_button0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock (resolution 1000000us) isa_probe_children: disabling PnP devices atrtc: atrtc0 already exists; skipping it sc: sc0 already exists; skipping it isa_probe_children: probing non-PnP devices orm0: at iomem 0xc0000-0xce7ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: scteken (teken terminal) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 kbd0: atkbd0, generic (0), config:0x0, flags:0x3f0000 ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 53 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: current command byte:0065 psm0: failed to reset the aux device. fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0 ppc0 failed to probe at irq 7 on isa0 uart0: failed to probe at port 0x3f8-0x3ff irq 4 on isa0 uart1: failed to probe at port 0x2f8-0x2ff irq 3 on isa0 isa_probe_children: probing PnP devices est0: on cpu0 est0: Invalid id16 (set, cur) = (19, 20) est0: Can't check freq 2533, it may be invalid est0: Invalid id16 (set, cur) = (18, 20) est0: Can't check freq 2400, it may be invalid est0: Invalid id16 (set, cur) = (17, 20) est0: Can't check freq 2267, it may be invalid est0: Invalid id16 (set, cur) = (16, 20) est0: Can't check freq 2133, it may be invalid est0: Invalid id16 (set, cur) = (15, 20) est0: Can't check freq 2000, it may be invalid est0: Invalid id16 (set, cur) = (14, 20) est0: Can't check freq 1867, it may be invalid est0: Invalid id16 (set, cur) = (13, 20) est0: Can't check freq 1733, it may be invalid est0: Invalid id16 (set, cur) = (12, 20) est0: Can't check freq 1600, it may be invalid est0: Invalid id16 (set, cur) = (11, 20) est0: Can't check freq 1467, it may be invalid est0: Invalid id16 (set, cur) = (10, 20) est0: Can't check freq 1333, it may be invalid est0: Invalid id16 (set, cur) = (9, 20) est0: Can't check freq 1200, it may be invalid p4tcc0: on cpu0 est1: on cpu1 est1: Invalid id16 (set, cur) = (19, 20) est1: Can't check freq 2533, it may be invalid est1: Invalid id16 (set, cur) = (18, 20) est1: Can't check freq 2400, it may be invalid est1: Invalid id16 (set, cur) = (17, 20) est1: Can't check freq 2267, it may be invalid est1: Invalid id16 (set, cur) = (16, 20) est1: Can't check freq 2133, it may be invalid est1: Invalid id16 (set, cur) = (15, 20) est1: Can't check freq 2000, it may be invalid est1: Invalid id16 (set, cur) = (14, 20) est1: Can't check freq 1867, it may be invalid est1: Invalid id16 (set, cur) = (13, 20) est1: Can't check freq 1733, it may be invalid est1: Invalid id16 (set, cur) = (12, 20) est1: Can't check freq 1600, it may be invalid est1: Invalid id16 (set, cur) = (11, 20) est1: Can't check freq 1467, it may be invalid est1: Invalid id16 (set, cur) = (10, 20) est1: Can't check freq 1333, it may be invalid est1: Invalid id16 (set, cur) = (9, 20) est1: Can't check freq 1200, it may be invalid p4tcc1: on cpu1 est2: on cpu2 est2: Invalid id16 (set, cur) = (19, 20) est2: Can't check freq 2533, it may be invalid est2: Invalid id16 (set, cur) = (18, 20) est2: Can't check freq 2400, it may be invalid est2: Invalid id16 (set, cur) = (17, 20) est2: Can't check freq 2267, it may be invalid est2: Invalid id16 (set, cur) = (16, 20) est2: Can't check freq 2133, it may be invalid est2: Invalid id16 (set, cur) = (15, 20) est2: Can't check freq 2000, it may be invalid est2: Invalid id16 (set, cur) = (14, 20) est2: Can't check freq 1867, it may be invalid est2: Invalid id16 (set, cur) = (13, 20) est2: Can't check freq 1733, it may be invalid est2: Invalid id16 (set, cur) = (12, 20) est2: Can't check freq 1600, it may be invalid est2: Invalid id16 (set, cur) = (11, 20) est2: Can't check freq 1467, it may be invalid est2: Invalid id16 (set, cur) = (10, 20) est2: Can't check freq 1333, it may be invalid est2: Invalid id16 (set, cur) = (9, 20) est2: Can't check freq 1200, it may be invalid p4tcc2: on cpu2 est3: on cpu3 est3: Invalid id16 (set, cur) = (19, 20) est3: Can't check freq 2533, it may be invalid est3: Invalid id16 (set, cur) = (18, 20) est3: Can't check freq 2400, it may be invalid est3: Invalid id16 (set, cur) = (17, 20) est3: Can't check freq 2267, it may be invalid est3: Invalid id16 (set, cur) = (16, 20) est3: Can't check freq 2133, it may be invalid est3: Invalid id16 (set, cur) = (15, 20) est3: Can't check freq 2000, it may be invalid est3: Invalid id16 (set, cur) = (14, 20) est3: Can't check freq 1867, it may be invalid est3: Invalid id16 (set, cur) = (13, 20) est3: Can't check freq 1733, it may be invalid est3: Invalid id16 (set, cur) = (12, 20) est3: Can't check freq 1600, it may be invalid est3: Invalid id16 (set, cur) = (11, 20) est3: Can't check freq 1467, it may be invalid est3: Invalid id16 (set, cur) = (10, 20) est3: Can't check freq 1333, it may be invalid est3: Invalid id16 (set, cur) = (9, 20) est3: Can't check freq 1200, it may be invalid p4tcc3: on cpu3 Device configuration finished. Reducing kern.maxvnodes 509903 -> 100000 procfs registered lapic: Divisor 2, Frequency 66500198 Hz Timecounter "TSC" frequency 2660007980 Hz quality -100 Timecounters tick every 1.000 msec lo0: bpf attached usbus0: 480Mbps High Speed USB v2.0 usbus1: 480Mbps High Speed USB v2.0 ata2: reset tp1 mask=03 ostat0=50 ostat1=00 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ata2: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata2: stat1=0x00 err=0x01 lsb=0x00 msb=0x00 ata2: reset tp2 stat0=50 stat1=00 devices=0x1 (aprobe0:ata2:0:0:0): SIGNATURE: 0000 ata3: reset tp1 mask=03 ostat0=50 ostat1=00 ata3: stat0=0x00 err=0x01 lsb=0x14 msb=0xeb ata3: stat1=0x00 err=0x01 lsb=0x14 msb=0xeb ata3: reset tp2 stat0=00 stat1=00 devices=0x30000 (aprobe1:ata3:0:0:0): SIGNATURE: eb14 ata4: reset tp1 mask=03 ostat0=7f ostat1=7f ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat1=0x7f err=0xff lsb=0xff msb=0xff ata4: reset tp2 stat0=ff stat1=ff devices=0x0 ata5: reset tp1 mask=03 ostat0=7f ostat1=7f ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat1=0x7f err=0xff lsb=0xff msb=0xff ata5: reset tp2 stat0=ff stat1=ff devices=0x0 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered (aprobe0:ata3:0:1:0): SIGNATURE: eb14 ata3: timeout waiting for read DRQ (aprobe0:ata3:0:1:0): Request completed with CAM_REQ_CMP_ERR (aprobe0:ata3:0:1:0): Retrying command ata3: timeout waiting for read DRQ (aprobe0:ata3:0:1:0): Request completed with CAM_REQ_CMP_ERR (aprobe0:ata3:0:1:0): Error 5, Retries exhausted GEOM: new disk ada0 ada0 at ata2 bus 0 scbus0 target 0 lun 0 ada0: ATA-8 SATA 2.x device ada0: Serial Number ........ ada0: 33.300MB/s transfers (UDMA2, PIO 8192bytes) ada0: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) pass0 at ata2 bus 0 scbus0 target 0 lun 0 pass0: ATA-8 SATA 2.x device pass0: Serial Number ........ pass0: 33.300MB/s transfers (UDMA2, PIO 8192bytes) pass1 at ata3 bus 0 scbus1 target 0 lun 0 pass1: Removable CD-ROM SCSI-0 device pass1: Serial Number .................... pass1: 33.300MB/s transfers (UDMA2, ATAPI 12bytes, PIO 8192bytes) SMP: AP CPU #3 Launched! cpu3 AP: ID: 0x06000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000200ef therm: 0x00010000 err: 0x00010000 pcm: 0x00010400 (cd0:ata3:0:0:0): SCSI status error (cd0:ata3:0:0:0): Requesting SCSI sense data SMP: AP CPU #2 Launched! cpu2 AP: ID: 0x04000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff(cd0: timer: 0x000200ef therm: 0x00010000 err: 0x00010000 pcm: 0x00010400ata3:0: 0:0): SCSI status error (cd0:ata3:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 (cd0:ata3:0:0:0): CAM status: SCSI Status Error (cd0:ata3:0:0:0): SCSI status: Check Condition (cd0:ata3:0:0:0): SCSI sense: NOT READY asc:3a,1 (Medium not present - tray closed) (cd0:ata3:0:0:0): Error 6, Unretryable errorSMP: AP CPU #1 Launched! cd0 at ata3 bus 0 scbus1 target 0 lun 0 cpu1 AP: cd0: ID: 0x02000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff Removable CD-ROM SCSI-0 device lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff cd0: Serial Number .................... timer: 0x000200ef therm: 0x00010000 err: 0x00010000 pcm: 0x00010400 cd0: 33.300MB/s transfers ioapic0: routing intpin 9 ( (ISA IRQ 9UDMA2, ) to lapic 2 vector 48ATAPI 12bytes, PIO 8192bytes) cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed ioapic0: routing intpin 16 (PCI IRQ 16) to lapic 4 vector 48 ioapic0: routing intpin 19 (PCI IRQ 19) to lapic 6 vector 48 msi: Assigning MSI IRQ 256 to local APIC 2 vector 49 WARNING: WITNESS option enabled, expect reduced performance. GEOM: ada0: partition 3 does not start on a track boundary. GEOM: ada0: partition 3 does not end on a track boundary. GEOM: ada0: partition 2 does not start on a track boundary. GEOM: ada0: partition 2 does not end on a track boundary. GEOM: new disk cd0 (cd0:ata3:0:0:0): SCSI status error (cd0:ata3:0:0:0): Requesting SCSI sense data (cd0:ata3:0:0:0): SCSI status error (cd0:ata3:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 (cd0:ata3:0:0:0): CAM status: SCSI Status Error (cd0:ata3:0:0:0): SCSI status: Check Condition (cd0:ata3:0:0:0): SCSI sense: NOT READY asc:3a,1 (Medium not present - tray closed) (cd0:ata3:0:0:0): Error 6, Unretryable error (cd0:ata3:0:0:0): SCSI status error (cd0:ata3:0:0:0): Requesting SCSI sense data (cd0:ata3:0:0:0): SCSI status error (cd0:ata3:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 (cd0:ata3:0:0:0): CAM status: SCSI Status Error (cd0:ata3:0:0:0): SCSI status: Check Condition (cd0:ata3:0:0:0): SCSI sense: NOT READY asc:3a,1 (Medium not present - tray closed) (cd0:ata3:0:0:0): Error 6, Unretryable error Root mount waiting for: usbus1 usbus0 interrupt storm detected on "irq19:"; throttling interrupt source ugen0.2: at usbus0 uhub2: on usbus0 ugen1.2: at usbus1 uhub3: on usbus1 Root mount waiting for: usbus1 usbus0 uhub2: 6 ports with 6 removable, self powered interrupt storm detected on "irq19:"; throttling interrupt source uhub3: 8 ports with 8 removable, self powered Root mount waiting for: usbus1 usbus0 ugen0.3: at usbus0 ukbd0: on usbus0 kbd2 at ukbd0 kbd2: ukbd0, generic (0), config:0x0, flags:0x3d0000 ugen1.3: at usbus1 Trying to mount root from ufs:/dev/ufs/root ct_to_ts([2010-03-12 09:48:15]) = 1268387295.000000000 start_init: trying /sbin/init interrupt storm detected on "irq19:"; throttling interrupt source interrupt storm detected on "irq19:"; throttling interrupt source interrupt storm detected on "irq19:"; throttling interrupt source interrupt storm detected on "irq19:"; throttling interrupt source interrupt storm detected on "irq19:"; throttling interrupt source interrupt storm detected on "irq19:"; throttling interrupt source interrupt storm detected on "irq19:"; throttling interrupt source interrupt storm detected on "irq19:"; throttling interrupt source interrupt storm detected on "irq19:"; throttling interrupt source interrupt storm detected on "irq19:"; throttling interrupt source bge0: Disabling fastboot bge0: Disabling fastboot interrupt storm detected on "irq19:"; throttling interrupt source bge0: link UP interrupt storm detected on "irq19:"; throttling interrupt source interrupt storm detected on "irq19:"; throttling interrupt source --EeQfGwPcQSOJBaQU Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=INSP580 cpu HAMMER ident INSP580 # To statically compile in device wiring instead of /boot/device.hints #hints "GENERIC.hints" # Default places to look for devices. # Use the following to compile in values accessible to the kernel # through getenv() (or kenv(1) in userland). The format of the file # is 'variable=value', see kenv(1) # # env "GENERIC.env" makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols options SCHED_ULE # ULE scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options INET6 # IPv6 communications protocols options SCTP # Stream Control Transmission Protocol options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_ACL # Support for access control lists options UFS_DIRHASH # Improve performance on big directories options UFS_GJOURNAL # Enable gjournal-based UFS journaling options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_PART_GPT # GUID Partition Tables. options GEOM_LABEL # Provides labelization options COMPAT_43TTY # BSD 4.3 TTY compat (sgtty) options COMPAT_IA32 # Compatible with i386 binaries options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options COMPAT_FREEBSD5 # Compatible with FreeBSD5 options COMPAT_FREEBSD6 # Compatible with FreeBSD6 options COMPAT_FREEBSD7 # Compatible with FreeBSD7 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options STACK # stack(9) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores options P1003_1B_SEMAPHORES # POSIX-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options PRINTF_BUFR_SIZE=128 # Prevent printf output being interspersed. options KBD_INSTALL_CDEV # install a CDEV entry in /dev options HWPMC_HOOKS # Necessary kernel hooks for hwpmc(4) options AUDIT # Security event auditing options MAC # TrustedBSD MAC Framework options FLOWTABLE # per-cpu routing cache #options KDTRACE_FRAME # Ensure frames are compiled in #options KDTRACE_HOOKS # Kernel DTrace hooks options INCLUDE_CONFIG_FILE # Include this file in kernel options WITNESS options INVARIANTS options INVARIANT_SUPPORT options DEADLKRES options KDB options KDB_UNATTENDED options DDB # Make an SMP-capable kernel by default options SMP # Symmetric MultiProcessor Kernel # CPU frequency control device cpufreq # Bus support. device acpi device pci # ATA and ATAPI devices device atacore device atapci device ataintel device atadisk # ATA disk drives device atapicd # ATAPI CDROM drives options ATA_STATIC_ID # Static device numbering options ATA_CAM # SCSI peripherals device scbus # SCSI bus (required for SCSI) device ch # SCSI media changers device da # Direct Access (disks) device sa # Sequential Access (tape etc) device cd # CD device pass # Passthrough device (direct SCSI access) device ses # SCSI Environmental Services (and SAF-TE) # atkbdc0 controls both the keyboard and the PS/2 mouse device atkbdc # AT keyboard controller device atkbd # AT keyboard device psm # PS/2 mouse device kbdmux # keyboard multiplexer device vga # VGA video card driver options VESA device splash # Splash screen and screen saver support # syscons is the default console driver, resembling an SCO console device sc # Serial (COM) ports device uart # Generic UART driver # PCI Ethernet NICs that use the common MII bus controller code. # NOTE: Be sure to keep the 'device miibus' line in order to use these NICs! device miibus # MII bus support device bge # Broadcom BCM570xx Gigabit Ethernet # Pseudo devices. device loop # Network loopback device random # Entropy device device ether # Ethernet support device pty # BSD-style compatibility pseudo ttys device firmware # firmware assist module # The `bpf' device enables the Berkeley Packet Filter. # Be aware of the administrative consequences of enabling this! # Note that 'bpf' is required for DHCP. device bpf # Berkeley packet filter --EeQfGwPcQSOJBaQU Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="vmstat.txt" interrupt total rate irq16: ehci0 3495 3 irq19: atapci0+ 54180102 48943 irq23: ehci1 2237 2 cpu0: timer 2191716 1979 irq256: bge0 390 0 cpu3: timer 2181718 1970 cpu2: timer 2181705 1970 cpu1: timer 2181707 1970 Total 62923070 56841 --EeQfGwPcQSOJBaQU Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="pciconf.txt" hostb0@pci0:0:0:0: class=0x060000 card=0x04381028 chip=0xd1318086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = bridge subclass = HOST-PCI pcib1@pci0:0:3:0: class=0x060400 card=0x04381028 chip=0xd1388086 rev=0x11 hdr=0x01 vendor = 'Intel Corporation' class = bridge subclass = PCI-PCI none0@pci0:0:8:0: class=0x088000 card=0x00380028 chip=0xd1558086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none1@pci0:0:8:1: class=0x088000 card=0x00380028 chip=0xd1568086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none2@pci0:0:8:2: class=0x088000 card=0x00380028 chip=0xd1578086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none3@pci0:0:8:3: class=0x088000 card=0x00380028 chip=0xd1588086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none4@pci0:0:16:0: class=0x088000 card=0x00380028 chip=0xd1508086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none5@pci0:0:16:1: class=0x088000 card=0x00380028 chip=0xd1518086 rev=0x11 hdr=0x00 vendor = 'Intel Corporation' class = base peripheral none6@pci0:0:22:0: class=0x078000 card=0x04381028 chip=0x3b648086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' class = simple comms ehci0@pci0:0:26:0: class=0x0c0320 card=0x04381028 chip=0x3b3c8086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' class = serial bus subclass = USB none7@pci0:0:27:0: class=0x040300 card=0x04381028 chip=0x3b568086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' class = multimedia subclass = HDA pcib2@pci0:0:28:0: class=0x060400 card=0x04381028 chip=0x3b428086 rev=0x06 hdr=0x01 vendor = 'Intel Corporation' class = bridge subclass = PCI-PCI pcib3@pci0:0:28:5: class=0x060400 card=0x04381028 chip=0x3b4c8086 rev=0x06 hdr=0x01 vendor = 'Intel Corporation' class = bridge subclass = PCI-PCI ehci1@pci0:0:29:0: class=0x0c0320 card=0x04381028 chip=0x3b348086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' class = serial bus subclass = USB pcib4@pci0:0:30:0: class=0x060401 card=0x04381028 chip=0x244e8086 rev=0xa6 hdr=0x01 vendor = 'Intel Corporation' device = '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge' class = bridge subclass = PCI-PCI isab0@pci0:0:31:0: class=0x060100 card=0x04381028 chip=0x3b088086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' class = bridge subclass = PCI-ISA atapci0@pci0:0:31:2: class=0x01018f card=0x04381028 chip=0x3b208086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = 'IBEX SATA Controller' class = mass storage subclass = ATA none8@pci0:0:31:3: class=0x0c0500 card=0x04381028 chip=0x3b308086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' class = serial bus subclass = SMBus atapci1@pci0:0:31:5: class=0x010185 card=0x04381028 chip=0x3b268086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = 'IBEX SATA Controller' class = mass storage subclass = ATA vgapci0@pci0:1:0:0: class=0x030000 card=0x010e1028 chip=0x68f91002 rev=0x00 hdr=0x00 vendor = 'ATI Technologies Inc. / Advanced Micro Devices, Inc.' class = display subclass = VGA none9@pci0:1:0:1: class=0x040300 card=0xaa681028 chip=0xaa681002 rev=0x00 hdr=0x00 vendor = 'ATI Technologies Inc. / Advanced Micro Devices, Inc.' class = multimedia subclass = HDA bge0@pci0:3:0:0: class=0x020000 card=0x04381028 chip=0x169114e4 rev=0x01 hdr=0x00 vendor = 'Broadcom Corporation' class = network subclass = ethernet --EeQfGwPcQSOJBaQU-- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 13:18:34 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BAD521065676 for ; Fri, 12 Mar 2010 13:18:34 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta10.emeryville.ca.mail.comcast.net (qmta10.emeryville.ca.mail.comcast.net [76.96.30.17]) by mx1.freebsd.org (Postfix) with ESMTP id A1DCE8FC19 for ; Fri, 12 Mar 2010 13:18:33 +0000 (UTC) Received: from omta10.emeryville.ca.mail.comcast.net ([76.96.30.28]) by qmta10.emeryville.ca.mail.comcast.net with comcast id sDC11d0040cQ2SLAADJa7K; Fri, 12 Mar 2010 13:18:34 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta10.emeryville.ca.mail.comcast.net with comcast id sDJZ1d0093S48mS8WDJZb5; Fri, 12 Mar 2010 13:18:34 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 221799B436; Fri, 12 Mar 2010 05:18:32 -0800 (PST) Date: Fri, 12 Mar 2010 05:18:32 -0800 From: Jeremy Chadwick To: Pierre Beyssac Message-ID: <20100312131832.GA54731@icarus.home.lan> References: <20100312121409.GA79294@fasterix.frmug.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100312121409.GA79294@fasterix.frmug.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: mav@freebsd.org, freebsd-stable@freebsd.org Subject: Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 13:18:34 -0000 On Fri, Mar 12, 2010 at 01:14:09PM +0100, Pierre Beyssac wrote: > I'm having "interrupt storm detected" messages on a Dell Inspiron > 580 running up-to-date 8-STABLE (amd64 arch). The interrupts seem > to come from one of the atapci controllers, apparently atapci0 (main > controller, with a SATA disk and an ATAPI optical drive). I'm a little confused by the kernel output. It appears as if you're using the new SATA-to-CAM layer (ahci.ko) for your SATA disks, rather than the ataahci.ko layer... but I don't see any indication of AHCI being available/used on your southbridge chipset. Possibly this is the source of the problem (specifically, it looks like FreeBSD doesn't have proper device ID knowledge of what this controller is. I believe that's because this system is *very* new, a Core i3/i5/i7 system)? If you disable use of ahci.ko and use the standard ata(4) layer, does the interrupt storm go away? -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 16:57:47 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5AE52106566B; Fri, 12 Mar 2010 16:57:47 +0000 (UTC) (envelope-from pb@fasterix.frmug.org) Received: from fasterix.frmug.org (unknown [IPv6:2a01:e35:8b50:2c40::2]) by mx1.freebsd.org (Postfix) with ESMTP id D0E678FC08; Fri, 12 Mar 2010 16:57:46 +0000 (UTC) Received: from fasterix.frmug.org (localhost [127.0.0.1]) by fasterix.frmug.org (8.14.3/8.14.3) with ESMTP id o2CGvjZK088572 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 12 Mar 2010 17:57:45 +0100 (CET) (envelope-from pb@fasterix.frmug.org) Received: (from pb@localhost) by fasterix.frmug.org (8.14.3/8.14.3/Submit) id o2CGvj1Q088571; Fri, 12 Mar 2010 17:57:45 +0100 (CET) (envelope-from pb) Date: Fri, 12 Mar 2010 17:57:44 +0100 From: Pierre Beyssac To: Jeremy Chadwick Message-ID: <20100312165744.GA86971@fasterix.frmug.org> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312131832.GA54731@icarus.home.lan> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="G4iJoqBmSsgzjUCe" Content-Disposition: inline In-Reply-To: <20100312131832.GA54731@icarus.home.lan> User-Agent: Mutt/1.4.2.3i X-message-flag: Warning! Use of Microsoft Outlook is dangerous and makes your system susceptible to worms and viruses X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: mav@freebsd.org, freebsd-stable@freebsd.org Subject: (fixed) Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 16:57:47 -0000 --G4iJoqBmSsgzjUCe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Mar 12, 2010 at 05:18:32AM -0800, Jeremy Chadwick wrote: > I'm a little confused by the kernel output. It appears as if you're > using the new SATA-to-CAM layer (ahci.ko) for your SATA disks, rather > than the ataahci.ko layer... but I don't see any indication of AHCI > being available/used on your southbridge chipset. ahci.ko is not compiled in or loaded as a module, however in the course of debugging the problem I added option ATA_CAM, which seems to result in /dev/ada0*. Removing the option to get back to /dev/ad4* doesn't fix the problem. BTW ATA_CAM seems to work fine with generic ATA but not with ataintel. > Possibly this is the source of the problem (specifically, it looks like > FreeBSD doesn't have proper device ID knowledge of what this controller > is. I believe that's because this system is *very* new, a Core i3/i5/i7 > system)? I didn't notice that, you're right! The system is brand new, got it delivered on Tuesday. Another odd thing is that the controllers have differents IDs, 0x3b208086 vs 0x3b268086. I added the IDs to ata-intel.c and it fixes the problem. Preparing a patch. Thanks for the hint! -- Pierre Beyssac pb@fasterix.frmug.org --G4iJoqBmSsgzjUCe-- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 17:09:39 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52BCC106566C for ; Fri, 12 Mar 2010 17:09:39 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta06.westchester.pa.mail.comcast.net (qmta06.westchester.pa.mail.comcast.net [76.96.62.56]) by mx1.freebsd.org (Postfix) with ESMTP id F24558FC15 for ; Fri, 12 Mar 2010 17:09:38 +0000 (UTC) Received: from omta04.westchester.pa.mail.comcast.net ([76.96.62.35]) by qmta06.westchester.pa.mail.comcast.net with comcast id sChA1d0010ldTLk56H9fKR; Fri, 12 Mar 2010 17:09:39 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta04.westchester.pa.mail.comcast.net with comcast id sH9d1d00c3S48mS3QH9e3i; Fri, 12 Mar 2010 17:09:38 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id AE57E9B436; Fri, 12 Mar 2010 09:09:36 -0800 (PST) Date: Fri, 12 Mar 2010 09:09:36 -0800 From: Jeremy Chadwick To: Pierre Beyssac Message-ID: <20100312170936.GA59343@icarus.home.lan> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312131832.GA54731@icarus.home.lan> <20100312165744.GA86971@fasterix.frmug.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100312165744.GA86971@fasterix.frmug.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: mav@freebsd.org, freebsd-stable@freebsd.org Subject: Re: (fixed) Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 17:09:39 -0000 On Fri, Mar 12, 2010 at 05:57:44PM +0100, Pierre Beyssac wrote: > > Possibly this is the source of the problem (specifically, it looks like > > FreeBSD doesn't have proper device ID knowledge of what this controller > > is. I believe that's because this system is *very* new, a Core i3/i5/i7 > > system)? > > I didn't notice that, you're right! The system is brand new, got > it delivered on Tuesday. Another odd thing is that the controllers > have differents IDs, 0x3b208086 vs 0x3b268086. > > I added the IDs to ata-intel.c and it fixes the problem. > > Preparing a patch. Thanks for the hint! > -- > Pierre Beyssac pb@fasterix.frmug.org > --- ata-intel.c.orig 2010-03-12 17:02:00.680011011 +0100 > +++ ata-intel.c 2010-03-12 16:55:54.773702505 +0100 > @@ -156,6 +156,8 @@ > { 0x3a2d8086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, > { 0x3a2e8086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, > { 0x3a2f8086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, > + { 0x3b208086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, > + { 0x3b268086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, > { ATA_I31244, 0, 0, 2, ATA_SA150, "31244" }, > { ATA_ISCH, 0, 0, 1, ATA_UDMA5, "SCH" }, > { 0, 0, 0, 0, 0, 0}}; I had a chance to review the Intel 5 Series and 3400 Series Chipset document, which is specific to the PCH. The breakdown for those two Device IDs, under Vendor ID 0x8086 (Intel), is: Device ID 0x3b20 - PCH SATA controller - Desktop revision, non-AHCI and non-RAID Mode - Ports 0,1,2,3 Device ID 0x3b26 - PCH SATA controller - Desktop revision, non-AHCI and non-RAID mode - Ports 4,5 So I'm not sure the setting of the INTEL_AHCI flag there is correct for these controllers. mav@ will need to chime in here. Regarding the "dual device IDs": what this means is that ports 0-3 are tied to one device on the PCI bus, and ports 4-5 are tied to another device on the PCI bus. They might have chosen to do this so they could segregate which ports do what or operate differently somehow. mav@ may also want to review the specification since it mentions a bunch of other Device IDs which don't appear to be in the above list: http://www.intel.com/Assets/PDF/specupdate/322170.pdf -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 17:31:54 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8DF2A106566C; Fri, 12 Mar 2010 17:31:54 +0000 (UTC) (envelope-from pb@fasterix.frmug.org) Received: from fasterix.frmug.org (unknown [IPv6:2a01:e35:8b50:2c40::2]) by mx1.freebsd.org (Postfix) with ESMTP id F18198FC16; Fri, 12 Mar 2010 17:31:53 +0000 (UTC) Received: from fasterix.frmug.org (localhost [127.0.0.1]) by fasterix.frmug.org (8.14.3/8.14.3) with ESMTP id o2CHVpIK089562 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 12 Mar 2010 18:31:52 +0100 (CET) (envelope-from pb@fasterix.frmug.org) Received: (from pb@localhost) by fasterix.frmug.org (8.14.3/8.14.3/Submit) id o2CHVpfZ089561; Fri, 12 Mar 2010 18:31:51 +0100 (CET) (envelope-from pb) Date: Fri, 12 Mar 2010 18:31:51 +0100 From: Pierre Beyssac To: Jeremy Chadwick Message-ID: <20100312173151.GA89136@fasterix.frmug.org> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312131832.GA54731@icarus.home.lan> <20100312165744.GA86971@fasterix.frmug.org> <20100312170936.GA59343@icarus.home.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100312170936.GA59343@icarus.home.lan> User-Agent: Mutt/1.4.2.3i X-message-flag: Warning! Use of Microsoft Outlook is dangerous and makes your system susceptible to worms and viruses Cc: mav@freebsd.org, freebsd-stable@freebsd.org Subject: Re: (fixed) Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 17:31:54 -0000 On Fri, Mar 12, 2010 at 09:09:36AM -0800, Jeremy Chadwick wrote: > > + { 0x3b208086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, > > + { 0x3b268086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, > Device ID 0x3b20 > - PCH SATA controller > - Desktop revision, non-AHCI and non-RAID Mode > - Ports 0,1,2,3 Great, I was just wondering about the values I put in the fields. So, thanks, non-AHCI. That explains why adding the same ids to ahci.c didn't yeld anything interesting :-). And the PCH name is correct OTOH. > So I'm not sure the setting of the INTEL_AHCI flag there is correct > for these controllers. mav@ will need to chime in here. Probably not, I'll remove it. Thanks. -- Pierre Beyssac pb@fasterix.frmug.org From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 17:47:01 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1EAED106566C; Fri, 12 Mar 2010 17:47:01 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.153]) by mx1.freebsd.org (Postfix) with ESMTP id 78D598FC08; Fri, 12 Mar 2010 17:46:59 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id e21so427052fga.13 for ; Fri, 12 Mar 2010 09:46:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=5B9dR/1UpT5Prv7lOp0tbpE08c0pK+eC9CH6bt+eWo8=; b=rf+z4M3fZVO29tOgK3aruy9dNXg5kUMvw598sNOuGSa/obnyoqzZN/cdB4rc6iZml1 /Obym/umLzcHsvK570x0fhtGAT9sRrIlcFSZAfow5qelxzZFhu6bqze5aei58+VaGcSq B+bLluKlDOyAx08W7Y63h4Qygf8tNjNo5wg0M= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=uas1t9mivSEt3hcNBv/6n2yQPJ49sQ3QeriCCe/GyzeEqXzNxaPHkPbbo+bjv4uf1Z mQS1DoQtJ2DoTXQ+TY3mOLdnFaROEUbk6J+Kf6n03SL5Eig2QmU/XYufyIa6+ZalkttC 8G4Rwv3uvigKkQWsCi2LOCGNmc0DKNvwNgTTw= Received: by 10.87.63.4 with SMTP id q4mr4266034fgk.59.1268416019083; Fri, 12 Mar 2010 09:46:59 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id e11sm1882708fga.1.2010.03.12.09.46.55 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 12 Mar 2010 09:46:57 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Fri, 12 Mar 2010 09:46:55 -0800 From: Pyun YongHyeon Date: Fri, 12 Mar 2010 09:46:55 -0800 To: Pierre Beyssac Message-ID: <20100312174655.GG16314@michelle.cdnetworks.com> References: <20100312121409.GA79294@fasterix.frmug.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="/04w6evG8XlLl3ft" Content-Disposition: inline In-Reply-To: <20100312121409.GA79294@fasterix.frmug.org> User-Agent: Mutt/1.4.2.3i Cc: mav@freebsd.org, freebsd-stable@freebsd.org Subject: Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 17:47:01 -0000 --/04w6evG8XlLl3ft Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Mar 12, 2010 at 01:14:09PM +0100, Pierre Beyssac wrote: > Hello, > > I'm having "interrupt storm detected" messages on a Dell Inspiron > 580 running up-to-date 8-STABLE (amd64 arch). The interrupts seem > to come from one of the atapci controllers, apparently atapci0 (main > controller, with a SATA disk and an ATAPI optical drive). > > ata_interrupt gets called at a variable rate, between 1000-150000 > times per second, constantly, even when the disk is not used. > > >From adding debug sysctl code in ata-all.c:ata_interrupt_locked() > I have been able to check that: > ch->running is NULL (breaks loop in "do we have a running request") > ch->state=0 > ch->unit=0 ch->devices=1 (ATA_ATA_MASTER) most of the time. > > Here's attached dmesg output, pciconf -lv output, kernel configuration > and vmstat -i output. A -current kernel exhibits the same behaviour. > > Any hint/idea how to debug this further would be really appreciated... > -- > Pierre Beyssac pb@fasterix.frmug.org [...] > bge0: mem 0xfbff0000-0xfbffffff irq 17 at device 0.0 on pci3 > bge0: Reserved 0x10000 bytes for rid 0x10 type 3 at 0xfbff0000 > bge0: adjust device control 0x2000 -> 0x5000 > bge0: attempting to allocate 1 MSI vectors (1 supported) > msi: routing MSI IRQ 256 to local APIC 0 vector 50 > bge0: using IRQ 256 for MSI > bge0: CHIP ID 0x57780001; ASIC REV 0x57780; CHIP REV 0x577800; PCI-E > bge0: Disabling fastboot > bge0: Disabling fastboot > miibus0: on bge0 > ukphy0: PHY 1 on miibus0 > ukphy0: OUI 0x00d897, model 0x0019, rev. 1 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This is not related with your interrupt storm issue but something is wrong here. I think brgphy(4) should be used for bge(4). Have no idea why the OUI has a different value. Would you try attached patch and let me know whether brgphy(4) get attached to the PHY? > ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto > bge0: bpf attached > bge0: Ethernet address: 00:25:64:f4:27:26 > bge0: [MPSAFE] > bge0: [FILTER] [...] --/04w6evG8XlLl3ft Content-Type: text/x-diff; charset=us-ascii Content-Disposition: attachment; filename="BCM57780.diff" Index: sys/dev/mii/miidevs =================================================================== --- sys/dev/mii/miidevs (revision 205052) +++ sys/dev/mii/miidevs (working copy) @@ -81,6 +81,7 @@ oui xxALTIMA 0x000895 Altima Communications oui xxBROADCOM 0x000818 Broadcom Corporation oui xxBROADCOM_ALT1 0x0050ef Broadcom Corporation +oui xxBROADCOM_ALT2 0x00d897 Broadcom Corporation oui xxICS 0x00057d Integrated Circuit Systems oui xxSEEQ 0x0005be Seeq oui xxSIS 0x000760 Silicon Integrated Systems @@ -150,6 +151,7 @@ model xxBROADCOM_ALT1 BCM5784 0x003a BCM5784 10/100/1000baseTX PHY model xxBROADCOM_ALT1 BCM5709C 0x003c BCM5709C 10/100/1000baseTX PHY model xxBROADCOM_ALT1 BCM5761 0x003d BCM5761 10/100/1000baseTX PHY +model xxBROADCOM_ALT2 BCM57780 0x0019 BCM57780 10/100/1000baseTX PHY model BROADCOM2 BCM5906 0x0004 BCM5906 10/100baseTX PHY /* Cicada Semiconductor PHYs (now owned by Vitesse?) */ Index: sys/dev/mii/brgphy.c =================================================================== --- sys/dev/mii/brgphy.c (revision 205052) +++ sys/dev/mii/brgphy.c (working copy) @@ -139,6 +139,7 @@ MII_PHY_DESC(xxBROADCOM_ALT1, BCM5784), MII_PHY_DESC(xxBROADCOM_ALT1, BCM5709C), MII_PHY_DESC(xxBROADCOM_ALT1, BCM5761), + MII_PHY_DESC(xxBROADCOM_ALT2, BCM57780), MII_PHY_DESC(BROADCOM2, BCM5906), MII_PHY_END }; --/04w6evG8XlLl3ft-- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 18:21:38 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BD7C7106566B for ; Fri, 12 Mar 2010 18:21:38 +0000 (UTC) (envelope-from pb@fasterix.frmug.org) Received: from fasterix.frmug.org (unknown [IPv6:2a01:e35:8b50:2c40::2]) by mx1.freebsd.org (Postfix) with ESMTP id 07F1C8FC17 for ; Fri, 12 Mar 2010 18:21:36 +0000 (UTC) Received: from fasterix.frmug.org (localhost [127.0.0.1]) by fasterix.frmug.org (8.14.3/8.14.3) with ESMTP id o2CILZkT091166 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 12 Mar 2010 19:21:35 +0100 (CET) (envelope-from pb@fasterix.frmug.org) Received: (from pb@localhost) by fasterix.frmug.org (8.14.3/8.14.3/Submit) id o2CILYQN091165; Fri, 12 Mar 2010 19:21:34 +0100 (CET) (envelope-from pb) Date: Fri, 12 Mar 2010 19:21:34 +0100 From: Pierre Beyssac To: Pyun YongHyeon Message-ID: <20100312182134.GA90606@fasterix.frmug.org> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312174655.GG16314@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="a8Wt8u1KmwUX3Y2C" Content-Disposition: inline In-Reply-To: <20100312174655.GG16314@michelle.cdnetworks.com> User-Agent: Mutt/1.4.2.3i X-message-flag: Warning! Use of Microsoft Outlook is dangerous and makes your system susceptible to worms and viruses Cc: freebsd-stable@freebsd.org Subject: Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 18:21:38 -0000 --a8Wt8u1KmwUX3Y2C Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Mar 12, 2010 at 09:46:55AM -0800, Pyun YongHyeon wrote: > This is not related with your interrupt storm issue but something > is wrong here. I think brgphy(4) should be used for bge(4). Have no > idea why the OUI has a different value. > Would you try attached patch and let me know whether brgphy(4) > get attached to the PHY? It sort of works (see attached dmesg) but not quite correctly, the ethernet link takes an awful lot of time to negotiate (> 10s) and it negotiates 10 Mbps instead of 100 Mbps previously. The message "Unrecognized OUI for PHY!" seems to indicate something's amiss, maybe I should regenerate some file(s) after applying your patches? -- Pierre Beyssac pb@fasterix.frmug.org --a8Wt8u1KmwUX3Y2C Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="dmesg.out" Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.0-STABLE #1: Fri Mar 12 19:02:49 CET 2010 root@inspiron:/usr/src/sys/amd64/compile/INSP580 amd64 WARNING: WITNESS option enabled, expect reduced performance. Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff80a2b000. Preloaded elf obj module "/boot/kernel/ehci.ko" at 0xffffffff80a2b240. Preloaded elf obj module "/boot/kernel/usb.ko" at 0xffffffff80a2b868. Preloaded elf obj module "/boot/kernel/ukbd.ko" at 0xffffffff80a2bf10. Timecounter "i8254" frequency 1193182 Hz quality 0 Calibrating TSC clock ... TSC clock: 2659996160 Hz CPU: Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz (2660.00-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x106e5 Stepping = 5 Features=0xbfebfbff Features2=0x98e3fd AMD Features=0x28100800 AMD Features2=0x1 TSC: P-state invariant real memory = 8589934592 (8192 MB) Physical memory chunk(s): 0x0000000000001000 - 0x000000000009bfff, 634880 bytes (155 pages) 0x0000000000a5f000 - 0x00000000bd77ffff, 3167883264 bytes (773409 pages) 0x0000000100000000 - 0x000000022f12ffff, 5084741632 bytes (1241392 pages) avail memory = 8210993152 (7830 MB) ACPI APIC Table: INTR: Adding local APIC 2 as a target INTR: Adding local APIC 4 as a target INTR: Adding local APIC 6 as a target FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 1 package(s) x 4 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 2 cpu2 (AP): APIC ID: 4 cpu3 (AP): APIC ID: 6 APIC: CPU 0 has ACPI ID 1 APIC: CPU 1 has ACPI ID 2 APIC: CPU 2 has ACPI ID 3 APIC: CPU 3 has ACPI ID 4 ULE: setup cpu 0 ULE: setup cpu 1 ULE: setup cpu 2 ULE: setup cpu 3 ACPI: RSDP 0xf9b00 00024 (v2 ACPIAM) ACPI: XSDT 0xbd780100 0006C (v1 DELL FX09 20091130 MSFT 00000097) ACPI: FACP 0xbd780290 000F4 (v4 DELL FX09 20091130 MSFT 00000097) ACPI: DSDT 0xbd780660 05B02 (v2 1AAAA 1AAAA000 00000000 INTL 20051117) ACPI: FACS 0xbd78e000 00040 ACPI: APIC 0xbd780390 0008C (v2 DELL FX09 20091130 MSFT 00000097) ACPI: MCFG 0xbd780420 0003C (v1 DELL OEMMCFG 20091130 MSFT 00000097) ACPI: SLIC 0xbd780460 00176 (v1 DELL FX09 20091130 MSFT 00000097) ACPI: OSFR 0xbd7805e0 00080 (v1 DELL FX09 20091130 MSFT 00000097) ACPI: OEMB 0xbd78e040 00072 (v1 DELL FX09 20091130 MSFT 00000097) ACPI: HPET 0xbd78a660 00038 (v1 DELL OEMHPET 20091130 MSFT 00000097) ACPI: ASF! 0xbd78a6a0 00099 (v32 LEGEND I865PASF 00000001 INTL 20051117) ACPI: SSDT 0xbd78f6f0 00363 (v1 DpgPmm CpuPm 00000012 INTL 20051117) MADT: Found IO APIC ID 7, Interrupt 0 at 0xfec00000 ioapic0: Changing APIC ID to 7 ioapic0: Routing external 8259A's -> intpin 0 MADT: Interrupt override: source 0, irq 2 ioapic0: Routing IRQ 0 -> intpin 2 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level ioapic0 irqs 0-23 on motherboard cpu0 BSP: ID: 0x00000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x0001000f pcm: 0x00010400 null: random: VESA: information block 0000 56 45 53 41 00 03 f0 01 00 c0 01 00 00 00 44 00 0010 00 01 00 01 0f 0c 29 01 00 c0 bb 00 00 c0 3a 4d 0020 00 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0040 00 00 00 00 00 01 01 01 03 01 05 01 07 01 10 01 0050 11 01 13 01 14 01 16 01 17 01 19 01 1a 01 0d 01 0060 0e 01 20 01 93 01 95 01 96 01 b3 01 b5 01 b6 01 0070 c3 01 c5 01 c6 01 33 01 35 01 36 01 53 01 55 01 0080 56 01 63 01 65 01 66 01 21 01 22 01 23 01 24 01 0090 43 01 45 01 46 01 73 01 75 01 76 01 83 01 85 01 00a0 86 01 d3 01 d5 01 d6 01 e3 01 e5 01 e6 01 ff ff 00b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0100 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0120 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0130 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0140 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0150 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0160 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0170 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0180 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0190 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01a0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01b0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01d0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01e0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01f0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 VESA: 53 mode(s) found VESA: v3.0, 16384k memory, flags:0x1, mode table:0xffffff8000099044 (1000044) VESA: ATI ATOMBIOS VESA: (C) 1988-2005, ATI Technologies Inc. CEDAR 01.00 io: kbd: new array size 4 kbd1 at kbdmux0 mem: acpi0: on motherboard PCIe: Memory Mapped configuration base @ 0xe0000000 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 acpi0: [MPSAFE] acpi0: [ITHREAD] ACPI: Executed 1 blocks of module-level executable AML code acpi0: Power Button (fixed) acpi0: wakeup code va 0xffffff800001b000 pa 0x4000 AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.IELK.RXA0 -> bus 0 dev 0 func 0 AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.FHR0 -> bus 0 dev 31 func 0 AcpiOsDerivePciId: \\_SB_.PCI0.SBRG.PIX0 -> bus 0 dev 31 func 0 acpi0: reservation of fee00000, 1000 (3) failed acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, bdf00000 (3) failed ACPI timer: 1/1 1/1 1/1 1/1 0/225 1/1 1/1 1/1 1/1 1/1 -> 9 Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 cpu0: on acpi0 ACPI: SSDT 0xbd78e0c0 01130 (v1 DpgPmm P001Ist 00000011 INTL 20051117) ACPI: SSDT 0xbd78f1f0 004F4 (v1 PmRef P001Cst 00003001 INTL 20051117) cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 pci_link0: Index IRQ Rtd Ref IRQs Initial Probe 0 10 N 0 3 4 6 7 10 11 12 14 15 Validation 0 10 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link1: Index IRQ Rtd Ref IRQs Initial Probe 0 5 N 0 5 Validation 0 5 N 0 5 After Disable 0 255 N 0 5 pci_link2: Index IRQ Rtd Ref IRQs Initial Probe 0 3 N 0 3 4 6 7 10 11 12 14 15 Validation 0 3 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link3: Index IRQ Rtd Ref IRQs Initial Probe 0 11 N 0 3 4 6 7 10 11 12 14 15 Validation 0 11 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link4: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 6 7 10 11 12 14 15 Validation 0 255 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link5: Index IRQ Rtd Ref IRQs Initial Probe 0 255 N 0 3 4 6 7 10 11 12 14 15 Validation 0 255 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link6: Index IRQ Rtd Ref IRQs Initial Probe 0 15 N 0 3 4 6 7 10 11 12 14 15 Validation 0 15 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 pci_link7: Index IRQ Rtd Ref IRQs Initial Probe 0 14 N 0 3 4 6 7 10 11 12 14 15 Validation 0 14 N 0 3 4 6 7 10 11 12 14 15 After Disable 0 255 N 0 3 4 6 7 10 11 12 14 15 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 acpi_hpet0: vend: 0x8086 rev: 0x1 num: 8 hz: 14318180 opts: legacy_route 64-bit Timecounter "HPET" frequency 14318180 Hz quality 900 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pci0: domain=0, physical bus=0 found-> vendor=0x8086, dev=0xd131, revid=0x11 domain=0, bus=0, slot=0, func=0 class=06-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd138, revid=0x11 domain=0, bus=0, slot=3, func=0 class=06-04-00, hdrtype=0x01, mfdev=0 cmdreg=0x0107, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x1a (6500 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 3 supports D0 D3 current D0 MSI supports 2 messages, vector masks pcib0: matched entry for 0.3.INTA pcib0: slot 3 INTA hardwired to IRQ 16 found-> vendor=0x8086, dev=0xd155, revid=0x11 domain=0, bus=0, slot=8, func=0 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd156, revid=0x11 domain=0, bus=0, slot=8, func=1 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd157, revid=0x11 domain=0, bus=0, slot=8, func=2 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd158, revid=0x11 domain=0, bus=0, slot=8, func=3 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd150, revid=0x11 domain=0, bus=0, slot=16, func=0 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0xd151, revid=0x11 domain=0, bus=0, slot=16, func=1 class=08-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0000, statreg=0x0000, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3b64, revid=0x06 domain=0, bus=0, slot=22, func=0 class=07-80-00, hdrtype=0x00, mfdev=1 cmdreg=0x0006, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xfbdfe000, size 4, enabled pcib0: matched entry for 0.22.INTA pcib0: slot 22 INTA hardwired to IRQ 16 found-> vendor=0x8086, dev=0x3b3c, revid=0x06 domain=0, bus=0, slot=26, func=0 class=0c-03-20, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xfbdfc000, size 10, enabled pcib0: matched entry for 0.26.INTA pcib0: slot 26 INTA hardwired to IRQ 16 unknown: Reserved 0x400 bytes for rid 0x10 type 3 at 0xfbdfc000 found-> vendor=0x8086, dev=0x3b56, revid=0x06 domain=0, bus=0, slot=27, func=0 class=04-03-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=15 powerspec 2 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xfbdf8000, size 14, enabled pcib0: matched entry for 0.27.INTA pcib0: slot 27 INTA hardwired to IRQ 22 found-> vendor=0x8086, dev=0x3b42, revid=0x06 domain=0, bus=0, slot=28, func=0 class=06-04-00, hdrtype=0x01, mfdev=1 cmdreg=0x0104, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) intpin=a, irq=5 powerspec 2 supports D0 D3 current D0 MSI supports 1 message pcib0: matched entry for 0.28.INTA pcib0: slot 28 INTA hardwired to IRQ 17 found-> vendor=0x8086, dev=0x3b4c, revid=0x06 domain=0, bus=0, slot=28, func=5 class=06-04-00, hdrtype=0x01, mfdev=1 cmdreg=0x0106, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) intpin=b, irq=10 powerspec 2 supports D0 D3 current D0 MSI supports 1 message pcib0: matched entry for 0.28.INTB pcib0: slot 28 INTB hardwired to IRQ 16 found-> vendor=0x8086, dev=0x3b34, revid=0x06 domain=0, bus=0, slot=29, func=0 class=0c-03-20, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0290, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=14 powerspec 2 supports D0 D3 current D0 map[10]: type Memory, range 32, base 0xfbdf6000, size 10, enabled pcib0: matched entry for 0.29.INTA pcib0: slot 29 INTA hardwired to IRQ 23 unknown: Reserved 0x400 bytes for rid 0x10 type 3 at 0xfbdf6000 found-> vendor=0x8086, dev=0x244e, revid=0xa6 domain=0, bus=0, slot=30, func=0 class=06-04-01, hdrtype=0x01, mfdev=0 cmdreg=0x0104, statreg=0x0010, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x02 (500 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3b08, revid=0x06 domain=0, bus=0, slot=31, func=0 class=06-01-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0210, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) found-> vendor=0x8086, dev=0x3b20, revid=0x06 domain=0, bus=0, slot=31, func=2 class=01-01-8f, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x02b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=11 powerspec 3 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xdc00, size 3, enabled map[14]: type I/O Port, range 32, base 0xd880, size 2, enabled map[18]: type I/O Port, range 32, base 0xd800, size 3, enabled map[1c]: type I/O Port, range 32, base 0xd480, size 2, enabled map[20]: type I/O Port, range 32, base 0xd400, size 4, enabled map[24]: type I/O Port, range 32, base 0xd080, size 4, enabled pcib0: matched entry for 0.31.INTB pcib0: slot 31 INTB hardwired to IRQ 19 found-> vendor=0x8086, dev=0x3b30, revid=0x06 domain=0, bus=0, slot=31, func=3 class=0c-05-00, hdrtype=0x00, mfdev=0 cmdreg=0x0003, statreg=0x0280, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=c, irq=3 map[10]: type Memory, range 64, base 0xfbdf4000, size 8, enabled map[20]: type I/O Port, range 32, base 0x400, size 5, enabled pcib0: matched entry for 0.31.INTC pcib0: slot 31 INTC hardwired to IRQ 18 found-> vendor=0x8086, dev=0x3b26, revid=0x06 domain=0, bus=0, slot=31, func=5 class=01-01-85, hdrtype=0x00, mfdev=0 cmdreg=0x0005, statreg=0x02b0, cachelnsz=0 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=11 powerspec 3 supports D0 D3 current D0 map[10]: type I/O Port, range 32, base 0xcc00, size 3, enabled map[14]: type I/O Port, range 32, base 0xc880, size 2, enabled map[18]: type I/O Port, range 32, base 0xc800, size 3, enabled map[1c]: type I/O Port, range 32, base 0xc480, size 2, enabled map[20]: type I/O Port, range 32, base 0xc400, size 4, enabled map[24]: type I/O Port, range 32, base 0xc080, size 4, enabled pcib0: matched entry for 0.31.INTB pcib0: slot 31 INTB hardwired to IRQ 19 pcib1: irq 16 at device 3.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: I/O decode 0xe000-0xefff pcib1: memory decode 0xfbe00000-0xfbefffff pcib1: prefetched decode 0xd0000000-0xdfffffff pcib1: could not get PCI interrupt routing table for \\_SB_.PCI0.P0P3 - AE_NOT_FOUND pci1: on pcib1 pci1: domain=0, physical bus=1 found-> vendor=0x1002, dev=0x68f9, revid=0x00 domain=0, bus=1, slot=0, func=0 class=03-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 3 supports D0 D1 D2 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Prefetchable Memory, range 64, base 0xd0000000, size 28, enabled pcib1: requested memory range 0xd0000000-0xdfffffff: good map[18]: type Memory, range 64, base 0xfbee0000, size 17, enabled pcib1: requested memory range 0xfbee0000-0xfbefffff: good map[20]: type I/O Port, range 32, base 0xe000, size 8, enabled pcib1: requested I/O range 0xe000-0xe0ff: in range pcib0: matched entry for 0.3.INTA pcib0: slot 3 INTA hardwired to IRQ 16 pcib1: slot 0 INTA is routed to irq 16 found-> vendor=0x1002, dev=0xaa68, revid=0x00 domain=0, bus=1, slot=0, func=1 class=04-03-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=5 powerspec 3 supports D0 D1 D2 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xfbebc000, size 14, enabled pcib1: requested memory range 0xfbebc000-0xfbebffff: good pcib0: matched entry for 0.3.INTB pcib0: slot 3 INTB hardwired to IRQ 17 pcib1: slot 0 INTB is routed to irq 17 vgapci0: port 0xe000-0xe0ff mem 0xd0000000-0xdfffffff,0xfbee0000-0xfbefffff irq 16 at device 0.0 on pci1 pci1: at device 0.1 (no driver attached) pci0: at device 8.0 (no driver attached) pci0: at device 8.1 (no driver attached) pci0: at device 8.2 (no driver attached) pci0: at device 8.3 (no driver attached) pci0: at device 16.0 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 22.0 (no driver attached) ehci0: mem 0xfbdfc000-0xfbdfc3ff irq 16 at device 26.0 on pci0 ioapic0: routing intpin 16 (PCI IRQ 16) to lapic 0 vector 49 ehci0: [MPSAFE] ehci0: [ITHREAD] usbus0: EHCI version 1.0 usbus0: on ehci0 pci0: at device 27.0 (no driver attached) pcib2: irq 17 at device 28.0 on pci0 pcib2: domain 0 pcib2: secondary bus 2 pcib2: subordinate bus 2 pcib2: I/O decode 0x0-0x0 pcib2: no prefetched decode pci2: on pcib2 pci2: domain=0, physical bus=2 pcib3: irq 16 at device 28.5 on pci0 pcib3: domain 0 pcib3: secondary bus 3 pcib3: subordinate bus 3 pcib3: I/O decode 0x0-0x0 pcib3: memory decode 0xfbf00000-0xfbffffff pcib3: no prefetched decode pci3: on pcib3 pci3: domain=0, physical bus=3 found-> vendor=0x14e4, dev=0x1691, revid=0x01 domain=0, bus=3, slot=0, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=5 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xfbff0000, size 16, enabled pcib3: requested memory range 0xfbff0000-0xfbffffff: good pcib3: matched entry for 3.0.INTA pcib3: slot 0 INTA hardwired to IRQ 17 bge0: mem 0xfbff0000-0xfbffffff irq 17 at device 0.0 on pci3 bge0: Reserved 0x10000 bytes for rid 0x10 type 3 at 0xfbff0000 bge0: adjust device control 0x2000 -> 0x5000 bge0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 256 to local APIC 0 vector 50 bge0: using IRQ 256 for MSI bge0: CHIP ID 0x57780001; ASIC REV 0x57780; CHIP REV 0x577800; PCI-E bge0: Disabling fastboot bge0: Disabling fastboot miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: OUI 0x00d897, model 0x0019, rev. 1 brgphy0: Unrecognized OUI for PHY! brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bge0: bpf attached bge0: Ethernet address: 00:25:64:f4:27:26 bge0: [MPSAFE] bge0: [FILTER] ehci1: mem 0xfbdf6000-0xfbdf63ff irq 23 at device 29.0 on pci0 ioapic0: routing intpin 23 (PCI IRQ 23) to lapic 0 vector 51 ehci1: [MPSAFE] ehci1: [ITHREAD] usbus1: EHCI version 1.0 usbus1: on ehci1 pcib4: at device 30.0 on pci0 pcib4: domain 0 pcib4: secondary bus 4 pcib4: subordinate bus 4 pcib4: I/O decode 0x0-0x0 pcib4: no prefetched decode pcib4: Subtractively decoded bridge. pci4: on pcib4 pci4: domain=0, physical bus=4 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0xdc00-0xdc07,0xd880-0xd883,0xd800-0xd807,0xd480-0xd483,0xd400-0xd40f,0xd080-0xd08f irq 19 at device 31.2 on pci0 atapci0: Reserved 0x10 bytes for rid 0x20 type 4 at 0xd400 ioapic0: routing intpin 19 (PCI IRQ 19) to lapic 0 vector 52 atapci0: [MPSAFE] atapci0: [ITHREAD] atapci0: Reserved 0x10 bytes for rid 0x24 type 4 at 0xd080 ata2: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x10 type 4 at 0xdc00 atapci0: Reserved 0x4 bytes for rid 0x14 type 4 at 0xd880 ata2: reset tp1 mask=01 ostat0=50 ostat1=00 ata2: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata2: reset tp2 stat0=50 stat1=00 devices=0x1 ata2: [MPSAFE] ata2: [ITHREAD] ata3: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x18 type 4 at 0xd800 atapci0: Reserved 0x4 bytes for rid 0x1c type 4 at 0xd480 ata3: reset tp1 mask=01 ostat0=00 ostat1=00 ata3: stat0=0x00 err=0x01 lsb=0x14 msb=0xeb ata3: reset tp2 stat0=00 stat1=00 devices=0x10000 ata3: [MPSAFE] ata3: [ITHREAD] pci0: at device 31.3 (no driver attached) atapci1: port 0xcc00-0xcc07,0xc880-0xc883,0xc800-0xc807,0xc480-0xc483,0xc400-0xc40f,0xc080-0xc08f irq 19 at device 31.5 on pci0 atapci1: Reserved 0x10 bytes for rid 0x20 type 4 at 0xc400 atapci1: [MPSAFE] atapci1: [ITHREAD] atapci1: Reserved 0x10 bytes for rid 0x24 type 4 at 0xc080 ata4: on atapci1 atapci1: Reserved 0x8 bytes for rid 0x10 type 4 at 0xcc00 atapci1: Reserved 0x4 bytes for rid 0x14 type 4 at 0xc880 ata4: reset tp1 mask=01 ostat0=7f ostat1=00 ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: reset tp2 stat0=ff stat1=00 devices=0x0 ata4: [MPSAFE] ata4: [ITHREAD] ata5: on atapci1 atapci1: Reserved 0x8 bytes for rid 0x18 type 4 at 0xc800 atapci1: Reserved 0x4 bytes for rid 0x1c type 4 at 0xc480 ata5: reset tp1 mask=01 ostat0=7f ostat1=00 ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: reset tp2 stat0=ff stat1=00 devices=0x0 ata5: [MPSAFE] ata5: [ITHREAD] acpi_button0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock (resolution 1000000us) isa_probe_children: disabling PnP devices atrtc: atrtc0 already exists; skipping it sc: sc0 already exists; skipping it isa_probe_children: probing non-PnP devices orm0: at iomem 0xc0000-0xce7ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: scteken (teken terminal) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 kbd0: atkbd0, generic (0), config:0x0, flags:0x3f0000 ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 53 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: current command byte:0065 psm0: failed to reset the aux device. fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0 ppc0 failed to probe at irq 7 on isa0 uart0: failed to probe at port 0x3f8-0x3ff irq 4 on isa0 uart1: failed to probe at port 0x2f8-0x2ff irq 3 on isa0 isa_probe_children: probing PnP devices est0: on cpu0 est0: Invalid id16 (set, cur) = (19, 20) est0: Can't check freq 2533, it may be invalid est0: Invalid id16 (set, cur) = (18, 20) est0: Can't check freq 2400, it may be invalid est0: Invalid id16 (set, cur) = (17, 20) est0: Can't check freq 2267, it may be invalid est0: Invalid id16 (set, cur) = (16, 20) est0: Can't check freq 2133, it may be invalid est0: Invalid id16 (set, cur) = (15, 20) est0: Can't check freq 2000, it may be invalid est0: Invalid id16 (set, cur) = (14, 20) est0: Can't check freq 1867, it may be invalid est0: Invalid id16 (set, cur) = (13, 20) est0: Can't check freq 1733, it may be invalid est0: Invalid id16 (set, cur) = (12, 20) est0: Can't check freq 1600, it may be invalid est0: Invalid id16 (set, cur) = (11, 20) est0: Can't check freq 1467, it may be invalid est0: Invalid id16 (set, cur) = (10, 20) est0: Can't check freq 1333, it may be invalid est0: Invalid id16 (set, cur) = (9, 20) est0: Can't check freq 1200, it may be invalid p4tcc0: on cpu0 est1: on cpu1 est1: Invalid id16 (set, cur) = (19, 20) est1: Can't check freq 2533, it may be invalid est1: Invalid id16 (set, cur) = (18, 20) est1: Can't check freq 2400, it may be invalid est1: Invalid id16 (set, cur) = (17, 20) est1: Can't check freq 2267, it may be invalid est1: Invalid id16 (set, cur) = (16, 20) est1: Can't check freq 2133, it may be invalid est1: Invalid id16 (set, cur) = (15, 20) est1: Can't check freq 2000, it may be invalid est1: Invalid id16 (set, cur) = (14, 20) est1: Can't check freq 1867, it may be invalid est1: Invalid id16 (set, cur) = (13, 20) est1: Can't check freq 1733, it may be invalid est1: Invalid id16 (set, cur) = (12, 20) est1: Can't check freq 1600, it may be invalid est1: Invalid id16 (set, cur) = (11, 20) est1: Can't check freq 1467, it may be invalid est1: Invalid id16 (set, cur) = (10, 20) est1: Can't check freq 1333, it may be invalid est1: Invalid id16 (set, cur) = (9, 20) est1: Can't check freq 1200, it may be invalid p4tcc1: on cpu1 est2: on cpu2 est2: Invalid id16 (set, cur) = (19, 20) est2: Can't check freq 2533, it may be invalid est2: Invalid id16 (set, cur) = (18, 20) est2: Can't check freq 2400, it may be invalid est2: Invalid id16 (set, cur) = (17, 20) est2: Can't check freq 2267, it may be invalid est2: Invalid id16 (set, cur) = (16, 20) est2: Can't check freq 2133, it may be invalid est2: Invalid id16 (set, cur) = (15, 20) est2: Can't check freq 2000, it may be invalid est2: Invalid id16 (set, cur) = (14, 20) est2: Can't check freq 1867, it may be invalid est2: Invalid id16 (set, cur) = (13, 20) est2: Can't check freq 1733, it may be invalid est2: Invalid id16 (set, cur) = (12, 20) est2: Can't check freq 1600, it may be invalid est2: Invalid id16 (set, cur) = (11, 20) est2: Can't check freq 1467, it may be invalid est2: Invalid id16 (set, cur) = (10, 20) est2: Can't check freq 1333, it may be invalid est2: Invalid id16 (set, cur) = (9, 20) est2: Can't check freq 1200, it may be invalid p4tcc2: on cpu2 est3: on cpu3 est3: Invalid id16 (set, cur) = (19, 20) est3: Can't check freq 2533, it may be invalid est3: Invalid id16 (set, cur) = (18, 20) est3: Can't check freq 2400, it may be invalid est3: Invalid id16 (set, cur) = (17, 20) est3: Can't check freq 2267, it may be invalid est3: Invalid id16 (set, cur) = (16, 20) est3: Can't check freq 2133, it may be invalid est3: Invalid id16 (set, cur) = (15, 20) est3: Can't check freq 2000, it may be invalid est3: Invalid id16 (set, cur) = (14, 20) est3: Can't check freq 1867, it may be invalid est3: Invalid id16 (set, cur) = (13, 20) est3: Can't check freq 1733, it may be invalid est3: Invalid id16 (set, cur) = (12, 20) est3: Can't check freq 1600, it may be invalid est3: Invalid id16 (set, cur) = (11, 20) est3: Can't check freq 1467, it may be invalid est3: Invalid id16 (set, cur) = (10, 20) est3: Can't check freq 1333, it may be invalid est3: Invalid id16 (set, cur) = (9, 20) est3: Can't check freq 1200, it may be invalid p4tcc3: on cpu3 Device configuration finished. Reducing kern.maxvnodes 509903 -> 100000 procfs registered lapic: Divisor 2, Frequency 66499904 Hz Timecounter "TSC" frequency 2659996160 Hz quality -100 Timecounters tick every 1.000 msec usbus0: 480Mbps High Speed USB v2.0 usbus1: 480Mbps High Speed USB v2.0 lo0: bpf attached ata2: Identifying devices: 00000001 ata2: New devices: 00000001 ata2-master: pio=PIO4 wdma=WDMA2 udma=UDMA133 cable=40 wire ad4: setting UDMA100 ad4: 953869MB at ata2-master UDMA100 SATA 3Gb/s ad4: 1953525168 sectors [1938021C/16H/63S] 16 sectors/interrupt 1 depth queue ata3: Identifying devices: 00010000 ata3: New devices: 00010000 GEOM: new disk ad4 ata3-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=40 wire acd0: setting UDMA100 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 acd0: DVDR drive at ata3 as master acd0: read 8269KB/s (8269KB/s) write 8269KB/s (8269KB/s), 2048KB buffer, UDMA100 SATA 1.5Gb/s acd0: Reads: CDR, CDRW, CDDA stream, DVDROM, DVDR, DVDRAM, packet acd0: Writes: CDR, CDRW, DVDR, DVDRAM, test write, burnproof acd0: Audio: play, 256 volume levels acd0: Mechanism: ejectable tray, unlocked acd0: Medium: no/blank disc ata4: Identifying devices: 00000000 ata4: New devices: 00000000 ata5: Identifying devices: 00000000 ata5: New devices: 00000000 SMP: AP CPU #3 Launched! cpu3 AP: ID: 0x06000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000200ef therm: 0x00010000 err: 0x00010000 pcm: 0x00010400 SMP: AP CPU #2 Launched! cpu2 AP: ID: 0x04000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000200ef therm: 0x00010000 err: 0x00010000 pcm: 0x00010400 SMP: AP CPU #1 Launched! cpu1 AP: ID: 0x02000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000200ef therm: 0x00010000 err: 0x00010000 pcm: 0x00010400 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 2 vector 48 ioapic0: routing intpin 16 (PCI IRQ 16) to lapic 4 vector 48 ioapic0: routing intpin 19 (PCI IRQ 19) to lapic 6 vector 48 msi: Assigning MSI IRQ 256 to local APIC 2 vector 49 WARNING: WITNESS option enabled, expect reduced performance. GEOM: ad4: partition 3 does not start on a track boundary. GEOM: ad4: partition 3 does not end on a track boundary. GEOM: ad4: partition 2 does not start on a track boundary. GEOM: ad4: partition 2 does not end on a track boundary. Root mount waiting for: usbus1 usbus0 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered Root mount waiting for: usbus1 usbus0 ugen0.2: at usbus0 uhub2: on usbus0 ugen1.2: at usbus1 uhub3: on usbus1 Root mount waiting for: usbus1 usbus0 uhub2: 6 ports with 6 removable, self powered uhub3: 8 ports with 8 removable, self powered Root mount waiting for: usbus1 usbus0 ugen0.3: at usbus0 ukbd0: on usbus0 kbd2 at ukbd0 kbd2: ukbd0, generic (0), config:0x0, flags:0x3d0000 ugen1.3: at usbus1 Trying to mount root from ufs:/dev/ufs/root ct_to_ts([2010-03-12 18:17:33]) = 1268417853.000000000 start_init: trying /sbin/init bge0: Disabling fastboot bge0: Disabling fastboot --a8Wt8u1KmwUX3Y2C-- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 18:32:20 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6F87F1065670 for ; Fri, 12 Mar 2010 18:32:20 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 30F578FC12 for ; Fri, 12 Mar 2010 18:32:17 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 56DEF19E023 for ; Fri, 12 Mar 2010 19:32:16 +0100 (CET) Received: from [192.168.1.2] (r5bb235.net.upc.cz [86.49.61.235]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 1574F19E019 for ; Fri, 12 Mar 2010 19:32:14 +0100 (CET) Message-ID: <4B9A88AD.1020601@quip.cz> Date: Fri, 12 Mar 2010 19:32:13 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.8) Gecko/20100205 SeaMonkey/2.0.3 MIME-Version: 1.0 To: freebsd-stable@FreeBSD.org Content-Type: text/plain; charset=ISO-8859-2; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Geom not found: "gm0" / Failed to write sector zero X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 18:32:20 -0000 I just installed 7.3-RC2 amd64 on new server. I created slice s1 (80GB on disk ad4 (500GB), then partitions for system (/, swap, /var, /usr, /tmp) by sysinstall. After base install I created gmirror gm0 as usual (I did it many times). Now I am no longer in datacenter and have only ssh access to this server and I need to create slice s2 with some partitions for data storage, but fdisk failed. fdisk -u /dev/mirror/gm0 At the end, I got this error: Should we write new partition table? [n] y fdisk: Geom not found: "gm0" fdisk: Failed to write sector zero Fdisk failed even if I used sysctl kern.geom.debugflags=16 Question #1 - why 'Geom not found: "gm0"'? Question #2 - is there any way to create slices + partitions on unused space if system is booted from this device? Or is the only way to boot it from some LiveFS / fixit? I found the same question on this list, but without reply http://lists.freebsd.org/pipermail/freebsd-stable/2009-June/050855.html I hope somebody can help / explain it. Miroslav Lachman From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 18:38:39 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B46FF106566B; Fri, 12 Mar 2010 18:38:39 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id 992518FC13; Fri, 12 Mar 2010 18:38:38 +0000 (UTC) Received: by fxm23 with SMTP id 23so1495594fxm.3 for ; Fri, 12 Mar 2010 10:38:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=2tSyOi4nNgSTPzhkO/2TebU726++GvNqCLZ0bMQmqJA=; b=FVMh/4AcGRUCrnko2IgXtXrTiOX+VfOxd0l3VkMABv3pE1Kul2VIdfEt1lccYUkGme qicz4Cvc7oesghoqr2dyE6YDRFwllEl2Upmw1xUeNxrpRQQ/Z0duKbDo9MOKDvo7j8ao H++q270RxZts9dSRPsETQmgzze/hilQ0MfcFo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=egaYpz25eFmIJYF3TCBe4rUjmeA85ZQTpqWEYdvXNC1r3Gx8EFFQ6ddnlKu+nuFOjD T4xYEFc62hrIaKE2wmeAMxw01CfpXDT3xfvToyOJO+w4Rtsu0bSoBh5RfwRWb3uDXOux 2nuSpS+DGL6h2LRt3vRTuiKhaHWUhw5S1HzmU= Received: by 10.223.164.75 with SMTP id d11mr1166233fay.68.1268419117543; Fri, 12 Mar 2010 10:38:37 -0800 (PST) Received: from mavbook.mavhome.dp.ua (s224.GtokyoFL6.vectant.ne.jp [222.228.90.224]) by mx.google.com with ESMTPS id 15sm1203260fxm.12.2010.03.12.10.38.34 (version=SSLv3 cipher=RC4-MD5); Fri, 12 Mar 2010 10:38:36 -0800 (PST) Sender: Alexander Motin Message-ID: <4B9A8A27.8050608@FreeBSD.org> Date: Fri, 12 Mar 2010 20:38:31 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Kai Gallasch References: <20100311133916.42ba69b0@orwell.free.de> <20100312115028.GG1819@garage.freebsd.pl> In-Reply-To: <20100312115028.GG1819@garage.freebsd.pl> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=KOI8-R Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, freebsd-stable@FreeBSD.org, Pawel Jakub Dawidek Subject: Re: proliant server lockups with freebsd-amd64-stable (2010-03-10) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 18:38:39 -0000 Pawel Jakub Dawidek wrote: > On Thu, Mar 11, 2010 at 01:39:16PM +0100, Kai Gallasch wrote: >> I have some trouble with an opteron server locking up spontaneously. It looses >> all networks connectivity and even through console I can get no shell. >> >> Lockups occur mostly under disk load (periodic daily, bacula backup >> running, make buildworld/buildkernel) and I can provoke them easily. > [...] >> 4 0 0 0 LL *cissmtx 0xffffff04ed820c00 [g_down] > [...] >> 100046 L *cissmtx 0xffffff04ed820c00 [irq257: ciss0] > [...] > > I was analizing similar problem as potential ZFS bug. It turned out to > be bug in ciss(4) and I believe mav@ (CCed) has fix for that. That my patch is already at 8-STABLE since r204873 of 2010-03-08. Make sure you have it. In this case trap stopped process at ciss_get_request(), which indeed called holding cissmtx lock. But there is no place to sleep or loop there, so may be it was just spontaneous. With bugs I was fixing there was a chance to loop indefinitely between ciss and CAM on resource constraint. That increases chance for such situation to be caught. You may try also look what's going on with `top -HS` and `systat -vm 1`. -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 18:44:35 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DAD30106564A for ; Fri, 12 Mar 2010 18:44:35 +0000 (UTC) (envelope-from korvus@comcast.net) Received: from mx04.pub.collaborativefusion.com (mx04.pub.collaborativefusion.com [206.210.72.84]) by mx1.freebsd.org (Postfix) with ESMTP id A95438FC0A for ; Fri, 12 Mar 2010 18:44:35 +0000 (UTC) Received: from [192.168.2.164] ([206.210.89.202]) by mx04.pub.collaborativefusion.com (StrongMail Enterprise 4.1.1.4(4.1.1.4-47689)); Fri, 12 Mar 2010 14:01:26 -0500 X-VirtualServerGroup: Default X-MailingID: 00000::00000::00000::00000::::523 X-SMHeaderMap: mid="X-MailingID" X-Destination-ID: freebsd-stable@FreeBSD.org X-SMFBL: ZnJlZWJzZC1zdGFibGVARnJlZUJTRC5vcmc= Message-ID: <4B9A8B92.6050704@comcast.net> Date: Fri, 12 Mar 2010 13:44:34 -0500 From: Steve Polyack User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.1.7) Gecko/20100311 Thunderbird/3.0.1 MIME-Version: 1.0 To: freebsd-rc@freebsd.org, freebsd-stable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: rc.d/rc.subr support for multiple FIBs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 18:44:35 -0000 With multiple FIB support generally available in FreeBSD 8.0-RELEASE it would be quite beneficial to have the ability to build routing tables in secondary FIBs as well as start certain applications in certain FIBs from within rc.conf(5). I've done some poking around and came across two PRs which implement exactly what I'm looking for: http://www.freebsd.org/cgi/query-pr.cgi?pr=132483 http://www.freebsd.org/cgi/query-pr.cgi?pr=132476 My question is whether there are any plans to commit these into -CURRENT and possibly MFC them to a future 8.x-RELEASE. Having multiple FIBs available has been great so far, it goes hand and hand with Multi-IP Jails. The only thing missing are native methods for constructing the routing tables on boot (Yes, there is rc.local, but I don't want to go there...). Thanks, Steve Polyack From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 19:11:33 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26253106567F for ; Fri, 12 Mar 2010 19:11:33 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id A7B7E8FC14 for ; Fri, 12 Mar 2010 19:11:32 +0000 (UTC) Received: by fxm23 with SMTP id 23so1526864fxm.3 for ; Fri, 12 Mar 2010 11:11:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=q5K8/dNMlZQgEuSV+5KgroWCLsu9ntzGcUK5JnS8ZRU=; b=rpLPuX/WZHnhxqhyLZtY/CvNmDe1/QDh1vqUOQ2+neUuZbali8vsaLA/AFHCjb6aKS O7A+9qn0N2i5PKoC6HyKk4kXY6+pnRW07JucCPm4JQBBf2M1v+QOMBnQbUs3kAYVijNe /yHICmIWXn61ee/PCataTZ3wGJn1PqxitAIig= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=NB+BDbmqe3ZnhGFk48+WxrSSMoXo3wMDOKadgN2j9gheaxPpzcibuPe2XMrEf+MCNB GLsFM983epNFBb47VhCelWoMXaw0umfqQu9QfXYTrdiThukGc3lngSkgfYDaSrrYTvRw qmKE+GsJafFg0QELMaPuE49qK69dSsuHNnSk0= Received: by 10.223.60.142 with SMTP id p14mr2292154fah.47.1268421091630; Fri, 12 Mar 2010 11:11:31 -0800 (PST) Received: from mavbook.mavhome.dp.ua (s224.GtokyoFL6.vectant.ne.jp [222.228.90.224]) by mx.google.com with ESMTPS id 16sm1227698fxm.11.2010.03.12.11.11.29 (version=SSLv3 cipher=RC4-MD5); Fri, 12 Mar 2010 11:11:31 -0800 (PST) Sender: Alexander Motin Message-ID: <4B9A8F14.4030907@FreeBSD.org> Date: Fri, 12 Mar 2010 20:59:32 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Jeremy Chadwick References: <20100312121409.GA79294@fasterix.frmug.org> <20100312131832.GA54731@icarus.home.lan> <20100312165744.GA86971@fasterix.frmug.org> <20100312170936.GA59343@icarus.home.lan> In-Reply-To: <20100312170936.GA59343@icarus.home.lan> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, Pierre Beyssac Subject: Re: (fixed) Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 19:11:33 -0000 Jeremy Chadwick wrote: > On Fri, Mar 12, 2010 at 05:57:44PM +0100, Pierre Beyssac wrote: >>> Possibly this is the source of the problem (specifically, it looks like >>> FreeBSD doesn't have proper device ID knowledge of what this controller >>> is. I believe that's because this system is *very* new, a Core i3/i5/i7 >>> system)? >> I didn't notice that, you're right! The system is brand new, got >> it delivered on Tuesday. Another odd thing is that the controllers >> have differents IDs, 0x3b208086 vs 0x3b268086. >> >> I added the IDs to ata-intel.c and it fixes the problem. >> >> Preparing a patch. Thanks for the hint! >> -- >> Pierre Beyssac pb@fasterix.frmug.org > >> --- ata-intel.c.orig 2010-03-12 17:02:00.680011011 +0100 >> +++ ata-intel.c 2010-03-12 16:55:54.773702505 +0100 >> @@ -156,6 +156,8 @@ >> { 0x3a2d8086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, >> { 0x3a2e8086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, >> { 0x3a2f8086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, >> + { 0x3b208086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, >> + { 0x3b268086, 0, INTEL_AHCI, 0, ATA_SA300, "PCH" }, >> { ATA_I31244, 0, 0, 2, ATA_SA150, "31244" }, >> { ATA_ISCH, 0, 0, 1, ATA_UDMA5, "SCH" }, >> { 0, 0, 0, 0, 0, 0}}; > > I had a chance to review the Intel 5 Series and 3400 Series Chipset > document, which is specific to the PCH. The breakdown for those > two Device IDs, under Vendor ID 0x8086 (Intel), is: > > Device ID 0x3b20 > - PCH SATA controller > - Desktop revision, non-AHCI and non-RAID Mode > - Ports 0,1,2,3 > > Device ID 0x3b26 > - PCH SATA controller > - Desktop revision, non-AHCI and non-RAID mode > - Ports 4,5 > > So I'm not sure the setting of the INTEL_AHCI flag there is correct > for these controllers. mav@ will need to chime in here. Except ICH6 Intel uses separate IDs for AHCI and non-AHCI SATA controller modes. So this flag is not completely correct. But probably it shoudn't be just removed, as it is checked in other place. I know about this and going to handle it later. It's not a problem now. > Regarding the "dual device IDs": what this means is that ports 0-3 are > tied to one device on the PCI bus, and ports 4-5 are tied to another > device on the PCI bus. They might have chosen to do this so they could > segregate which ports do what or operate differently somehow. > > mav@ may also want to review the specification since it mentions a bunch > of other Device IDs which don't appear to be in the above list: > > http://www.intel.com/Assets/PDF/specupdate/322170.pdf Thanks. PS: Check BIOS settings for enabling AHCI mode. -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 19:21:49 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9350F106564A; Fri, 12 Mar 2010 19:21:49 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id EC92F8FC19; Fri, 12 Mar 2010 19:21:48 +0000 (UTC) Received: by fxm23 with SMTP id 23so1536810fxm.3 for ; Fri, 12 Mar 2010 11:21:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=lIAVvVUVjSvXGT5Ex3FrkSPRNgw4BX3Ua+CRW9NpNHM=; b=E+gO91+w/2ba66wtzx44rPkuZwUc4UqsxTBmE9uZh14kfUTfUZqfee5T7q475lxwX6 sDKSyL8I3LMvtOF3p8NU8RAyYKigJsvd3nBQBFFjwBQwlbDCWouaUEMFPZgpPAlQPbAq 5PxNXYFdGM4BcAaInxVzbukiLJNEJyjo/Wpyo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=Wpo1VyCyv5b1rp1NbF0EG0Lz0DUtrgQ7kD3xzuK0OfQ87FSml/GUtYlqLa3tuxtNgn VCFyykoFIchblMMuKzP2NkkzFF81gDoOBtzwKxLAFjvt4EhlIqT/1KDNWcO99IyUrl5W Vt0ZqxVuRPxa4+8zUy1pyvKdVOClH0qp1Zt3g= Received: by 10.87.47.1 with SMTP id z1mr2235090fgj.74.1268421707927; Fri, 12 Mar 2010 11:21:47 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id e11sm1990831fga.16.2010.03.12.11.21.45 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 12 Mar 2010 11:21:46 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Fri, 12 Mar 2010 11:21:44 -0800 From: Pyun YongHyeon Date: Fri, 12 Mar 2010 11:21:44 -0800 To: Pierre Beyssac Message-ID: <20100312192144.GL16314@michelle.cdnetworks.com> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312174655.GG16314@michelle.cdnetworks.com> <20100312182134.GA90606@fasterix.frmug.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="1UWUbFP1cBYEclgG" Content-Disposition: inline In-Reply-To: <20100312182134.GA90606@fasterix.frmug.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@freebsd.org Subject: Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 19:21:49 -0000 --1UWUbFP1cBYEclgG Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Mar 12, 2010 at 07:21:34PM +0100, Pierre Beyssac wrote: > On Fri, Mar 12, 2010 at 09:46:55AM -0800, Pyun YongHyeon wrote: > > This is not related with your interrupt storm issue but something > > is wrong here. I think brgphy(4) should be used for bge(4). Have no > > idea why the OUI has a different value. > > Would you try attached patch and let me know whether brgphy(4) > > get attached to the PHY? > > It sort of works (see attached dmesg) but not quite correctly, the > ethernet link takes an awful lot of time to negotiate (> 10s) and > it negotiates 10 Mbps instead of 100 Mbps previously. > > The message "Unrecognized OUI for PHY!" seems to indicate something's > amiss, maybe I should regenerate some file(s) after applying your > patches? No, it seems there is other issue in brgphy(4). I noticed brgphy(4) blindly try to set jumbo frame related registers. I guess the PHY may not have the register. Back out previous patch and try this one. --1UWUbFP1cBYEclgG Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="BCM57780.diff2" Index: sys/dev/mii/miidevs =================================================================== --- sys/dev/mii/miidevs (revision 205052) +++ sys/dev/mii/miidevs (working copy) @@ -81,6 +81,7 @@ oui xxALTIMA 0x000895 Altima Communications oui xxBROADCOM 0x000818 Broadcom Corporation oui xxBROADCOM_ALT1 0x0050ef Broadcom Corporation +oui xxBROADCOM_ALT2 0x00d897 Broadcom Corporation oui xxICS 0x00057d Integrated Circuit Systems oui xxSEEQ 0x0005be Seeq oui xxSIS 0x000760 Silicon Integrated Systems @@ -150,6 +151,7 @@ model xxBROADCOM_ALT1 BCM5784 0x003a BCM5784 10/100/1000baseTX PHY model xxBROADCOM_ALT1 BCM5709C 0x003c BCM5709C 10/100/1000baseTX PHY model xxBROADCOM_ALT1 BCM5761 0x003d BCM5761 10/100/1000baseTX PHY +model xxBROADCOM_ALT2 BCM57780 0x0019 BCM57780 10/100/1000baseTX PHY model BROADCOM2 BCM5906 0x0004 BCM5906 10/100baseTX PHY /* Cicada Semiconductor PHYs (now owned by Vitesse?) */ Index: sys/dev/mii/brgphy.c =================================================================== --- sys/dev/mii/brgphy.c (revision 205052) +++ sys/dev/mii/brgphy.c (working copy) @@ -139,6 +139,7 @@ MII_PHY_DESC(xxBROADCOM_ALT1, BCM5784), MII_PHY_DESC(xxBROADCOM_ALT1, BCM5709C), MII_PHY_DESC(xxBROADCOM_ALT1, BCM5761), + MII_PHY_DESC(xxBROADCOM_ALT2, BCM57780), MII_PHY_DESC(BROADCOM2, BCM5906), MII_PHY_END }; @@ -213,6 +214,7 @@ switch (bsc->mii_oui) { case MII_OUI_BROADCOM: case MII_OUI_BROADCOM2: + case MII_OUI_xxBROADCOM_ALT2: break; case MII_OUI_xxBROADCOM: switch (bsc->mii_model) { @@ -1021,7 +1023,8 @@ if (bge_sc->bge_flags & BGE_FLAG_JITTER_BUG) brgphy_fixup_jitter_bug(sc); - brgphy_jumbo_settings(sc, ifp->if_mtu); + if (bge_sc->bge_flags & BGE_FLAG_JUMBO) + brgphy_jumbo_settings(sc, ifp->if_mtu); if (bge_sc->bge_flags & BGE_FLAG_WIRESPEED) brgphy_ethernet_wirespeed(sc); --1UWUbFP1cBYEclgG-- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 19:28:01 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6FFCC106566B; Fri, 12 Mar 2010 19:28:01 +0000 (UTC) (envelope-from pb@fasterix.frmug.org) Received: from fasterix.frmug.org (unknown [IPv6:2a01:e35:8b50:2c40::2]) by mx1.freebsd.org (Postfix) with ESMTP id E9F218FC0A; Fri, 12 Mar 2010 19:28:00 +0000 (UTC) Received: from fasterix.frmug.org (localhost [127.0.0.1]) by fasterix.frmug.org (8.14.3/8.14.3) with ESMTP id o2CJRxiE094003 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 12 Mar 2010 20:27:59 +0100 (CET) (envelope-from pb@fasterix.frmug.org) Received: (from pb@localhost) by fasterix.frmug.org (8.14.3/8.14.3/Submit) id o2CJRxDa094002; Fri, 12 Mar 2010 20:27:59 +0100 (CET) (envelope-from pb) Date: Fri, 12 Mar 2010 20:27:59 +0100 From: Pierre Beyssac To: Alexander Motin Message-ID: <20100312192759.GA91317@fasterix.frmug.org> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312131832.GA54731@icarus.home.lan> <20100312165744.GA86971@fasterix.frmug.org> <20100312170936.GA59343@icarus.home.lan> <4B9A8F14.4030907@FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4B9A8F14.4030907@FreeBSD.org> User-Agent: Mutt/1.4.2.3i X-message-flag: Warning! Use of Microsoft Outlook is dangerous and makes your system susceptible to worms and viruses Cc: freebsd-stable@freebsd.org, Jeremy Chadwick Subject: Re: (fixed) Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 19:28:01 -0000 On Fri, Mar 12, 2010 at 08:59:32PM +0200, Alexander Motin wrote: > Except ICH6 Intel uses separate IDs for AHCI and non-AHCI SATA > controller modes. So this flag is not completely correct. But probably > it shoudn't be just removed, as it is checked in other place. I know > about this and going to handle it later. It's not a problem now. I was wondering exactly about that given the wording of the Intel document sent by Jeremy... > > http://www.intel.com/Assets/PDF/specupdate/322170.pdf > Thanks. > PS: Check BIOS settings for enabling AHCI mode. Problem is, I can't find any setting to do that, either it's not there or it's well hidden. The BIOS is up-to-date. Could it be possible for Dell to lock the chip in non-AHCI mode? -- Pierre Beyssac pb@fasterix.frmug.org From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 19:45:28 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AF72F106566C for ; Fri, 12 Mar 2010 19:45:28 +0000 (UTC) (envelope-from pb@fasterix.frmug.org) Received: from fasterix.frmug.org (unknown [IPv6:2a01:e35:8b50:2c40::2]) by mx1.freebsd.org (Postfix) with ESMTP id 687DD8FC1C for ; Fri, 12 Mar 2010 19:45:26 +0000 (UTC) Received: from fasterix.frmug.org (localhost [127.0.0.1]) by fasterix.frmug.org (8.14.3/8.14.3) with ESMTP id o2CJjPb7095116 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 12 Mar 2010 20:45:25 +0100 (CET) (envelope-from pb@fasterix.frmug.org) Received: (from pb@localhost) by fasterix.frmug.org (8.14.3/8.14.3/Submit) id o2CJjOCZ095115; Fri, 12 Mar 2010 20:45:25 +0100 (CET) (envelope-from pb) Date: Fri, 12 Mar 2010 20:45:24 +0100 From: Pierre Beyssac To: Pyun YongHyeon Message-ID: <20100312194524.GB91694@fasterix.frmug.org> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312174655.GG16314@michelle.cdnetworks.com> <20100312182134.GA90606@fasterix.frmug.org> <20100312192144.GL16314@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="pf9I7BMVVzbSWLtt" Content-Disposition: inline In-Reply-To: <20100312192144.GL16314@michelle.cdnetworks.com> User-Agent: Mutt/1.4.2.3i X-message-flag: Warning! Use of Microsoft Outlook is dangerous and makes your system susceptible to worms and viruses Cc: freebsd-stable@freebsd.org Subject: Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 19:45:28 -0000 --pf9I7BMVVzbSWLtt Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Mar 12, 2010 at 11:21:44AM -0800, Pyun YongHyeon wrote: > No, it seems there is other issue in brgphy(4). I noticed brgphy(4) > blindly try to set jumbo frame related registers. I guess the PHY > may not have the register. Back out previous patch and try this > one. Thanks, works better (the error message is gone), but still negotiates at 10baseT/UTP fdx... See attached dmesg.out. -- Pierre Beyssac pb@fasterix.frmug.org --pf9I7BMVVzbSWLtt Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="dmesg.out" map[20]: type I/O Port, range 32, base 0xc400, size 4, enabled map[24]: type I/O Port, range 32, base 0xc080, size 4, enabled pcib0: matched entry for 0.31.INTB pcib0: slot 31 INTB hardwired to IRQ 19 pcib1: irq 16 at device 3.0 on pci0 pcib1: domain 0 pcib1: secondary bus 1 pcib1: subordinate bus 1 pcib1: I/O decode 0xe000-0xefff pcib1: memory decode 0xfbe00000-0xfbefffff pcib1: prefetched decode 0xd0000000-0xdfffffff pcib1: could not get PCI interrupt routing table for \\_SB_.PCI0.P0P3 - AE_NOT_FOUND pci1: on pcib1 pci1: domain=0, physical bus=1 found-> vendor=0x1002, dev=0x68f9, revid=0x00 domain=0, bus=1, slot=0, func=0 class=03-00-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=10 powerspec 3 supports D0 D1 D2 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Prefetchable Memory, range 64, base 0xd0000000, size 28, enabled pcib1: requested memory range 0xd0000000-0xdfffffff: good map[18]: type Memory, range 64, base 0xfbee0000, size 17, enabled pcib1: requested memory range 0xfbee0000-0xfbefffff: good map[20]: type I/O Port, range 32, base 0xe000, size 8, enabled pcib1: requested I/O range 0xe000-0xe0ff: in range pcib0: matched entry for 0.3.INTA pcib0: slot 3 INTA hardwired to IRQ 16 pcib1: slot 0 INTA is routed to irq 16 found-> vendor=0x1002, dev=0xaa68, revid=0x00 domain=0, bus=1, slot=0, func=1 class=04-03-00, hdrtype=0x00, mfdev=1 cmdreg=0x0007, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=b, irq=5 powerspec 3 supports D0 D1 D2 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xfbebc000, size 14, enabled pcib1: requested memory range 0xfbebc000-0xfbebffff: good pcib0: matched entry for 0.3.INTB pcib0: slot 3 INTB hardwired to IRQ 17 pcib1: slot 0 INTB is routed to irq 17 vgapci0: port 0xe000-0xe0ff mem 0xd0000000-0xdfffffff,0xfbee0000-0xfbefffff irq 16 at device 0.0 on pci1 hdac0: mem 0xfbebc000-0xfbebffff irq 17 at device 0.1 on pci1 hdac0: HDA Driver Revision: 20100226_0142 hdac0: Reserved 0x4000 bytes for rid 0x10 type 3 at 0xfbebc000 hdac0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 256 to local APIC 0 vector 49 hdac0: using IRQ 256 for MSI hdac0: [MPSAFE] hdac0: [ITHREAD] hdac0: Caps: OSS 1, ISS 0, BSS 0, NSDO 1, 64bit, CORB 256, RIRB 256 pci0: at device 8.0 (no driver attached) pci0: at device 8.1 (no driver attached) pci0: at device 8.2 (no driver attached) pci0: at device 8.3 (no driver attached) pci0: at device 16.0 (no driver attached) pci0: at device 16.1 (no driver attached) pci0: at device 22.0 (no driver attached) ehci0: mem 0xfbdfc000-0xfbdfc3ff irq 16 at device 26.0 on pci0 ioapic0: routing intpin 16 (PCI IRQ 16) to lapic 0 vector 50 ehci0: [MPSAFE] ehci0: [ITHREAD] usbus0: EHCI version 1.0 usbus0: on ehci0 hdac1: mem 0xfbdf8000-0xfbdfbfff irq 22 at device 27.0 on pci0 hdac1: HDA Driver Revision: 20100226_0142 hdac1: Reserved 0x4000 bytes for rid 0x10 type 3 at 0xfbdf8000 hdac1: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 257 to local APIC 0 vector 51 hdac1: using IRQ 257 for MSI hdac1: [MPSAFE] hdac1: [ITHREAD] hdac1: Caps: OSS 4, ISS 4, BSS 0, NSDO 1, 64bit, CORB 256, RIRB 256 pcib2: irq 17 at device 28.0 on pci0 pcib2: domain 0 pcib2: secondary bus 2 pcib2: subordinate bus 2 pcib2: I/O decode 0x0-0x0 pcib2: no prefetched decode pci2: on pcib2 pci2: domain=0, physical bus=2 pcib3: irq 16 at device 28.5 on pci0 pcib3: domain 0 pcib3: secondary bus 3 pcib3: subordinate bus 3 pcib3: I/O decode 0x0-0x0 pcib3: memory decode 0xfbf00000-0xfbffffff pcib3: no prefetched decode pci3: on pcib3 pci3: domain=0, physical bus=3 found-> vendor=0x14e4, dev=0x1691, revid=0x01 domain=0, bus=3, slot=0, func=0 class=02-00-00, hdrtype=0x00, mfdev=0 cmdreg=0x0006, statreg=0x0010, cachelnsz=8 (dwords) lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns) intpin=a, irq=5 powerspec 3 supports D0 D3 current D0 MSI supports 1 message, 64 bit map[10]: type Memory, range 64, base 0xfbff0000, size 16, enabled pcib3: requested memory range 0xfbff0000-0xfbffffff: good pcib3: matched entry for 3.0.INTA pcib3: slot 0 INTA hardwired to IRQ 17 bge0: mem 0xfbff0000-0xfbffffff irq 17 at device 0.0 on pci3 bge0: Reserved 0x10000 bytes for rid 0x10 type 3 at 0xfbff0000 bge0: adjust device control 0x2000 -> 0x5000 bge0: attempting to allocate 1 MSI vectors (1 supported) msi: routing MSI IRQ 258 to local APIC 0 vector 52 bge0: using IRQ 258 for MSI bge0: CHIP ID 0x57780001; ASIC REV 0x57780; CHIP REV 0x577800; PCI-E bge0: Disabling fastboot bge0: Disabling fastboot miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: OUI 0x00d897, model 0x0019, rev. 1 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto bge0: bpf attached bge0: Ethernet address: 00:25:64:f4:27:26 bge0: [MPSAFE] bge0: [FILTER] ehci1: mem 0xfbdf6000-0xfbdf63ff irq 23 at device 29.0 on pci0 ioapic0: routing intpin 23 (PCI IRQ 23) to lapic 0 vector 53 ehci1: [MPSAFE] ehci1: [ITHREAD] usbus1: EHCI version 1.0 usbus1: on ehci1 pcib4: at device 30.0 on pci0 pcib4: domain 0 pcib4: secondary bus 4 pcib4: subordinate bus 4 pcib4: I/O decode 0x0-0x0 pcib4: no prefetched decode pcib4: Subtractively decoded bridge. pci4: on pcib4 pci4: domain=0, physical bus=4 isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0xdc00-0xdc07,0xd880-0xd883,0xd800-0xd807,0xd480-0xd483,0xd400-0xd40f,0xd080-0xd08f irq 19 at device 31.2 on pci0 atapci0: Reserved 0x10 bytes for rid 0x20 type 4 at 0xd400 ioapic0: routing intpin 19 (PCI IRQ 19) to lapic 0 vector 54 atapci0: [MPSAFE] atapci0: [ITHREAD] atapci0: Reserved 0x10 bytes for rid 0x24 type 4 at 0xd080 ata2: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x10 type 4 at 0xdc00 atapci0: Reserved 0x4 bytes for rid 0x14 type 4 at 0xd880 ata2: reset tp1 mask=01 ostat0=50 ostat1=00 ata2: stat0=0x50 err=0x01 lsb=0x00 msb=0x00 ata2: reset tp2 stat0=50 stat1=00 devices=0x1 ata2: [MPSAFE] ata2: [ITHREAD] ata3: on atapci0 atapci0: Reserved 0x8 bytes for rid 0x18 type 4 at 0xd800 atapci0: Reserved 0x4 bytes for rid 0x1c type 4 at 0xd480 ata3: reset tp1 mask=01 ostat0=00 ostat1=00 ata3: stat0=0x00 err=0x01 lsb=0x14 msb=0xeb ata3: reset tp2 stat0=00 stat1=00 devices=0x10000 ata3: [MPSAFE] ata3: [ITHREAD] ichsmb0: port 0x400-0x41f mem 0xfbdf4000-0xfbdf40ff irq 18 at device 31.3 on pci0 ichsmb0: Reserved 0x20 bytes for rid 0x20 type 4 at 0x400 ioapic0: routing intpin 18 (PCI IRQ 18) to lapic 0 vector 55 ichsmb0: [MPSAFE] ichsmb0: [ITHREAD] smbus0: on ichsmb0 atapci1: port 0xcc00-0xcc07,0xc880-0xc883,0xc800-0xc807,0xc480-0xc483,0xc400-0xc40f,0xc080-0xc08f irq 19 at device 31.5 on pci0 atapci1: Reserved 0x10 bytes for rid 0x20 type 4 at 0xc400 atapci1: [MPSAFE] atapci1: [ITHREAD] atapci1: Reserved 0x10 bytes for rid 0x24 type 4 at 0xc080 ata4: on atapci1 atapci1: Reserved 0x8 bytes for rid 0x10 type 4 at 0xcc00 atapci1: Reserved 0x4 bytes for rid 0x14 type 4 at 0xc880 ata4: reset tp1 mask=01 ostat0=7f ostat1=00 ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: stat0=0x7f err=0xff lsb=0xff msb=0xff ata4: reset tp2 stat0=ff stat1=00 devices=0x0 ata4: [MPSAFE] ata4: [ITHREAD] ata5: on atapci1 atapci1: Reserved 0x8 bytes for rid 0x18 type 4 at 0xc800 atapci1: Reserved 0x4 bytes for rid 0x1c type 4 at 0xc480 ata5: reset tp1 mask=01 ostat0=7f ostat1=00 ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: stat0=0x7f err=0xff lsb=0xff msb=0xff ata5: reset tp2 stat0=ff stat1=00 devices=0x0 ata5: [MPSAFE] ata5: [ITHREAD] acpi_button0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock (resolution 1000000us) isa_probe_children: disabling PnP devices atrtc: atrtc0 already exists; skipping it sc: sc0 already exists; skipping it isa_probe_children: probing non-PnP devices orm0: at iomem 0xc0000-0xce7ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> sc0: fb0, kbd1, terminal emulator: scteken (teken terminal) vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 kbd0: atkbd0, generic (0), config:0x0, flags:0x3f0000 ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 56 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: current command byte:0065 psm0: failed to reset the aux device. fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0 ppc0 failed to probe at irq 7 on isa0 uart0: failed to probe at port 0x3f8-0x3ff irq 4 on isa0 uart1: failed to probe at port 0x2f8-0x2ff irq 3 on isa0 isa_probe_children: probing PnP devices est0: on cpu0 est0: Invalid id16 (set, cur) = (19, 20) est0: Can't check freq 2533, it may be invalid est0: Invalid id16 (set, cur) = (18, 20) est0: Can't check freq 2400, it may be invalid est0: Invalid id16 (set, cur) = (17, 20) est0: Can't check freq 2267, it may be invalid est0: Invalid id16 (set, cur) = (16, 20) est0: Can't check freq 2133, it may be invalid est0: Invalid id16 (set, cur) = (15, 20) est0: Can't check freq 2000, it may be invalid est0: Invalid id16 (set, cur) = (14, 20) est0: Can't check freq 1867, it may be invalid est0: Invalid id16 (set, cur) = (13, 20) est0: Can't check freq 1733, it may be invalid est0: Invalid id16 (set, cur) = (12, 20) est0: Can't check freq 1600, it may be invalid est0: Invalid id16 (set, cur) = (11, 20) est0: Can't check freq 1467, it may be invalid est0: Invalid id16 (set, cur) = (10, 20) est0: Can't check freq 1333, it may be invalid est0: Invalid id16 (set, cur) = (9, 20) est0: Can't check freq 1200, it may be invalid p4tcc0: on cpu0 est1: on cpu1 est1: Invalid id16 (set, cur) = (19, 20) est1: Can't check freq 2533, it may be invalid est1: Invalid id16 (set, cur) = (18, 20) est1: Can't check freq 2400, it may be invalid est1: Invalid id16 (set, cur) = (17, 20) est1: Can't check freq 2267, it may be invalid est1: Invalid id16 (set, cur) = (16, 20) est1: Can't check freq 2133, it may be invalid est1: Invalid id16 (set, cur) = (15, 20) est1: Can't check freq 2000, it may be invalid est1: Invalid id16 (set, cur) = (14, 20) est1: Can't check freq 1867, it may be invalid est1: Invalid id16 (set, cur) = (13, 20) est1: Can't check freq 1733, it may be invalid est1: Invalid id16 (set, cur) = (12, 20) est1: Can't check freq 1600, it may be invalid est1: Invalid id16 (set, cur) = (11, 20) est1: Can't check freq 1467, it may be invalid est1: Invalid id16 (set, cur) = (10, 20) est1: Can't check freq 1333, it may be invalid est1: Invalid id16 (set, cur) = (9, 20) est1: Can't check freq 1200, it may be invalid p4tcc1: on cpu1 est2: on cpu2 est2: Invalid id16 (set, cur) = (19, 20) est2: Can't check freq 2533, it may be invalid est2: Invalid id16 (set, cur) = (18, 20) est2: Can't check freq 2400, it may be invalid est2: Invalid id16 (set, cur) = (17, 20) est2: Can't check freq 2267, it may be invalid est2: Invalid id16 (set, cur) = (16, 20) est2: Can't check freq 2133, it may be invalid est2: Invalid id16 (set, cur) = (15, 20) est2: Can't check freq 2000, it may be invalid est2: Invalid id16 (set, cur) = (14, 20) est2: Can't check freq 1867, it may be invalid est2: Invalid id16 (set, cur) = (13, 20) est2: Can't check freq 1733, it may be invalid est2: Invalid id16 (set, cur) = (12, 20) est2: Can't check freq 1600, it may be invalid est2: Invalid id16 (set, cur) = (11, 20) est2: Can't check freq 1467, it may be invalid est2: Invalid id16 (set, cur) = (10, 20) est2: Can't check freq 1333, it may be invalid est2: Invalid id16 (set, cur) = (9, 20) est2: Can't check freq 1200, it may be invalid p4tcc2: on cpu2 est3: on cpu3 est3: Invalid id16 (set, cur) = (19, 20) est3: Can't check freq 2533, it may be invalid est3: Invalid id16 (set, cur) = (18, 20) est3: Can't check freq 2400, it may be invalid est3: Invalid id16 (set, cur) = (17, 20) est3: Can't check freq 2267, it may be invalid est3: Invalid id16 (set, cur) = (16, 20) est3: Can't check freq 2133, it may be invalid est3: Invalid id16 (set, cur) = (15, 20) est3: Can't check freq 2000, it may be invalid est3: Invalid id16 (set, cur) = (14, 20) est3: Can't check freq 1867, it may be invalid est3: Invalid id16 (set, cur) = (13, 20) est3: Can't check freq 1733, it may be invalid est3: Invalid id16 (set, cur) = (12, 20) est3: Can't check freq 1600, it may be invalid est3: Invalid id16 (set, cur) = (11, 20) est3: Can't check freq 1467, it may be invalid est3: Invalid id16 (set, cur) = (10, 20) est3: Can't check freq 1333, it may be invalid est3: Invalid id16 (set, cur) = (9, 20) est3: Can't check freq 1200, it may be invalid p4tcc3: on cpu3 Device configuration finished. Reducing kern.maxvnodes 509848 -> 100000 procfs registered lapic: Divisor 2, Frequency 66500156 Hz Timecounter "TSC" frequency 2660006168 Hz quality -100 Timecounters tick every 1.000 msec lo0: bpf attached ata2: Identifying devices: 00000001 ata2: New devices: 00000001 usbus0: 480Mbps High Speed USB v2.0 usbus1: 480Mbps High Speed USB v2.0 ata2-master: pio=PIO4 wdma=WDMA2 udma=UDMA133 cable=40 wire ad4: setting UDMA100 ad4: 953869MB at ata2-master UDMA100 SATA 3Gb/s ad4: 1953525168 sectors [1938021C/16H/63S] 16 sectors/interrupt 1 depth queue ata3: Identifying devices: 00010000 ata3: New devices: 00010000 GEOM: new disk ad4 ata3-master: pio=PIO4 wdma=WDMA2 udma=UDMA100 cable=40 wire acd0: setting UDMA100 ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 acd0: DVDR drive at ata3 as master acd0: read 8269KB/s (8269KB/s) write 8269KB/s (8269KB/s), 2048KB buffer, UDMA100 SATA 1.5Gb/s acd0: Reads: CDR, CDRW, CDDA stream, DVDROM, DVDR, DVDRAM, packet acd0: Writes: CDR, CDRW, DVDR, DVDRAM, test write, burnproof acd0: Audio: play, 256 volume levels acd0: Mechanism: ejectable tray, unlocked acd0: Medium: no/blank disc ata4: Identifying devices: 00000000 ata4: New devices: 00000000 ata5: Identifying devices: 00000000 ata5: New devices: 00000000 hdac0: Probing codec #0... hdac0: HDA Codec #0: ATI R6xx HDMI hdac0: HDA Codec ID: 0x1002aa01 hdac0: Vendor: 0x1002 hdac0: Device: 0xaa01 hdac0: Revision: 0x02 hdac0: Stepping: 0x00 hdac0: PCI Subvendor: 0xaa681028 hdac0: Found audio FG nid=1 startnode=2 endnode=4 total=2 hdac0: hdac0: Processing audio FG cad=0 nid=1... hdac0: GPIO: 0x00000000 NumGPIO=0 NumGPO=0 NumGPI=0 GPIWake=0 GPIUnsol=0 hdac0: nid 3 0x18560010 as 1 seq 0 Digital-out Jack jack 6 loc 24 color Unknown misc 0 hdac0: Patched pins configuration: hdac0: nid 3 0x18560010 as 1 seq 0 Digital-out Jack jack 6 loc 24 color Unknown misc 0 hdac0: 1 associations found: hdac0: Association 0 (1) out: hdac0: Pin nid=3 seq=0 hdac0: Tracing association 0 (1) hdac0: Pin 3 traced to DAC 2 hdac0: Association 0 (1) trace succeeded hdac0: Tracing input monitor hdac0: Tracing other input monitors hdac0: Tracing beeper hdac0: FG config/quirks: forcestereo ivref50 ivref80 ivref100 ivref hdac0: hdac0: +-------------------+ hdac0: | DUMPING HDA NODES | hdac0: +-------------------+ hdac0: hdac0: Default Parameter hdac0: ----------------- hdac0: Stream cap: 0x00000001 hdac0: PCM hdac0: PCM cap: 0x00020070 hdac0: 16 bits, 32 44 48 KHz hdac0: IN amp: 0x00000000 hdac0: OUT amp: 0x00000000 hdac0: hdac0: nid: 2 hdac0: Name: audio output hdac0: Widget cap: 0x00000201 hdac0: DIGITAL STEREO hdac0: Association: 0 (0x00000001) hdac0: OSS: pcm (pcm) hdac0: Stream cap: 0x00000001 hdac0: PCM hdac0: PCM cap: 0x00020070 hdac0: 16 bits, 32 44 48 KHz hdac0: hdac0: nid: 3 hdac0: Name: pin: Digital-out (Jack) hdac0: Widget cap: 0x00400381 hdac0: DIGITAL UNSOL STEREO hdac0: Association: 0 (0x00000001) hdac0: Pin cap: 0x00000094 hdac0: PDC OUT HDMI hdac0: Pin config: 0x18560010 hdac0: Pin control: 0x00000040 OUT hdac0: connections: 1 hdac0: | hdac0: + <- nid=2 [audio output] hdac0: pcm0: at cad 0 nid 1 on hdac0 pcm0: +--------------------------------------+ pcm0: | DUMPING PCM Playback/Record Channels | pcm0: +--------------------------------------+ pcm0: pcm0: Playback: pcm0: pcm0: Stream cap: 0x00000005 pcm0: AC3 PCM pcm0: PCM cap: 0x00020070 pcm0: 16 bits, 32 44 48 KHz pcm0: DAC: 2 pcm0: pcm0: +-------------------------------+ pcm0: | DUMPING Playback/Record Paths | pcm0: +-------------------------------+ pcm0: pcm0: Playback: pcm0: pcm0: nid=3 [pin: Digital-out (Jack)] pcm0: | pcm0: + <- nid=2 [audio output] [src: pcm] pcm0: pcm0: +-------------------------+ pcm0: | DUMPING Volume Controls | pcm0: +-------------------------+ pcm0: pcm0: Forcing Soft PCM volume pcm0: Forcing master volume with PCM pcm0: Mixer "vol" -> "none": child=0x00000010 pcm0: Mixer "pcm": parent="vol" pcm0: Soft PCM mixer ENABLED pcm0: clone manager: deadline=750ms flags=0x8000001e pcm0: sndbuf_setmap 1bd0000, 4000; 0xffffff80e669a000 -> 1bd0000 hdac1: Probing codec #0... hdac1: HDA Codec #0: Realtek ALC887 hdac1: HDA Codec ID: 0x10ec0887 hdac1: Vendor: 0x10ec hdac1: Device: 0x0887 hdac1: Revision: 0x02 hdac1: Stepping: 0x02 hdac1: PCI Subvendor: 0x04381028 hdac1: Found audio FG nid=1 startnode=2 endnode=39 total=37 hdac1: Probing codec #3... hdac1: HDA Codec #3: Intel (Unknown) hdac1: HDA Codec ID: 0x80862804 hdac1: Vendor: 0x8086 hdac1: Device: 0x2804 hdac1: Revision: 0x00 hdac1: Stepping: 0x00 hdac1: PCI Subvendor: 0x04381028 hdac1: Found audio FG nid=1 startnode=2 endnode=8 total=6 hdac1: hdac1: Processing audio FG cad=0 nid=1... hdac1: GPIO: 0xc0000002 NumGPIO=2 NumGPO=0 NumGPI=0 GPIWake=1 GPIUnsol=1 hdac1: nid 17 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1 hdac1: nid 18 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1 hdac1: nid 20 0x01014010 as 1 seq 0 Line-out Jack jack 1 loc 1 color Green misc 0 hdac1: nid 21 0x01011012 as 1 seq 2 Line-out Jack jack 1 loc 1 color Black misc 0 hdac1: nid 22 0x01016011 as 1 seq 1 Line-out Jack jack 1 loc 1 color Orange misc 0 hdac1: nid 23 0x01012014 as 1 seq 4 Line-out Jack jack 1 loc 1 color Grey misc 0 hdac1: nid 24 0x01a19830 as 3 seq 0 Mic Jack jack 1 loc 1 color Pink misc 8 hdac1: nid 25 0x02a19840 as 4 seq 0 Mic Jack jack 1 loc 2 color Pink misc 8 hdac1: nid 26 0x0181303f as 3 seq 15 Line-in Jack jack 1 loc 1 color Blue misc 0 hdac1: nid 27 0x02214020 as 2 seq 0 Headphones Jack jack 1 loc 2 color Green misc 0 hdac1: nid 28 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1 hdac1: nid 29 0x4015e601 as 0 seq 1 Speaker None jack 5 loc 0 color White misc 6 hdac1: nid 30 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1 hdac1: nid 31 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1 hdac1: Patched pins configuration: hdac1: nid 17 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1 [DISABLED] hdac1: nid 18 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1 [DISABLED] hdac1: nid 20 0x01014010 as 1 seq 0 Line-out Jack jack 1 loc 1 color Green misc 0 hdac1: nid 21 0x01011012 as 1 seq 2 Line-out Jack jack 1 loc 1 color Black misc 0 hdac1: nid 22 0x01016011 as 1 seq 1 Line-out Jack jack 1 loc 1 color Orange misc 0 hdac1: nid 23 0x01012014 as 1 seq 4 Line-out Jack jack 1 loc 1 color Grey misc 0 hdac1: nid 24 0x01a19830 as 3 seq 0 Mic Jack jack 1 loc 1 color Pink misc 8 hdac1: nid 25 0x02a19840 as 4 seq 0 Mic Jack jack 1 loc 2 color Pink misc 8 hdac1: nid 26 0x0181303f as 3 seq 15 Line-in Jack jack 1 loc 1 color Blue misc 0 hdac1: nid 27 0x02214020 as 2 seq 0 Headphones Jack jack 1 loc 2 color Green misc 0 hdac1: nid 28 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1 [DISABLED] hdac1: nid 29 0x4015e601 as 0 seq 1 Speaker None jack 5 loc 0 color White misc 6 [DISABLED] hdac1: nid 30 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1 [DISABLED] hdac1: nid 31 0x411111f0 as 15 seq 0 Speaker None jack 1 loc 1 color Black misc 1 [DISABLED] hdac1: 4 associations found: hdac1: Association 0 (1) out: hdac1: Pin nid=20 seq=0 hdac1: Pin nid=22 seq=1 hdac1: Pin nid=21 seq=2 hdac1: Pin nid=23 seq=4 hdac1: Association 1 (2) out: hdac1: Pin nid=27 seq=0 hdac1: Association 2 (3) in: hdac1: Pin nid=24 seq=0 hdac1: Pin nid=26 seq=15 hdac1: Association 3 (4) in: hdac1: Pin nid=25 seq=0 hdac1: Tracing association 0 (1) hdac1: Pin 20 traced to DAC 2 hdac1: Pin 22 traced to DAC 3 hdac1: Pin 21 traced to DAC 4 hdac1: Pin 23 traced to DAC 5 hdac1: Association 0 (1) trace succeeded hdac1: Tracing association 1 (2) hdac1: Pin 27 traced to DAC 37 hdac1: Association 1 (2) trace succeeded hdac1: Tracing association 2 (3) hdac1: Pin 24 traced to ADC 8 hdac1: Pin 26 traced to ADC 8 hdac1: Association 2 (3) trace succeeded hdac1: Tracing association 3 (4) hdac1: Pin 25 traced to ADC 9 hdac1: Association 3 (4) trace succeeded hdac1: Tracing input monitor hdac1: Tracing nid 11 to out hdac1: nid 11 is input monitor hdac1: Tracing nid 34 to out hdac1: Tracing nid 35 to out hdac1: Tracing other input monitors hdac1: Tracing nid 24 to out hdac1: Tracing nid 25 to out hdac1: Tracing nid 26 to out hdac1: Tracing beeper hdac1: FG config/quirks: forcestereo ivref50 ivref80 ivref100 ivref hdac1: hdac1: +-------------------+ hdac1: | DUMPING HDA NODES | hdac1: +-------------------+ hdac1: hdac1: Default Parameter hdac1: ----------------- hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e0560 hdac1: 16 20 24 bits, 44 48 96 192 KHz hdac1: IN amp: 0x00000000 hdac1: OUT amp: 0x00000000 hdac1: hdac1: nid: 2 hdac1: Name: audio output hdac1: Widget cap: 0x00000411 hdac1: PWR STEREO hdac1: Association: 0 (0x00000001) hdac1: OSS: pcm (pcm) hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e0560 hdac1: 16 20 24 bits, 44 48 96 192 KHz hdac1: hdac1: nid: 3 hdac1: Name: audio output hdac1: Widget cap: 0x00000411 hdac1: PWR STEREO hdac1: Association: 0 (0x00000002) hdac1: OSS: pcm (pcm) hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e0560 hdac1: 16 20 24 bits, 44 48 96 192 KHz hdac1: hdac1: nid: 4 hdac1: Name: audio output hdac1: Widget cap: 0x00000411 hdac1: PWR STEREO hdac1: Association: 0 (0x00000004) hdac1: OSS: pcm (pcm) hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e0560 hdac1: 16 20 24 bits, 44 48 96 192 KHz hdac1: hdac1: nid: 5 hdac1: Name: audio output hdac1: Widget cap: 0x00000411 hdac1: PWR STEREO hdac1: Association: 0 (0x00000010) hdac1: OSS: pcm (pcm) hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e0560 hdac1: 16 20 24 bits, 44 48 96 192 KHz hdac1: hdac1: nid: 6 [DISABLED] hdac1: Name: audio output hdac1: Widget cap: 0x00000611 hdac1: PWR DIGITAL STEREO hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e05e0 hdac1: 16 20 24 bits, 44 48 88 96 192 KHz hdac1: hdac1: nid: 7 [DISABLED] hdac1: Name: vendor widget hdac1: Widget cap: 0x00f00000 hdac1: hdac1: nid: 8 hdac1: Name: audio input hdac1: Widget cap: 0x0010051b hdac1: PWR STEREO hdac1: Association: 2 (0x00008001) hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e0560 hdac1: 16 20 24 bits, 44 48 96 192 KHz hdac1: Input amp: 0x80051f0b hdac1: mute=1 step=31 size=5 offset=11 hdac1: connections: 1 hdac1: | hdac1: + <- nid=35 [audio mixer] hdac1: hdac1: nid: 9 hdac1: Name: audio input hdac1: Widget cap: 0x0010051b hdac1: PWR STEREO hdac1: Association: 3 (0x00000001) hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e0560 hdac1: 16 20 24 bits, 44 48 96 192 KHz hdac1: Input amp: 0x80051f0b hdac1: mute=1 step=31 size=5 offset=11 hdac1: connections: 1 hdac1: | hdac1: + <- nid=34 [audio mixer] hdac1: hdac1: nid: 10 [DISABLED] hdac1: Name: audio input hdac1: Widget cap: 0x00100711 hdac1: PWR DIGITAL STEREO hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e0560 hdac1: 16 20 24 bits, 44 48 96 192 KHz hdac1: connections: 1 hdac1: | hdac1: + [DISABLED] <- nid=31 [pin: Speaker (None)] [DISABLED] hdac1: hdac1: nid: 11 hdac1: Name: audio mixer hdac1: Widget cap: 0x0020010b hdac1: STEREO hdac1: Association: 2 (0x00008001) hdac1: OSS: mix (mix) hdac1: Input amp: 0x80051f17 hdac1: mute=1 step=31 size=5 offset=23 hdac1: connections: 10 hdac1: | hdac1: + <- nid=24 [pin: Mic (Pink Jack)] hdac1: + [DISABLED] <- nid=25 [pin: Mic (Pink Jack)] hdac1: + <- nid=26 [pin: Line-in (Blue Jack)] hdac1: + [DISABLED] <- nid=27 [pin: Headphones (Green Jack)] hdac1: + [DISABLED] <- nid=28 [pin: Speaker (None)] [DISABLED] hdac1: + [DISABLED] <- nid=29 [pin: Speaker (None)] [DISABLED] hdac1: + [DISABLED] <- nid=20 [pin: Line-out (Green Jack)] hdac1: + [DISABLED] <- nid=21 [pin: Line-out (Black Jack)] hdac1: + [DISABLED] <- nid=22 [pin: Line-out (Orange Jack)] hdac1: + [DISABLED] <- nid=23 [pin: Line-out (Grey Jack)] hdac1: hdac1: nid: 12 hdac1: Name: audio mixer hdac1: Widget cap: 0x0020010f hdac1: STEREO hdac1: Association: 0 (0x00000001) hdac1: OSS: pcm, mix hdac1: Output amp: 0x00051f1f hdac1: mute=0 step=31 size=5 offset=31 hdac1: Input amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: connections: 2 hdac1: | hdac1: + <- nid=2 [audio output] hdac1: + <- nid=11 [audio mixer] hdac1: hdac1: nid: 13 hdac1: Name: audio mixer hdac1: Widget cap: 0x0020010f hdac1: STEREO hdac1: Association: 0 (0x00000002) hdac1: OSS: pcm, mix hdac1: Output amp: 0x00051f1f hdac1: mute=0 step=31 size=5 offset=31 hdac1: Input amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: connections: 2 hdac1: | hdac1: + <- nid=3 [audio output] hdac1: + <- nid=11 [audio mixer] hdac1: hdac1: nid: 14 hdac1: Name: audio mixer hdac1: Widget cap: 0x0020010f hdac1: STEREO hdac1: Association: 0 (0x00000004) hdac1: OSS: pcm, mix hdac1: Output amp: 0x00051f1f hdac1: mute=0 step=31 size=5 offset=31 hdac1: Input amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: connections: 2 hdac1: | hdac1: + <- nid=4 [audio output] hdac1: + <- nid=11 [audio mixer] hdac1: hdac1: nid: 15 hdac1: Name: audio mixer hdac1: Widget cap: 0x0020010f hdac1: STEREO hdac1: Association: 0 (0x00000010) hdac1: OSS: pcm, mix hdac1: Output amp: 0x00051f1f hdac1: mute=0 step=31 size=5 offset=31 hdac1: Input amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: connections: 2 hdac1: | hdac1: + <- nid=5 [audio output] hdac1: + <- nid=11 [audio mixer] hdac1: hdac1: nid: 16 [DISABLED] hdac1: Name: audio output hdac1: Widget cap: 0x00000611 hdac1: PWR DIGITAL STEREO hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e05e0 hdac1: 16 20 24 bits, 44 48 88 96 192 KHz hdac1: hdac1: nid: 17 [DISABLED] hdac1: Name: pin: Speaker (None) hdac1: Widget cap: 0x00400780 hdac1: PWR DIGITAL UNSOL hdac1: Pin cap: 0x00000014 hdac1: PDC OUT hdac1: Pin config: 0x411111f0 hdac1: Pin control: 0x00000000 hdac1: connections: 1 hdac1: | hdac1: + <- nid=16 [audio output] [DISABLED] hdac1: hdac1: nid: 18 [DISABLED] hdac1: Name: pin: Speaker (None) hdac1: Widget cap: 0x00400401 hdac1: PWR STEREO hdac1: Pin cap: 0x00000020 hdac1: IN hdac1: Pin config: 0x411111f0 hdac1: Pin control: 0x00000000 hdac1: hdac1: nid: 19 [DISABLED] hdac1: Name: vendor widget hdac1: Widget cap: 0x00f00000 hdac1: hdac1: nid: 20 hdac1: Name: pin: Line-out (Green Jack) hdac1: Widget cap: 0x0040058f hdac1: PWR UNSOL STEREO hdac1: Association: 0 (0x00000001) hdac1: Pin cap: 0x0001003e hdac1: TRQD PDC HP OUT IN EAPD hdac1: Pin config: 0x01014010 hdac1: Pin control: 0x00000040 OUT hdac1: EAPD: 0x00000002 hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: Input amp: 0x00270300 hdac1: mute=0 step=3 size=39 offset=0 hdac1: connections: 5 hdac1: | hdac1: + <- nid=12 [audio mixer] (selected) hdac1: + [DISABLED] <- nid=13 [audio mixer] hdac1: + [DISABLED] <- nid=14 [audio mixer] hdac1: + [DISABLED] <- nid=15 [audio mixer] hdac1: + [DISABLED] <- nid=38 [audio mixer] hdac1: hdac1: nid: 21 hdac1: Name: pin: Line-out (Black Jack) hdac1: Widget cap: 0x0040058f hdac1: PWR UNSOL STEREO hdac1: Association: 0 (0x00000004) hdac1: Pin cap: 0x0001003e hdac1: TRQD PDC HP OUT IN EAPD hdac1: Pin config: 0x01011012 hdac1: Pin control: 0x00000040 OUT hdac1: EAPD: 0x00000002 hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: Input amp: 0x00270300 hdac1: mute=0 step=3 size=39 offset=0 hdac1: connections: 5 hdac1: | hdac1: + [DISABLED] <- nid=12 [audio mixer] hdac1: + [DISABLED] <- nid=13 [audio mixer] hdac1: + <- nid=14 [audio mixer] (selected) hdac1: + [DISABLED] <- nid=15 [audio mixer] hdac1: + [DISABLED] <- nid=38 [audio mixer] hdac1: hdac1: nid: 22 hdac1: Name: pin: Line-out (Orange Jack) hdac1: Widget cap: 0x0040058f hdac1: PWR UNSOL STEREO hdac1: Association: 0 (0x00000002) hdac1: Pin cap: 0x00000036 hdac1: TRQD PDC OUT IN hdac1: Pin config: 0x01016011 hdac1: Pin control: 0x00000040 OUT hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: Input amp: 0x00270300 hdac1: mute=0 step=3 size=39 offset=0 hdac1: connections: 5 hdac1: | hdac1: + [DISABLED] <- nid=12 [audio mixer] hdac1: + <- nid=13 [audio mixer] (selected) hdac1: + [DISABLED] <- nid=14 [audio mixer] hdac1: + [DISABLED] <- nid=15 [audio mixer] hdac1: + [DISABLED] <- nid=38 [audio mixer] hdac1: hdac1: nid: 23 hdac1: Name: pin: Line-out (Grey Jack) hdac1: Widget cap: 0x0040058f hdac1: PWR UNSOL STEREO hdac1: Association: 0 (0x00000010) hdac1: Pin cap: 0x00000036 hdac1: TRQD PDC OUT IN hdac1: Pin config: 0x01012014 hdac1: Pin control: 0x00000040 OUT hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: Input amp: 0x00270300 hdac1: mute=0 step=3 size=39 offset=0 hdac1: connections: 5 hdac1: | hdac1: + [DISABLED] <- nid=12 [audio mixer] hdac1: + [DISABLED] <- nid=13 [audio mixer] hdac1: + [DISABLED] <- nid=14 [audio mixer] hdac1: + <- nid=15 [audio mixer] (selected) hdac1: + [DISABLED] <- nid=38 [audio mixer] hdac1: hdac1: nid: 24 hdac1: Name: pin: Mic (Pink Jack) hdac1: Widget cap: 0x0040058f hdac1: PWR UNSOL STEREO hdac1: Association: 2 (0x00000001) hdac1: OSS: mic (mic) hdac1: Pin cap: 0x0000373e hdac1: TRQD PDC HP OUT IN VREF[ 50 80 100 GROUND HIZ ] hdac1: Pin config: 0x01a19830 hdac1: Pin control: 0x00000025 IN VREFs hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: Input amp: 0x00270300 hdac1: mute=0 step=3 size=39 offset=0 hdac1: connections: 5 hdac1: | hdac1: + [DISABLED] <- nid=12 [audio mixer] (selected) hdac1: + [DISABLED] <- nid=13 [audio mixer] hdac1: + [DISABLED] <- nid=14 [audio mixer] hdac1: + [DISABLED] <- nid=15 [audio mixer] hdac1: + [DISABLED] <- nid=38 [audio mixer] hdac1: hdac1: nid: 25 hdac1: Name: pin: Mic (Pink Jack) hdac1: Widget cap: 0x0040058f hdac1: PWR UNSOL STEREO hdac1: Association: 3 (0x00000001) hdac1: OSS: monitor (monitor) hdac1: Pin cap: 0x0000373e hdac1: TRQD PDC HP OUT IN VREF[ 50 80 100 GROUND HIZ ] hdac1: Pin config: 0x02a19840 hdac1: Pin control: 0x00000025 IN VREFs hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: Input amp: 0x00270300 hdac1: mute=0 step=3 size=39 offset=0 hdac1: connections: 5 hdac1: | hdac1: + [DISABLED] <- nid=12 [audio mixer] (selected) hdac1: + [DISABLED] <- nid=13 [audio mixer] hdac1: + [DISABLED] <- nid=14 [audio mixer] hdac1: + [DISABLED] <- nid=15 [audio mixer] hdac1: + [DISABLED] <- nid=38 [audio mixer] hdac1: hdac1: nid: 26 hdac1: Name: pin: Line-in (Blue Jack) hdac1: Widget cap: 0x0040058f hdac1: PWR UNSOL STEREO hdac1: Association: 2 (0x00008000) hdac1: OSS: line (line) hdac1: Pin cap: 0x0000373e hdac1: TRQD PDC HP OUT IN VREF[ 50 80 100 GROUND HIZ ] hdac1: Pin config: 0x0181303f hdac1: Pin control: 0x00000025 IN VREFs hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: Input amp: 0x00270300 hdac1: mute=0 step=3 size=39 offset=0 hdac1: connections: 5 hdac1: | hdac1: + [DISABLED] <- nid=12 [audio mixer] (selected) hdac1: + [DISABLED] <- nid=13 [audio mixer] hdac1: + [DISABLED] <- nid=14 [audio mixer] hdac1: + [DISABLED] <- nid=15 [audio mixer] hdac1: + [DISABLED] <- nid=38 [audio mixer] hdac1: hdac1: nid: 27 hdac1: Name: pin: Headphones (Green Jack) hdac1: Widget cap: 0x0040058f hdac1: PWR UNSOL STEREO hdac1: Association: 1 (0x00000001) hdac1: Pin cap: 0x0000373e hdac1: TRQD PDC HP OUT IN VREF[ 50 80 100 GROUND HIZ ] hdac1: Pin config: 0x02214020 hdac1: Pin control: 0x000000c0 HP OUT hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: Input amp: 0x00270300 hdac1: mute=0 step=3 size=39 offset=0 hdac1: connections: 5 hdac1: | hdac1: + [DISABLED] <- nid=12 [audio mixer] hdac1: + [DISABLED] <- nid=13 [audio mixer] hdac1: + [DISABLED] <- nid=14 [audio mixer] hdac1: + [DISABLED] <- nid=15 [audio mixer] hdac1: + <- nid=38 [audio mixer] (selected) hdac1: hdac1: nid: 28 [DISABLED] hdac1: Name: pin: Speaker (None) hdac1: Widget cap: 0x00400481 hdac1: PWR UNSOL STEREO hdac1: Pin cap: 0x00000024 hdac1: PDC IN hdac1: Pin config: 0x411111f0 hdac1: Pin control: 0x00000000 hdac1: hdac1: nid: 29 [DISABLED] hdac1: Name: pin: Speaker (None) hdac1: Widget cap: 0x00400400 hdac1: PWR hdac1: Pin cap: 0x00000020 hdac1: IN hdac1: Pin config: 0x4015e601 hdac1: Pin control: 0x00000000 hdac1: hdac1: nid: 30 [DISABLED] hdac1: Name: pin: Speaker (None) hdac1: Widget cap: 0x00400780 hdac1: PWR DIGITAL UNSOL hdac1: Pin cap: 0x00000014 hdac1: PDC OUT hdac1: Pin config: 0x411111f0 hdac1: Pin control: 0x00000000 hdac1: connections: 1 hdac1: | hdac1: + <- nid=6 [audio output] [DISABLED] hdac1: hdac1: nid: 31 [DISABLED] hdac1: Name: pin: Speaker (None) hdac1: Widget cap: 0x00400680 hdac1: PWR DIGITAL UNSOL hdac1: Pin cap: 0x00000024 hdac1: PDC IN hdac1: Pin config: 0x411111f0 hdac1: Pin control: 0x00000000 hdac1: hdac1: nid: 32 [DISABLED] hdac1: Name: vendor widget hdac1: Widget cap: 0x00f00040 hdac1: PROC hdac1: hdac1: nid: 33 [DISABLED] hdac1: Name: vendor widget hdac1: Widget cap: 0x00f00000 hdac1: hdac1: nid: 34 hdac1: Name: audio mixer hdac1: Widget cap: 0x0020010b hdac1: STEREO hdac1: Association: 3 (0x00000001) hdac1: OSS: monitor hdac1: Input amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: connections: 12 hdac1: | hdac1: + [DISABLED] <- nid=24 [pin: Mic (Pink Jack)] hdac1: + <- nid=25 [pin: Mic (Pink Jack)] hdac1: + [DISABLED] <- nid=26 [pin: Line-in (Blue Jack)] hdac1: + [DISABLED] <- nid=27 [pin: Headphones (Green Jack)] hdac1: + [DISABLED] <- nid=28 [pin: Speaker (None)] [DISABLED] hdac1: + [DISABLED] <- nid=29 [pin: Speaker (None)] [DISABLED] hdac1: + [DISABLED] <- nid=20 [pin: Line-out (Green Jack)] hdac1: + [DISABLED] <- nid=21 [pin: Line-out (Black Jack)] hdac1: + [DISABLED] <- nid=22 [pin: Line-out (Orange Jack)] hdac1: + [DISABLED] <- nid=23 [pin: Line-out (Grey Jack)] hdac1: + [DISABLED] <- nid=11 [audio mixer] hdac1: + [DISABLED] <- nid=18 [pin: Speaker (None)] [DISABLED] hdac1: hdac1: nid: 35 hdac1: Name: audio mixer hdac1: Widget cap: 0x0020010b hdac1: STEREO hdac1: Association: 2 (0x00008001) hdac1: OSS: line, mic, mix hdac1: Input amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: connections: 11 hdac1: | hdac1: + <- nid=24 [pin: Mic (Pink Jack)] hdac1: + [DISABLED] <- nid=25 [pin: Mic (Pink Jack)] hdac1: + <- nid=26 [pin: Line-in (Blue Jack)] hdac1: + [DISABLED] <- nid=27 [pin: Headphones (Green Jack)] hdac1: + [DISABLED] <- nid=28 [pin: Speaker (None)] [DISABLED] hdac1: + [DISABLED] <- nid=29 [pin: Speaker (None)] [DISABLED] hdac1: + [DISABLED] <- nid=20 [pin: Line-out (Green Jack)] hdac1: + [DISABLED] <- nid=21 [pin: Line-out (Black Jack)] hdac1: + [DISABLED] <- nid=22 [pin: Line-out (Orange Jack)] hdac1: + [DISABLED] <- nid=23 [pin: Line-out (Grey Jack)] hdac1: + <- nid=11 [audio mixer] hdac1: hdac1: nid: 36 [DISABLED] hdac1: Name: vendor widget hdac1: Widget cap: 0x00f00000 hdac1: hdac1: nid: 37 hdac1: Name: audio output hdac1: Widget cap: 0x00000411 hdac1: PWR STEREO hdac1: Association: 1 (0x00000001) hdac1: OSS: pcm (pcm) hdac1: Stream cap: 0x00000001 hdac1: PCM hdac1: PCM cap: 0x000e0560 hdac1: 16 20 24 bits, 44 48 96 192 KHz hdac1: hdac1: nid: 38 hdac1: Name: audio mixer hdac1: Widget cap: 0x0020010f hdac1: STEREO hdac1: Association: 1 (0x00000001) hdac1: OSS: pcm, mix hdac1: Output amp: 0x00051f1f hdac1: mute=0 step=31 size=5 offset=31 hdac1: Input amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: connections: 2 hdac1: | hdac1: + <- nid=37 [audio output] hdac1: + <- nid=11 [audio mixer] hdac1: hdac1: Processing audio FG cad=3 nid=1... hdac1: GPIO: 0x00000000 NumGPIO=0 NumGPO=0 NumGPI=0 GPIWake=0 GPIUnsol=0 hdac1: nid 4 0x58560010 as 1 seq 0 Digital-out None jack 6 loc 24 color Unknown misc 0 hdac1: nid 5 0x18560020 as 2 seq 0 Digital-out Jack jack 6 loc 24 color Unknown misc 0 hdac1: nid 6 0x58560030 as 3 seq 0 Digital-out None jack 6 loc 24 color Unknown misc 0 hdac1: Patched pins configuration: hdac1: nid 4 0x58560010 as 1 seq 0 Digital-out None jack 6 loc 24 color Unknown misc 0 [DISABLED] hdac1: nid 5 0x18560020 as 2 seq 0 Digital-out Jack jack 6 loc 24 color Unknown misc 0 hdac1: nid 6 0x58560030 as 3 seq 0 Digital-out None jack 6 loc 24 color Unknown misc 0 [DISABLED] hdac1: 1 associations found: hdac1: Association 0 (2) out: hdac1: Pin nid=5 seq=0 hdac1: Tracing association 0 (2) hdac1: Pin 5 traced to DAC 2 hdac1: Association 0 (2) trace succeeded hdac1: Tracing input monitor hdac1: Tracing other input monitors hdac1: Tracing beeper hdac1: FG config/quirks: forcestereo ivref50 ivref80 ivref100 ivref hdac1: hdac1: +-------------------+ hdac1: | DUMPING HDA NODES | hdac1: +-------------------+ hdac1: hdac1: Default Parameter hdac1: ----------------- hdac1: IN amp: 0x00000000 hdac1: OUT amp: 0x00000000 hdac1: hdac1: nid: 2 hdac1: Name: audio output hdac1: Widget cap: 0x00006611 hdac1: PWR DIGITAL 8CH hdac1: Association: 0 (0x00000001) hdac1: OSS: pcm (pcm) hdac1: Stream cap: 0x00000005 hdac1: AC3 PCM hdac1: PCM cap: 0x001e07f0 hdac1: 16 20 24 32 bits, 32 44 48 88 96 176 192 KHz hdac1: hdac1: nid: 3 [DISABLED] hdac1: Name: audio output hdac1: Widget cap: 0x00006611 hdac1: PWR DIGITAL 8CH hdac1: Stream cap: 0x00000005 hdac1: AC3 PCM hdac1: PCM cap: 0x001e07f0 hdac1: 16 20 24 32 bits, 32 44 48 88 96 176 192 KHz hdac1: hdac1: nid: 4 [DISABLED] hdac1: Name: pin: Digital-out (None) hdac1: Widget cap: 0x0040778d hdac1: PWR DIGITAL UNSOL 8CH hdac1: Pin cap: 0x09000094 hdac1: PDC OUT HDMI DP HBR hdac1: Pin config: 0x58560010 hdac1: Pin control: 0x00000000 hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: connections: 2 hdac1: | hdac1: + [DISABLED] <- nid=2 [audio output] (selected) hdac1: + <- nid=3 [audio output] [DISABLED] hdac1: hdac1: nid: 5 hdac1: Name: pin: Digital-out (Jack) hdac1: Widget cap: 0x0040778d hdac1: PWR DIGITAL UNSOL 8CH hdac1: Association: 0 (0x00000001) hdac1: Pin cap: 0x09000094 hdac1: PDC OUT HDMI DP HBR hdac1: Pin config: 0x18560020 hdac1: Pin control: 0x00000040 OUT hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: connections: 2 hdac1: | hdac1: + <- nid=2 [audio output] (selected) hdac1: + [DISABLED] <- nid=3 [audio output] [DISABLED] hdac1: hdac1: nid: 6 [DISABLED] hdac1: Name: pin: Digital-out (None) hdac1: Widget cap: 0x0040778d hdac1: PWR DIGITAL UNSOL 8CH hdac1: Pin cap: 0x09000094 hdac1: PDC OUT HDMI DP HBR hdac1: Pin config: 0x58560030 hdac1: Pin control: 0x00000000 hdac1: Output amp: 0x80000000 hdac1: mute=1 step=0 size=0 offset=0 hdac1: connections: 2 hdac1: | hdac1: + [DISABLED] <- nid=2 [audio output] (selected) hdac1: + <- nid=3 [audio output] [DISABLED] hdac1: hdac1: nid: 7 [DISABLED] hdac1: Name: vendor widget hdac1: Widget cap: 0x00f00000 hdac1: pcm1: at cad 0 nid 1 on hdac1 pcm1: +--------------------------------------+ pcm1: | DUMPING PCM Playback/Record Channels | pcm1: +--------------------------------------+ pcm1: pcm1: Playback: pcm1: pcm1: Stream cap: 0x00000001 pcm1: PCM pcm1: PCM cap: 0x000e0560 pcm1: 16 20 24 bits, 44 48 96 192 KHz pcm1: DAC: 2 3 4 5 pcm1: pcm1: Record: pcm1: pcm1: Stream cap: 0x00000001 pcm1: PCM pcm1: PCM cap: 0x000e0560 pcm1: 16 20 24 bits, 44 48 96 192 KHz pcm1: ADC: 8 pcm1: pcm1: +-------------------------------+ pcm1: | DUMPING Playback/Record Paths | pcm1: +-------------------------------+ pcm1: pcm1: Playback: pcm1: pcm1: nid=20 [pin: Line-out (Green Jack)] pcm1: | pcm1: + <- nid=12 [audio mixer] [src: pcm, mix] pcm1: | pcm1: + <- nid=2 [audio output] [src: pcm] pcm1: + <- nid=11 [audio mixer] [src: mix] pcm1: pcm1: nid=22 [pin: Line-out (Orange Jack)] pcm1: | pcm1: + <- nid=13 [audio mixer] [src: pcm, mix] pcm1: | pcm1: + <- nid=3 [audio output] [src: pcm] pcm1: + <- nid=11 [audio mixer] [src: mix] pcm1: pcm1: nid=21 [pin: Line-out (Black Jack)] pcm1: | pcm1: + <- nid=14 [audio mixer] [src: pcm, mix] pcm1: | pcm1: + <- nid=4 [audio output] [src: pcm] pcm1: + <- nid=11 [audio mixer] [src: mix] pcm1: pcm1: nid=23 [pin: Line-out (Grey Jack)] pcm1: | pcm1: + <- nid=15 [audio mixer] [src: pcm, mix] pcm1: | pcm1: + <- nid=5 [audio output] [src: pcm] pcm1: + <- nid=11 [audio mixer] [src: mix] pcm1: pcm1: Record: pcm1: pcm1: nid=8 [audio input] pcm1: | pcm1: + <- nid=35 [audio mixer] [src: line, mic, mix] pcm1: | pcm1: + <- nid=24 [pin: Mic (Pink Jack)] [src: mic] pcm1: + <- nid=26 [pin: Line-in (Blue Jack)] [src: line] pcm1: + <- nid=11 [audio mixer] [src: mix] pcm1: pcm1: Input Mix: pcm1: pcm1: nid=11 [audio mixer] pcm1: | pcm1: + <- nid=24 [pin: Mic (Pink Jack)] [src: mic] pcm1: + <- nid=26 [pin: Line-in (Blue Jack)] [src: line] pcm1: pcm1: +-------------------------+ pcm1: | DUMPING Volume Controls | pcm1: +-------------------------+ pcm1: pcm1: Master Volume (OSS: vol) pcm1: | pcm1: +- ctl 13 (nid 12 out): -46/0dB (32 steps) pcm1: +- ctl 14 (nid 12 in 0): mute pcm1: +- ctl 15 (nid 12 in 1): mute pcm1: +- ctl 16 (nid 13 out): -46/0dB (32 steps) pcm1: +- ctl 17 (nid 13 in 0): mute pcm1: +- ctl 18 (nid 13 in 1): mute pcm1: +- ctl 19 (nid 14 out): -46/0dB (32 steps) pcm1: +- ctl 20 (nid 14 in 0): mute pcm1: +- ctl 21 (nid 14 in 1): mute pcm1: +- ctl 22 (nid 15 out): -46/0dB (32 steps) pcm1: +- ctl 23 (nid 15 in 0): mute pcm1: +- ctl 24 (nid 15 in 1): mute pcm1: +- ctl 25 (nid 20 in ): mute pcm1: +- ctl 27 (nid 21 in ): mute pcm1: +- ctl 29 (nid 22 in ): mute pcm1: +- ctl 31 (nid 23 in ): mute pcm1: pcm1: PCM Volume (OSS: pcm) pcm1: | pcm1: +- ctl 14 (nid 12 in 0): mute pcm1: +- ctl 17 (nid 13 in 0): mute pcm1: +- ctl 20 (nid 14 in 0): mute pcm1: +- ctl 23 (nid 15 in 0): mute pcm1: pcm1: Microphone Volume (OSS: mic) pcm1: | pcm1: +- ctl 34 (nid 24 out): 0/30dB (4 steps) pcm1: +- ctl 53 (nid 35 in 0): mute pcm1: pcm1: Line-in Volume (OSS: line) pcm1: | pcm1: +- ctl 38 (nid 26 out): 0/30dB (4 steps) pcm1: +- ctl 55 (nid 35 in 2): mute pcm1: pcm1: Recording Level (OSS: rec) pcm1: | pcm1: +- ctl 1 (nid 8 in 0): -16/30dB (32 steps) + mute pcm1: +- ctl 53 (nid 35 in 0): mute pcm1: +- ctl 55 (nid 35 in 2): mute pcm1: +- ctl 63 (nid 35 in 10): mute pcm1: pcm1: Input Mix Level (OSS: mix) pcm1: | pcm1: +- ctl 3 (nid 11 in 0): -34/12dB (32 steps) + mute pcm1: +- ctl 5 (nid 11 in 2): -34/12dB (32 steps) + mute pcm1: +- ctl 15 (nid 12 in 1): mute pcm1: +- ctl 18 (nid 13 in 1): mute pcm1: +- ctl 21 (nid 14 in 1): mute pcm1: +- ctl 24 (nid 15 in 1): mute pcm1: +- ctl 63 (nid 35 in 10): mute pcm1: pcm1: Input Monitoring Level (OSS: igain) pcm1: | pcm1: +- ctl 15 (nid 12 in 1): mute pcm1: +- ctl 18 (nid 13 in 1): mute pcm1: +- ctl 21 (nid 14 in 1): mute pcm1: +- ctl 24 (nid 15 in 1): mute pcm1: pcm1: Enabling Soft PCM volume pcm1: Mixer "vol": pcm1: Mixer "pcm": pcm1: Mixer "line": pcm1: Mixer "mic": pcm1: Mixer "mix": pcm1: Mixer "rec": pcm1: Mixer "igain": pcm1: Mixer "ogain": pcm1: Soft PCM mixer ENABLED pcm1: clone manager: deadline=750ms flags=0x8000001e pcm1: sndbuf_setmap 1be0000, 4000; 0xffffff80e66aa000 -> 1be0000 pcm1: sndbuf_setmap 1bf0000, 4000; 0xffffff80e66ba000 -> 1bf0000 pcm2: at cad 0 nid 1 on hdac1 pcm2: +--------------------------------------+ pcm2: | DUMPING PCM Playback/Record Channels | pcm2: +--------------------------------------+ pcm2: pcm2: Playback: pcm2: pcm2: Stream cap: 0x00000001 pcm2: PCM pcm2: PCM cap: 0x000e0560 pcm2: 16 20 24 bits, 44 48 96 192 KHz pcm2: DAC: 37 pcm2: pcm2: Record: pcm2: pcm2: Stream cap: 0x00000001 pcm2: PCM pcm2: PCM cap: 0x000e0560 pcm2: 16 20 24 bits, 44 48 96 192 KHz pcm2: ADC: 9 pcm2: pcm2: +-------------------------------+ pcm2: | DUMPING Playback/Record Paths | pcm2: +-------------------------------+ pcm2: pcm2: Playback: pcm2: pcm2: nid=27 [pin: Headphones (Green Jack)] pcm2: | pcm2: + <- nid=38 [audio mixer] [src: pcm, mix] pcm2: | pcm2: + <- nid=37 [audio output] [src: pcm] pcm2: + <- nid=11 [audio mixer] [src: mix] pcm2: pcm2: Record: pcm2: pcm2: nid=9 [audio input] pcm2: | pcm2: + <- nid=34 [audio mixer] [src: monitor] pcm2: | pcm2: + <- nid=25 [pin: Mic (Pink Jack)] [src: monitor] pcm2: pcm2: +-------------------------+ pcm2: | DUMPING Volume Controls | pcm2: +-------------------------+ pcm2: pcm2: Master Volume (OSS: vol) pcm2: | pcm2: +- ctl 39 (nid 27 in ): mute pcm2: +- ctl 64 (nid 38 out): -46/0dB (32 steps) pcm2: +- ctl 65 (nid 38 in 0): mute pcm2: +- ctl 66 (nid 38 in 1): mute pcm2: pcm2: PCM Volume (OSS: pcm) pcm2: | pcm2: +- ctl 65 (nid 38 in 0): mute pcm2: pcm2: Microphone2 Volume (OSS: monitor) pcm2: | pcm2: +- ctl 36 (nid 25 out): 0/30dB (4 steps) pcm2: +- ctl 42 (nid 34 in 1): mute pcm2: pcm2: Recording Level (OSS: rec) pcm2: | pcm2: +- ctl 2 (nid 9 in 0): -16/30dB (32 steps) + mute pcm2: +- ctl 42 (nid 34 in 1): mute pcm2: pcm2: Input Mix Level (OSS: mix) pcm2: | pcm2: +- ctl 66 (nid 38 in 1): mute pcm2: pcm2: Input Monitoring Level (OSS: igain) pcm2: | pcm2: +- ctl 66 (nid 38 in 1): mute pcm2: pcm2: Enabling Soft PCM volume pcm2: Mixer "vol": pcm2: Mixer "pcm": pcm2: Mixer "mix": pcm2: Mixer "rec": pcm2: Mixer "igain": pcm2: Mixer "monitor": pcm2: Soft PCM mixer ENABLED pcm2: clone manager: deadline=750ms flags=0x8000001e pcm2: sndbuf_setmap 1d70000, 4000; 0xffffff80e66ca000 -> 1d70000 pcm2: sndbuf_setmap 1d80000, 4000; 0xffffff80e66da000 -> 1d80000 pcm3: at cad 3 nid 1 on hdac1 pcm3: +--------------------------------------+ pcm3: | DUMPING PCM Playback/Record Channels | pcm3: +--------------------------------------+ pcm3: pcm3: Playback: pcm3: pcm3: Stream cap: 0x00000005 pcm3: AC3 PCM pcm3: PCM cap: 0x001e07f0 pcm3: 16 20 24 32 bits, 32 44 48 88 96 176 192 KHz pcm3: DAC: 2 pcm3: pcm3: +-------------------------------+ pcm3: | DUMPING Playback/Record Paths | pcm3: +-------------------------------+ pcm3: pcm3: Playback: pcm3: pcm3: nid=5 [pin: Digital-out (Jack)] pcm3: | pcm3: + <- nid=2 [audio output] [src: pcm] pcm3: pcm3: +-------------------------+ pcm3: | DUMPING Volume Controls | pcm3: +-------------------------+ pcm3: pcm3: Master Volume (OSS: vol) pcm3: | pcm3: +- ctl 2 (nid 5 in ): mute pcm3: pcm3: PCM Volume (OSS: pcm) pcm3: | pcm3: +- ctl 2 (nid 5 in ): mute pcm3: pcm3: Enabling Soft PCM volume pcm3: Mixer "vol": pcm3: Mixer "pcm": pcm3: Soft PCM mixer ENABLED pcm3: clone manager: deadline=750ms flags=0x8000001e pcm3: sndbuf_setmap 1d90000, 4000; 0xffffff80e66ea000 -> 1d90000 SMP: AP CPU #1 Launched! cpu1 AP: ID: 0x02000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000200ef therm: 0x00010000 err: 0x00010000 pcm: 0x00010400 SMP: AP CPU #3 Launched! cpu3 AP: ID: 0x06000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000200ef therm: 0x00010000 err: 0x00010000 pcm: 0x00010400 SMP: AP CPU #2 Launched! cpu2 AP: ID: 0x04000000 VER: 0x00060015 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000200ef therm: 0x00010000 err: 0x00010000 pcm: 0x00010400 ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 2 vector 48 ioapic0: routing intpin 16 (PCI IRQ 16) to lapic 4 vector 48 ioapic0: routing intpin 18 (PCI IRQ 18) to lapic 6 vector 48 ioapic0: routing intpin 23 (PCI IRQ 23) to lapic 2 vector 49 msi: Assigning MSI IRQ 256 to local APIC 4 vector 49 msi: Assigning MSI IRQ 257 to local APIC 6 vector 49 WARNING: WITNESS option enabled, expect reduced performance. GEOM: ad4: partition 3 does not start on a track boundary. GEOM: ad4: partition 3 does not end on a track boundary. GEOM: ad4: partition 2 does not start on a track boundary. GEOM: ad4: partition 2 does not end on a track boundary. Root mount waiting for: usbus1 usbus0 uhub1: 2 ports with 2 removable, self powered uhub0: 2 ports with 2 removable, self powered Root mount waiting for: usbus1 usbus0 ugen1.2: at usbus1 uhub2: on usbus1 ugen0.2: at usbus0 uhub3: on usbus0 Root mount waiting for: usbus1 usbus0 uhub3: 6 ports with 6 removable, self powered uhub2: 8 ports with 8 removable, self powered Root mount waiting for: usbus1 usbus0 ugen0.3: at usbus0 ukbd0: on usbus0 kbd2 at ukbd0 kbd2: ukbd0, generic (0), config:0x0, flags:0x3d0000 ugen1.3: at usbus1 Trying to mount root from ufs:/dev/ufs/root ct_to_ts([2010-03-12 19:36:04]) = 1268422564.000000000 start_init: trying /sbin/init acd0: FAILURE - INQUIRY ILLEGAL REQUEST asc=0x24 ascq=0x00 (probe0:ata3:0:0:0): SCSI status error (probe0:ata3:0:0:0): INQUIRY. CDB: 12 1 0 0 ff 0 (probe0:ata3:0:0:0): CAM status: SCSI Status Error (probe0:ata3:0:0:0): SCSI status: Check Condition (probe0:ata3:0:0:0): SCSI sense: ILLEGAL REQUEST asc:24,0 (Invalid field in CDB) (probe0:ata3:0:0:0): Error 22, Unretryable error (probe0:ata3:0:0:0): Down reving Protocol Version from 2 to 0? (probe0:ata3:0:0:0): SCSI status error (probe0:ata3:0:0:0): TEST UNIT READY. CDB: 0 0 0 0 0 0 (probe0:ata3:0:0:0): CAM status: SCSI Status Error (probe0:ata3:0:0:0): SCSI status: Check Condition (probe0:ata3:0:0:0): SCSI sense: NOT READY asc:3a,1 (Medium not present - tray closed) (probe0:ata3:0:0:0): Error 6, Unretryable error pass0 at ata3 bus 0 scbus1 target 0 lun 0 pass0: Removable CD-ROM SCSI-0 device pass0: 100.000MB/s transfers GEOM: new disk cd0 (cd0:ata3:0:0:0): SCSI status error (cd0:ata3:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 (cd0:ata3:0:0:0): CAM status: SCSI Status Error (cd0:ata3:0:0:0): SCSI status: Check Condition (cd0:ata3:0:0:0): SCSI sense: NOT READY asc:3a,1 (Medium not present - tray closed) (cd0:ata3:0:0:0): Error 6, Unretryable error cd0 at ata3 bus 0 scbus1 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 100.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed (cd0:ata3:0:0:0): SCSI status error (cd0:ata3:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 (cd0:ata3:0:0:0): CAM status: SCSI Status Error (cd0:ata3:0:0:0): SCSI status: Check Condition (cd0:ata3:0:0:0): SCSI sense: NOT READY asc:3a,1 (Medium not present - tray closed) (cd0:ata3:0:0:0): Error 6, Unretryable error (cd0:ata3:0:0:0): SCSI status error (cd0:ata3:0:0:0): READ CAPACITY. CDB: 25 0 0 0 0 0 0 0 0 0 (cd0:ata3:0:0:0): CAM status: SCSI Status Error (cd0:ata3:0:0:0): SCSI status: Check Condition (cd0:ata3:0:0:0): SCSI sense: NOT READY asc:3a,1 (Medium not present - tray closed) (cd0:ata3:0:0:0): Error 6, Unretryable error bge0: Disabling fastboot bge0: Disabling fastboot bge0: bge0: link state changed to UP link UP bge0: link state changed to DOWN bge0: link DOWN --pf9I7BMVVzbSWLtt-- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 19:56:46 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6D86B106564A; Fri, 12 Mar 2010 19:56:46 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id C5FCA8FC1B; Fri, 12 Mar 2010 19:56:45 +0000 (UTC) Received: by fxm23 with SMTP id 23so1569831fxm.3 for ; Fri, 12 Mar 2010 11:56:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=oU0CUS1FEMI/QVx3S6eAYECBgSwhQkjwEgWRzRxviMQ=; b=ktnCD2PiYEt5zMNU1jbFCCFwEqHiKHMOHu1aPOEjck7ib/DssIAAv61Kh/JOjmXy8a XHxcLb9XmE0uwhrBwGfcemIT3rwMNjJwQRtQkyXdjB1Y7trywCHKYSFL/EBrVMZR5bjl IEuuySKsm/yszzKOCrRLz6HrrePaE+vGmX7QA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=ZwP19WlcRTmKRQjVp9UDMkeeRiCFXfuZZ781V00JDJkhruEEHV11p5z39hIj6cKDV7 XtwAukhc1W30y03AAKYc88dmoEG/2Rc0CCzQuiYyPn1QTaNUxOzGoFNYZsPG1a/VxQjJ PE7FQptfqRBAbJN/obTlqI5VZigdOm76IBgUc= Received: by 10.223.76.74 with SMTP id b10mr1574935fak.55.1268423804635; Fri, 12 Mar 2010 11:56:44 -0800 (PST) Received: from mavbook.mavhome.dp.ua (s224.GtokyoFL6.vectant.ne.jp [222.228.90.224]) by mx.google.com with ESMTPS id 14sm1269240fxm.1.2010.03.12.11.56.42 (version=SSLv3 cipher=RC4-MD5); Fri, 12 Mar 2010 11:56:43 -0800 (PST) Sender: Alexander Motin Message-ID: <4B9A9C77.1030100@FreeBSD.org> Date: Fri, 12 Mar 2010 21:56:39 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Pierre Beyssac References: <20100312121409.GA79294@fasterix.frmug.org> <20100312131832.GA54731@icarus.home.lan> <20100312165744.GA86971@fasterix.frmug.org> <20100312170936.GA59343@icarus.home.lan> <4B9A8F14.4030907@FreeBSD.org> <20100312192759.GA91317@fasterix.frmug.org> In-Reply-To: <20100312192759.GA91317@fasterix.frmug.org> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-stable@freebsd.org, Jeremy Chadwick Subject: Re: (fixed) Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 19:56:46 -0000 Pierre Beyssac wrote: > On Fri, Mar 12, 2010 at 08:59:32PM +0200, Alexander Motin wrote: >> Except ICH6 Intel uses separate IDs for AHCI and non-AHCI SATA >> controller modes. So this flag is not completely correct. But probably >> it shoudn't be just removed, as it is checked in other place. I know >> about this and going to handle it later. It's not a problem now. > > I was wondering exactly about that given the wording of the Intel > document sent by Jeremy... > >>> http://www.intel.com/Assets/PDF/specupdate/322170.pdf >> Thanks. >> PS: Check BIOS settings for enabling AHCI mode. > > Problem is, I can't find any setting to do that, either it's not > there or it's well hidden. The BIOS is up-to-date. Could it be > possible for Dell to lock the chip in non-AHCI mode? It is BIOS job to configure SATA controller mode. It is difficult to do from driver level and very vendor-specific. Yes, it happens sometimes to see BIOSes without AHCI support. -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 20:01:52 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E3A81065673 for ; Fri, 12 Mar 2010 20:01:52 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta10.emeryville.ca.mail.comcast.net (qmta10.emeryville.ca.mail.comcast.net [76.96.30.17]) by mx1.freebsd.org (Postfix) with ESMTP id 641388FC0C for ; Fri, 12 Mar 2010 20:01:52 +0000 (UTC) Received: from omta18.emeryville.ca.mail.comcast.net ([76.96.30.74]) by qmta10.emeryville.ca.mail.comcast.net with comcast id sKHy1d0091bwxycAAL1sV5; Fri, 12 Mar 2010 20:01:52 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta18.emeryville.ca.mail.comcast.net with comcast id sL521d00Z3S48mS8eL53jj; Fri, 12 Mar 2010 20:05:03 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id A9B9F9B436; Fri, 12 Mar 2010 12:01:50 -0800 (PST) Date: Fri, 12 Mar 2010 12:01:50 -0800 From: Jeremy Chadwick To: Pierre Beyssac Message-ID: <20100312200150.GA62481@icarus.home.lan> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312131832.GA54731@icarus.home.lan> <20100312165744.GA86971@fasterix.frmug.org> <20100312170936.GA59343@icarus.home.lan> <4B9A8F14.4030907@FreeBSD.org> <20100312192759.GA91317@fasterix.frmug.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100312192759.GA91317@fasterix.frmug.org> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: Alexander Motin , freebsd-stable@freebsd.org Subject: Re: (fixed) Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 20:01:52 -0000 On Fri, Mar 12, 2010 at 08:27:59PM +0100, Pierre Beyssac wrote: > On Fri, Mar 12, 2010 at 08:59:32PM +0200, Alexander Motin wrote: > > Except ICH6 Intel uses separate IDs for AHCI and non-AHCI SATA > > controller modes. So this flag is not completely correct. But probably > > it shoudn't be just removed, as it is checked in other place. I know > > about this and going to handle it later. It's not a problem now. > > I was wondering exactly about that given the wording of the Intel > document sent by Jeremy... > > > > http://www.intel.com/Assets/PDF/specupdate/322170.pdf > > Thanks. > > PS: Check BIOS settings for enabling AHCI mode. > > Problem is, I can't find any setting to do that, either it's not > there or it's well hidden. The BIOS is up-to-date. Could it be > possible for Dell to lock the chip in non-AHCI mode? The system uses the Intel H57 (Ibex) chipset, which is documented here: http://www.intel.com/Products/Desktop/Chipsets/H57/h57-overview.htm The official datasheet is here (~4.4MBytes): http://www.intel.com/Assets/PDF/datasheet/322169.pdf Which does state AHCI is available, however Intel is known for making separate SKUs (revisions/models with different tweaks to the chips) that support different features. Table 1.3 on the PDF outlines what the H57 is capable of: Intel 5 Series Chipset SKUs (Desktop) SKU Name AHCI RAID 0/1/5/10 Support ----------------------------------------- Q57 Yes Yes H57 Yes Yes <==== H55 Yes No P55 Yes Yes ----------------------------------------- Intel 3400 Series Chipset SKUs (Server) SKU Name AHCI RAID 0/1/5/10 Support ----------------------------------------- 3400 No No 3420 Yes Yes 3450 Yes Yes ----------------------------------------- Further within the documentation, it states that for eSATA hot-plug to work, AHCI has to be enabled/in use. That's about all I can discern from the PDF without getting into technical specifics. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 20:02:29 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8EBBD106567E; Fri, 12 Mar 2010 20:02:29 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id E855A8FC13; Fri, 12 Mar 2010 20:02:28 +0000 (UTC) Received: by fxm23 with SMTP id 23so1575199fxm.3 for ; Fri, 12 Mar 2010 12:02:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=QeObv8RgKiQhMvQBTh+JzIWYl9o26JX5/9MlSwkHEfM=; b=Ni5WoB09SaJCh/5KVMj2nzQxNadcKHreQ2T4onPFAOBbeZtGHhyIuIQQ90pV7uvuUK x5jLVFpbkK33mBJlviUNZyAO1mfZN9XDC+eFOXquFgGEnBzjrQnIixSm/Q6mSCTLWVL2 9VWuAbJDk5Pom5CKDRWNbyE1bxDcUZyefkBbc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=pnJIaBFIfhD4eSZTYESqUVG2x+5EsflINioZ5JUiT8vCyHacE/I4HIM0hVX7xPpMkL vAK433SjZl9o1CzU5YKVvaTf2Pv7GzBNA5/cmDorLoBZz8psyCyoncxAEViPQZ0T+jEH o3oW6j2NAPU+GWRuMI5xlVbUEaicWcFgjhzlE= Received: by 10.87.63.20 with SMTP id q20mr2719182fgk.27.1268424147291; Fri, 12 Mar 2010 12:02:27 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id e20sm2043609fga.0.2010.03.12.12.02.24 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 12 Mar 2010 12:02:25 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Fri, 12 Mar 2010 12:02:24 -0800 From: Pyun YongHyeon Date: Fri, 12 Mar 2010 12:02:24 -0800 To: Pierre Beyssac Message-ID: <20100312200224.GO16314@michelle.cdnetworks.com> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312174655.GG16314@michelle.cdnetworks.com> <20100312182134.GA90606@fasterix.frmug.org> <20100312192144.GL16314@michelle.cdnetworks.com> <20100312194524.GB91694@fasterix.frmug.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="NU0Ex4SbNnrxsi6C" Content-Disposition: inline In-Reply-To: <20100312194524.GB91694@fasterix.frmug.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@freebsd.org Subject: Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 20:02:29 -0000 --NU0Ex4SbNnrxsi6C Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Mar 12, 2010 at 08:45:24PM +0100, Pierre Beyssac wrote: > On Fri, Mar 12, 2010 at 11:21:44AM -0800, Pyun YongHyeon wrote: > > No, it seems there is other issue in brgphy(4). I noticed brgphy(4) > > blindly try to set jumbo frame related registers. I guess the PHY > > may not have the register. Back out previous patch and try this > > one. > > Thanks, works better (the error message is gone), but still negotiates > at 10baseT/UTP fdx... See attached dmesg.out. Hmm, try this one and let me know it make any differences. --NU0Ex4SbNnrxsi6C Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="BCM57780.diff3" Index: sys/dev/mii/miidevs =================================================================== --- sys/dev/mii/miidevs (revision 205052) +++ sys/dev/mii/miidevs (working copy) @@ -81,6 +81,7 @@ oui xxALTIMA 0x000895 Altima Communications oui xxBROADCOM 0x000818 Broadcom Corporation oui xxBROADCOM_ALT1 0x0050ef Broadcom Corporation +oui xxBROADCOM_ALT2 0x00d897 Broadcom Corporation oui xxICS 0x00057d Integrated Circuit Systems oui xxSEEQ 0x0005be Seeq oui xxSIS 0x000760 Silicon Integrated Systems @@ -150,6 +151,7 @@ model xxBROADCOM_ALT1 BCM5784 0x003a BCM5784 10/100/1000baseTX PHY model xxBROADCOM_ALT1 BCM5709C 0x003c BCM5709C 10/100/1000baseTX PHY model xxBROADCOM_ALT1 BCM5761 0x003d BCM5761 10/100/1000baseTX PHY +model xxBROADCOM_ALT2 BCM57780 0x0019 BCM57780 10/100/1000baseTX PHY model BROADCOM2 BCM5906 0x0004 BCM5906 10/100baseTX PHY /* Cicada Semiconductor PHYs (now owned by Vitesse?) */ Index: sys/dev/mii/brgphy.c =================================================================== --- sys/dev/mii/brgphy.c (revision 205052) +++ sys/dev/mii/brgphy.c (working copy) @@ -139,6 +139,7 @@ MII_PHY_DESC(xxBROADCOM_ALT1, BCM5784), MII_PHY_DESC(xxBROADCOM_ALT1, BCM5709C), MII_PHY_DESC(xxBROADCOM_ALT1, BCM5761), + MII_PHY_DESC(xxBROADCOM_ALT2, BCM57780), MII_PHY_DESC(BROADCOM2, BCM5906), MII_PHY_END }; @@ -213,6 +214,7 @@ switch (bsc->mii_oui) { case MII_OUI_BROADCOM: case MII_OUI_BROADCOM2: + case MII_OUI_xxBROADCOM_ALT2: break; case MII_OUI_xxBROADCOM: switch (bsc->mii_model) { @@ -678,16 +680,18 @@ brgphy_mii_phy_auto(struct mii_softc *sc) { struct brgphy_softc *bsc = (struct brgphy_softc *)sc; + uint16_t anar; int ktcr = 0; brgphy_reset(sc); /* Enable flow control in the advertisement register. */ if ((sc->mii_flags & MIIF_HAVEFIBER) == 0) { + anar = PHY_READ(sc, BRGPHY_MII_ANAR) & BRGPHY_ANAR_NP; /* Pause capability advertisement (pause capable & asymmetric) */ PHY_WRITE(sc, BRGPHY_MII_ANAR, BMSR_MEDIA_TO_ANAR(sc->mii_capabilities) | ANAR_CSMA | - BRGPHY_ANAR_ASP | BRGPHY_ANAR_PC); + BRGPHY_ANAR_ASP | BRGPHY_ANAR_PC | anar); } else { PHY_WRITE(sc, BRGPHY_SERDES_ANAR, BRGPHY_SERDES_ANAR_FDX | BRGPHY_SERDES_ANAR_HDX | BRGPHY_SERDES_ANAR_BOTH_PAUSE); @@ -1021,7 +1025,8 @@ if (bge_sc->bge_flags & BGE_FLAG_JITTER_BUG) brgphy_fixup_jitter_bug(sc); - brgphy_jumbo_settings(sc, ifp->if_mtu); + if (bge_sc->bge_flags & BGE_FLAG_JUMBO) + brgphy_jumbo_settings(sc, ifp->if_mtu); if (bge_sc->bge_flags & BGE_FLAG_WIRESPEED) brgphy_ethernet_wirespeed(sc); --NU0Ex4SbNnrxsi6C-- From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 21:27:04 2010 Return-Path: Delivered-To: freebsd-stable@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 84E8B1065677 for ; Fri, 12 Mar 2010 21:27:04 +0000 (UTC) (envelope-from gallasch@free.de) Received: from smtp.free.de (smtp.free.de [91.204.6.103]) by mx1.freebsd.org (Postfix) with ESMTP id 764A18FC23 for ; Fri, 12 Mar 2010 21:27:03 +0000 (UTC) Received: (qmail 31273 invoked from network); 12 Mar 2010 22:27:01 +0100 Received: from smtp.free.de (HELO orwell.free.de) (gallasch@free.de@[91.204.4.103]) (envelope-sender ) by smtp.free.de (qmail-ldap-1.03) with AES128-SHA encrypted SMTP for ; 12 Mar 2010 22:27:01 +0100 Date: Fri, 12 Mar 2010 22:26:59 +0100 From: Kai Gallasch To: Alexander Motin Message-ID: <20100312222659.0198dd03@orwell.free.de> In-Reply-To: <4B9A8A27.8050608@FreeBSD.org> References: <20100311133916.42ba69b0@orwell.free.de> <20100312115028.GG1819@garage.freebsd.pl> <4B9A8A27.8050608@FreeBSD.org> X-Mailer: Claws Mail 3.7.3 (GTK+ 2.18.7; powerpc-apple-darwin9.8.0) X-Face: 7"x0zA5=*cXGZw-xjU<">'+!3(KXTUXZVLD42KVN{'go[UQr"Mc.e(XW92N8plZ(9x.{x; I<|95e+b&GH-36\15F~L$YD*Y +u}o&KV?6.%"mJIkaY3G>BKNt`1|Y+%K1P4t; 47D65&(Y7h5Ll-[ltkhamx.-; ,jggK'}oMpUgEHFG YQ"9oXKAl>!d,J}T{)@uxvfu?YFWC*\~h+,^f Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, Pawel Jakub Dawidek , freebsd-stable@FreeBSD.org Subject: Re: proliant server lockups with freebsd-amd64-stable (2010-03-10) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 21:27:04 -0000 Am Fri, 12 Mar 2010 20:38:31 +0200 schrieb Alexander Motin : > Pawel Jakub Dawidek wrote: > > On Thu, Mar 11, 2010 at 01:39:16PM +0100, Kai Gallasch wrote: > >> I have some trouble with an opteron server locking up > >> spontaneously. It looses all networks connectivity and even > >> through console I can get no shell. > >> > >> Lockups occur mostly under disk load (periodic daily, bacula backup > >> running, make buildworld/buildkernel) and I can provoke them > >> easily. > > [...] > >> 4 0 0 0 LL *cissmtx 0xffffff04ed820c00 > >> [g_down] > > [...] > >> 100046 L *cissmtx 0xffffff04ed820c00 > >> [irq257: ciss0] > > [...] > > > > I was analizing similar problem as potential ZFS bug. It turned out > > to be bug in ciss(4) and I believe mav@ (CCed) has fix for that. > > That my patch is already at 8-STABLE since r204873 of 2010-03-08. Make > sure you have it. Updating collection src-all/cvs .. .. Edit src/sys/dev/ciss/ciss.c Edit src/sys/dev/ciss/cissvar.h Didn't have it! Must have been just a few hours I missed it, when I built the last kernel. So I will rebuild my kernel and give it a spin later on. > In this case trap stopped process at ciss_get_request(), which indeed > called holding cissmtx lock. But there is no place to sleep or loop > there, so may be it was just spontaneous. With bugs I was fixing there > was a chance to loop indefinitely between ciss and CAM on resource > constraint. That increases chance for such situation to be caught. > > You may try also look what's going on with `top -HS` and `systat -vm > 1`. FYI. Without your patch of ciss.c and cissvar.h a lockup with top -HS and systat -vm running gives following information below. Is there a pattern visible relating to your patch of the ciss driver? Kai. ------ vmstat -vm ------ 3 users Load 0.21 0.36 0.45 Mar 12 21:47 Mem:KB REAL VIRTUAL VN PAGER SWAP PAGER Tot Share Tot Share Free in out in out Act 2805456 40804 62699320 79936 12358k count All 6182560 95796 1136820k 212452 pages Proc: Interrupts r p d s w Csw Trp Sys Int Sof Flt cow 16018 total 491 533 28 576 19 179 174 zfod atkbd0 1 ozfod uart0 irq4 12.9%Sys 12.5%Intr 0.1%User 0.0%Nice 74.5%Idle %ozfod ata0 irq14 | | | | | | | | | | | daefr uhci0 45 ======+++++++ prcfr 2000 cpu0: time 87 dtbuf totfr 19 bce0 256 Namei Name-cache Dir-cache 100000 desvn react ciss0 257 Calls hits % hits % 40811 numvn pdwak 2000 cpu1: time 17300 frevn pdpgs 2000 cpu4: time intrn 2000 cpu5: time Disks da0 da1 da2 da3 da4 pass0 pass1 4995516 wire 2000 cpu6: time KB/t 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2593276 act 1999 cpu7: time tps 0 0 0 0 0 0 0 369560 inact 2000 cpu3: time MB/s 0.00 0.00 0.00 0.00 0.00 0.00 0.00 9568 cache 2000 cpu2: time %busy 0 0 0 0 0 0 0 12348752 free 1252272 buf ----------- top -HS ------------ last pid: 42561; load averages: 0.35, 0.38, 0.46 up 0+11:24:36 21:53:39 658 processes: 13 running, 623 sleeping, 21 waiting, 1 lock CPU: 0.6% user, 0.0% nice, 12.6% system, 25.0% interrupt, 61.8% idle Mem: 2559M Active, 363M Inact, 4892M Wired, 9548K Cache, 1223M Buf, 12G Free Swap: 21G Total, 21G Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 11 root 171 ki31 0K 128K CPU3 3 672:22 100.00% {idle: cpu3} 11 root 171 ki31 0K 128K CPU1 1 663:50 100.00% {idle: cpu1} 11 root 171 ki31 0K 128K RUN 4 649:18 100.00% {idle: cpu4} 12 root -32 - 0K 384K CPU7 0 6:38 100.00% {swi4: clock} 4 root -8 - 0K 16K CPU5 5 1:16 100.00% g_down 12 root -64 - 0K 384K CPU2 0 1:15 100.00% {swi2: cambio} 11 root 171 ki31 0K 128K CPU6 6 672:13 97.27% {idle: cpu6} 11 root 171 ki31 0K 128K CPU0 0 622:18 96.29% {idle: cpu0} 11 root 171 ki31 0K 128K RUN 7 676:57 10.99% {idle: cpu7} 2046 zope1 46 0 468M 379M ucond 6 7:30 1.76% {python} 11 root 171 ki31 0K 128K RUN 5 663:27 0.00% {idle: cpu5} 11 root 171 ki31 0K 128K RUN 2 656:03 0.00% {idle: cpu2} 12 root -68 - 0K 384K WAIT 0 14:54 0.00% {irq256: bce0} 2046 zope1 46 0 468M 379M ucond 4 6:02 0.00% {python} 1719 zope2 48 0 263M 178M ucond 3 2:00 0.00% {python2.4} 1724 zope2 46 0 321M 235M ucond 5 1:52 0.00% {python2.4} 6314 root 45 0 10432K 2564K ttyin 4 1:36 0.00% systat 3 root -8 - 0K 16K - 2 1:29 0.00% g_up 6301 zope4 46 0 311M 225M ucond 1 1:28 0.00% {python2.4} 2320 zope5 44 0 281M 197M ucond 1 1:22 0.00% {python2.4} 20 root 44 - 0K 16K zio->i 6 1:21 0.00% syncer 6301 zope4 46 0 311M 225M ucond 5 1:19 0.00% {python2.4} 1729 zope6 44 0 311M 228M ucond 4 1:15 0.00% {python2.4} 2320 zope5 45 0 281M 197M ucond 4 1:13 0.00% {python2.4} 2320 zope5 50 0 281M 197M ucond 7 1:08 0.00% {python2.4} 6301 zope4 44 0 311M 225M ucond 4 1:02 0.00% {python2.4} 5138 dave 44 0 341M 248M select 4 0:58 0.00% {initial thread} 1729 zope6 45 0 311M 228M ucond 6 0:56 0.00% {python2.4} 8158 19421 44 0 27644K 7560K select 4 0:52 0.00% centerim 1743 zope7 44 0 287M 201M zio->i 1 0:51 0.00% {python} 2046 zope1 44 0 468M 379M select 1 0:48 0.00% {initial thread} 19943 19420 44 0 31740K 13044K select 4 0:46 0.00% centerim 6301 zope4 44 0 311M 225M ucond 0 0:40 0.00% {python2.4} 1743 zope7 44 0 287M 201M select 0 0:38 0.00% {initial thread} 2052 zope8 44 0 300M 212M select 3 0:37 0.00% {initial thread} 1724 zope3 46 0 321M 235M ucond 4 0:33 0.00% {python2.4} 2320 zope5 44 0 281M 197M ucond 1 0:31 0.00% {python2.4} 2052 zope8 45 0 300M 212M ucond 7 0:30 0.00% {python} 2320 zope5 44 0 281M 197M select 6 0:29 0.00% {initial thread} 12 root -64 - 0K 384K *cissm 1 0:28 0.00% {irq257: ciss0} 6301 zope4 44 0 311M 225M select 6 0:28 0.00% {initial thread} 0 root -16 0 0K 816K - 0 0:25 0.00% {spa_zio} 2052 zope8 47 0 300M 212M ucond 2 0:24 0.00% {python} 1729 zope6 44 0 311M 228M select 3 0:24 0.00% {initial thread} 1719 zope2 44 0 263M 178M select 1 0:23 0.00% {initial thread} 1724 zope3 44 0 321M 235M select 1 0:23 0.00% {initial thread} 12 root -48 - 0K 384K WAIT 0 0:22 0.00% {swi0: uart uart} 1734 zope9 44 0 249M 163M select 4 0:21 0.00% {initial thread} 0 root -16 0 0K 816K - 4 0:16 0.00% {spa_zio_6} 0 root -16 0 0K 816K - 3 0:16 0.00% {spa_zio_4} 0 root -16 0 0K 816K - 0 0:16 0.00% {spa_zio_1} 0 root -16 0 0K 816K - 4 0:16 0.00% {spa_zio_2} 0 root -16 0 0K 816K - 1 0:16 0.00% {spa_zio_5} 0 root -16 0 0K 816K - 2 0:16 0.00% {spa_zio_0} 0 root -16 0 0K 816K - 1 0:16 0.00% {spa_zio_7} 0 root -16 0 0K 816K - 5 0:16 0.00% {spa_zio_3} 4288 mysql 44 0 62460K 21608K select 4 0:16 0.00% {mysqld} 59 root -8 - 0K 92K tx->tx 0 0:12 0.00% {txg_thread_enter} 6129 mysql 44 0 62788K 23420K select 0 0:12 0.00% {mysqld} 59 root -8 - 0K 92K vgeom: 5 0:09 0.00% {vdev da2} 59 root -8 - 0K 92K vgeom: 6 0:09 0.00% {vdev da3} 4288 mysql 44 0 62460K 21608K select 0 0:08 0.00% {mysqld} 6129 mysql 44 0 62788K 23420K select 0 0:08 0.00% {mysqld} 13 root 44 - 0K 16K - 0 0:08 0.00% yarrow 0 root -16 0 0K 816K - 0 0:08 0.00% {spa_zio_6} 0 root -16 0 0K 816K - 7 0:08 0.00% {spa_zio_5} 0 root -16 0 0K 816K - 5 0:08 0.00% {spa_zio_2} 0 root -16 0 0K 816K - 2 0:08 0.00% {spa_zio_4} 0 root -16 0 0K 816K - 3 0:08 0.00% {spa_zio_3} 0 root -16 0 0K 816K - 4 0:07 0.00% {spa_zio_1} 0 root -16 0 0K 816K - 0 0:07 0.00% {spa_zio_7} 0 root -16 0 0K 816K - 1 0:07 0.00% {spa_zio_0} 5099 root 44 0 11380K 4292K CPU4 4 0:07 0.00% top 12 root -44 - 0K 384K WAIT 0 0:07 0.00% {swi1: netisr 0} 4864 news 45 0 24964K 12108K nanslp 6 0:06 0.00% perl 5138 dave 76 0 341M 248M ucond 7 0:05 0.00% {python} 4863 news 76 0 8260K 1920K wait 5 0:05 0.00% sh 5155 root 44 0 442M 6592K kqread 1 0:04 0.00% {cache-kqueue} 1734 zope9 44 0 249M 163M ucond 4 0:04 0.00% {python2.4} 4862 news 44 0 139M 43180K select 6 0:04 0.00% innd 8173 19421 44 0 30928K 8496K select 4 0:04 0.00% irssi From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 21:52:33 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3F663106566C; Fri, 12 Mar 2010 21:52:33 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-fx0-f223.google.com (mail-fx0-f223.google.com [209.85.220.223]) by mx1.freebsd.org (Postfix) with ESMTP id 7488B8FC16; Fri, 12 Mar 2010 21:52:32 +0000 (UTC) Received: by fxm23 with SMTP id 23so1688832fxm.3 for ; Fri, 12 Mar 2010 13:52:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :user-agent:mime-version:to:cc:subject:references:in-reply-to :x-enigmail-version:content-type:content-transfer-encoding; bh=NV8tJFdp2BUjPxtwWpup2plB6nu8xJsEpuJRVbWmCy0=; b=VnfcaRrEPFriydgFxiWF7V+R80N2c1OPZYRHoAYFkKL7wZsaM8jHbezk5At23Zd3kE rPCrp2YTQnYGmucdgXtchDUryXvubqWbDu8zJS9S0FqJLCzXMnjQyrI9pra7w5YFzSCu +ovLWS3GnH0CsEO8aOHV24uZTFnBYNGD2PjqY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=OksR/9/dQBtaAWqlAEjI78JLrOsPB/jtcqhxs2GEDPbePEySim+rmkz9wCR1QHyWBw vtXARDVTKiVHCscuy6EH57nIIJDuheHwYe4pbo2MatW89bWVr5HAD/+5eNmoRGpdaMpH FZzQzj34kp4Nv7sXgscMm4C79oNSVIevKLLx4= Received: by 10.223.143.18 with SMTP id s18mr1651141fau.64.1268430751344; Fri, 12 Mar 2010 13:52:31 -0800 (PST) Received: from mavbook.mavhome.dp.ua (s224.GtokyoFL6.vectant.ne.jp [222.228.90.224]) by mx.google.com with ESMTPS id 16sm1342126fxm.7.2010.03.12.13.52.29 (version=SSLv3 cipher=RC4-MD5); Fri, 12 Mar 2010 13:52:30 -0800 (PST) Sender: Alexander Motin Message-ID: <4B9AB799.6070906@FreeBSD.org> Date: Fri, 12 Mar 2010 23:52:25 +0200 From: Alexander Motin User-Agent: Thunderbird 2.0.0.23 (X11/20091212) MIME-Version: 1.0 To: Kai Gallasch References: <20100311133916.42ba69b0@orwell.free.de> <20100312115028.GG1819@garage.freebsd.pl> <4B9A8A27.8050608@FreeBSD.org> <20100312222659.0198dd03@orwell.free.de> In-Reply-To: <20100312222659.0198dd03@orwell.free.de> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org, Pawel Jakub Dawidek , freebsd-stable@FreeBSD.org Subject: Re: proliant server lockups with freebsd-amd64-stable (2010-03-10) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 21:52:33 -0000 Kai Gallasch wrote: > Am Fri, 12 Mar 2010 20:38:31 +0200 > schrieb Alexander Motin : >> Pawel Jakub Dawidek wrote: >>> I was analizing similar problem as potential ZFS bug. It turned out >>> to be bug in ciss(4) and I believe mav@ (CCed) has fix for that. >> That my patch is already at 8-STABLE since r204873 of 2010-03-08. Make >> sure you have it. > > Updating collection src-all/cvs > .. > .. > Edit src/sys/dev/ciss/ciss.c > Edit src/sys/dev/ciss/cissvar.h > > Didn't have it! Must have been just a few hours I missed it, > when I built the last kernel. So I will rebuild my kernel and > give it a spin later on. > >> In this case trap stopped process at ciss_get_request(), which indeed >> called holding cissmtx lock. But there is no place to sleep or loop >> there, so may be it was just spontaneous. With bugs I was fixing there >> was a chance to loop indefinitely between ciss and CAM on resource >> constraint. That increases chance for such situation to be caught. >> >> You may try also look what's going on with `top -HS` and `systat -vm >> 1`. > > FYI. Without your patch of ciss.c and cissvar.h a lockup with top -HS > and systat -vm running gives following information below. Is there a > pattern visible relating to your patch of the ciss driver? May be. You should try, -- Alexander Motin From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 22:38:16 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 898661065670; Fri, 12 Mar 2010 22:38:16 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.159]) by mx1.freebsd.org (Postfix) with ESMTP id E23AD8FC1D; Fri, 12 Mar 2010 22:38:15 +0000 (UTC) Received: by fg-out-1718.google.com with SMTP id e21so528743fga.13 for ; Fri, 12 Mar 2010 14:38:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=4gE7GlZ0+gnUAFF6Fbf6z/tEzplRV735b3FCI1vwe+4=; b=MH8Oy7uR8JmARwoOIrf7EfaQQC7wwKZoE15EBM9ojfsUIHb090P6uLM+qAHEDgwyJe 3PYTbdPhecdwLWl+zvU1anQ2J31OTzQn24dvzgu08GlapAKzJKlImXhcoCpQVuZp5Dq6 5d1rYKJAIA4aSGzj+wKokbCLUkUvcyvwq/SDg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=gC9JeoF353rjIpRPctt5wul3OQqpPRcm6KJEPw+o91Pu6umLH/MYUUhIhBNxdYcK8Y z2snZzn4q2Qvw73c3zGBsA9bj0LDMHWwksox7j9TfAd65wiJukJlbrsE3VCnI4R065Lb lMHRxCI3HucfK2Bi0LteUFrrxZKAO/wZMm/94= Received: by 10.87.48.6 with SMTP id a6mr8564000fgk.40.1268433494562; Fri, 12 Mar 2010 14:38:14 -0800 (PST) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id e3sm2163280fga.23.2010.03.12.14.38.11 (version=TLSv1/SSLv3 cipher=RC4-MD5); Fri, 12 Mar 2010 14:38:13 -0800 (PST) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Fri, 12 Mar 2010 14:38:11 -0800 From: Pyun YongHyeon Date: Fri, 12 Mar 2010 14:38:11 -0800 To: Pierre Beyssac Message-ID: <20100312223811.GS16314@michelle.cdnetworks.com> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312174655.GG16314@michelle.cdnetworks.com> <20100312182134.GA90606@fasterix.frmug.org> <20100312192144.GL16314@michelle.cdnetworks.com> <20100312194524.GB91694@fasterix.frmug.org> <20100312200224.GO16314@michelle.cdnetworks.com> <20100312202852.GD91694@fasterix.frmug.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100312202852.GD91694@fasterix.frmug.org> User-Agent: Mutt/1.4.2.3i Cc: freebsd-stable@freebsd.org Subject: Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 22:38:16 -0000 On Fri, Mar 12, 2010 at 09:28:52PM +0100, Pierre Beyssac wrote: > On Fri, Mar 12, 2010 at 12:02:24PM -0800, Pyun YongHyeon wrote: > > Hmm, try this one and let me know it make any differences. > > No, still the same, negotiates at 10baseT/UTP. It seems the PHY has no BRGPHY_MII_AUXSTS register as it does not seem to manufactured by Broadcom. Adding a special case to brgphy(4) does not look right. Does ukphy(4) work without problem? If so I think you can live with ukphy(4). From owner-freebsd-stable@FreeBSD.ORG Fri Mar 12 22:53:30 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7F5A9106566C; Fri, 12 Mar 2010 22:53:30 +0000 (UTC) (envelope-from jamesbrandongooch@gmail.com) Received: from mail-ew0-f228.google.com (mail-ew0-f228.google.com [209.85.219.228]) by mx1.freebsd.org (Postfix) with ESMTP id BD8748FC0C; Fri, 12 Mar 2010 22:53:29 +0000 (UTC) Received: by ewy28 with SMTP id 28so271405ewy.13 for ; Fri, 12 Mar 2010 14:53:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=THDrcVFcZmXPNWgF4Xy8V8vvvDqsla9vuc0KVmoJlTQ=; b=VjaaVZYcgryxOBsXy+2lWSgRqZPbbKvXK0GDH/N/VYY/7gXsciWQjnSnkPZTD7agfN OO3VYkzKJowGAXKLKyUWLQpTVupjZT70p27oFna+GFHdt75pf7tu6k1io2kQo+LS53E9 FyKkMAOpuzVwpdkS6HzYsdvhs7QBtqYlYJwW8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=dcjCG6oAsLSY8P2Z37n60dctdGPQM2UZfbq/mkM/9uftehbl+6J+9YLRD+jKuSlrgR /aub2rKrkxq4/vA6A9ewJVsipB3qXnC+hPmClo0cpRQzJh8A99OJbFFEuGmZBeLhR/HG Q4/NAYPrp4eWAqMqg/0kMNKo03teWcnRBhdMI= MIME-Version: 1.0 Received: by 10.213.66.1 with SMTP id l1mr776781ebi.91.1268434408525; Fri, 12 Mar 2010 14:53:28 -0800 (PST) In-Reply-To: <4B9A8B92.6050704@comcast.net> References: <4B9A8B92.6050704@comcast.net> Date: Fri, 12 Mar 2010 16:53:28 -0600 Message-ID: <179b97fb1003121453r5f6bec26wdc39550115f32af0@mail.gmail.com> From: Brandon Gooch To: Steve Polyack Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-stable , freebsd-rc@freebsd.org, freebsd-docs@freebsd.org Subject: Re: rc.d/rc.subr support for multiple FIBs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2010 22:53:30 -0000 On Fri, Mar 12, 2010 at 12:44 PM, Steve Polyack wrote: > With multiple FIB support generally available in FreeBSD 8.0-RELEASE it > would be quite beneficial to have the ability to build routing tables in > secondary FIBs as well as start certain applications in certain FIBs from > within rc.conf(5). > > I've done some poking around and came across two PRs which implement exac= tly > what I'm looking for: > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D132483 > http://www.freebsd.org/cgi/query-pr.cgi?pr=3D132476 > > My question is whether there are any plans to commit these into -CURRENT = and > possibly MFC them to a future 8.x-RELEASE. =A0Having multiple FIBs availa= ble > has been great so far, it goes hand and hand with Multi-IP Jails. =A0The = only > thing missing are native methods for constructing the routing tables on b= oot > (Yes, there is rc.local, but I don't want to go there...). > > Thanks, > Steve Polyack > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > I too would love to see this integrated; I've hacked together my own set of rc scripts (elegant in no way) to accomplish something similar on my systems. Also, general documentation regarding multiple FIBs (if that's what they are to be referred to as) in the official docs would be cool -- I'm not knowledgeable enough to do it myself (although I'm a pretty decent hand at "Documentation Testing" :) -Brandon From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 00:26:38 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4F404106564A for ; Sat, 13 Mar 2010 00:26:38 +0000 (UTC) (envelope-from weongyo.jeong@gmail.com) Received: from mail-bw0-f216.google.com (mail-bw0-f216.google.com [209.85.218.216]) by mx1.freebsd.org (Postfix) with ESMTP id CC2508FC1B for ; Sat, 13 Mar 2010 00:26:37 +0000 (UTC) Received: by bwz8 with SMTP id 8so1535851bwz.3 for ; Fri, 12 Mar 2010 16:26:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:mail-followup-to:references :mime-version:content-type:content-disposition:in-reply-to :user-agent:organization:x-operation-sytem; bh=ZLrqulF7OcG7DvpHgow0nfiKWyB/gOeYE6jDgs+Q7a4=; b=h2kQG5VJiiOtnywMnHt2Ymz4Ip1mqb1PtvFUd6vD6ZaL/a8w9LDnypTcEcrC3HD5cv NDf9GpSeZ5nme+pFuFxAX0bSKQXiHUxNZQ/PptnzHX3Z6oUMgD2j0fiVuO75Ok/8hLrp RvM1jmvnyUXJyENnychtz57GEL1Flfx0m3Z9c= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:mail-followup-to :references:mime-version:content-type:content-disposition :in-reply-to:user-agent:organization:x-operation-sytem; b=iKru8dtLtnvSWvPzf1+EJ14EW7P8eZ+JIw5LCq6YsKs6LzJNNGXYg0co7l5tfy6u4e AKikPIKSq3Jge9FB3fqjkjEM4sddqDxbMDjI43biTjEfPkzu4u3zf5rFK/h/tDnJcpIj CNPeSVHoi2FKj66REd7jdXvVKYyhWom4aA8Y8= Received: by 10.204.35.1 with SMTP id n1mr429614bkd.98.1268438574244; Fri, 12 Mar 2010 16:02:54 -0800 (PST) Received: from weongyo ([174.35.1.224]) by mx.google.com with ESMTPS id 24sm7485651bkr.18.2010.03.12.16.02.50 (version=SSLv3 cipher=RC4-MD5); Fri, 12 Mar 2010 16:02:52 -0800 (PST) Received: by weongyo (sSMTP sendmail emulation); Fri, 12 Mar 2010 16:03:04 -0800 From: Weongyo Jeong Date: Fri, 12 Mar 2010 16:03:03 -0800 To: Jeff Dowsley Message-ID: <20100313000303.GA1295@weongyo> Mail-Followup-To: Jeff Dowsley , freebsd-stable@freebsd.org References: <1AAFC89E-D1B3-49A7-805F-8DA753C4934E@mac.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1AAFC89E-D1B3-49A7-805F-8DA753C4934E@mac.com> User-Agent: Mutt/1.4.2.3i Organization: CDNetworks. X-Operation-Sytem: FreeBSD Cc: freebsd-stable@freebsd.org Subject: Re: Broadcom USB wireless support? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Weongyo Jeong List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 00:26:38 -0000 On Tue, Feb 16, 2010 at 11:39:53AM +1100, Jeff Dowsley wrote: > Gentles > > I have an old HP Pavillion DV6000 laptop, which has a Broadcom USB > wireless device. Worked under Windows Vista. I installed freeBSD 8- > stable, and see as the last line in dmesg > > ugen2.2: at usbus2 > > Ferreting with google suggests that 8.0 might have usb support for > the ndis wrapper, but I am unable to get any joy either using the HP > bcmwl5 drivers for the DV6000, or in attempting to use the bwi driver. FYI recently OpenBSD commited urndis(4) driver into their tree that looks it could support your device if the device uses Broadcom 4320 chipset. But it looks nobody is working on it to port to FreeBSD. regards, Weongyo Jeong From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 13:11:17 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 42CE5106564A for ; Sat, 13 Mar 2010 13:11:17 +0000 (UTC) (envelope-from torfinn.ingolfsen@broadpark.no) Received: from eterpe-smout.broadpark.no (eterpe-smout.broadpark.no [80.202.8.16]) by mx1.freebsd.org (Postfix) with ESMTP id F21228FC0A for ; Sat, 13 Mar 2010 13:11:15 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from ignis-smin.broadpark.no ([unknown] [80.202.8.11]) by eterpe-smout.broadpark.no (Sun Java(tm) System Messaging Server 7u3-12.01 64bit (built Oct 15 2009)) with ESMTP id <0KZ8005RD1YQJ000@eterpe-smout.broadpark.no> for freebsd-stable@freebsd.org; Sat, 13 Mar 2010 14:11:14 +0100 (CET) Received: from kg-v2.kg4.no ([unknown] [80.203.92.186]) by ignis-smin.broadpark.no (Sun Java(tm) System Messaging Server 7u3-12.01 64bit (built Oct 15 2009)) with SMTP id <0KZ8006IX1YPPV70@ignis-smin.broadpark.no> for freebsd-stable@freebsd.org; Sat, 13 Mar 2010 14:11:14 +0100 (CET) Date: Sat, 13 Mar 2010 14:11:13 +0100 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Message-id: <20100313141113.cd215e62.torfinn.ingolfsen@broadpark.no> In-reply-to: <20100307122913.2f634018.torfinn.ingolfsen@broadpark.no> References: <20100131144217.ca08e965.torfinn.ingolfsen@broadpark.no> <20100306141944.95ec8cb6.torfinn.ingolfsen@broadpark.no> <20100307122913.2f634018.torfinn.ingolfsen@broadpark.no> X-Mailer: Sylpheed 2.7.1 (GTK+ 2.18.7; amd64-portbld-freebsd8.0) X-Face: "t9w2,-X@O^I`jVW\sonI3.,36KBLZE*AL[y9lL[PyFD*r_S:dIL9c[8Y>V42R0"!"yb_zN,f#%.[PYYNq; m"_0v; ~rUM2Yy!zmkh)3&U|u!=T(zyv,MHJv"nDH>OJ`t(@mil461d_B'Uo|'nMwlKe0Mv=kvV?Nh@>Hb<3s_z2jYgZhPb@?Wi^x1a~Hplz1.zH Subject: Re: panic - sleeping thread on FreeBSD 8.0-stable / amd64 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 13:11:17 -0000 Update: Perhaps the problems I have with this machine[1] are related to the MB-455SPF[1] hard drive cage I am using. It has three power connectors, which distributes power to all five drives. The documentation doesn't say more about which ports gives power to which drives, but since it is always drive 2 and 3 in the cage (ata5 and 6) that I've had problems with, I rearranged power connections to the cage. We will see if that makes any difference. References: 1) http://sites.google.com/site/tingox/ga-ma74gm-s2h 2) http://www.icydock.com/product/mb455spf.html -- Torfinn From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 13:34:42 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 25C88106564A for ; Sat, 13 Mar 2010 13:34:42 +0000 (UTC) (envelope-from pb@fasterix.frmug.org) Received: from fasterix.frmug.org (unknown [IPv6:2a01:e35:8b50:2c40::2]) by mx1.freebsd.org (Postfix) with ESMTP id 9F8E88FC18 for ; Sat, 13 Mar 2010 13:34:41 +0000 (UTC) Received: from fasterix.frmug.org (localhost [127.0.0.1]) by fasterix.frmug.org (8.14.3/8.14.3) with ESMTP id o2DDYedM024648 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 13 Mar 2010 14:34:40 +0100 (CET) (envelope-from pb@fasterix.frmug.org) Received: (from pb@localhost) by fasterix.frmug.org (8.14.3/8.14.3/Submit) id o2DDYerM024647; Sat, 13 Mar 2010 14:34:40 +0100 (CET) (envelope-from pb) Date: Sat, 13 Mar 2010 14:34:40 +0100 From: Pierre Beyssac To: Pyun YongHyeon Message-ID: <20100313133439.GA24616@fasterix.frmug.org> References: <20100312121409.GA79294@fasterix.frmug.org> <20100312174655.GG16314@michelle.cdnetworks.com> <20100312182134.GA90606@fasterix.frmug.org> <20100312192144.GL16314@michelle.cdnetworks.com> <20100312194524.GB91694@fasterix.frmug.org> <20100312200224.GO16314@michelle.cdnetworks.com> <20100312202852.GD91694@fasterix.frmug.org> <20100312223811.GS16314@michelle.cdnetworks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100312223811.GS16314@michelle.cdnetworks.com> User-Agent: Mutt/1.4.2.3i X-message-flag: Warning! Use of Microsoft Outlook is dangerous and makes your system susceptible to worms and viruses Cc: freebsd-stable@freebsd.org Subject: Re: 8-STABLE interrupt storm on atapci(?), Dell Inspiron 580 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 13:34:42 -0000 On Fri, Mar 12, 2010 at 02:38:11PM -0800, Pyun YongHyeon wrote: > It seems the PHY has no BRGPHY_MII_AUXSTS register as it does not > seem to manufactured by Broadcom. Adding a special case to > brgphy(4) does not look right. Does ukphy(4) work without problem? > If so I think you can live with ukphy(4). Oh yes, it works like a charm, no worry. -- Pierre Beyssac pb@fasterix.frmug.org From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 15:57:35 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBF43106566C; Sat, 13 Mar 2010 15:57:34 +0000 (UTC) (envelope-from svein-listmail@stillbilde.net) Received: from mail.stillbilde.net (unknown [IPv6:2002:51af:3dc3:0:20c:29ff:fece:79f3]) by mx1.freebsd.org (Postfix) with ESMTP id 629108FC12; Sat, 13 Mar 2010 15:57:34 +0000 (UTC) Received: from [IPv6:2002:51af:3dc3:0:79cc:a126:7e59:782f] (unknown [IPv6:2002:51af:3dc3:0:79cc:a126:7e59:782f]) (Authenticated sender: svein-listmail) by mail.stillbilde.net (Familien Skogens mail) with ESMTPSA id EE59F23; Sat, 13 Mar 2010 16:57:53 +0100 (CET) Message-ID: <4B9BB600.5080200@stillbilde.net> Date: Sat, 13 Mar 2010 16:57:52 +0100 From: "Svein Skogen (Listmail Account)" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.8) Gecko/20100227 Lightning/1.0b1 Thunderbird/3.0.3 MIME-Version: 1.0 To: freebsd-questions@freebsd.org, "Stable, FreeBSD" , current@FreeBSD.org References: <4B9A4D9D.6040805@stillbilde.net> In-Reply-To: <4B9A4D9D.6040805@stillbilde.net> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Amanda, FreeBSD8, amtype, hairpulling, etc. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 15:57:35 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12.03.2010 15:20, Svein Skogen wrote: > I'm having trouble getting Amanda (2.6.1p2 from ports) to play nicely > with my hardware. > > Devices are: > at scbus0 target 1 lun 0 (sa0,pass0) > at scbus0 target 1 lun 1 (pass1,ch0) > connected via: > mpt0: port 0x9000-0x90ff mem > 0xfe4fc000-0xfe4fffff,0xfe4e0000-0xfe4effff irq 16 at device 0.0 on pci2 > mpt0: [ITHREAD] > mpt0: MPI Version=1.5.20.0 > > os and number in question is: > FreeBSD storage.stillbilde.net 8.0-STABLE FreeBSD 8.0-STABLE #0: Tue Mar > 9 07:01:59 UTC 2010 > svein@storage.stillbilde.net:/usr/obj/usr/src/sys/GENERIC amd64 > > Tapes are regular LTO-3 (HP C7973A). > > amtapetype simply hangs (after writing 3-4 tps for 5 seconds, then > simple silence both on sa0 and console) > > Has anybody run into this problem with FreeBSD8+mpt+autoloader? > > //Svein > I'm finally starting to make sense of what I'm seeing (hence the crossposting) Seems I've stumbled onto some strange incompatibility between my SAS controller (LSI 3801E), Tape-library, and FreeBSD+Solaris. The behavior I'm seeing is consistent with this solaris bug: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6894775 but luckily (?) for me, my disks are on an MFI controller. I'm seeing the exact same behavior in both FreeBSD and OpenSolaris 133 and 134. Linux and Windows installs on the box (this box is currently being set up, so I'm rather liberated from reinstall-concerns) seems unaffected, atleast the HP Software doesn't fail the way tar/dump/dd/amanda/bacula/whatnot does at random intervals. The errors only occur when the device I'm reading/writing from is 100% laoded (reading or writing 56mb/sec + compression), which when fed from a raid capable of more than 6 times that is quite likely to happen during backups. The Solaris bug seems to be around MSI handling, but there are several reports "over there" about this error occuring even with MSI disabled. Right now I'm dumbstruck about this, and might install Linux "just to get backups up and running this year", because I'm too tired of this entire process, but I'd REALLY rather run FreeBSD or OpenSolaris. This is based on a personal preference and nothing else, but if anybody has some blinding insights on how to get this working, I'm open for suggestions. //Svein - -- - --------+-------------------+------------------------------- /"\ |Svein Skogen | svein@d80.iso100.no \ / |Solberg Østli 9 | PGP Key: 0xE5E76831 X |2020 Skedsmokorset | svein@jernhuset.no / \ |Norway | PGP Key: 0xCE96CE13 | | svein@stillbilde.net ascii | | PGP Key: 0x58CD33B6 ribbon |System Admin | svein-listmail@stillbilde.net Campaign|stillbilde.net | PGP Key: 0x22D494A4 +-------------------+------------------------------- |msn messenger: | Mobile Phone: +47 907 03 575 |svein@jernhuset.no | RIPE handle: SS16503-RIPE - --------+-------------------+------------------------------- If you really are in a hurry, mail me at svein-mobile@stillbilde.net This mailbox goes directly to my cellphone and is checked even when I'm not in front of my computer. - ------------------------------------------------------------ Picture Gallery: https://gallery.stillbilde.net/v/svein/ - ------------------------------------------------------------ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkubtf8ACgkQODUnwSLUlKTfGgCgndwhQdAjFjlUp2jCh5POr0jp 0rYAoKUsR2AjzlBCM/eqMfUjGfKtWXof =2lnD -----END PGP SIGNATURE----- From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 16:38:57 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DC0A106564A for ; Sat, 13 Mar 2010 16:38:57 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta11.emeryville.ca.mail.comcast.net (qmta11.emeryville.ca.mail.comcast.net [76.96.27.211]) by mx1.freebsd.org (Postfix) with ESMTP id 236C48FC08 for ; Sat, 13 Mar 2010 16:38:56 +0000 (UTC) Received: from omta18.emeryville.ca.mail.comcast.net ([76.96.30.74]) by qmta11.emeryville.ca.mail.comcast.net with comcast id sfwv1d00C1bwxycABgaVrR; Sat, 13 Mar 2010 16:34:29 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta18.emeryville.ca.mail.comcast.net with comcast id sgi31d0023S48mS8egi8Nq; Sat, 13 Mar 2010 16:42:09 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 275B69B436; Sat, 13 Mar 2010 08:38:48 -0800 (PST) Date: Sat, 13 Mar 2010 08:38:48 -0800 From: Jeremy Chadwick To: Torfinn Ingolfsen Message-ID: <20100313163848.GA94056@icarus.home.lan> References: <20100131144217.ca08e965.torfinn.ingolfsen@broadpark.no> <20100306141944.95ec8cb6.torfinn.ingolfsen@broadpark.no> <20100307122913.2f634018.torfinn.ingolfsen@broadpark.no> <20100313141113.cd215e62.torfinn.ingolfsen@broadpark.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100313141113.cd215e62.torfinn.ingolfsen@broadpark.no> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-stable@freebsd.org Subject: Re: panic - sleeping thread on FreeBSD 8.0-stable / amd64 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 16:38:57 -0000 On Sat, Mar 13, 2010 at 02:11:13PM +0100, Torfinn Ingolfsen wrote: > Update: > Perhaps the problems I have with this machine[1] are related to the > MB-455SPF[1] hard drive cage I am using. It has three power connectors, > which distributes power to all five drives. The documentation doesn't > say more about which ports gives power to which drives, but since it is > always drive 2 and 3 in the cage (ata5 and 6) that I've had problems > with, I rearranged power connections to the cage. We will see > if that makes any difference. > > > References: > 1) http://sites.google.com/site/tingox/ga-ma74gm-s2h > 2) http://www.icydock.com/product/mb455spf.html That enclosure also doesn't state if it has a SAF-TE or SES-2 chip on it. It's impossible to tell from the photos since the metallic enclosure cover up the backplane. If it does, such chips can/will yank devices off the bus when the chip considers the drive "faulty". At least in the SCSI world with SES-2, I've seen QLogic GEM318 and GEM354 chips do this. Here's an example of what I'm referring to: http://www.supermicro.com/manuals/chassis/mobile_rack/CSE-M34.pdf -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 17:33:01 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 90FF610656F2 for ; Sat, 13 Mar 2010 17:33:01 +0000 (UTC) (envelope-from torfinn.ingolfsen@broadpark.no) Received: from eterpe-smout.broadpark.no (eterpe-smout.broadpark.no [80.202.8.16]) by mx1.freebsd.org (Postfix) with ESMTP id 4AD218FC1B for ; Sat, 13 Mar 2010 17:33:01 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from ignis-smin.broadpark.no ([unknown] [80.202.8.11]) by eterpe-smout.broadpark.no (Sun Java(tm) System Messaging Server 7u3-12.01 64bit (built Oct 15 2009)) with ESMTP id <0KZ8009UPE305G00@eterpe-smout.broadpark.no> for freebsd-stable@freebsd.org; Sat, 13 Mar 2010 18:33:00 +0100 (CET) Received: from kg-v2.kg4.no ([unknown] [80.203.92.186]) by ignis-smin.broadpark.no (Sun Java(tm) System Messaging Server 7u3-12.01 64bit (built Oct 15 2009)) with SMTP id <0KZ800C9EE2Z75E0@ignis-smin.broadpark.no> for freebsd-stable@freebsd.org; Sat, 13 Mar 2010 18:33:00 +0100 (CET) Date: Sat, 13 Mar 2010 18:32:59 +0100 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Message-id: <20100313183259.40fe1db2.torfinn.ingolfsen@broadpark.no> In-reply-to: <20100313163848.GA94056@icarus.home.lan> References: <20100131144217.ca08e965.torfinn.ingolfsen@broadpark.no> <20100306141944.95ec8cb6.torfinn.ingolfsen@broadpark.no> <20100307122913.2f634018.torfinn.ingolfsen@broadpark.no> <20100313141113.cd215e62.torfinn.ingolfsen@broadpark.no> <20100313163848.GA94056@icarus.home.lan> X-Mailer: Sylpheed 2.7.1 (GTK+ 2.18.7; amd64-portbld-freebsd8.0) X-Face: "t9w2,-X@O^I`jVW\sonI3.,36KBLZE*AL[y9lL[PyFD*r_S:dIL9c[8Y>V42R0"!"yb_zN,f#%.[PYYNq; m"_0v; ~rUM2Yy!zmkh)3&U|u!=T(zyv,MHJv"nDH>OJ`t(@mil461d_B'Uo|'nMwlKe0Mv=kvV?Nh@>Hb<3s_z2jYgZhPb@?Wi^x1a~Hplz1.zH Subject: Re: panic - sleeping thread on FreeBSD 8.0-stable / amd64 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 17:33:01 -0000 On Sat, 13 Mar 2010 08:38:48 -0800 Jeremy Chadwick wrote: > That enclosure also doesn't state if it has a SAF-TE or SES-2 chip on > it. It's impossible to tell from the photos since the metallic > enclosure cover up the backplane. Unfortunately, the "manual" is very brief and lacks such technical details / specs. This is a very simple enclosure; I don't think it even has any sensors except for the fan and temperature sensors, which are available at the signal out connector on the backplane. > If it does, such chips can/will yank devices off the bus when the chip > considers the drive "faulty". At least in the SCSI world with SES-2, Well, the manual states that this is a host function: 2.2 HDD FAIL system 2.2.1 This product can not provide the signal, unless the HOST supports this function. 2.2.2 If the Host can provide HDD fail signal, connect the cable to the HOST and to the backplane HDD FAIL 1 - 2 - 3 - 4 - 5. 2.2.3 When the HDD is damaged, the light will show in red. There is a separate connector for HDD fail input signals on the backplane. There are also output signals for power, access for each drive. The only thing I have connected the backplane is power (3 connectors) and sata (5 connectors). Since each drive has its own sata connector there really shouldn't be any reason to disconnect drives, but I guess it is possible. -- Torfinn From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 17:47:24 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2D021065674 for ; Sat, 13 Mar 2010 17:47:23 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta15.emeryville.ca.mail.comcast.net (qmta15.emeryville.ca.mail.comcast.net [76.96.27.228]) by mx1.freebsd.org (Postfix) with ESMTP id A5A5B8FC1D for ; Sat, 13 Mar 2010 17:47:23 +0000 (UTC) Received: from omta19.emeryville.ca.mail.comcast.net ([76.96.30.76]) by qmta15.emeryville.ca.mail.comcast.net with comcast id sg3x1d0061eYJf8AFhnQkw; Sat, 13 Mar 2010 17:47:24 +0000 Received: from koitsu.dyndns.org ([98.248.46.159]) by omta19.emeryville.ca.mail.comcast.net with comcast id shnP1d0033S48mS01hnPKj; Sat, 13 Mar 2010 17:47:23 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 2BA779B436; Sat, 13 Mar 2010 09:47:22 -0800 (PST) Date: Sat, 13 Mar 2010 09:47:22 -0800 From: Jeremy Chadwick To: Torfinn Ingolfsen Message-ID: <20100313174722.GA95395@icarus.home.lan> References: <20100131144217.ca08e965.torfinn.ingolfsen@broadpark.no> <20100306141944.95ec8cb6.torfinn.ingolfsen@broadpark.no> <20100307122913.2f634018.torfinn.ingolfsen@broadpark.no> <20100313141113.cd215e62.torfinn.ingolfsen@broadpark.no> <20100313163848.GA94056@icarus.home.lan> <20100313183259.40fe1db2.torfinn.ingolfsen@broadpark.no> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100313183259.40fe1db2.torfinn.ingolfsen@broadpark.no> User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-stable@freebsd.org Subject: Re: panic - sleeping thread on FreeBSD 8.0-stable / amd64 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 17:47:24 -0000 On Sat, Mar 13, 2010 at 06:32:59PM +0100, Torfinn Ingolfsen wrote: > On Sat, 13 Mar 2010 08:38:48 -0800 > Jeremy Chadwick wrote: > > > That enclosure also doesn't state if it has a SAF-TE or SES-2 chip on > > it. It's impossible to tell from the photos since the metallic > > enclosure cover up the backplane. > > Unfortunately, the "manual" is very brief and lacks such technical details / specs. > This is a very simple enclosure; I don't think it even has any sensors except for the fan and temperature sensors, > which are available at the signal out connector on the backplane. ...which is probably wired to some sort of interface chip that would toggle a buzzer/etc. if something goes awry. SAF-TE and/or SES-2 provide this capability, but so can a very simple circuit. How do you tune the thresholds for the temperature or fan? If they're DIP switches, then chances are SAF-TE or SES-2 aren't involved and it's probably just some cheap/generic logic chip that does the work. > > If it does, such chips can/will yank devices off the bus when the chip > > considers the drive "faulty". At least in the SCSI world with SES-2, > > Well, the manual states that this is a host function: > > 2.2 HDD FAIL system > 2.2.1 This product can not provide the signal, unless the HOST supports this function. > 2.2.2 If the Host can provide HDD fail signal, connect the cable to the HOST and to the backplane HDD FAIL 1 - 2 - 3 - 4 - 5. > 2.2.3 When the HDD is damaged, the light will show in red. > > There is a separate connector for HDD fail input signals on the backplane. There are also output signals for power, access for each drive. Is it a 4-pin connector? If so, what the manual is (horribly) trying to document is probably an SGPIO connector. SGPIO is a 4-pin connector which connects a SATA/SAS enclosure to a SATA/SAS HBA (controller) so that the two have a direct way to signal that a disk has physically failed -- otherwise, the controller has to make guesses about the state of things, and timeouts can take a while. SGPIO can also be used to provide other things; it's a generic communication interface with an official specification. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 18:40:41 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F314D106564A; Sat, 13 Mar 2010 18:40:40 +0000 (UTC) (envelope-from dak.col@gmail.com) Received: from mail-iw0-f185.google.com (mail-iw0-f185.google.com [209.85.223.185]) by mx1.freebsd.org (Postfix) with ESMTP id A993A8FC21; Sat, 13 Mar 2010 18:40:40 +0000 (UTC) Received: by iwn15 with SMTP id 15so2147734iwn.7 for ; Sat, 13 Mar 2010 10:40:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type; bh=thIBgUJ59dCvLSKZ6GjQoyTZFBDPRM6yIl/IXHRbBVo=; b=EJ7jHCtAljwnq8cwiVviZ64bT3HWEiKi7ABuhwfd20sO/N3cBosll+z8Qf6zua9pON mO7HowhTwn06nPOImawld76VmcsxHaBF8dCJGzqvrGND/bizam1LCbgiVo9ARPqAVvwQ 75ra72hk1KWCuOFUBitqAk3GCCY03BcMXR5iE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=UOywl5DwLZx23bMUxC/mlnSoAfWkIZouG9jY/S83cQt9zcA+DQH2Qd+JvJgQ9y50iY CDyi8c9jdO1PmAqeWi/T7ymR3la8TZXG2hNmP/Ws4wsiL7/dBi/YphOvEmaCLY6E48tK y07Icfyd2WgQ9TfT9vszmQ6q1sbA0U5qrUqZc= MIME-Version: 1.0 Received: by 10.231.190.204 with SMTP id dj12mr1612646ibb.9.1268503919634; Sat, 13 Mar 2010 10:11:59 -0800 (PST) In-Reply-To: <4B9BB600.5080200@stillbilde.net> References: <4B9A4D9D.6040805@stillbilde.net> <4B9BB600.5080200@stillbilde.net> Date: Sat, 13 Mar 2010 13:11:59 -0500 Message-ID: <3b93bd111003131011i8bba45tf7258d92b63b562@mail.gmail.com> From: "Diego F. Arias R." To: "Svein Skogen (Listmail Account)" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: "Stable, FreeBSD" , freebsd-questions@freebsd.org, current@freebsd.org Subject: Re: Amanda, FreeBSD8, amtype, hairpulling, etc. X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 18:40:41 -0000 On Sat, Mar 13, 2010 at 10:57 AM, Svein Skogen (Listmail Account) < svein-listmail@stillbilde.net> wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 12.03.2010 15:20, Svein Skogen wrote: > > I'm having trouble getting Amanda (2.6.1p2 from ports) to play nicely > > with my hardware. > > > > Devices are: > > at scbus0 target 1 lun 0 (sa0,pass0) > > at scbus0 target 1 lun 1 (pass1,ch0) > > connected via: > > mpt0: port 0x9000-0x90ff mem > > 0xfe4fc000-0xfe4fffff,0xfe4e0000-0xfe4effff irq 16 at device 0.0 on pci2 > > mpt0: [ITHREAD] > > mpt0: MPI Version=1.5.20.0 > > > > os and number in question is: > > FreeBSD storage.stillbilde.net 8.0-STABLE FreeBSD 8.0-STABLE #0: Tue Mar > > 9 07:01:59 UTC 2010 > > svein@storage.stillbilde.net:/usr/obj/usr/src/sys/GENERIC amd64 > > > > Tapes are regular LTO-3 (HP C7973A). > > > > amtapetype simply hangs (after writing 3-4 tps for 5 seconds, then > > simple silence both on sa0 and console) > > > > Has anybody run into this problem with FreeBSD8+mpt+autoloader? > > > > //Svein > > > > I'm finally starting to make sense of what I'm seeing (hence the > crossposting) > > Seems I've stumbled onto some strange incompatibility between my SAS > controller (LSI 3801E), Tape-library, and FreeBSD+Solaris. > > The behavior I'm seeing is consistent with this solaris bug: > > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6894775 > > but luckily (?) for me, my disks are on an MFI controller. I'm seeing > the exact same behavior in both FreeBSD and OpenSolaris 133 and 134. > > Linux and Windows installs on the box (this box is currently being set > up, so I'm rather liberated from reinstall-concerns) seems unaffected, > atleast the HP Software doesn't fail the way > tar/dump/dd/amanda/bacula/whatnot does at random intervals. > > The errors only occur when the device I'm reading/writing from is 100% > laoded (reading or writing 56mb/sec + compression), which when fed from > a raid capable of more than 6 times that is quite likely to happen > during backups. > > The Solaris bug seems to be around MSI handling, but there are several > reports "over there" about this error occuring even with MSI disabled. > > Right now I'm dumbstruck about this, and might install Linux "just to > get backups up and running this year", because I'm too tired of this > entire process, but I'd REALLY rather run FreeBSD or OpenSolaris. This > is based on a personal preference and nothing else, but if anybody has > some blinding insights on how to get this working, I'm open for > suggestions. > > if there is a hardware/driver bug isnt possible to get another cheap sas controller for your tape, maybe a second hand? -- mmm, interesante..... From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 19:39:52 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DD77B106566C for ; Sat, 13 Mar 2010 19:39:52 +0000 (UTC) (envelope-from torfinn.ingolfsen@broadpark.no) Received: from eterpe-smout.broadpark.no (eterpe-smout.broadpark.no [80.202.8.16]) by mx1.freebsd.org (Postfix) with ESMTP id 9555A8FC12 for ; Sat, 13 Mar 2010 19:39:52 +0000 (UTC) MIME-version: 1.0 Content-transfer-encoding: 7BIT Content-type: text/plain; CHARSET=US-ASCII Received: from ignis-smin.broadpark.no ([unknown] [80.202.8.11]) by eterpe-smout.broadpark.no (Sun Java(tm) System Messaging Server 7u3-12.01 64bit (built Oct 15 2009)) with ESMTP id <0KZ8009V1JYF5GA0@eterpe-smout.broadpark.no> for freebsd-stable@freebsd.org; Sat, 13 Mar 2010 20:39:51 +0100 (CET) Received: from kg-v2.kg4.no ([unknown] [80.203.92.186]) by ignis-smin.broadpark.no (Sun Java(tm) System Messaging Server 7u3-12.01 64bit (built Oct 15 2009)) with SMTP id <0KZ800H9FJYE8330@ignis-smin.broadpark.no> for freebsd-stable@freebsd.org; Sat, 13 Mar 2010 20:39:51 +0100 (CET) Date: Sat, 13 Mar 2010 20:39:50 +0100 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Message-id: <20100313203950.ac227fc3.torfinn.ingolfsen@broadpark.no> In-reply-to: <20100313174722.GA95395@icarus.home.lan> References: <20100131144217.ca08e965.torfinn.ingolfsen@broadpark.no> <20100306141944.95ec8cb6.torfinn.ingolfsen@broadpark.no> <20100307122913.2f634018.torfinn.ingolfsen@broadpark.no> <20100313141113.cd215e62.torfinn.ingolfsen@broadpark.no> <20100313163848.GA94056@icarus.home.lan> <20100313183259.40fe1db2.torfinn.ingolfsen@broadpark.no> <20100313174722.GA95395@icarus.home.lan> X-Mailer: Sylpheed 2.7.1 (GTK+ 2.18.7; amd64-portbld-freebsd8.0) X-Face: "t9w2,-X@O^I`jVW\sonI3.,36KBLZE*AL[y9lL[PyFD*r_S:dIL9c[8Y>V42R0"!"yb_zN,f#%.[PYYNq; m"_0v; ~rUM2Yy!zmkh)3&U|u!=T(zyv,MHJv"nDH>OJ`t(@mil461d_B'Uo|'nMwlKe0Mv=kvV?Nh@>Hb<3s_z2jYgZhPb@?Wi^x1a~Hplz1.zH Subject: Re: panic - sleeping thread on FreeBSD 8.0-stable / amd64 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 19:39:52 -0000 On Sat, 13 Mar 2010 09:47:22 -0800 Jeremy Chadwick wrote: > How do you tune the thresholds for the temperature or fan? If they're > DIP switches, then chances are SAF-TE or SES-2 aren't involved and it's > probably just some cheap/generic logic chip that does the work. There is a switch for the temp. The fan can't be adjusted. Luckily it is quite quiet. > Is it a 4-pin connector? If so, what the manual is (horribly) trying to > document is probably an SGPIO connector. Nope, eight pin. I've scanned the manual and put it up at the page for the machine[1], in case anyone have a need for it. > SGPIO is a 4-pin connector which connects a SATA/SAS enclosure to a > SATA/SAS HBA (controller) so that the two have a direct way to signal > that a disk has physically failed -- otherwise, the controller has to > make guesses about the state of things, and timeouts can take a while. > > SGPIO can also be used to provide other things; it's a generic > communication interface with an official specification. This one seems much more simple; one signal (two pins) per drive. References: 1) http://sites.google.com/site/tingox/ga-ma74gm-s2h -- Torfinn From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 21:28:13 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D834C106566B; Sat, 13 Mar 2010 21:28:13 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from host.omnilan.net (host.omnilan.net [62.245.232.135]) by mx1.freebsd.org (Postfix) with ESMTP id C10588FC19; Sat, 13 Mar 2010 21:28:12 +0000 (UTC) Received: from [172.21.1.33] (akima-win.flintsbach.schmalzbauer.de [172.21.1.33]) (authenticated bits=0) by host.omnilan.net (8.13.8/8.13.8) with ESMTP id o2DLS7Ki025275 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 13 Mar 2010 22:28:11 +0100 (CET) (envelope-from h.schmalzbauer@omnilan.de) X-Authentication-Warning: smtp.dmz.omnilan.net: Host akima-win.flintsbach.schmalzbauer.de [172.21.1.33] claimed to be [172.21.1.33] Message-ID: <4B9C034B.90900@omnilan.de> Date: Sat, 13 Mar 2010 22:27:39 +0100 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; de; rv:1.9.1.5) Gecko/20091204 Lightning/1.0b1 Thunderbird/3.0 MIME-Version: 1.0 To: Jeremy Chadwick References: <1266934981.00222684.1266922202@10.7.7.3> <4B83EFD4.8050403@FreeBSD.org> <4B8E1489.2070306@omnilan.de> <4B8E1B3D.306@FreeBSD.org> <4B8E1DA9.2090406@omnilan.de> <20100303110647.GA51588@icarus.home.lan> In-Reply-To: <20100303110647.GA51588@icarus.home.lan> X-Enigmail-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig99F70BB8ABD28A47139A7119" Cc: Alexander Motin , freebsd-stable@freebsd.org Subject: Re: ahcich timeouts, only with ahci, not with ataahci X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 21:28:14 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig99F70BB8ABD28A47139A7119 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Am 03.03.2010 12:06, schrieb Jeremy Chadwick: > On Wed, Mar 03, 2010 at 09:28:25AM +0100, Harald Schmalzbauer wrote: >> Alexander Motin schrieb am 03.03.2010 09:18 (localtime): >>> Harald Schmalzbauer wrote: >>>> Alexander Motin schrieb am 23.02.2010 16:10 (localtime): >>>>> Harald Schmalzbauer wrote: >>>>>> I'm frequently getting my machine locked with ahcichX timeouts: >>>>>> ahcich2: Timeout on slot 0 >>>>>> ahcich2: is 00000000 cs 00000001 ss 00000000 rs 00000001 tfd c0 se= rr >>>>>> 00000000 >>>>>> ahcich2: Timeout on slot 8 >>>>>> ahcich2: is 00000000 cs 00000100 ss 00000000 rs 00000100 tfd c0 se= rr >>>>>> 00000000 >>>>>> ahcich2: Timeout on slot 8 >>>>>> ahcich2: is 00000000 cs fffff07f ss ffffff7f rs ffffff7f tfd c0 se= rr >>>>>> 00000000 >>>>>> ... >>>>> Looking that is (Interrupt status) is zero and `rs =3D=3D cs | ss` = (running >>>>> command bitmasks in driver and hardware), controller doesn't report= >>>>> command completion. Looking on TFD status 0xc0 with BUSY bit set, I= >>>>> would suppose that either disk stuck in command processing for some= >>>>> reason, or controller missed command completion status. >>>>> >>>>> Have you noticed 30 second (default ATA timeout) pause before timeo= ut >>>>> message printed? Just want to be sure that driver waited enough bef= ore >>>>> give up. >>>>> >>>>>> This happens when backup over GbE overloads ZFS/HDD capabilities. >>>>>> I reduced vfs.zfs.txg.timeout to 1 to prevent the machine from loc= king >>>>>> up almost immediately, but from it still happens. >>>>>> When I don't use ahci but ataahci (the old driver if I understand = things >>>>>> correct) I also see the ZFS burst write congestion, but this doesn= 't >>>>>> lead to controller timeouts, thus blocking the machine. >>>>>> >>>>>> Sometimes the machine recovers from the disk lock, but most often = I have >>>>>> to reboot. >>>>> How it looks when it doesn't? Can you send me full log messages? >>>> Hello, this morning I had a stall, but the machine recovered after a= bout >>>> one Minute. Here's what I got from the kernel: >>>> ahcich2: Timeout on slot 29 >>>> ahcich2: is 00000000 cs 00000003 ss e0000003 rs e0000003 tfd c0 serr= >>>> 00000000 >>>> em1: watchdog timeout -- resetting >>>> em1: watchdog timeout -- resetting >>>> ahcich2: Timeout on slot 10 >>>> ahcich2: is 00000000 cs 00006000 ss 00007c00 rs 00007c00 tfd c0 serr= >>>> 00000000 >>>> ahcich2: Timeout on slot 18 >>>> ahcich2: is 00000000 cs 00040000 ss 00000000 rs 00040000 tfd c0 serr= >>>> 00000000 >>>> ahcich2: Timeout on slot 2 >>>> ahcich2: is 00000000 cs 00000004 ss 00000000 rs 00000004 tfd c0 serr= >>>> 00000000 >>>> ahcich2: Timeout on slot 2 >>>> ahcich2: is 00000000 cs 00000000 ss 0000000c rs 0000000c tfd 40 serr= >>>> 00000000 >>>> >>>> Does this tell you something useful? >>> >>> It doesn't. Looking on logged register content - commands are indeed >>> still running and no interrupts requested. Interesting to see em1 >>> watchdog timeout there. Aren't they related somehow? >> >> dmesg | grep "irq 18": >> uhci0: port 0x20c0-0x20df irq >> 18 at device 26.0 on pci0 >> uhci4: port 0x2040-0x205f irq >> 18 at device 29.2 on pci0 >> em1: port >> 0x1000-0x103f mem 0xe1920000-0xe193ffff,0xe1900000-0xe191ffff irq 18 >> at device 2.0 on pci3 >> ichsmb0: port 0x2000-0x201f >> mem 0xe1a22000-0xe1a220ff irq 18 at device 31.3 on pci0 >> >> The don't share the same IRQ at least. >> dmesg | grep "irq 21" >> uhci1: port 0x20a0-0x20bf irq >> 21 at device 26.1 on pci0 >> ahci0: port >> 0x2408-0x240f,0x2414-0x2417,0x2400-0x2407,0x2410-0x2413,0x2020-0x203f >> mem 0xe1a21000-0xe1a217ff irq 21 at device 31.2 on pci0 >> >> The em1 has no cable attached. I get many of these em watchdog >> timeouts. Never thought they could be related to ahci. I'll see if >> the em watchdog timeouts happens in any relation to disk usage. >=20 > Please provide output from the commands I provided. dmesg|grep is not > sufficient for helping track this down, specifically with regards to th= e > em1 watchdog timeouts. Sorry for the delay, here's the details: hostb0@pci0:0:0:0: class=3D0x060000 card=3D0x34d08086 chip=3D0x29f08= 086 rev=3D0x00 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '3200 Chipset (Bearlake) Processor to I/O Controller' class =3D bridge subclass =3D HOST-PCI em0@pci0:0:25:0: class=3D0x020000 card=3D0x34d08086 chip=3D0x10bd8= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Intel 82566DM Gigabit Ethernet Adapter (82566DM)' class =3D network subclass =3D ethernet uhci0@pci0:0:26:0: class=3D0x0c0300 card=3D0x34d08086 chip=3D0x29378= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Contro= ller' class =3D serial bus subclass =3D USB uhci1@pci0:0:26:1: class=3D0x0c0300 card=3D0x34d08086 chip=3D0x29388= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Contro= ller' class =3D serial bus subclass =3D USB ehci0@pci0:0:26:7: class=3D0x0c0320 card=3D0x34d08086 chip=3D0x293c8= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Contro= ller' class =3D serial bus subclass =3D USB pcib1@pci0:0:28:0: class=3D0x060400 card=3D0x29408086 chip=3D0x29408= 086 rev=3D0x02 hdr=3D0x01 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) PCIe Root Port 1' class =3D bridge subclass =3D PCI-PCI pcib2@pci0:0:28:4: class=3D0x060400 card=3D0x29488086 chip=3D0x29488= 086 rev=3D0x02 hdr=3D0x01 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) PCIe Root Port 5' class =3D bridge subclass =3D PCI-PCI uhci2@pci0:0:29:0: class=3D0x0c0300 card=3D0x34d08086 chip=3D0x29348= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Contro= ller' class =3D serial bus subclass =3D USB uhci3@pci0:0:29:1: class=3D0x0c0300 card=3D0x34d08086 chip=3D0x29358= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Contro= ller' class =3D serial bus subclass =3D USB uhci4@pci0:0:29:2: class=3D0x0c0300 card=3D0x34d08086 chip=3D0x29368= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) USB Universal Host Contro= ller' class =3D serial bus subclass =3D USB ehci1@pci0:0:29:7: class=3D0x0c0320 card=3D0x34d08086 chip=3D0x293a8= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) USB2 Enhanced Host Contro= ller' class =3D serial bus subclass =3D USB pcib3@pci0:0:30:0: class=3D0x060401 card=3D0x34d08086 chip=3D0x244e8= 086 rev=3D0x92 hdr=3D0x01 vendor =3D 'Intel Corporation' device =3D '82801 Family (ICH2/3/4/5/6/7/8/9,63xxESB) Hub Interface to PCI Bridge' class =3D bridge subclass =3D PCI-PCI isab0@pci0:0:31:0: class=3D0x060100 card=3D0x34d08086 chip=3D0x29168= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IR (ICH9R) LPC Interface Controller' class =3D bridge subclass =3D PCI-ISA ahci0@pci0:0:31:2: class=3D0x010601 card=3D0x34d08086 chip=3D0x29228= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) 6 port SATA AHCI Controll= er' class =3D mass storage subclass =3D SATA ichsmb0@pci0:0:31:3: class=3D0x0c0500 card=3D0x34d08086 chip=3D0x29308= 086 rev=3D0x02 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D '82801IB/IR/IH (ICH9 Family) SMBus Controller' class =3D serial bus subclass =3D SMBus vgapci0@pci0:2:0:0: class=3D0x030000 card=3D0x01018086 chip=3D0x05221= 02b rev=3D0x02 hdr=3D0x00 vendor =3D 'Matrox Electronic Systems Ltd.' device =3D 'Matrox G200e (ServerEngines) - English (G200e)' class =3D display subclass =3D VGA em1@pci0:3:2:0: class=3D0x020000 card=3D0x34d08086 chip=3D0x10768086 rev=3D= 0x05 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Gigabit Ethernet Controller (82541EI)' class =3D network subclass =3D ethernet banana:~>20: vmstat -i interrupt total rate irq4: uart0 27392 0 irq18: em1 uhci0++ 178 0 irq19: uhci3 22 0 irq23: uhci2 ehci1 2 0 cpu0: timer 231629257 1917 irq256: em0 2559286 21 irq257: ahci0 1602196 13 cpu1: timer 231628629 1917 Total 467446962 3869 Looks very similar to the problem here: http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2010-03/msg00143.h= tml Thanks, -Harry --------------enig99F70BB8ABD28A47139A7119 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.12 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkucA2wACgkQLDqVQ9VXb8gK/gCgor2kS8lJcVdSi2zqI+aMA68S DisAn3To1/x7eJpkU20Dxqp7wCQjfYA/ =W7lO -----END PGP SIGNATURE----- --------------enig99F70BB8ABD28A47139A7119-- From owner-freebsd-stable@FreeBSD.ORG Sat Mar 13 21:49:33 2010 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4BA1E1065672; Sat, 13 Mar 2010 21:49:33 +0000 (UTC) (envelope-from tinderbox@freebsd.org) Received: from freebsd-current.sentex.ca (freebsd-current.sentex.ca [64.7.128.98]) by mx1.freebsd.org (Postfix) with ESMTP id 14DFC8FC12; Sat, 13 Mar 2010 21:49:32 +0000 (UTC) Received: from freebsd-current.sentex.ca (localhost [127.0.0.1]) by freebsd-current.sentex.ca (8.14.4/8.14.3) with ESMTP id o2DLnWXb051221; Sat, 13 Mar 2010 16:49:32 -0500 (EST) (envelope-from tinderbox@freebsd.org) Received: (from tinderbox@localhost) by freebsd-current.sentex.ca (8.14.4/8.14.3/Submit) id o2DLnWrU051220; Sat, 13 Mar 2010 21:49:32 GMT (envelope-from tinderbox@freebsd.org) Date: Sat, 13 Mar 2010 21:49:32 GMT Message-Id: <201003132149.o2DLnWrU051220@freebsd-current.sentex.ca> X-Authentication-Warning: freebsd-current.sentex.ca: tinderbox set sender to FreeBSD Tinderbox using -f Sender: FreeBSD Tinderbox From: FreeBSD Tinderbox To: FreeBSD Tinderbox , , Precedence: bulk Cc: Subject: [releng_8_0 tinderbox] failure on amd64/amd64 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Mar 2010 21:49:33 -0000 TB --- 2010-03-13 21:43:38 - tinderbox 2.6 running on freebsd-current.sentex.ca TB --- 2010-03-13 21:43:38 - starting RELENG_8_0 tinderbox run for amd64/amd64 TB --- 2010-03-13 21:43:38 - cleaning the object tree TB --- 2010-03-13 21:44:04 - cvsupping the source tree TB --- 2010-03-13 21:44:04 - /usr/bin/csup -z -r 3 -g -L 1 -h cvsup.sentex.ca /tinderbox/RELENG_8_0/amd64/amd64/supfile TB --- 2010-03-13 21:49:32 - WARNING: /usr/bin/csup returned exit code 1 TB --- 2010-03-13 21:49:32 - ERROR: unable to cvsup the source tree TB --- 2010-03-13 21:49:32 - 1.61 user 14.37 system 353.16 real http://tinderbox.freebsd.org/tinderbox-releng_8-RELENG_8_0-amd64-amd64.full