From owner-freebsd-stable@freebsd.org Sun Oct 16 09:29:04 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CF4DC12FC4; Sun, 16 Oct 2016 09:29:04 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: from mail-oi0-x231.google.com (mail-oi0-x231.google.com [IPv6:2607:f8b0:4003:c06::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F09F3D6; Sun, 16 Oct 2016 09:29:04 +0000 (UTC) (envelope-from v.maffione@gmail.com) Received: by mail-oi0-x231.google.com with SMTP id d132so182801939oib.2; Sun, 16 Oct 2016 02:29:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=1mzgs/x2ybum58ghVWHFPJD2QjKo7A2h1u3IUBSovDo=; b=pB2iBQ8ZDeGn7VNIE4ETQ6WWkqO8ALECyl3M5h1BFrg0hD3JcPMdB6O7S8Z9mkJGlR 5bXuF8UbGxHkbIZoABUk6ZiU2IvDsqS9tHaRRDyFpWa7TDDd+UwgHyx23rH8ADfRn4bC HJfLA5FyPcw1MZ/smsZMT5jeEkRNwCXFDNMREyJIr4a7M2Yxjiv28LlgxBh7sYG5jetj pj1O7NFs9SQL0wXWwe9ixyfte5th5hfoEVwz/M+wWyG3n9+em9L8nIaSV8N62lYxU2Gw j5wufg9myvU//59WP0rEhJWp8XLzP0HuSKmmQCuTWEhTcuwaEh4wmpALFJyabwAKLOAj Auqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=1mzgs/x2ybum58ghVWHFPJD2QjKo7A2h1u3IUBSovDo=; b=mo8eYKXyySivzsiSSW+gsnBBn2cTC2Q5gbKMJWzIWNgtUIQJVsqsRQ9Oy3a3D8rMvc to/iyjUr2UhRzcQJNKyCulMZrAiPf6Hf+nU5OMfyyafX+qS7tpQoIi39dZpG5st30vYr 2ji3l9S0dFGYdByFTXVAXhM7uPEhF4gB06kRU1fiWBJZz+ZCUOwn+uHyYqQbw61mVf1+ amfpU6Ln4FqZAVOqgFEwD3R1J1X9Z3FLjl8XfkbT9pId6HbmXg8++q8ZbQ+6POLr4OnX xtv8cxb/pBOl1cPIzst64AArxN1ouRmQaY8RTR+8OLiKbrLpLRfxnfMCuA8/XQ3Rm4rz oKxw== X-Gm-Message-State: AA6/9RmvlhKd/NZmVM2EU4jqGO8A6UQd1lVXf5KPK24UAb8RpBKrO8iE7amoaGe7g1tOIs1kPqo9+SpMVmzOmQ== X-Received: by 10.202.74.79 with SMTP id x76mr13574558oia.77.1476610143656; Sun, 16 Oct 2016 02:29:03 -0700 (PDT) MIME-Version: 1.0 Received: by 10.157.61.52 with HTTP; Sun, 16 Oct 2016 02:29:03 -0700 (PDT) In-Reply-To: <5801F9EE.1000407@omnilan.de> References: <58009EB4.30708@omnilan.de> <5800DFB9.5030102@omnilan.de> <5801F9EE.1000407@omnilan.de> From: Vincenzo Maffione Date: Sun, 16 Oct 2016 11:29:03 +0200 Message-ID: Subject: Re: vale-ctl(-8), ifconfig(8), SIOCAIFADDR: Invalid argument [utilizing netmap(4) providing virtual switches+interfaces to BHyVe] To: Harry Schmalzbauer Cc: FreeBSD Net , FreeBSD Stable , Luigi Rizzo Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 09:29:04 -0000 2016-10-15 11:42 GMT+02:00 Harry Schmalzbauer : > Bez=C3=BCglich Vincenzo Maffione's Nachricht vom 15.10.2016 09:32 (localt= ime): > > 2016-10-14 15:38 GMT+02:00 Harry Schmalzbauer : > > =E2=80=A6 > >> I'm familar with epair(4), but not with tap(4). > >> I don't understand the man page for tap, perhaps I should read pty(4)= =E2=80=A6 > >> But I guess I don't have to know the details of tap(4), since you > >> confirmed that it can be connected to VALE. > >> > > > > It's not necessary to understand the details. However, a TAP device is > > conceptually similar to the two ends of an epair, with the difference > that > > in the TAP a network interface (e.g. tap0) is conecptually "connected" > > back-to-back to a file descriptor. The file descriptor is written/read = by > > the hypervisor (to inject/intercept packets to/from the network stack), > > while the tap0 interface can be attached to if_bridge. > > Hi Vincenzo, thanks for your explanation! > > > >> > >> So one could summarize: > >> VALE (as part of netmap(4)) can act as a if_bridge(4) replacement in > >> FreeBSD-10/11, keeping everything else involved untouched. > >> Please correct me if I'm wrong. > >> > > > > For simple cases yes. if_bridge may have features that are not supporte= d > by > > netmap (i.e. configure ports as VLAN access ports). Moreover, if_bridge > has > > a interface (br0), whereas VALE bridges doesn't. > > Again, thank you for your time! (R)STP comes to my mind (which I don't > need any more). And I'm not sure if VALE really lacks that, but I guess > it wouldn't match VALEs philosophy/design at all=E2=80=A6 > Well, VALE is a programmable switch, meaning that you can plug in a custom forwarding function (which is just a C function). The default forwarding function implements an L2 learning bridge without STP, because it is thought to be used as the preferred hypervisor virtual switch, so no L2 connectivity loops, and the STP comes with an overhead. If you really wish to have VALE with STP, you could define your own custom forwarding function that implements it. > > =E2=80=A6 > >>> https://github.com/luigirizzo/netmap). Among the new features, there > is > >> a > >>> new solution for bhyve networking, which will let you attach your bhy= ve > >> VMs > >>> directly to a VALE switch, without paying additional overheads relate= d > to > >>> TAPs, epairs, and vtnet emulation. You can find additional informatio= n, > >>> code and performance numbers here: > >>> https://wiki.freebsd.org/SummerOfCode2016/PtnetDriverAndDeviceModel. > >> > >> Thanks for that hint! > >> I guess it's about ptnetmap(4)? I read papers but haven't considered i= t > >> could be production-ready for FreeBSD in the near future. > >> It's extremely interesting and I'd love to be eraly adopter, but my > >> (ESXi) setups are currently doing well and I don't have spare time or > >> any business project to try out=E2=80=A6 :-( > >> > > > > Yes, it's ptnetmap. However, bhyve is going to have support for VALE > ports > > anyway (even without ptnetmap), as QEMU already does, so at least you > will > > be able to replace TAPs with VALE ports (while still using vtnet device= s > > for the VM). > > Oic, I wasn't aware that there will be a VALE-vtnet direct path! That is > really great news :-) And a big achievment for guests preferring > "standard" drivers, ptnetmap could limit the guest OS choice I guess. > Yes ptnetmap is supported on QEMU and bhyve, and as a gues you can use only FreeBSD and Linux, at the moment. > > For now, I'm happy having been in touch with netmap(4) =E2=80=93 at least= with a > very little fraction of natmap =E2=80=93 but I'll stay the legacy way uti= lizing > if_bridge(4) and see if there are still oddities and try to find some > time to track them down (involving LACP, VLANs, Jumbo-Frames and IPv6 =E2= =80=93 > that was the problematic constellation) > > Since I have extra PHYs, I can do PCIe-passthrough like before (with > ESXi) for some special guests. I'm looking forward to find out how this > works with bhyve! > > No idea, but also ptnetmap is able to do that: not only you can pass-through a VALE port, but also you can pass-through a physical interface. Cheers, Vincenzo > Best, > > -Harry > --=20 Vincenzo Maffione From owner-freebsd-stable@freebsd.org Sun Oct 16 10:20:52 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6EA20C06D97 for ; Sun, 16 Oct 2016 10:20:52 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from bouvier.getmail.no (bouvier.getmail.no [84.210.184.8]) by mx1.freebsd.org (Postfix) with ESMTP id 1966DD1A for ; Sun, 16 Oct 2016 10:20:51 +0000 (UTC) (envelope-from torfinn.ingolfsen@getmail.no) Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id E092347A9C for ; Sun, 16 Oct 2016 12:20:42 +0200 (CEST) Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id h6FEzvNYPTQg for ; Sun, 16 Oct 2016 12:20:42 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by bouvier.getmail.no (Postfix) with ESMTP id 5AF5A4856B for ; Sun, 16 Oct 2016 12:20:42 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.9.2 bouvier.getmail.no 5AF5A4856B DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=getmail.no; s=8A9C8B4C-D727-11E2-8095-B6466E6B3FA2; t=1476613242; bh=3ORIYdEgpsNRGbd8yk0lHUyP2hoCFP2H9IEISb5bf5M=; h=Date:From:To:Subject:Message-Id:Mime-Version:Content-Type: Content-Transfer-Encoding; b=bFUEEF0KnzCGo9mpBMbK/GGAa52A27xUVaagI0B3oz5GVYXpw75oyYDi5IWUb6Ta0 mRU6F9ZmtfXybu/KB6exVcOoPX1WzWXde3B8cw76siI3yJ8Hj8MXRXbjBMNpoTnPvE xpK+QVtqz4qyhV/siWqCYFF4maDkhY/AMoauaxW0= X-Virus-Scanned: amavisd-new at bouvier.get.c.bitbit.net Received: from bouvier.getmail.no ([127.0.0.1]) by localhost (bouvier.get.c.bitbit.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id g60RqQ60EHtp for ; Sun, 16 Oct 2016 12:20:42 +0200 (CEST) Received: from kg-core1.kg4.no (cm-84.209.39.108.getinternet.no [84.209.39.108]) by bouvier.getmail.no (Postfix) with ESMTPSA id 2C03A47A9C for ; Sun, 16 Oct 2016 12:20:42 +0200 (CEST) Date: Sun, 16 Oct 2016 12:20:41 +0200 From: Torfinn Ingolfsen To: freebsd-stable@freebsd.org Subject: Building FreeBSD 11.0-stable on FreeBSD 10.1-stable fails Message-Id: <20161016122041.e20ae504c58d4f9224eec6bf@getmail.no> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.29; amd64-portbld-freebsd9.3) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 10:20:52 -0000 I am trying to build FreeBSD 11.0-stable on a machine which runs: tingo@kg-v7$ uname -a FreeBSD kg-v7.kg4.no 10.1-STABLE FreeBSD 10.1-STABLE #0 r278322: Fri Feb 6 21:36:01 CET 2015 root@kg-v7.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 I have emptied /usr/src and /usr/obj and fetched the latest stable/11 via subversion: tingo@kg-v7$ egrep "^BRANCH|^REVISION" /usr/src/sys/conf/newvers.sh REVISION="11.0" BRANCH="STABLE" But building it (per the procedure in the handbook) fails at the buildworld stage. Both 'make -j5 buildworld' and 'make buildworld' fails, like this: c++: error: unable to execute command: Segmentation fault (core dumped) c++: error: clang frontend command failed due to signal (use -v to see invocation) FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 Target: x86_64-unknown-freebsd10.1 Thread model: posix c++: note: diagnostic msg: PLEASE submit a bug report to https://bugs.freebsd.org/submit/ and include the crash backtrace, preprocessed source, and associated run script. c++: note: diagnostic msg: ******************** PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT: Preprocessed source(s) and associated run script(s) are located at: c++: note: diagnostic msg: /tmp/CGBlocks-abcdc1.cpp c++: note: diagnostic msg: /tmp/CGBlocks-abcdc1.sh c++: note: diagnostic msg: ******************** *** Error code 254 Stop. bmake[4]: stopped in /usr/src/lib/clang/libclangcodegen *** Error code 1 Stop. bmake[3]: stopped in /usr/src/lib/clang *** Error code 1 Stop. bmake[2]: stopped in /usr/src *** Error code 1 Stop. bmake[1]: stopped in /usr/src *** Error code 1 Stop. make: stopped in /usr/src So I cleaned out /usr/obj again and tried this (fro /usr/src/UPDATING): root@kg-v7# make WITHOUT_CLANG=y WITH_GCC=y -j5 buildworld [..] ******************** *** [CGClass.o] Error code 254 bmake[4]: stopped in /usr/src/lib/clang/libclangcodegen 2 errors bmake[4]: stopped in /usr/src/lib/clang/libclangcodegen *** [all_subdir_lib/clang/libclangcodegen] Error code 2 bmake[3]: stopped in /usr/src/lib/clang --- all_subdir_lib/clang/libclangedit --- A failure has been detected in another branch of the parallel make bmake[4]: stopped in /usr/src/lib/clang/libclangedit *** [all_subdir_lib/clang/libclangedit] Error code 2 bmake[3]: stopped in /usr/src/lib/clang --- all_subdir_lib/clang/libclangast --- A failure has been detected in another branch of the parallel make bmake[4]: stopped in /usr/src/lib/clang/libclangast *** [all_subdir_lib/clang/libclangast] Error code 2 bmake[3]: stopped in /usr/src/lib/clang --- all_subdir_lib/clang/libclangfrontend --- A failure has been detected in another branch of the parallel make bmake[4]: stopped in /usr/src/lib/clang/libclangfrontend *** [all_subdir_lib/clang/libclangfrontend] Error code 2 bmake[3]: stopped in /usr/src/lib/clang 4 errors bmake[3]: stopped in /usr/src/lib/clang *** [cross-tools] Error code 2 bmake[2]: stopped in /usr/src 1 error bmake[2]: stopped in /usr/src *** [_cross-tools] Error code 2 bmake[1]: stopped in /usr/src 1 error bmake[1]: stopped in /usr/src *** [buildworld] Error code 2 make: stopped in /usr/src 1 error make: stopped in /usr/src So, is there a way to fix this without building an updated stable/10 as an intermediary step? References: FreeBSD forum thread: https://forums.freebsd.org/threads/58082/ -- Torfinn Ingolfsen From owner-freebsd-stable@freebsd.org Sun Oct 16 11:35:09 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65061BEDD38 for ; Sun, 16 Oct 2016 11:35:09 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [IPv6:2001:7b8:3a7:1:2d0:b7ff:fea0:8c26]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 22B4B18D6 for ; Sun, 16 Oct 2016 11:35:09 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from [IPv6:2001:7b8:3a7::c36:9109:5a15:ae78] (unknown [IPv6:2001:7b8:3a7:0:c36:9109:5a15:ae78]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id CE264122D; Sun, 16 Oct 2016 13:35:05 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_44568974-F021-4633-9A90-F613FCC963AA"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Building FreeBSD 11.0-stable on FreeBSD 10.1-stable fails From: Dimitry Andric In-Reply-To: <20161016122041.e20ae504c58d4f9224eec6bf@getmail.no> Date: Sun, 16 Oct 2016 13:34:53 +0200 Cc: freebsd-stable@freebsd.org Message-Id: <426D0291-D785-4CFE-A15D-9BE688499ACB@FreeBSD.org> References: <20161016122041.e20ae504c58d4f9224eec6bf@getmail.no> To: Torfinn Ingolfsen X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 11:35:09 -0000 --Apple-Mail=_44568974-F021-4633-9A90-F613FCC963AA Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 16 Oct 2016, at 12:20, Torfinn Ingolfsen = wrote: > I am trying to build FreeBSD 11.0-stable on a machine which runs: > tingo@kg-v7$ uname -a > FreeBSD kg-v7.kg4.no 10.1-STABLE FreeBSD 10.1-STABLE #0 r278322: Fri = Feb 6 21:36:01 CET 2015 > root@kg-v7.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 >=20 > I have emptied /usr/src and /usr/obj and fetched the latest stable/11 = via subversion: > tingo@kg-v7$ egrep "^BRANCH|^REVISION" /usr/src/sys/conf/newvers.sh > REVISION=3D"11.0" > BRANCH=3D"STABLE" >=20 > But building it (per the procedure in the handbook) fails at the = buildworld stage. Both 'make -j5 buildworld' and 'make buildworld' = fails, like this: >=20 > c++: error: unable to execute command: Segmentation fault (core = dumped) > c++: error: clang frontend command failed due to signal (use -v to see = invocation) > FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) = 20140512 > Target: x86_64-unknown-freebsd10.1 > Thread model: posix > c++: note: diagnostic msg: PLEASE submit a bug report to = https://bugs.freebsd.org/submit/ and include the crash backtrace, = preprocessed source, and associated run script. > c++: note: diagnostic msg: > ******************** >=20 > PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT: > Preprocessed source(s) and associated run script(s) are located at: > c++: note: diagnostic msg: /tmp/CGBlocks-abcdc1.cpp > c++: note: diagnostic msg: /tmp/CGBlocks-abcdc1.sh Please make sure your stable/10 is at least r286033. -Dimitry --Apple-Mail=_44568974-F021-4633-9A90-F613FCC963AA Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.30 iEYEARECAAYFAlgDZewACgkQsF6jCi4glqOVgQCeLUE4Wvxucd+fYQrUpD+WxyvK erAAoKlQB9N6Ips92fClni5G0uFAsyWQ =OD82 -----END PGP SIGNATURE----- --Apple-Mail=_44568974-F021-4633-9A90-F613FCC963AA-- From owner-freebsd-stable@freebsd.org Sun Oct 16 13:26:44 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABD53C123E2; Sun, 16 Oct 2016 13:26:44 +0000 (UTC) (envelope-from george@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [IPv6:2001:418:3fd::f7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6D1641464; Sun, 16 Oct 2016 13:26:44 +0000 (UTC) (envelope-from george@m5p.com) Received: from [10.100.0.31] (haymarket.m5p.com [10.100.0.31]) by mailhost.m5p.com (8.14.5/8.14.5) with ESMTP id u9GDQb2A037652; Sun, 16 Oct 2016 09:26:42 -0400 (EDT) (envelope-from george@m5p.com) Subject: Re: Clandestine USB SD card slot To: Anthony Jenkins , freebsd-usb@freebsd.org, FreeBSD Stable Mailing List References: <832db721-d03a-a21e-11ed-701190140d17@yahoo.com> <2bf09be6-e2ad-2d38-ffd8-d80b45d4c3d8@m5p.com> Reply-To: George Mitchell From: George Mitchell Message-ID: <82d2acb3-12cd-678e-c90a-21b9d62b5721@m5p.com> Date: Sun, 16 Oct 2016 09:26:37 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <2bf09be6-e2ad-2d38-ffd8-d80b45d4c3d8@m5p.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.73 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (mailhost.m5p.com [10.100.0.247]); Sun, 16 Oct 2016 09:26:43 -0400 (EDT) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 13:26:44 -0000 Whoops, I should send this to freebsd-stable instead of freebsd-usb. Sorry! -- George On 10/16/16 09:22, George Mitchell wrote: > On 10/15/16 23:10, Anthony Jenkins wrote: >> On 10/15/16 18:28, George Mitchell wrote: >>> FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016 >>> Acer Aspire E15 >>> There is a slot on the front of this laptop which sure looks like an >>> SD card slot, which I hope corresponds to one of these: >>> >>> ugen1.1: at usbus1, cfg=0 md=HOST spd=HIGH (480Mbps) >>> pwr=SAVE (0mA) >>> ugen0.1: at usbus0, cfg=0 md=HOST spd=SUPER >>> (5.0Gbps) pwr=SAVE (0mA) >>> ugen2.1: at usbus2, cfg=0 md=HOST spd=HIGH (480Mbps) >>> pwr=SAVE (0mA) >>> ugen1.2: at usbus1, cfg=0 md=HOST >>> spd=HIGH (480Mbps) pwr=SAVE (100mA) >>> ugen2.2: at usbus2, cfg=0 md=HOST >>> spd=HIGH (480Mbps) pwr=SAVE (100mA) >>> ugen0.2: at usbus0, cfg=0 md=HOST >>> spd=SUPER (5.0Gbps) pwr=ON (224mA) >>> ugen1.3: at usbus1, cfg=0 md=HOST spd=FULL >>> (12Mbps) pwr=SAVE (100mA) >>> ugen2.3: at usbus2, cfg=0 md=HOST >>> spd=FULL (12Mbps) pwr=ON (100mA) >>> ugen1.4: at usbus1, cfg=0 md=HOST >>> spd=LOW (1.5Mbps) pwr=ON (100mA) >>> ugen1.5: at usbus1, cfg=0 md=HOST spd=FULL >>> (12Mbps) pwr=SAVE (100mA) >>> ugen1.6: at usbus1, cfg=0 md=HOST >>> spd=FULL (12Mbps) pwr=ON (100mA) >>> ugen1.7: at usbus1, cfg=0 md=HOST >>> spd=FULL (12Mbps) pwr=ON (100mA) >>> ugen1.8: at usbus1, cfg=0 md=HOST >>> spd=HIGH (480Mbps) pwr=ON (500mA) >>> >>> But inserting a card into the slot produces no results, even with >>> sysctl hw.usb.umass.debug=1 hw.usb.ugen.debug=1 hw.usb.dev.debug=1. >>> (sysctl hw.usb.debug=1 produces way too much output all the time.) >>> Any suggestions on how I can get this slot to overcome its shyness? >>> (I am not subscribed to the list; please CC me.) -- George >> Your card reader is probably on the PCI bus (and likely not supported by >> FreeBSD); I couldn't find any of your USB Pid:Vids as card readers. >> >> [ajenkins@ajenkins-hplaptop ~]$ pciconf -lv >> ... >> none2@pci0:3:0:0: class=0xff0000 card=0x1995103c chip=0x522910ec >> rev=0x01 hdr=0x00 >> vendor = 'Realtek Semiconductor Co., Ltd.' >> device = 'RTS5229 PCI Express Card Reader' >> >> Anthony Jenkins >> > Ah! It didn't occur to me to run pciconf. "pciconf -lv" shows this: > > sdhci_pci0@pci0:0:20:7: class=0x080501 card=0x08651025 chip=0x78131022 > rev=0x01 hdr=0x00 > vendor = 'Advanced Micro Devices, Inc. [AMD]' > device = 'FCH SD Flash Controller' > class = base peripheral > subclass = SD host controller > > So not only is it (apparently) recognized, but the sdhci_pci driver > attached to it! But inserting or removing a card shows no activity. > What's my next step? -- George > From owner-freebsd-stable@freebsd.org Sun Oct 16 14:23:44 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83E70C141C4 for ; Sun, 16 Oct 2016 14:23:44 +0000 (UTC) (envelope-from zkolic@sbb.rs) Received: from mproxy19.sbb.rs (mproxy19.sbb.rs [89.216.2.104]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "smtp.sbb.rs", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E21741C6B for ; Sun, 16 Oct 2016 14:23:43 +0000 (UTC) (envelope-from zkolic@sbb.rs) Received: from knossos ([87.116.189.62]) by mproxy19.sbb.rs (8.15.2/8.15.2) with ESMTP id u9GEDSFL019321 for ; Sun, 16 Oct 2016 16:13:28 +0200 X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.97.6 at SBB mail Received: from localhost (knossos [local]) by knossos (OpenSMTPD) with ESMTPA id ce7c2b00 for ; Sun, 16 Oct 2016 16:13:28 +0200 (CEST) Date: Sun, 16 Oct 2016 16:13:28 +0200 From: Zoran Kolic To: freebsd-stable@freebsd.org Subject: update from 9.3 to 11.0 Message-ID: <20161016141328.GA57825@knossos> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mproxy19.sbb.rs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 14:23:44 -0000 I would like to know what experience and tips people on this list have regarding this update. Doug Hardie made it successfully. To my eyes, it is the mere change of one charracter in the file. I'm a bit late due to lack of time to do the task. Best regards all Zoran From owner-freebsd-stable@freebsd.org Sun Oct 16 15:22:27 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA2F1C14D7B for ; Sun, 16 Oct 2016 15:22:27 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x233.google.com (mail-io0-x233.google.com [IPv6:2607:f8b0:4001:c06::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D032147C for ; Sun, 16 Oct 2016 15:22:27 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x233.google.com with SMTP id r30so165685232ioi.1 for ; Sun, 16 Oct 2016 08:22:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=OpzjnSLvhru1K+3ZK2y90D/yntU6tLmXK2Of2sK5Hvg=; b=EfP1DGjvAiga5WXCWhhfm7O3lksmPv2g0tGpe/JpmvqplNNIprUQ7Hk/IvXYtdqqcZ ssK8t5t11A2UbW0FDFyaGkm6a1c8MxjVmz5b+jJ9YmgbAKolUKjwFKEJ2JLmKBKzd09K gCo68R2L8CBb2dM10S8PU9mRD73LK9RX1kGczr2KNZE6JcyqEQJfG0SrmPGZZ7jy08m4 6ZCES9NPG2qJa3xfZ0iQ5cKkB6/jlA1kWWp8hEsPhA4p6T39K8G1ccBpSBIaFyTMrE+4 J3JU5k/ycvsLXIp7xRVa/6oZ+rN6atPkwbnfhaDH/z6/KeKlnVJ0mt4Apl++Pe05brkW 7tiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=OpzjnSLvhru1K+3ZK2y90D/yntU6tLmXK2Of2sK5Hvg=; b=dQzoNBKoy2k3OELu4zunCsiKK1wprXnGOJAnZduJH8JYTK2N2iKva4pi8zSsDKbWZ2 l+bqzFg/+GX4ejoTR87gaZuDrcegtld/srQe/TghI2ow7mFjF7o1H/LJqIxAaF2qHqs7 sBH18MZZtpkvGqLUH7MnkWjD1N1Nfs/IkE7yIBHGdVJhHVd8HsoBWwK4tY831NgSWX2n pmxO5lsh1OynpT7ddc2RduRa/vuJDRaAg9vqxX9V9pD0n8oWeRODCmFfrTwz4TTt1mnD JqMb1JEg8kc0ZWhJ2S9epVjCrnDqPOwEtNdlJqDMVM4JnB42ESAl9hig8A0AeWnJByHk so9Q== X-Gm-Message-State: AA6/9Rm7wDz0vd9QEQyfGz/XuE6lU9sik41SMWU9Rs8O3Sk7QPlMNmt9MgyJFN6XWVyZPZfsfA+YGzvNePzqQg== X-Received: by 10.107.139.79 with SMTP id n76mr22162086iod.166.1476631346707; Sun, 16 Oct 2016 08:22:26 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.77.85 with HTTP; Sun, 16 Oct 2016 08:22:25 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: <426D0291-D785-4CFE-A15D-9BE688499ACB@FreeBSD.org> References: <20161016122041.e20ae504c58d4f9224eec6bf@getmail.no> <426D0291-D785-4CFE-A15D-9BE688499ACB@FreeBSD.org> From: Warner Losh Date: Sun, 16 Oct 2016 09:22:25 -0600 X-Google-Sender-Auth: AOr2Za9uCfwz9qsQwuF5mBqPUXE Message-ID: Subject: Re: Building FreeBSD 11.0-stable on FreeBSD 10.1-stable fails To: Dimitry Andric , Bryan Drewery Cc: Torfinn Ingolfsen , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 15:22:27 -0000 On Sun, Oct 16, 2016 at 5:34 AM, Dimitry Andric wrote: > On 16 Oct 2016, at 12:20, Torfinn Ingolfsen wrote: >> I am trying to build FreeBSD 11.0-stable on a machine which runs: >> tingo@kg-v7$ uname -a >> FreeBSD kg-v7.kg4.no 10.1-STABLE FreeBSD 10.1-STABLE #0 r278322: Fri Feb 6 21:36:01 CET 2015 >> root@kg-v7.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 >> >> I have emptied /usr/src and /usr/obj and fetched the latest stable/11 via subversion: >> tingo@kg-v7$ egrep "^BRANCH|^REVISION" /usr/src/sys/conf/newvers.sh >> REVISION="11.0" >> BRANCH="STABLE" >> >> But building it (per the procedure in the handbook) fails at the buildworld stage. Both 'make -j5 buildworld' and 'make buildworld' fails, like this: >> >> c++: error: unable to execute command: Segmentation fault (core dumped) >> c++: error: clang frontend command failed due to signal (use -v to see invocation) >> FreeBSD clang version 3.4.1 (tags/RELEASE_34/dot1-final 208032) 20140512 >> Target: x86_64-unknown-freebsd10.1 >> Thread model: posix >> c++: note: diagnostic msg: PLEASE submit a bug report to https://bugs.freebsd.org/submit/ and include the crash backtrace, preprocessed source, and associated run script. >> c++: note: diagnostic msg: >> ******************** >> >> PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT: >> Preprocessed source(s) and associated run script(s) are located at: >> c++: note: diagnostic msg: /tmp/CGBlocks-abcdc1.cpp >> c++: note: diagnostic msg: /tmp/CGBlocks-abcdc1.sh > > Please make sure your stable/10 is at least r286033. What's the issue this fixes? Right now we have safeties in place in buildworld that indicate we 'support' back to 9 sometime. If that's not really the case, we need to fix something (either fix so we don't need this specific revision, or fix the Makefile to indicate we don't support back that far). Warner From owner-freebsd-stable@freebsd.org Sun Oct 16 15:24:11 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 81108C14E57 for ; Sun, 16 Oct 2016 15:24:11 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x230.google.com (mail-it0-x230.google.com [IPv6:2607:f8b0:4001:c0b::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47912161E for ; Sun, 16 Oct 2016 15:24:11 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x230.google.com with SMTP id 139so22938307itm.1 for ; Sun, 16 Oct 2016 08:24:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=F5hqdY4b9uXSUCto8hq4jCMQsluVx7CL2rTu0k1xUrg=; b=yqlIUUSIH280gbWf5YpzFTA7hnSMmzETOPb3ujCRe1myAseHAXT41Dt9iO8nEcjfGM snXSqBrAY2vzYI5KR3BZKBfDi1e2hdM/W1uRJ4k1AeQ8p9aJamFfcGH5dB8owWITIL6k vYEBltydYTd/QPaVtTKdv5ofzcDvuURdkO7Gm6upXLdrBEPNqYyjmxkur0ModPB9eCYn 1fYbkYUWWEj+dVhby6hy+MrYGA1IY9+U//L3hRZMPCbC6V/upmI5AcLfD6zIdAAeZXFi tmM0BnNUMCf8Vpb4i/zLBk1SUTxmaklYhGAxJEQFNz/dIyFR4ox6ZZWBEo6plpe0hH34 cnog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=F5hqdY4b9uXSUCto8hq4jCMQsluVx7CL2rTu0k1xUrg=; b=jX1HUByqU/yxP9LwmmnuphqfAapPQBdgwYCI3URZJTM+De5JbGYWpXCp7FlFWUTyNw /a0bTIDyNRmj80a6jLTKjQADZ3eKErLiR1sJftCZOhFDBZ97tiZfWkYtw+VvCPzxWe1x 4JjhSOXLNCGYL/DX3HY/QGAKnAWph8ZalPJUrG2J7EIL8mkjSE0v0hrDe+Q9jcXImKKl A+pW3dRj1xe32CxfHScDu4uGwz43yyrkAzRmOPrgulD5ci37I3/Gy+ljWjdaWeYSGuVx dMRK7KnSBykNALwMD4mN3CWjeyzayGLX1663hmQrxWUmIZ+RvNwBcNop9f7ZEFXA/61H PdtA== X-Gm-Message-State: AA6/9RmUWFbEjkncm2hEz4m1+cBjAgHGI43JA9bvOv+B4oT+3VlfbHZfyFrwTs9upOVBi1bNZKfB/+R1Nycwtw== X-Received: by 10.36.53.214 with SMTP id k205mr5181276ita.85.1476631450726; Sun, 16 Oct 2016 08:24:10 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.77.85 with HTTP; Sun, 16 Oct 2016 08:24:10 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: <20161016141328.GA57825@knossos> References: <20161016141328.GA57825@knossos> From: Warner Losh Date: Sun, 16 Oct 2016 09:24:10 -0600 X-Google-Sender-Auth: TJEzPmqGv4FCNd0hSbpCX9U9o6w Message-ID: Subject: Re: update from 9.3 to 11.0 To: Zoran Kolic Cc: FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 15:24:11 -0000 On Sun, Oct 16, 2016 at 8:13 AM, Zoran Kolic wrote: > I would like to know what experience and tips people on this list have > regarding this update. Doug Hardie made it successfully. To my eyes, > it is the mere change of one charracter in the file. > I'm a bit late due to lack of time to do the task. > Best regards all I missed the original post. what's the one character change? Warner From owner-freebsd-stable@freebsd.org Sun Oct 16 16:26:08 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 48AC8C14B17 for ; Sun, 16 Oct 2016 16:26:08 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 339311D6F for ; Sun, 16 Oct 2016 16:26:08 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id 32E89C14B16; Sun, 16 Oct 2016 16:26:08 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 328C9C14B15 for ; Sun, 16 Oct 2016 16:26:08 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0ADDE1D6E for ; Sun, 16 Oct 2016 16:26:07 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id u9GGQ5JJ005403 for ; Sun, 16 Oct 2016 16:26:05 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id u9GGQ5il005402 for stable@freebsd.org; Sun, 16 Oct 2016 09:26:05 -0700 (PDT) (envelope-from david) Date: Sun, 16 Oct 2016 09:26:05 -0700 From: David Wolfskill To: stable@freebsd.org Subject: sshd whines & dies after releng/10 "freebsd-update" run Message-ID: <20161016162605.GG1069@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="2NLGdgz3UMHa/lqP" Content-Disposition: inline User-Agent: Mutt/1.7.0 (2016-08-17) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 16:26:08 -0000 --2NLGdgz3UMHa/lqP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable For most of my experience with FreeBSD (since 1998) and for most of my machines, I build from source (either on the machine itself or a dedicated "build machine"); this has been ... occasionally turbulent, but overall, a fairly stable approach for me (and it's a great deal less turbulent -- usually! -- now than it was a decade ago). However, I have one machine that is pretty much dedicated to one specific function, and for it, I thought I'd try freebsd-update. And over the last year or so, it's worked pretty well: I have the machine set up (as is usually my approach) to be able to boot from either of a couple of slices. I use a "dump | restore" pipeline to copy the / and /usr file systems from the "active" slice to the "inactive" slice, adjust /etc/fstab on the inactive slice to reflect reality for when it's the boot slice, then (while the file systemms =66rom the other slice are still mounted -- e.g., on /S2) run "freebsd-update -b /S2 fetch install", then reboot from the newly-updated slice. In the past, that's Just Worked. This weekend, though, I was planning to update my other systems tfrom stable/10 to stable/11, so I figured I'd try freebsd-update on this machine first. But before I tried going to stable/11, I thought it might be good to first get to the latest releng/10. Running freebsd-update seemed to go well. I rebooted from the updated slice... and found that I could not ssh to the machine. (I only physically login to a machine other than my laptop if there's a problem that's so bad that I can't login from the laptop....) And I found that sshd wasn't running. Indeed, on attempting to start it by hand: root@sisboombah: # service sshd start Performing sanity check on sshd configuration. Undefined symbol "ssh_compat13" referenced from COPY relocation in /usr/sbi= n/sshd /etc/rc.d/sshd: WARNING: failed precmd routine for sshd Attempting to start it in "debug" mode was of no help: root@sisboombah:/tmp # `which sshd` -d Undefined symbol "ssh_compat13" referenced from COPY relocation in /usr/sbi= n/sshd Any clues? I have placed both a typescript of the freebsd-update run (actually, a pair of them: one yesterday; another, today), as well as a typescript =66rom some poking around a bit, under . Thanks! Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --2NLGdgz3UMHa/lqP Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJYA6odXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRDQ0I3Q0VGOTE3QTgwMUY0MzA2NEQ3N0Ix NTM5Q0M0MEEwNDlFRTE3AAoJEBU5zECgSe4XV6EH/AvpHlmKFSmZWK4DJyPHkmSb ICqo+WHgnkR5QBvgVeYqMRsFB6mgNjJ92ZpW7XRaVnAs1L9TSZau8jbrSsrfmJ4E SolsMp8UmGNfFQCWC2ZVSt+fw1hI2Ly+BVPTq4kiUlH/9OQt0Roazf/Cn9cpxDBn eYc3HUQCOiKw2lZdU9k9WSSlxmkayamuuRwLcLD/sdNseXBR9qQ7B/doJsSlkSlg WjvnYqrefU/Uv055SwBIVZvAkXGbdBHLEIeP4n1FRlDEDo2465r/uoRuc+984onS JfiBwWBTOnhDaynvfMyv9TCYTCw3UoX2CeleDeQC55YGGimeD16IpJj2X3ut8Zk= =iACX -----END PGP SIGNATURE----- --2NLGdgz3UMHa/lqP-- From owner-freebsd-stable@freebsd.org Sun Oct 16 16:35:03 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8370DC14D2E for ; Sun, 16 Oct 2016 16:35:03 +0000 (UTC) (envelope-from allbery.b@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 62A81120D for ; Sun, 16 Oct 2016 16:35:03 +0000 (UTC) (envelope-from allbery.b@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 5EEACC14D2D; Sun, 16 Oct 2016 16:35:03 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E8C1C14D2C for ; Sun, 16 Oct 2016 16:35:03 +0000 (UTC) (envelope-from allbery.b@gmail.com) Received: from mail-qk0-x22d.google.com (mail-qk0-x22d.google.com [IPv6:2607:f8b0:400d:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19B3D120C for ; Sun, 16 Oct 2016 16:35:03 +0000 (UTC) (envelope-from allbery.b@gmail.com) Received: by mail-qk0-x22d.google.com with SMTP id o68so246879553qkf.3 for ; Sun, 16 Oct 2016 09:35:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=bAXr5ywrl4QM3hvLA8T7BkWgcg/Xi7X7t/RrV9OsGkc=; b=FXByvk8pgN6KMTGZnmPuCh+3kdzLM6ZtuuBlISdgHWGm6v31IeMm0XLgU8CnGOGk0Z Kzh3YjiC21vFb/dCmDlN7nqRw6e18E4Vxp6leEaany0cIboUZysV23GQSBoKtCbvJTv9 NZ42GBDKiZyJygLh/di0Kd3lT6byL9qdX61TI18jVcXcdLWO2S0rNvtC23iUGUxZjmvD 44rfg0TFj4STPu3q5nrD/BHLY0weGYQW00rMp3JZuRolanedT6rUyA4Lzat5XWRUdseq 32ViJW7LuHG7M9ySW6H66iuUzr9EBcTFcSWm58EtUYln0AxVAfB8Qkm9d/c+S9g9Zb3x Pwrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=bAXr5ywrl4QM3hvLA8T7BkWgcg/Xi7X7t/RrV9OsGkc=; b=IXZ4bklwI7bZu7iG2uJ7MTrhFJvixuHeTfdP0eTukKVkso3VHtoLEFOzP3d1sCEWFf thThEtBoHU57cOKfbHrtOYsJ79NMscjH/B8hgUNnv4uviCUytmww5ObYYbb0qvyfP9Ov zRD9Uw5vloed+ArB2CjZjOaF1OTedS/7iCLMB3095IWWruywl6+1dDBrRmh/CX5TNf46 HzhN/HZJu7YKWjOolP9W4GcuEoLm8GnH2ixSyrCWnCbeIwLa50L+DD2mtVFXBWnGzEcC cOc4p4inMHVPYo89I0Ew5ESBBWPyZyaFy44nf1IACYwvv/uU4oJgEHIsKFXuyW1cLTS4 bNgg== X-Gm-Message-State: AA6/9RnpUFTtCPfDfQmeeHU6cesVbWCsZcOGOoMwKtbDQG3XvmJzVC9R0K9G8/oMYHHLbRE0GbEJOz/D6y1KqQ== X-Received: by 10.233.235.85 with SMTP id b82mr23019323qkg.156.1476635702311; Sun, 16 Oct 2016 09:35:02 -0700 (PDT) MIME-Version: 1.0 Received: by 10.233.216.132 with HTTP; Sun, 16 Oct 2016 09:35:01 -0700 (PDT) In-Reply-To: <20161016162605.GG1069@albert.catwhisker.org> References: <20161016162605.GG1069@albert.catwhisker.org> From: Brandon Allbery Date: Sun, 16 Oct 2016 12:35:01 -0400 Message-ID: Subject: Re: sshd whines & dies after releng/10 "freebsd-update" run To: David Wolfskill , "stable@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 16:35:03 -0000 On Sun, Oct 16, 2016 at 12:26 PM, David Wolfskill wrote: > This weekend, though, I was planning to update my other systems tfrom > stable/10 to stable/11, so I figured I'd try freebsd-update on this > machine first. > Wait, you used freebsd-update on a machine running stable? It only supports releases. IOW you may well have *downgraded* the machine in some sense. (Although really it should have just failed in that case.) Also make sure you are not using an sshd from ports; even if such a down/sidegrade works for base, I'd expect it to screw up installed ports. -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net From owner-freebsd-stable@freebsd.org Sun Oct 16 16:41:25 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 132D3C14E36 for ; Sun, 16 Oct 2016 16:41:25 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id EF9811459 for ; Sun, 16 Oct 2016 16:41:24 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id EBE12C14E35; Sun, 16 Oct 2016 16:41:24 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB86CC14E34 for ; Sun, 16 Oct 2016 16:41:24 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A00E31458 for ; Sun, 16 Oct 2016 16:41:24 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id u9GGfN6t005507; Sun, 16 Oct 2016 16:41:23 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id u9GGfMTP005506; Sun, 16 Oct 2016 09:41:22 -0700 (PDT) (envelope-from david) Date: Sun, 16 Oct 2016 09:41:22 -0700 From: David Wolfskill To: Brandon Allbery Cc: "stable@freebsd.org" Subject: Re: sshd whines & dies after releng/10 "freebsd-update" run Message-ID: <20161016164122.GH1069@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , Brandon Allbery , "stable@freebsd.org" References: <20161016162605.GG1069@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="YrlhzR9YrZtruaFS" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.0 (2016-08-17) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 16:41:25 -0000 --YrlhzR9YrZtruaFS Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Oct 16, 2016 at 12:35:01PM -0400, Brandon Allbery wrote: > On Sun, Oct 16, 2016 at 12:26 PM, David Wolfskill > wrote: >=20 > > This weekend, though, I was planning to update my other systems tfrom > > stable/10 to stable/11, so I figured I'd try freebsd-update on this > > machine first. > > >=20 > Wait, you used freebsd-update on a machine running stable? No. My *other* machines run stable (though a couple also sometimes run head). This machine runs releng/10. > It only supports > releases. IOW you may well have *downgraded* the machine in some sense. > (Although really it should have just failed in that case.) Right. > Also make sure you are not using an sshd from ports; even if such a > down/sidegrade works for base, I'd expect it to screw up installed ports. > .... I'm certain that I wouldn't have placed a "ports" sshd in /sur/sbin (even had I installed one), and pkg reports negatively: sisboombah(10.3-RELEASE-p7)[5] pkg info ssh\* pkg: No package(s) matching ssh* sisboombah(10.3-RELEASE-p7)[6]=20 Thanks! Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --YrlhzR9YrZtruaFS Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJYA62yXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRDQ0I3Q0VGOTE3QTgwMUY0MzA2NEQ3N0Ix NTM5Q0M0MEEwNDlFRTE3AAoJEBU5zECgSe4XxEwH/1wj88/qKNrkODj6GuaGA2ST BoMCSH11KoFnnB3M+fkbO/Y56Wpl6Hgv0KZd4KAp/FDavEDLgwx4OKmea5eVy15T 9uA0gFNU3eRb3w4GQTtlPsbVG6ZPxYTgrZN6keUxFMLFPRKJFvBzRbI9Yo/btz/d JyH/1wqNT3L9RlhXOY/wOzlf1uxWR9mOaM64ugk49mWqc2lOfR9IwSMdlLwAt1lC wotToVyVXoCPygpFqTqJvz1oTY+aqkgDQZtKwJ9ADkM0AsO9+EX8xqlrp8+EKFO7 q7WHNNR5QTlgtsqH95cxdh6pbGZ5jv+hOTitHKY/rcqUaL68pagdXBbOgS0oXRQ= =EbjY -----END PGP SIGNATURE----- --YrlhzR9YrZtruaFS-- From owner-freebsd-stable@freebsd.org Sun Oct 16 16:56:16 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30159C14185 for ; Sun, 16 Oct 2016 16:56:16 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 1A1481281 for ; Sun, 16 Oct 2016 16:56:16 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: by mailman.ysv.freebsd.org (Postfix) id 168F4C14184; Sun, 16 Oct 2016 16:56:16 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1643EC14183 for ; Sun, 16 Oct 2016 16:56:16 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 87B80127A for ; Sun, 16 Oct 2016 16:56:15 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:c47f:381:9c23:2301] ([IPv6:2a02:2698:27:b4d9:c47f:381:9c23:2301]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9GGu9mI037484 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Sun, 16 Oct 2016 21:56:10 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1476636970; bh=Pz3vlT8LimicJwbDoXfwOLVAQSFRneTm3aCB7QHRVbA=; h=To:From:Subject:Date; b=HR2UtbwqEyJ3VEI8vQlaPdhDCyiHcrk2/JD3kLesxdzPmxdESMAVAYQiHgZTDsmvE 7HRqlVTPGDEzGESwHZwXUcU/sl0K0PgvfOWVk+g9JB34y+iIu3hkNkROvTv78OwQ/x vqf9O7XxOoLlqtv++OJ3zls7bSKoMvv3piw5RbnI= To: "stable@freebsd.org" From: "Eugene M. Zheganin" Subject: zfs/raidz: seems like I'm failing with math Message-ID: Date: Sun, 16 Oct 2016 21:56:10 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 16:56:16 -0000 Hi. FreeBSD 11.0-RC1 r303979, zfs raidz1: ===Cut=== # zpool status gamestop pool: gamestop state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM gamestop ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 da0 ONLINE 0 0 0 da1 ONLINE 0 0 0 da2 ONLINE 0 0 0 da3 ONLINE 0 0 0 da4 ONLINE 0 0 0 da5 ONLINE 0 0 0 ===Cut=== 6 disks 960 Gbs each: ===Cut=== # smartctl -a /dev/da0 smartctl 6.5 2016-05-07 r4318 [FreeBSD 11.0-RC1 amd64] (local build) Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Samsung based SSDs Device Model: SAMSUNG MZ7KM960HAHP-00005 Serial Number: S2HTNX0H507466 LU WWN Device Id: 5 002538 c402bdac1 Firmware Version: GXM1003Q User Capacity: 960 197 124 096 bytes [960 GB] Sector Size: 512 bytes logical/physical [...] ===Cut=== But: ===Cut=== # zpool list gamestop NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT gamestop 5,22T 4,38T 861G - 24% 83% 1.00x ONLINE - ===Cut=== Why 5.22T ? If zpool is displaying raw size, it should be 960 x 6 = 5760 Gb = 5.65 T. If it's displaying the actual data, then it should be 960 x 5 = 4800 Gb = 4.68 T. 5.22 T is neither of these. I'm stuck, please explain. :) Thanks. Eugene. From owner-freebsd-stable@freebsd.org Sun Oct 16 17:02:46 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B61AC14676 for ; Sun, 16 Oct 2016 17:02:46 +0000 (UTC) (envelope-from doug@mail.sermon-archive.info) Received: from zoom.lafn.org (zoom.lafn.org [108.92.93.123]) by mx1.freebsd.org (Postfix) with ESMTP id 65A431DC2 for ; Sun, 16 Oct 2016 17:02:45 +0000 (UTC) (envelope-from doug@mail.sermon-archive.info) Received: from [10.0.1.4] (unknown [71.177.216.148]) by zoom.lafn.org (Postfix) with ESMTPSA id B2F4A34A9F4; Sun, 16 Oct 2016 10:02:35 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 10.0 \(3226\)) Subject: Re: update from 9.3 to 11.0 From: Doug Hardie In-Reply-To: Date: Sun, 16 Oct 2016 10:02:34 -0700 Cc: Zoran Kolic , FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: <2E339839-8BD3-4B96-ACE3-41FFAD6F8484@mail.sermon-archive.info> References: <20161016141328.GA57825@knossos> To: Warner Losh X-Mailer: Apple Mail (2.3226) X-Virus-Scanned: clamav-milter 0.98 at zoom.lafn.org X-Virus-Status: Clean X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 17:02:46 -0000 > On 16 October 2016, at 08:24, Warner Losh wrote: >=20 > On Sun, Oct 16, 2016 at 8:13 AM, Zoran Kolic wrote: >> I would like to know what experience and tips people on this list = have >> regarding this update. Doug Hardie made it successfully. To my eyes, >> it is the mere change of one charracter in the file. >> I'm a bit late due to lack of time to do the task. >> Best regards all >=20 > I missed the original post. what's the one character change? I was sent the patch below. However, it doesn't apply directly to 9.3. = Edit freebsd-update and go down about 1231 lines. Generally the = comment is found a few lines before that. Add the comma, and all works. -- Doug https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D211398 --- Comment #5 from Xin LI --- (In reply to bc979 from comment #4) Can you try applying this? (r279901) Index: head/usr.sbin/freebsd-update/freebsd-update.sh =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- head/usr.sbin/freebsd-update/freebsd-update.sh (revision = 279900) +++ head/usr.sbin/freebsd-update/freebsd-update.sh (revision = 279901) @@ -1231,7 +1231,7 @@ fetch_metadata_sanity () { # Some aliases to save space later: ${P} is a character which can # appear in a path; ${M} is the four numeric metadata fields; and # ${H} is a sha256 hash. - P=3D"[-+./:=3D%@_[~[:alnum:]]" + P=3D"[-+./:=3D,%@_[~[:alnum:]]" M=3D"[0-9]+\|[0-9]+\|[0-9]+\|[0-9]+" H=3D"[0-9a-f]{64}" From owner-freebsd-stable@freebsd.org Sun Oct 16 17:06:52 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 36418C147F5 for ; Sun, 16 Oct 2016 17:06:52 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 15F591FA7 for ; Sun, 16 Oct 2016 17:06:52 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 123D6C147F0; Sun, 16 Oct 2016 17:06:52 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 11DB9C147EF for ; Sun, 16 Oct 2016 17:06:52 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qk0-x233.google.com (mail-qk0-x233.google.com [IPv6:2607:f8b0:400d:c09::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C1F171FA6 for ; Sun, 16 Oct 2016 17:06:51 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qk0-x233.google.com with SMTP id n189so201568063qke.0 for ; Sun, 16 Oct 2016 10:06:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=ihWESpVP5c1xoXPS6s4+v3R745IYQx+UwW7u+I1egoI=; b=JfCIWRqNmfOVMv7uGAmbYW/Zn6azAPjPQjgHDTTzwrNbXGst3NbIvsPVUrccgVyuMN nttLDx9ux9fPZjGHqGAYIDIPl9hB063K6PIiFAPkYZ94G6e6dcMQgGMZMudfqaqAQwYi WNEIyL9clvo0sZekgBrKramwaJfF4d0KqHOSp+1VZApX96UDRP31oL11FnVHGZPcEZjN S9ADfah8Va0b+hT0+jUfjW7kELsgRfjIV8kH65voZyL7RCvkGX289dGfGPyKS6P63qOb IJn+Ff/eLFm3ZUmmyMa0dsoPsMIePR8/7BPwdK8D6E0FbmQ0302Lu2tQkf5I2Sl7a9M5 w0qA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=ihWESpVP5c1xoXPS6s4+v3R745IYQx+UwW7u+I1egoI=; b=mugc9YTvghpQFRidU9gUw9AdKb7zNL7F35VZD3HtT1Hxt5KE05eviI4qkuhXK5uHqV YHM2qL/EXaaX/afGIjXtIsOKDXV67WcvO8lZweyW6/kUVuAsw1d2HucNUiJUhDFeJkTg 8zRxUJgvhBdChvimWSwoKcEmi+Qg0QWInPDLe1nSV7OqOhGVm5st4pW2MpLbocxtDJjw ORHxONIrczrWhHsA7QCYcTFGGcQN4Qjb5Ua+7BqfcPaclwQVU32pmhV0iDDLrsY07pZq 1gWpPwoW6U8kdnCNlwaDWBEIyk3Z5BW0OO/e18j+SU/CczuE6DkfiZRIrnfUgV0lWtd4 Qd8g== X-Gm-Message-State: AA6/9Rk6YZzXppmTyfrkGZvtqt9KrT5QyMlP9a7FiclZM0e/0lI32g82vuSJWSvNnbHkfwL6GvSl8rpbGmVi6Q== X-Received: by 10.55.164.141 with SMTP id n135mr20027842qke.222.1476637610923; Sun, 16 Oct 2016 10:06:50 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.12.138.251 with HTTP; Sun, 16 Oct 2016 10:06:50 -0700 (PDT) In-Reply-To: References: From: Alan Somers Date: Sun, 16 Oct 2016 11:06:50 -0600 X-Google-Sender-Auth: yKaObTKrkGLBfaYUwnQf2bbsC98 Message-ID: Subject: Re: zfs/raidz: seems like I'm failing with math To: "Eugene M. Zheganin" Cc: "stable@freebsd.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 17:06:52 -0000 On Sun, Oct 16, 2016 at 10:56 AM, Eugene M. Zheganin wrote: > Hi. > > FreeBSD 11.0-RC1 r303979, zfs raidz1: > > ===Cut=== > > # zpool status gamestop > pool: gamestop > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > gamestop ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > da0 ONLINE 0 0 0 > da1 ONLINE 0 0 0 > da2 ONLINE 0 0 0 > da3 ONLINE 0 0 0 > da4 ONLINE 0 0 0 > da5 ONLINE 0 0 0 > > ===Cut=== > > 6 disks 960 Gbs each: > > ===Cut=== > > # smartctl -a /dev/da0 > smartctl 6.5 2016-05-07 r4318 [FreeBSD 11.0-RC1 amd64] (local build) > Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org > > === START OF INFORMATION SECTION === > Model Family: Samsung based SSDs > Device Model: SAMSUNG MZ7KM960HAHP-00005 > Serial Number: S2HTNX0H507466 > LU WWN Device Id: 5 002538 c402bdac1 > Firmware Version: GXM1003Q > User Capacity: 960 197 124 096 bytes [960 GB] > Sector Size: 512 bytes logical/physical > > [...] > > ===Cut=== > > But: > > ===Cut=== > > # zpool list gamestop > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > gamestop 5,22T 4,38T 861G - 24% 83% 1.00x ONLINE - > > ===Cut=== > > Why 5.22T ? If zpool is displaying raw size, it should be 960 x 6 = 5760 Gb > = 5.65 T. If it's displaying the actual data, then it should be 960 x 5 = > 4800 Gb = 4.68 T. 5.22 T is neither of these. I'm stuck, please explain. :) > > > Thanks. > > Eugene. It's raw size, but the discrepancy is between 1000 and 1024. Smartctl is reporting base 10 size, but zpool is reporting base 1024.. 960197124096.0*6/1024**4 = 5.24 TB, which is pretty close to what zpool says. -Alan From owner-freebsd-stable@freebsd.org Sun Oct 16 17:29:11 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3041C14E0F for ; Sun, 16 Oct 2016 17:29:11 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "anubis.delphij.net", Issuer "StartCom Class 1 DV Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B4D12194E for ; Sun, 16 Oct 2016 17:29:11 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from Xins-MBP.iot.rhv.delphij.net (unknown [IPv6:2601:646:8882:7525:c1ee:9a8:c78c:6bd0]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by anubis.delphij.net (Postfix) with ESMTPSA id 999A3D182; Sun, 16 Oct 2016 10:29:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphij.net; s=anubis; t=1476638944; x=1476653344; bh=2uhehaLTzLdu2i3gZQILf9deQ3pDPlOy39qrBvHKSnk=; h=Subject:To:References:Cc:From:Date:In-Reply-To; b=L2f3Xij6ZWIyzAETuGWWfhgFp/X5LK6/RwkU0S6jgLxcf5hHSIwHeG3ygueNRz6b/ Vil4kb4zbwusKsCWijwM1tMXXIvYFpQ6ElMPx7Fa0ewHuADrG+RR0VXPUVQQ5UIgP6 4Z/AmjbyyNOT1vIfIuh+Z80I99vJvsQLp/A05sAI= Subject: Re: sshd whines & dies after releng/10 "freebsd-update" run To: freebsd-stable@freebsd.org References: <20161016162605.GG1069@albert.catwhisker.org> Cc: d@delphij.net From: Xin Li Message-ID: Date: Sun, 16 Oct 2016 10:29:00 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161016162605.GG1069@albert.catwhisker.org> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="6KaIaepDKmP8CqG8GgLmLs2KmCwLs0pxe" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 17:29:11 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --6KaIaepDKmP8CqG8GgLmLs2KmCwLs0pxe Content-Type: multipart/mixed; boundary="B3lE4bjic3FjH3tFPJQ1K3ucgNxkXqCHa"; protected-headers="v1" From: Xin Li To: freebsd-stable@freebsd.org Cc: d@delphij.net Message-ID: Subject: Re: sshd whines & dies after releng/10 "freebsd-update" run References: <20161016162605.GG1069@albert.catwhisker.org> In-Reply-To: <20161016162605.GG1069@albert.catwhisker.org> --B3lE4bjic3FjH3tFPJQ1K3ucgNxkXqCHa Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 10/16/16 09:26, David Wolfskill wrote: > And over the last year or so, it's worked pretty well: I have the > machine set up (as is usually my approach) to be able to boot from > either of a couple of slices. I use a "dump | restore" pipeline > to copy the / and /usr file systems from the "active" slice to the > "inactive" slice, adjust /etc/fstab on the inactive slice to reflect > reality for when it's the boot slice, then (while the file systemms > from the other slice are still mounted -- e.g., on /S2) run > "freebsd-update -b /S2 fetch install", then reboot from the > newly-updated slice. >=20 > In the past, that's Just Worked. Your usage probably worked because you were lucky for a few times in the past. (details below) > This weekend, though, I was planning to update my other systems tfrom > stable/10 to stable/11, so I figured I'd try freebsd-update on this > machine first. >=20 [...] > root@sisboombah:/tmp # `which sshd` -d > Undefined symbol "ssh_compat13" referenced from COPY relocation in /usr= /sbin/sshd >=20 > Any clues? I think this is not going to work (stable/10 -> releng/10.3) due to ABI incompatibility in a downgrade. Basically, freebsd-update is treating your stable/10 as a 10.3-RELEASE installation and will fetch only changes from 10.3-RELEASE to the latest patchlevel. Because of a SSH vulnerability that affects 10.3, freebsd-update would patch libssh (shared library used by sshd and friends), however the change does not affect the main binary. This worked by replacing your existing libssh with the one shipped by freebsd-update (effectively downgraded the library) and that would break sshd. I think upgrade -r 10.2-RELEASE (ideally, 11.0-RELEASE though as it would eliminate the possibility of any potential incompatibility) would work because that would result in a full rewrite of all files. Cheers, --B3lE4bjic3FjH3tFPJQ1K3ucgNxkXqCHa-- --6KaIaepDKmP8CqG8GgLmLs2KmCwLs0pxe Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQIcBAEBCgAGBQJYA7jfAAoJEJW2GBstM+nsDKQQAKnxEzD9J9ZDI0dz84Ijcdh9 +3IkL7dVOsAwjDbw6Pn4W/DLvpXoBKDDd/zwEwuHpAaz/hhk841q5j+LTUFcZ/GC hgNcmM7reETr7GKr9QQMZX//xEnhfONMK/b3zuEcbvVxZvbhxbMSLux6KRqBF6y0 O6tGxTWt4icanVVEeUw6lfgcI6vby8TYT+zBPs0gYjRgHEs5robPEs/qG2lCuRtv tzf/iWKLeTm+vRlL+XIu9xBPz3yjyfUv8+UY6bnNCb/79QKv3FZnmgTBnB01XfCA 6ZBP45EDK/LrqsBGcLzon+ugSF5detq+YoP0yuWgoCfHgEzZBxSwQkQSggCPFksp yOneHtk4Dh15MlR4yHOhyJosCyqU4XVN5hdS2MJpev3RhdvVFEhonjY4fwBX5gi5 Eudlhxsqqk+e54+EM+cIGLHRsDMgbLaKqZYBtblDqXOcrwspGpVKNiPIdUu5tDRH lRFdG7ZIgJLCBnG2gjhAlMiI5inAgLs9omonmXSFHBG5kWH0aDaCCKiwmJmbeAKi kUqj8TQwsYApMWtM1zRRlPHaA4OLFUYH6aRZh6IxZjQvKkYZpFabKnHv22cRBpsQ WKA9QcySmPCFIoy5tdWkx/dXtTpHRTu5rOPEWU0SDJSzHuiWjXgX0cqgeM3BFSHX Xn32o3nRvA7/O7IunhbG =TbFP -----END PGP SIGNATURE----- --6KaIaepDKmP8CqG8GgLmLs2KmCwLs0pxe-- From owner-freebsd-stable@freebsd.org Sun Oct 16 17:45:53 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B7C4C1425B for ; Sun, 16 Oct 2016 17:45:53 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 149E41196 for ; Sun, 16 Oct 2016 17:45:52 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id u9GHjemL006285; Sun, 16 Oct 2016 17:45:40 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id u9GHjegQ006284; Sun, 16 Oct 2016 10:45:40 -0700 (PDT) (envelope-from david) Date: Sun, 16 Oct 2016 10:45:40 -0700 From: David Wolfskill To: Xin Li Cc: freebsd-stable@freebsd.org, d@delphij.net Subject: Re: sshd whines & dies after releng/10 "freebsd-update" run Message-ID: <20161016174540.GI1069@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , Xin Li , freebsd-stable@freebsd.org, d@delphij.net References: <20161016162605.GG1069@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="ijf6z65S790CMqo8" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.0 (2016-08-17) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 17:45:53 -0000 --ijf6z65S790CMqo8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Oct 16, 2016 at 10:29:00AM -0700, Xin Li wrote: > ...=20 > On 10/16/16 09:26, David Wolfskill wrote: > > And over the last year or so, it's worked pretty well: I have the > > machine set up (as is usually my approach) to be able to boot from > > either of a couple of slices. I use a "dump | restore" pipeline > > to copy the / and /usr file systems from the "active" slice to the > > "inactive" slice, adjust /etc/fstab on the inactive slice to reflect > > reality for when it's the boot slice, then (while the file systemms > > from the other slice are still mounted -- e.g., on /S2) run > > "freebsd-update -b /S2 fetch install", then reboot from the > > newly-updated slice. > >=20 > > In the past, that's Just Worked. >=20 > Your usage probably worked because you were lucky for a few times in the > past. (details below) >=20 > > This weekend, though, I was planning to update my other systems tfrom > > stable/10 to stable/11, so I figured I'd try freebsd-update on this > > machine first. > >=20 > [...] > > root@sisboombah:/tmp # `which sshd` -d > > Undefined symbol "ssh_compat13" referenced from COPY relocation in /usr= /sbin/sshd > >=20 > > Any clues? >=20 > I think this is not going to work (stable/10 -> releng/10.3) due to ABI > incompatibility in a downgrade. I seem to have failed to commnunicate clearly: The machine in question does not, and has not, run "stable". It runs releng. At the moment (on the "old" slice), it reports: sisboombah(10.3-RELEASE-p7)[1] uname -a FreeBSD sisboombah.catwhisker.org 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 #= 0: Thu Aug 11 18:38:15 UTC 2016 root@amd64-builder.daemonology.net:/usr= /obj/usr/src/sys/GENERIC amd64 sisboombah(10.3-RELEASE-p7)[2]=20 > Basically, freebsd-update is treating your stable/10 as a 10.3-RELEASE > installation and will fetch only changes from 10.3-RELEASE to the latest > patchlevel. I can see that... if the machine were running stable. > Because of a SSH vulnerability that affects 10.3, freebsd-update would > patch libssh (shared library used by sshd and friends), however the > change does not affect the main binary. This worked by replacing your > existing libssh with the one shipped by freebsd-update (effectively > downgraded the library) and that would break sshd. As a reality check: sisboombah(10.3-RELEASE-p7)[4] sudo mount /S2 Password: sisboombah(10.3-RELEASE-p7)[5] sudo mount /S2/usr sisboombah(10.3-RELEASE-p7)[6] ls -lT {,/S2}/usr/lib/private/libssh.so.* -r--r--r-- 1 root wheel 634232 Oct 16 11:57:32 2016 /S2/usr/lib/private/= libssh.so.5 -r--r--r-- 1 root wheel 569864 Jun 5 13:37:52 2016 /usr/lib/private/lib= ssh.so.5 sisboombah(10.3-RELEASE-p7)[7] ls -lT {,/S2}/usr/sbin/ssh* -r-xr-xr-x 1 root wheel 297736 Jun 5 13:38:35 2016 /S2/usr/sbin/sshd -r-xr-xr-x 1 root wheel 297736 Jun 5 13:38:35 2016 /usr/sbin/sshd sisboombah(10.3-RELEASE-p7)[8]=20 > I think upgrade -r 10.2-RELEASE (ideally, 11.0-RELEASE though as it > would eliminate the possibility of any potential incompatibility) would > work because that would result in a full rewrite of all files. Well, I had seen reports of folks having "issues" with attempts to use freebsd-update to get to releng/11 from systems that weren't as up-to-date as they might be; I was actually trying to avoid a problem.... :-} Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --ijf6z65S790CMqo8 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJYA7zEXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRDQ0I3Q0VGOTE3QTgwMUY0MzA2NEQ3N0Ix NTM5Q0M0MEEwNDlFRTE3AAoJEBU5zECgSe4X4g4H/jhmqtzk5w0tdx4ZXM/AQL2K weSyfcBo/Z2UkNbY2eI4xa8tGNdrePQBLv8Ezp33aDSJ1/nltbUqC3FdtxQrkWmZ EDm8OdA9n9zUD7dFz3gof/Pr7onB2CcCZ7XAzfdXr/pLx3qQ1qLEM81Tv3rfN4+O pmo3SaBTxLN8qqjQPBJJmN00uH5r94gJUMaTlkQzJXgBllOCytVyy0bOyJNfRakv C6nvIT4aNXtzXyKUxGblQah1ckQb8JGf6Z1i3izGgO1zM111r9nbjdqCX8vuAQZM teueoaSPoVzFqigZV/ycpLMyUA3z+ylDq5OL7f6W2UuLnVKWmzeEKvp+pKJDswk= =XWKl -----END PGP SIGNATURE----- --ijf6z65S790CMqo8-- From owner-freebsd-stable@freebsd.org Sun Oct 16 18:08:28 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 881BFC145BD; Sun, 16 Oct 2016 18:08:28 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4D11F1919; Sun, 16 Oct 2016 18:08:27 +0000 (UTC) (envelope-from wblock@wonkity.com) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.15.2/8.15.2) with ESMTPS id u9GI8QqZ016081 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 16 Oct 2016 12:08:26 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.15.2/8.15.2/Submit) with ESMTP id u9GI8Qom016078; Sun, 16 Oct 2016 12:08:26 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Sun, 16 Oct 2016 12:08:26 -0600 (MDT) From: Warren Block To: George Mitchell cc: Anthony Jenkins , freebsd-usb@freebsd.org, FreeBSD Stable Mailing List Subject: Re: Clandestine USB SD card slot In-Reply-To: <82d2acb3-12cd-678e-c90a-21b9d62b5721@m5p.com> Message-ID: References: <832db721-d03a-a21e-11ed-701190140d17@yahoo.com> <2bf09be6-e2ad-2d38-ffd8-d80b45d4c3d8@m5p.com> <82d2acb3-12cd-678e-c90a-21b9d62b5721@m5p.com> User-Agent: Alpine 2.20 (BSF 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Sun, 16 Oct 2016 12:08:26 -0600 (MDT) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 18:08:28 -0000 On Sun, 16 Oct 2016, George Mitchell wrote: >> So not only is it (apparently) recognized, but the sdhci_pci driver >> attached to it! But inserting or removing a card shows no activity. >> What's my next step? -- George Is a device created for the empty reader? It's worth trying to force a retaste of that device with 'true > /dev/daX' after the card is inserted. From owner-freebsd-stable@freebsd.org Sun Oct 16 18:16:24 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 326E1C148F8 for ; Sun, 16 Oct 2016 18:16:24 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x230.google.com (mail-it0-x230.google.com [IPv6:2607:f8b0:4001:c0b::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EE6F31DAB for ; Sun, 16 Oct 2016 18:16:23 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x230.google.com with SMTP id e187so8249207itc.1 for ; Sun, 16 Oct 2016 11:16:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=ny2IceM840p1dnAk7wVF0iTv5eA2bhi1lZEVKPlf1TI=; b=hmevlfEm60QcxxN5tpEhtuRKc2Xw5DpT3+ySZ1TWZuyiEpY4umTjfYa5h2K9QMgdUT dqiE2NdCA484Rt7RQLiqoIm1suuqfWrIz0h8pQ2RYu7FMQJiIL7gxAOzNxXJPkxOdz0M lZCMIjvs2wvEaUba+GkIBQb0c4ZnfQOw8Etwikr4utvU3l5qD5qiRp2ER9VWfBBNwckM R1fWofCGhm4IG8ZclWoVp+7x1npGigAR9V5vdP4gH5Hi8kntyx4ggSrCCqEyBlzLmvpr 9dvm+gs4qa81IldxKCYieCMynAQMv1VuJ+Uyl0NwyN35yOXTeAOy/pRkrRtaxMo4zd5j OnpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=ny2IceM840p1dnAk7wVF0iTv5eA2bhi1lZEVKPlf1TI=; b=eCdZwUjXVQnPklkMie6C5CTvY07n2mgaDqlMCeGGREdd2uD1nqf4qQxxg7LenpLnXe onYCNSDg3KbOnm0u5LGpC6Wm0e9Ws+Vt1VfgNfMAsN5LMOwvnEhT2igV3ihDe5EuHLW0 iVGtEmd8UUAV8lPyLlwEKQJ4bezo1YQuHYwrhGk13hKBYjpckT13q0czHivj40pjBo2g MABcWsYI6o2k147NdhvShfwFh58S+GhI7N3HO8V/p03iOYIF5vS9MVdo61LvMVRegmW3 u+owu34SG3aXh1JmfLkBV/DUKWmSgrOI2rbwRaNxhq5knz7tQ9+e9BR1wfiqaJyxRleb Ug6g== X-Gm-Message-State: AA6/9RkVbvnzxR2ovXNxgfS/coPpCFioBNrpC9gIK5Jfiar4YGzCOL6dQkags7fEP6JV9wbleHm3vcldVxqJJQ== X-Received: by 10.36.43.82 with SMTP id h79mr6342776ita.60.1476641783300; Sun, 16 Oct 2016 11:16:23 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.77.85 with HTTP; Sun, 16 Oct 2016 11:16:22 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: References: <832db721-d03a-a21e-11ed-701190140d17@yahoo.com> <2bf09be6-e2ad-2d38-ffd8-d80b45d4c3d8@m5p.com> <82d2acb3-12cd-678e-c90a-21b9d62b5721@m5p.com> From: Warner Losh Date: Sun, 16 Oct 2016 12:16:22 -0600 X-Google-Sender-Auth: bpK0nWBnWFjHzsl8TjLSAwwr2G0 Message-ID: Subject: Re: Clandestine USB SD card slot To: Warren Block Cc: George Mitchell , FreeBSD Stable Mailing List , "freebsd-usb@FreeBSD.org" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 18:16:24 -0000 On Sun, Oct 16, 2016 at 12:08 PM, Warren Block wrote: > On Sun, 16 Oct 2016, George Mitchell wrote: > >>> So not only is it (apparently) recognized, but the sdhci_pci driver >>> attached to it! But inserting or removing a card shows no activity. >>> What's my next step? -- George > > > Is a device created for the empty reader? It's worth trying to force a > retaste of that device with 'true > /dev/daX' after the card is inserted. Don't look for da anything. Look for mmcsd something. The sdhci_pci driver provides disks that are mmcsdX. Looks like card change interrupts aren't happening, or there's something else making the driver unhappy with the SDHCI controller though... Warner From owner-freebsd-stable@freebsd.org Sun Oct 16 18:19:06 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 63B91C14AFD for ; Sun, 16 Oct 2016 18:19:06 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "tensor.andric.com", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2193D10DB; Sun, 16 Oct 2016 18:19:06 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from [IPv6:2001:7b8:3a7::c36:9109:5a15:ae78] (unknown [IPv6:2001:7b8:3a7:0:c36:9109:5a15:ae78]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id EC5F11266; Sun, 16 Oct 2016 20:19:03 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_CE82138D-E19C-40DD-BF2D-FA7086897965"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: Building FreeBSD 11.0-stable on FreeBSD 10.1-stable fails From: Dimitry Andric In-Reply-To: Date: Sun, 16 Oct 2016 20:18:45 +0200 Cc: Bryan Drewery , Torfinn Ingolfsen , FreeBSD-STABLE Mailing List Message-Id: <84890108-CB52-4B2B-B795-C650879469E9@FreeBSD.org> References: <20161016122041.e20ae504c58d4f9224eec6bf@getmail.no> <426D0291-D785-4CFE-A15D-9BE688499ACB@FreeBSD.org> To: Warner Losh X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 18:19:06 -0000 --Apple-Mail=_CE82138D-E19C-40DD-BF2D-FA7086897965 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 16 Oct 2016, at 17:22, Warner Losh wrote: >=20 > On Sun, Oct 16, 2016 at 5:34 AM, Dimitry Andric = wrote: >> On 16 Oct 2016, at 12:20, Torfinn Ingolfsen = wrote: >>> I am trying to build FreeBSD 11.0-stable on a machine which runs: >>> tingo@kg-v7$ uname -a >>> FreeBSD kg-v7.kg4.no 10.1-STABLE FreeBSD 10.1-STABLE #0 r278322: Fri = Feb 6 21:36:01 CET 2015 >>> root@kg-v7.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 >>>=20 >>> I have emptied /usr/src and /usr/obj and fetched the latest = stable/11 via subversion: >>> tingo@kg-v7$ egrep "^BRANCH|^REVISION" /usr/src/sys/conf/newvers.sh >>> REVISION=3D"11.0" >>> BRANCH=3D"STABLE" >>>=20 >>> But building it (per the procedure in the handbook) fails at the = buildworld stage. Both 'make -j5 buildworld' and 'make buildworld' = fails, like this: >>>=20 >>> c++: error: unable to execute command: Segmentation fault (core = dumped) ... >> Please make sure your stable/10 is at least r286033. >=20 > What's the issue this fixes? It fixes a possible crash in clang 3.4, which can occur if newer versions of llvm are compiled. Unfortunately this fix only went in after 10.3-RELEASE. > Right now we have safeties in place in > buildworld that indicate we 'support' back to 9 sometime. If that's > not really the case, we need to fix something (either fix so we don't > need this specific revision, or fix the Makefile to indicate we don't > support back that far). The fix was also merged to stable/9 in r286035. As far as I know, we have always required people to upgrade to the latest revision in stable branches before attempting to hop to the next stable branch (or head). -Dimitry --Apple-Mail=_CE82138D-E19C-40DD-BF2D-FA7086897965 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.30 iEYEARECAAYFAlgDxJcACgkQsF6jCi4glqOBlgCbB9sw/whzahXYqwhg2CfvwBCB 9+8AoIgfTLI1DKCKXte9qyH2n5is8/tT =oTUE -----END PGP SIGNATURE----- --Apple-Mail=_CE82138D-E19C-40DD-BF2D-FA7086897965-- From owner-freebsd-stable@freebsd.org Sun Oct 16 18:26:23 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 85D31C14D19 for ; Sun, 16 Oct 2016 18:26:23 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7360915E2 for ; Sun, 16 Oct 2016 18:26:23 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: by mailman.ysv.freebsd.org (Postfix) id 6FAA0C14D18; Sun, 16 Oct 2016 18:26:23 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F40CC14D17 for ; Sun, 16 Oct 2016 18:26:23 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B0BD15E1 for ; Sun, 16 Oct 2016 18:26:23 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bvq8T-0006Bh-QW for stable@freebsd.org; Sun, 16 Oct 2016 21:26:13 +0300 Date: Sun, 16 Oct 2016 21:26:13 +0300 From: Slawa Olhovchenkov To: "stable@freebsd.org" Subject: stable/11: lock contention on zone_fetch_slab Message-ID: <20161016182613.GE57876@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 18:26:23 -0000 @ CPU_CLK_UNHALTED_CORE [271718 samples] 22.48% [61081] lock_delay @ /boot/kernel/kernel 99.72% [60908] __mtx_lock_sleep 67.69% [41230] zone_fetch_slab 100.0% [41230] zone_import 100.0% [41230] zone_alloc_item 99.99% [41226] uma_zalloc_arg 74.55% [30732] m_getm2 100.0% [30732] m_uiotombuf 100.0% [30732] sosend_generic 99.93% [30711] soo_write 100.0% [30711] dofilewrite 100.0% [30711] kern_writev 99.37% [30518] sys_writev 100.0% [30518] amd64_syscall 00.63% [193] sys_write 100.0% [193] amd64_syscall 00.07% [21] kern_sendit 100.0% [21] sendit 100.0% [21] sys_sendto 100.0% [21] amd64_syscall 20.84% [8590] m_copym 100.0% [8590] tcp_output 96.18% [8262] tcp_usr_send 100.0% [8262] sosend_generic 99.67% [8235] soo_write 100.0% [8235] dofilewrite 100.0% [8235] kern_writev 99.38% [8184] sys_writev 100.0% [8184] amd64_syscall 00.62% [51] sys_write 100.0% [51] amd64_syscall 00.33% [27] kern_sendit 100.0% [27] sendit 100.0% [27] sys_sendto 100.0% [27] amd64_syscall 03.82% [328] tcp_timer_rexmt 100.0% [328] softclock_call_cc 100.0% [328] softclock 100.0% [328] intr_event_execute_handlers 100.0% [328] ithread_loop 100.0% [328] fork_exit 04.55% [1874] tcp_output 75.72% [1419] tcp_usr_send 100.0% [1419] sosend_generic 98.38% [1396] soo_write 100.0% [1396] dofilewrite 100.0% [1396] kern_writev 87.11% [1216] sys_writev 100.0% [1216] amd64_syscall 12.89% [180] sys_write 100.0% [180] amd64_syscall 01.62% [23] kern_sendit 100.0% [23] sendit 100.0% [23] sys_sendto 100.0% [23] amd64_syscall 13.07% [245] tcp_timer_rexmt 100.0% [245] softclock_call_cc 100.0% [245] softclock 100.0% [245] intr_event_execute_handlers 100.0% [245] ithread_loop 100.0% [245] fork_exit 06.46% [121] tcp_timer_delack 100.0% [121] softclock_call_cc 100.0% [121] softclock 100.0% [121] intr_event_execute_handlers 100.0% [121] ithread_loop 100.0% [121] fork_exit 02.99% [56] tcp_do_segment 100.0% [56] tcp_input 100.0% [56] ip_input 100.0% [56] swi_net 100.0% [56] intr_event_execute_handlers 100.0% [56] ithread_loop 100.0% [56] fork_exit 01.55% [29] tcp_usr_disconnect 100.0% [29] soclose 100.0% [29] _fdrop 100.0% [29] closef 100.0% [29] closefp 100.0% [29] amd64_syscall 00.11% [2] tcp_timer_persist 100.0% [2] softclock_call_cc 100.0% [2] softclock 100.0% [2] intr_event_execute_handlers 100.0% [2] ithread_loop 100.0% [2] fork_exit 00.11% [2] tcp_drop 100.0% [2] tcp_timer_rexmt 100.0% [2] softclock_call_cc 100.0% [2] softclock 100.0% [2] intr_event_execute_handlers 100.0% [2] ithread_loop 100.0% [2] fork_exit 00.07% [28] syncache_respond 100.0% [28] syncache_timer 100.0% [28] softclock_call_cc 100.0% [28] softclock 100.0% [28] intr_event_execute_handlers 100.0% [28] ithread_loop 100.0% [28] fork_exit 00.00% [1] kqueue_register 100.0% [1] kqfd_register 100.0% [1] aio_aqueue 100.0% [1] amd64_syscall 00.00% [1] mb_zinit_pack 100.0% [1] zone_alloc_item 100.0% [1] uma_zalloc_arg 100.0% [1] m_getm2 100.0% [1] m_uiotombuf 100.0% [1] sosend_generic 100.0% [1] soo_write 100.0% [1] dofilewrite 100.0% [1] kern_writev 100.0% [1] sys_writev 00.01% [4] keg_alloc_slab 100.0% [4] keg_fetch_slab 100.0% [4] zone_fetch_slab 100.0% [4] zone_import 100.0% [4] zone_alloc_item 100.0% [4] uma_zalloc_arg 100.0% [4] m_getm2 100.0% [4] m_uiotombuf 100.0% [4] sosend_generic 100.0% [4] soo_write 100.0% [4] dofilewrite 16.50% [10051] ixgbe_deferred_mq_start @ /boot/kernel/if_ix.ko 100.0% [10051] taskqueue_run_locked @ /boot/kernel/kernel 100.0% [10051] taskqueue_thread_loop 100.0% [10051] fork_exit 12.37% [7537] zone_release 100.0% [7537] uma_zfree_arg 90.83% [6846] m_freem 97.14% [6650] ixgbe_txeof @ /boot/kernel/if_ix.ko 100.0% [6650] ixgbe_mq_start_locked 92.33% [6140] ixgbe_mq_start 100.0% [6140] lagg_transmit @ /boot/kernel/if_lagg.ko 100.0% [6140] ether_output @ /boot/kernel/kernel 100.0% [6140] ip_output 100.0% [6140] tcp_output 94.72% [5816] tcp_usr_send 100.0% [5816] sosend_generic 100.0% [5816] soo_write 100.0% [5816] dofilewrite 05.28% [324] tcp_timer_rexmt 100.0% [324] softclock_call_cc 100.0% [324] softclock 100.0% [324] intr_event_execute_handlers 07.67% [510] ixgbe_deferred_mq_start @ /boot/kernel/if_ix.ko 100.0% [510] taskqueue_run_locked @ /boot/kernel/kernel 100.0% [510] taskqueue_thread_loop 100.0% [510] fork_exit 02.86% [196] tcp_do_segment 100.0% [196] tcp_input 100.0% [196] ip_input 100.0% [196] swi_net 100.0% [196] intr_event_execute_handlers 100.0% [196] ithread_loop 100.0% [196] fork_exit 05.04% [380] mb_free_ext 82.63% [314] soreceive_generic 100.0% [314] kern_recvit 100.0% [314] sys_recvfrom 100.0% [314] amd64_syscall 13.42% [51] m_freem 70.59% [36] soisdisconnected 100.0% [36] tcp_close 100.0% [36] tcp_timer_rexmt 100.0% [36] softclock_call_cc 100.0% [36] softclock 100.0% [36] intr_event_execute_handlers 100.0% [36] ithread_loop 100.0% [36] fork_exit 23.53% [12] ixgbe_txeof @ /boot/kernel/if_ix.ko 100.0% [12] ixgbe_mq_start_locked 100.0% [12] ixgbe_mq_start 100.0% [12] lagg_transmit @ /boot/kernel/if_lagg.ko 100.0% [12] ether_output @ /boot/kernel/kernel 100.0% [12] ip_output 100.0% [12] tcp_output 91.67% [11] tcp_usr_send 100.0% [11] sosend_generic 100.0% [11] soo_write 08.33% [1] tcp_timer_rexmt 100.0% [1] softclock_call_cc 100.0% [1] softclock 03.92% [2] tcp_usr_send 100.0% [2] sosend_generic 100.0% [2] soo_write 100.0% [2] dofilewrite 100.0% [2] kern_writev 100.0% [2] sys_writev 100.0% [2] amd64_syscall 01.96% [1] sbflush 100.0% [1] tcp_disconnect 100.0% [1] tcp_usr_disconnect 100.0% [1] soclose 100.0% [1] _fdrop 100.0% [1] closef 100.0% [1] closefp 100.0% [1] amd64_syscall 03.95% [15] sbcompress 100.0% [15] sbappendstream 100.0% [15] tcp_usr_send 100.0% [15] sosend_generic 100.0% [15] soo_write 100.0% [15] dofilewrite 100.0% [15] kern_writev 86.67% [13] sys_write 100.0% [13] amd64_syscall 13.33% [2] sys_writev 100.0% [2] amd64_syscall 02.12% [160] sbcompress 56.25% [90] sbappendstream_locked 100.0% [90] tcp_do_segment 100.0% [90] tcp_input 100.0% [90] ip_input 100.0% [90] swi_net 100.0% [90] intr_event_execute_handlers 100.0% [90] ithread_loop 100.0% [90] fork_exit 43.75% [70] sbappendstream 100.0% [70] tcp_usr_send 100.0% [70] sosend_generic 100.0% [70] soo_write 100.0% [70] dofilewrite 100.0% [70] kern_writev 98.57% [69] sys_write 100.0% [69] amd64_syscall 01.43% [1] sys_writev 100.0% [1] amd64_syscall 02.00% [151] soreceive_generic 51.66% [78] kern_recvit 100.0% [78] sys_recvfrom 100.0% [78] amd64_syscall 48.34% [73] dofileread 100.0% [73] kern_readv 100.0% [73] sys_read 100.0% [73] amd64_syscall 00.70% [427] aio_aqueue 100.0% [427] amd64_syscall 00.65% [396] aio_clear_cancel_function 100.0% [396] aio_daemon 100.0% [396] fork_exit 00.53% [322] _vm_page_deactivate 100.0% [322] vm_object_madvise 100.0% [322] vm_map_madvise 100.0% [322] sys_madvise 100.0% [322] amd64_syscall 00.38% [234] keg_alloc_slab 100.0% [234] keg_fetch_slab 100.0% [234] zone_fetch_slab 100.0% [234] zone_import 100.0% [234] zone_alloc_item 100.0% [234] uma_zalloc_arg 100.0% [234] m_getm2 100.0% [234] m_uiotombuf 100.0% [234] sosend_generic 100.0% [234] soo_write 100.0% [234] dofilewrite 100.0% [234] kern_writev 100.0% [234] sys_writev 100.0% [234] amd64_syscall 00.21% [129] aio_complete 100.0% [129] aio_process_rw 100.0% [129] aio_daemon 100.0% [129] fork_exit 00.19% [115] vm_page_alloc 66.96% [77] uma_small_alloc 100.0% [77] keg_alloc_slab 100.0% [77] keg_fetch_slab 100.0% [77] zone_fetch_slab 100.0% [77] zone_import 100.0% [77] zone_alloc_item 100.0% [77] uma_zalloc_arg 100.0% [77] m_getm2 100.0% [77] m_uiotombuf 100.0% [77] sosend_generic 100.0% [77] soo_write 100.0% [77] dofilewrite 100.0% [77] kern_writev 20.00% [23] vm_fault_hold 91.30% [21] vm_fault_quick_hold_pages 100.0% [21] vn_io_fault1 100.0% [21] vn_io_fault 100.0% [21] aio_process_rw 100.0% [21] aio_daemon 100.0% [21] fork_exit 08.70% [2] vm_fault 100.0% [2] trap_pfault 100.0% [2] trap 13.04% [15] kmem_back 100.0% [15] kmem_malloc 100.0% [15] keg_alloc_slab 100.0% [15] keg_fetch_slab 100.0% [15] zone_fetch_slab 100.0% [15] zone_import 53.33% [8] zone_alloc_item 100.0% [8] uma_zalloc_arg 100.0% [8] arc_get_data_buf @ /boot/kernel/zfs.ko 100.0% [8] arc_buf_alloc 100.0% [8] dbuf_read 100.0% [8] dmu_tx_check_ioerr 100.0% [8] dmu_tx_count_write 46.67% [7] uma_zalloc_arg @ /boot/kernel/kernel 100.0% [7] arc_get_data_buf @ /boot/kernel/zfs.ko 100.0% [7] arc_buf_alloc 100.0% [7] dbuf_read 100.0% [7] dmu_tx_check_ioerr 100.0% [7] dmu_tx_count_write 100.0% [7] dmu_tx_hold_write From owner-freebsd-stable@freebsd.org Sun Oct 16 18:34:48 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BA2E6C14106 for ; Sun, 16 Oct 2016 18:34:48 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7FE281D16 for ; Sun, 16 Oct 2016 18:34:48 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22b.google.com with SMTP id 139so25174000itm.1 for ; Sun, 16 Oct 2016 11:34:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=BxsxLLCkxXHeUmKj+JnC6PRMpD65EoO7ZKqnPEOZPYY=; b=ZDMmvv6Fh70TnqSNTDyl86e1ebey8qifAXGkSiELmagiNKFdZQ45fnE31K3tKOrUty BjGuxubcEAXyqmCgQ45GapLnB4MyOWs2vrguA4LmdNdnuEoft4nQzuCagxAtpbKoL7f3 VPe2r6qUrNCU9DPEQ3rsRDUMntUkkaayVCdWDcDLQy6MjW5+Mr0k3stM2zR3diNcyiNl MlG1vJ56A1CtaPPevgVU3YLxdEx9Xn73VF76Qp/my+1R4hvKTIVXxhGjGKboyCUDbdZf aJ4RMpAbxZkmhWKNzfZMm37UMI+xatwsg36KON1uARXQdaKPSibYzidMQBaIOmIdnggp u1sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=BxsxLLCkxXHeUmKj+JnC6PRMpD65EoO7ZKqnPEOZPYY=; b=fhz2/V0ox1EtjyD0zlS1IveZFVNzE4KmlAcoqCQBkWm2WLFFqAYf3Mvsq4opW3TZHw 6s0sBHynY3GKED2rbFTbG9jWo0deYpHWc+rcp1/a9N0E60KkWpOU91w+m2bwetHLMzxS 4iyftJrtLHpw2QAd7WPgMZZwFUKl5g4xu24e0BDM9lROqFO9N6fYDHI+95GZYpoqVd0M dCC7gdHuIs/2RBQUoTvXn0LABu9xzYdZlToBDdlPMPWtpBSyui4wVohxh8eMbt6hiANh LIQS0cOgNIrfKxvqF8aZPssiBLOy7SfgYJvDtRLqAYWr1604cfRG4aOZknxr48kWuIPb DMeA== X-Gm-Message-State: AA6/9RkzpmYmn9C/DYOhgpc+3TsVTmI1dg5XuOXZPbEAqrcYN/LduKj08sUUSCSZ4c/3a8O2tt9iVSC5vGptHQ== X-Received: by 10.36.43.82 with SMTP id h79mr6388816ita.60.1476642887811; Sun, 16 Oct 2016 11:34:47 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.77.85 with HTTP; Sun, 16 Oct 2016 11:34:47 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: <84890108-CB52-4B2B-B795-C650879469E9@FreeBSD.org> References: <20161016122041.e20ae504c58d4f9224eec6bf@getmail.no> <426D0291-D785-4CFE-A15D-9BE688499ACB@FreeBSD.org> <84890108-CB52-4B2B-B795-C650879469E9@FreeBSD.org> From: Warner Losh Date: Sun, 16 Oct 2016 12:34:47 -0600 X-Google-Sender-Auth: yYd3oYSdgIJpL_e2X3YE7TLRhRk Message-ID: Subject: Re: Building FreeBSD 11.0-stable on FreeBSD 10.1-stable fails To: Dimitry Andric Cc: Bryan Drewery , Torfinn Ingolfsen , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 18:34:48 -0000 On Sun, Oct 16, 2016 at 12:18 PM, Dimitry Andric wrote: > On 16 Oct 2016, at 17:22, Warner Losh wrote: >> >> On Sun, Oct 16, 2016 at 5:34 AM, Dimitry Andric wrote: >>> On 16 Oct 2016, at 12:20, Torfinn Ingolfsen wrote: >>>> I am trying to build FreeBSD 11.0-stable on a machine which runs: >>>> tingo@kg-v7$ uname -a >>>> FreeBSD kg-v7.kg4.no 10.1-STABLE FreeBSD 10.1-STABLE #0 r278322: Fri Feb 6 21:36:01 CET 2015 >>>> root@kg-v7.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 >>>> >>>> I have emptied /usr/src and /usr/obj and fetched the latest stable/11 via subversion: >>>> tingo@kg-v7$ egrep "^BRANCH|^REVISION" /usr/src/sys/conf/newvers.sh >>>> REVISION="11.0" >>>> BRANCH="STABLE" >>>> >>>> But building it (per the procedure in the handbook) fails at the buildworld stage. Both 'make -j5 buildworld' and 'make buildworld' fails, like this: >>>> >>>> c++: error: unable to execute command: Segmentation fault (core dumped) > ... >>> Please make sure your stable/10 is at least r286033. >> >> What's the issue this fixes? > > It fixes a possible crash in clang 3.4, which can occur if newer > versions of llvm are compiled. Unfortunately this fix only went in > after 10.3-RELEASE. > > >> Right now we have safeties in place in >> buildworld that indicate we 'support' back to 9 sometime. If that's >> not really the case, we need to fix something (either fix so we don't >> need this specific revision, or fix the Makefile to indicate we don't >> support back that far). > > The fix was also merged to stable/9 in r286035. As far as I know, we > have always required people to upgrade to the latest revision in stable > branches before attempting to hop to the next stable branch (or head). No. that's not always been the case. It hasn't been the case since Ruslan Ermilov's efforts to support from the 4.x branch point upgrades to 5, 6 and 7. We've well supported upgrading from older versions for 15 years or so. It's been documented in Makefile.inc1 for all that time, and the documented version has generally worked for much of that time. There was a policy that pre-dated this stating only tip of stable to next stable, but that policy has been found to be needlessly restrictive so people like Juniper have made sure that older -> newer builds have been working for maybe the last 5-8 years. It causes them big logistical issues to upgrade their build servers too often due to dependencies that most people don't have on non-public third-party code that doesn't gracefully upgrade. So not since the 3.x -> 4.x days has there been a requirement to upgrade to tip of stable of X-1 before building X. It's usually X-2 or X-3 and generally all the release points on the branch, or at least most of the branch. If we can't do it now because of old bugs in releases that we can't work around (which this sounds like), we'll need to fix that in the versions we allow to be correct. Generally, we've been able to work around issues, but this one sounds to be almost impossible w/o host modification. In addition to fixing the version in Makefile,inc1, we'll need to document this change from past expectations in the release notes (retroactively), in UPDATING and in the handbook. Suffice to say, this is a big problem that sadly we can only fix with documentation. Warner From owner-freebsd-stable@freebsd.org Sun Oct 16 18:42:58 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 74CE1C14500 for ; Sun, 16 Oct 2016 18:42:58 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 626F2138F for ; Sun, 16 Oct 2016 18:42:58 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 61CFEC144FF; Sun, 16 Oct 2016 18:42:58 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 61780C144FE for ; Sun, 16 Oct 2016 18:42:58 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2DBFD138E for ; Sun, 16 Oct 2016 18:42:58 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bvqOe-000CsX-2d; Sun, 16 Oct 2016 19:42:56 +0100 Date: Sun, 16 Oct 2016 19:42:55 +0100 From: Gary Palmer To: "Eugene M. Zheganin" Cc: "stable@freebsd.org" Subject: Re: zfs/raidz: seems like I'm failing with math Message-ID: <20161016184255.GA2241@in-addr.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 18:42:58 -0000 On Sun, Oct 16, 2016 at 09:56:10PM +0500, Eugene M. Zheganin wrote: > Hi. > > FreeBSD 11.0-RC1 r303979, zfs raidz1: > > ===Cut=== > > # zpool status gamestop > pool: gamestop > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > gamestop ONLINE 0 0 0 > raidz1-0 ONLINE 0 0 0 > da0 ONLINE 0 0 0 > da1 ONLINE 0 0 0 > da2 ONLINE 0 0 0 > da3 ONLINE 0 0 0 > da4 ONLINE 0 0 0 > da5 ONLINE 0 0 0 > > ===Cut=== > > 6 disks 960 Gbs each: > > ===Cut=== > > # smartctl -a /dev/da0 > smartctl 6.5 2016-05-07 r4318 [FreeBSD 11.0-RC1 amd64] (local build) > Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org > > === START OF INFORMATION SECTION === > Model Family: Samsung based SSDs > Device Model: SAMSUNG MZ7KM960HAHP-00005 > Serial Number: S2HTNX0H507466 > LU WWN Device Id: 5 002538 c402bdac1 > Firmware Version: GXM1003Q > User Capacity: 960 197 124 096 bytes [960 GB] You're confusing disk manufacturer gigabytes with real (power of two) gigabytes. The below turns 960 197 124 096 into real gigabytes % bc scale = 4 960197124096 / 1024 / 1024 / 1024 894.2532 > Sector Size: 512 bytes logical/physical > > [...] > > ===Cut=== > > But: > > ===Cut=== > > # zpool list gamestop > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > gamestop 5,22T 4,38T 861G - 24% 83% 1.00x ONLINE - > > ===Cut=== > > Why 5.22T ? If zpool is displaying raw size, it should be 960 x 6 = 5760 > Gb = 5.65 T. If it's displaying the actual data, then it should be 960 x > 5 = 4800 Gb = 4.68 T. 5.22 T is neither of these. I'm stuck, please > explain. :) Take 6 disks, size calculated above, and turn that into power of two TB: 894.2532 * 6 / 1024 5.2397 Close enough to 5.22, especially after ZFS has some overhead for it's own data structures Regards, Gary From owner-freebsd-stable@freebsd.org Sun Oct 16 19:49:44 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09709C12687 for ; Sun, 16 Oct 2016 19:49:44 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id E5B08139C for ; Sun, 16 Oct 2016 19:49:43 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: by mailman.ysv.freebsd.org (Postfix) id E22A7C12686; Sun, 16 Oct 2016 19:49:43 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E1CD6C12685 for ; Sun, 16 Oct 2016 19:49:43 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 54A84139B for ; Sun, 16 Oct 2016 19:49:43 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:c47f:381:9c23:2301] ([IPv6:2a02:2698:27:b4d9:c47f:381:9c23:2301]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9GJndTE046973 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 17 Oct 2016 00:49:40 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1476647380; bh=SbXtF85XLdva8wB9ehe97GhHUr1VucMZFIvHNvECz7U=; h=Subject:To:References:From:Date:In-Reply-To; b=VuHY0IISjszCzSE8fFo7mWGYLM8jC1aXMqIsF2KIG1heLv6GTK0uTBO7zQgwpZlTN NcWnDzOIb76V6zvigkh98HXSVjXhgvfvIJYQip+Ot+HASTRx39f+VyHkGWSBZeHM3H gfS3IPB34ovFc2frKikKOAeqdBkpGeijd3LxZpLc= Subject: Re: zfs/raidz: seems like I'm failing with math To: "stable@freebsd.org" References: From: "Eugene M. Zheganin" Message-ID: <75ef50fa-0e0a-c576-c558-145ddcb32f0b@norma.perm.ru> Date: Mon, 17 Oct 2016 00:49:40 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 19:49:44 -0000 Hi. On 16.10.2016 22:06, Alan Somers wrote: > It's raw size, but the discrepancy is between 1000 and 1024. Smartctl > is reporting base 10 size, but zpool is reporting base 1024.. > 960197124096.0*6/1024**4 = 5.24 TB, which is pretty close to what > zpool says. Thanks ! It does explain it. But then again, on a pool that has been just created, I check the properties of the root dataset (I'm posting all the properties, just to display there's no child datasets or data on the pool): ===Cut=== # zfs get all gamestop NAME PROPERTY VALUE SOURCE gamestop type filesystem - gamestop creation sun oct 16 19:02 2016 - gamestop used 403K - gamestop available 4,04T - gamestop referenced 153K - gamestop compressratio 1.00x - gamestop mounted yes - gamestop quota none default gamestop reservation none default gamestop recordsize 128K default gamestop mountpoint /gamestop default gamestop sharenfs off default gamestop checksum on default gamestop compression off default gamestop atime on default gamestop devices on default gamestop exec on default gamestop setuid on default gamestop readonly off default gamestop jailed off default gamestop snapdir hidden default gamestop aclmode discard default gamestop aclinherit restricted default gamestop canmount on default gamestop xattr off temporary gamestop copies 1 default gamestop version 5 - gamestop utf8only off - gamestop normalization none - gamestop casesensitivity sensitive - gamestop vscan off default gamestop nbmand off default gamestop sharesmb off default gamestop refquota none default gamestop refreservation none default gamestop primarycache all default gamestop secondarycache all default gamestop usedbysnapshots 0 - gamestop usedbydataset 153K - gamestop usedbychildren 249K - gamestop usedbyrefreservation 0 - gamestop logbias latency default gamestop dedup off default gamestop mlslabel - gamestop sync standard default gamestop refcompressratio 1.00x - gamestop written 153K - gamestop logicalused 26,5K - gamestop logicalreferenced 9,50K - gamestop volmode default default gamestop filesystem_limit none default gamestop snapshot_limit none default gamestop filesystem_count none default gamestop snapshot_count none default gamestop redundant_metadata all default ===Cut=== Only 4.03T is available. Looks like it's the actual size, since it's zfs and not zpool. But 960197124096 bytes * 5 / 1024^4 gives me 4.366 Tb, and not the 4.03 T. Where did about 300 gigs go ? I'm really trying to understand, not to catch some questionable logic or find errors. Thanks. Eugene. From owner-freebsd-stable@freebsd.org Sun Oct 16 20:02:25 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BFDBDC12C6B for ; Sun, 16 Oct 2016 20:02:25 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A83A51F20 for ; Sun, 16 Oct 2016 20:02:25 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: by mailman.ysv.freebsd.org (Postfix) id A7998C12C69; Sun, 16 Oct 2016 20:02:25 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A74CBC12C67 for ; Sun, 16 Oct 2016 20:02:25 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 22F351F19 for ; Sun, 16 Oct 2016 20:02:24 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:c47f:381:9c23:2301] ([IPv6:2a02:2698:27:b4d9:c47f:381:9c23:2301]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9GK2LYW048256 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 17 Oct 2016 01:02:22 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1476648142; bh=8iqudVXmtKs67dNRwHjHoU9RXzcNsBgwSJQWUtNcR5U=; h=Subject:References:Cc:From:Date:In-Reply-To; b=ppg4g1bsdBRK/rat5kz96LoWxTofoNKUFPvmc4r2Xv5eWDVCpdstTTJq826wfM83k dXAhgtUnWdvkWddrWdRvl60GDGNSa/Z+epF/m85FDUTLCirf5cVdE2G6ch3q+41F1B WvVr6B/TJJQ7iNfwI4YIYdJFpjrL6QLHcNUd8dNY= Subject: Re: zfs/raidz: seems like I'm failing with math References: <20161016184255.GA2241@in-addr.com> Cc: "stable@freebsd.org" From: "Eugene M. Zheganin" Message-ID: Date: Mon, 17 Oct 2016 01:02:22 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161016184255.GA2241@in-addr.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 20:02:25 -0000 Hi. On 16.10.2016 23:42, Gary Palmer wrote: > > You're confusing disk manufacturer gigabytes with real (power of two) > gigabytes. The below turns 960 197 124 096 into real gigabytes > Yup, I thought that smartctl is better than that and already displayed the size with base 1024. :) Thanks. Eugene. From owner-freebsd-stable@freebsd.org Sun Oct 16 20:04:19 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 37F6BC12DFE; Sun, 16 Oct 2016 20:04:19 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-io0-x234.google.com (mail-io0-x234.google.com [IPv6:2607:f8b0:4001:c06::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 09A5310F0; Sun, 16 Oct 2016 20:04:19 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mail-io0-x234.google.com with SMTP id q192so169747839iod.0; Sun, 16 Oct 2016 13:04:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=uS8dV8psLjAoYTz2HQQUa6J78/P8taavBb+HwafZcPg=; b=ZHN7aK6WWZUVl6vkhApfiauYe0tGJ8sdHg55CUV3Nb/yywghibRllfPR7Z46Yu8c/J 8ZkX5qeChJOzDDME5ZReOCkrtwun4L2IxKzXbITzyKZ0oP7AO91ig8w8fUw+nkmwQECZ K1bGUI5VwAg0+bWCDnYX4ez9m5D6LyxilCEvN8sCZF2sEdfUbkjDxGrLlVWPX/q0RpqZ JY223IaFKkqrKfZ1obZcA0v/ITWLUjvIWB0UjUiPN2SzUBe8HW/3MQ7aEohTrooAOxpY 98UrrM7JS8YpDamsw6/IIlpBxShe3fFenmz+E8uLoO2ZjMV5syF3U0aX9UgSNICbtUSo OIXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=uS8dV8psLjAoYTz2HQQUa6J78/P8taavBb+HwafZcPg=; b=JRDCFta3H3R+2mFp/zJuCvghB9yUSIlky+VAGZWmEp8mMqAjmGfPH8l7FN66g9+Rsn zbjb7mirFeQEgH9MXneyhv2xaDztlW6y/CXZ+Tk+ODlQsAhAMvVj86OorTKwO85wWMZ2 4K4eC6GFvhe0ehwazPLvz43rBENi5ea4rbExfklSjw1e5drsrIFR6NbiA3U+hD3T1PeF E7Lh/mi6SdsBZxehHukzPmI02bt4KJZgJHiojLRuQUTurvoNtuGcqWYZS5iXnA07MZnj m0yBHR6j2oy19J4ghfE/YypfVryxblWsxsLS3QIV9A7Xk0/9jFOb0LiYlP6sEJWxxaUp lOoA== X-Gm-Message-State: AA6/9RlQ5dXGrIcWh7Wn3Jy8OC2ukXcu1upaSqIFvbbYjXgNQPLTu6meumUDiPdUJM/RnoXcLMYPL1myfmJXRw== X-Received: by 10.107.37.212 with SMTP id l203mr20940269iol.101.1476648258132; Sun, 16 Oct 2016 13:04:18 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.7.129 with HTTP; Sun, 16 Oct 2016 13:04:17 -0700 (PDT) From: Aryeh Friedman Date: Sun, 16 Oct 2016 16:04:17 -0400 Message-ID: Subject: help install new kernel on root-on-ZFS on 11-STABLE To: FreeBSD Mailing List , FreeBSD Stable List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 20:04:19 -0000 I just installed 11-STABLE on a brand new system and can't find a valid /boot in which to do "make installkernel" to. I am used to how to do make world on UFS. How do I do it on root-on-ZFS? -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-stable@freebsd.org Sun Oct 16 21:40:45 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 65A75C13A13 for ; Sun, 16 Oct 2016 21:40:45 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [IPv6:2001:418:3fd::f7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 076951ACA for ; Sun, 16 Oct 2016 21:40:44 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [10.100.0.31] (haymarket.m5p.com [10.100.0.31]) by mailhost.m5p.com (8.14.5/8.14.5) with ESMTP id u9GLeZRH040661; Sun, 16 Oct 2016 17:40:41 -0400 (EDT) (envelope-from george+freebsd@m5p.com) Subject: Re: Clandestine USB SD card slot To: Warner Losh , Warren Block References: <832db721-d03a-a21e-11ed-701190140d17@yahoo.com> <2bf09be6-e2ad-2d38-ffd8-d80b45d4c3d8@m5p.com> <82d2acb3-12cd-678e-c90a-21b9d62b5721@m5p.com> Cc: FreeBSD Stable Mailing List From: George Mitchell Message-ID: Date: Sun, 16 Oct 2016 17:40:35 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.73 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (mailhost.m5p.com [10.100.0.247]); Sun, 16 Oct 2016 17:40:42 -0400 (EDT) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 21:40:45 -0000 On 10/16/16 14:16, Warner Losh wrote: > On Sun, Oct 16, 2016 at 12:08 PM, Warren Block wrote: >> On Sun, 16 Oct 2016, George Mitchell wrote: >> >>>> So not only is it (apparently) recognized, but the sdhci_pci driver >>>> attached to it! But inserting or removing a card shows no activity. >>>> What's my next step? -- George >> >> >> Is a device created for the empty reader? It's worth trying to force a >> retaste of that device with 'true > /dev/daX' after the card is inserted. > > Don't look for da anything. Look for mmcsd something. The sdhci_pci > driver provides disks that are mmcsdX. Looks like card change > interrupts aren't happening, or there's something else making the > driver unhappy with the SDHCI controller though... > > Warner > [...] No /dev/mm*; no log output on card insertion/removal even with sysctl hw.sdhci.debug=1. Other sysctl info: sysctl -a | grep sdhci device sdhci hw.sdhci.enable_msi: 1 hw.sdhci.debug: 1 dev.sdhci_pci.0.%parent: pci0 dev.sdhci_pci.0.%pnpinfo: vendor=0x1022 device=0x7813 subvendor=0x1025 subdevice=0x0865 class=0x080501 dev.sdhci_pci.0.%location: pci0:0:20:7 dev.sdhci_pci.0.%driver: sdhci_pci dev.sdhci_pci.0.%desc: Generic SD HCI dev.sdhci_pci.%parent: -- George From owner-freebsd-stable@freebsd.org Sun Oct 16 22:21:35 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B603C1445C; Sun, 16 Oct 2016 22:21:35 +0000 (UTC) (envelope-from brandon.wandersee@gmail.com) Received: from mail-it0-f47.google.com (mail-it0-f47.google.com [209.85.214.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E54B91B43; Sun, 16 Oct 2016 22:21:34 +0000 (UTC) (envelope-from brandon.wandersee@gmail.com) Received: by mail-it0-f47.google.com with SMTP id 139so27722213itm.1; Sun, 16 Oct 2016 15:21:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:user-agent:from:to:cc:subject :in-reply-to:date:message-id:mime-version; bh=HbP1+U3CMN1iY1buXQxhHO5XfOoPZqijjA9O7kfvt/A=; b=IBSMJQm9NFtyAKzsZ56M38/0l3u9nn64B+I3nXudhxSJijDIHG+pqqMyEWdbcOSgPD bBHqNkoen+5wuT7hcn9VLZSE6xwGIknPUfWdgMSpJzIU1A17ajt726o6RkdWzvGhBI2V cCaRMrnYXvFmHlpaZhOP96b6Czg4JwEcx2iIwDQGE/eN2KDBdsrsMcJPEQvVmw7vdDpG g/s7IkDt+yhWO5GDWVeA13Vzj+VcuvVNLx2RYxvgd3dwq57kzHxXIu1j81Ec8aUifLyn zcARvQfYB/zG2AfxyXt7HdX+BsW/EogWvq/C7Aut7GlEgHTthj4a7TMtbR2/CblTyUKD F0ww== X-Gm-Message-State: AA6/9Rk1j3BOlPVWa7q3jd/Pd79/XauoORL+3WBHXYqcnL/f9TwZWmD9gKRASzfS2VShhA== X-Received: by 10.36.219.138 with SMTP id c132mr6942978itg.16.1476656488384; Sun, 16 Oct 2016 15:21:28 -0700 (PDT) Received: from WorkBox.Home (75-161-214-135.mpls.qwest.net. [75.161.214.135]) by smtp.gmail.com with ESMTPSA id f9sm3962406itc.10.2016.10.16.15.21.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 16 Oct 2016 15:21:26 -0700 (PDT) Received: from localhost (WorkBox.Home [local]) by WorkBox.Home (OpenSMTPD) with ESMTPA id 3547f3e2; Sun, 16 Oct 2016 17:21:25 -0500 (CDT) References: User-agent: mu4e 0.9.16; emacs 24.5.1 From: Brandon J. Wandersee To: Aryeh Friedman Cc: FreeBSD Mailing List , FreeBSD Stable List Subject: Re: help install new kernel on root-on-ZFS on 11-STABLE In-reply-to: Date: Sun, 16 Oct 2016 17:21:25 -0500 Message-ID: <864m4cszoa.fsf@WorkBox.Home> MIME-Version: 1.0 Content-Type: text/plain X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 22:21:35 -0000 Aryeh Friedman writes: > I just installed 11-STABLE on a brand new system and can't find a valid > /boot in which to do "make installkernel" to. I am used to how to do make > world on UFS. How do I do it on root-on-ZFS? Last I checked, the automated installer created a separate pool called "bootpool," with a symlink from /boot to /bootpool. Check to see "bootpool" exists; if so, it's not mounted, which is another problem that needs addressing. If it doesn't exist, then something has changed and I have no idea what to do. -- :: Brandon J. Wandersee :: brandon.wandersee@gmail.com :: -------------------------------------------------- :: 'The best design is as little design as possible.' :: --- Dieter Rams ---------------------------------- From owner-freebsd-stable@freebsd.org Sun Oct 16 22:30:37 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 50133C14902; Sun, 16 Oct 2016 22:30:37 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-it0-x22c.google.com (mail-it0-x22c.google.com [IPv6:2607:f8b0:4001:c0b::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 199EC1339; Sun, 16 Oct 2016 22:30:37 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mail-it0-x22c.google.com with SMTP id e187so9013158itc.1; Sun, 16 Oct 2016 15:30:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=HK4A3HkbpYi5WOOctAf0EryJRH+pF8640BZ/aQYEKiI=; b=d/QlD4roz0DnOHurl5xPCQYPD9rOm4y5O+MjCsgrp6vqXtWFHvcggxz6IqujWOb5PA 2ujshSClej59Ht8nPxVoP+EJowWv+QlLpGttRbmRDNsKsLAUsMTTUOHPoHFRYoT/0slD 39I5QsM4b3IyTB0Pv3tZ0FHaIqy7EJok2+0oxwOusNoEBGbU8I+XN7ZHyFr/55ItZvoK oGVrLbi5US1MS5nH/PJ61wheaVrBA3CmGKnT7n7EcLWyqPXSgssWr7uWCxft5hAjPina cSoaojmvxmeLLs8iPDx96pfAeKYSA5iWXeqgsGAVfN7af3lSDdOnAsfxPeyv7pgbQrzZ Et4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=HK4A3HkbpYi5WOOctAf0EryJRH+pF8640BZ/aQYEKiI=; b=e2/MS0LUKwsh6ZHTZ5eHs6aUjD0kJdyoZPlpN+4ldLo0XHZ61oqc9Jz37f9tv71X3Y Sd4tLQz0RhrlaF9tMSk9pc87aRZf4CidDbVWMFhdtQYSIFNPeOYwZswbGgU4dQ0/nSOZ uwv5X9voRlmF8i3mrvIgdD+XpQSAr36rNBUKo2rY55GGkP+cb93vi3UD6uxMnTb7aga9 6RrioA9Jw6dbSa82rTt4wW7cz4DQX7Bv1IEVkxk+aRGgw/YIPiQhPtcDtte1WPN8vkDC qyItegoiAXxygyt8jWq4EhbDbfaX267t73ypDVjbphSFMGRbC6G8O+2mBSWYQ8iCwKds oqDQ== X-Gm-Message-State: AA6/9Rk2hPi3Ntp9vrvRkltdecFjcV1DbVkp8rbdkF+a7J7fnFBaovyg4rypD+n9gQ8vbhlf+tWBa6ySJf0pZQ== X-Received: by 10.36.225.77 with SMTP id n74mr6246651ith.105.1476657036497; Sun, 16 Oct 2016 15:30:36 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.7.129 with HTTP; Sun, 16 Oct 2016 15:30:36 -0700 (PDT) In-Reply-To: <864m4cszoa.fsf@WorkBox.Home> References: <864m4cszoa.fsf@WorkBox.Home> From: Aryeh Friedman Date: Sun, 16 Oct 2016 18:30:36 -0400 Message-ID: Subject: Re: help install new kernel on root-on-ZFS on 11-STABLE To: "Brandon J. Wandersee" Cc: FreeBSD Mailing List , FreeBSD Stable List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 22:30:37 -0000 The bug report pointed to says the same thing but the new issue is it is not documented in the Handbook and/or not mounted by default (for someone who is brand new to ZFS it is very counter intutive to see a symlink pointing off into space) On Sun, Oct 16, 2016 at 6:21 PM, Brandon J. Wandersee < brandon.wandersee@gmail.com> wrote: > > Aryeh Friedman writes: > > > I just installed 11-STABLE on a brand new system and can't find a valid > > /boot in which to do "make installkernel" to. I am used to how to do > make > > world on UFS. How do I do it on root-on-ZFS? > > Last I checked, the automated installer created a separate pool called > "bootpool," with a symlink from /boot to /bootpool. Check to see > "bootpool" exists; if so, it's not mounted, which is another problem > that needs addressing. If it doesn't exist, then something has changed > and I have no idea what to do. > > -- > :: Brandon J. Wandersee > :: brandon.wandersee@gmail.com > :: -------------------------------------------------- > :: 'The best design is as little design as possible.' > :: --- Dieter Rams ---------------------------------- > -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-stable@freebsd.org Sun Oct 16 22:31:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CF98BC149DE; Sun, 16 Oct 2016 22:31:05 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: from mail-io0-x22f.google.com (mail-io0-x22f.google.com [IPv6:2607:f8b0:4001:c06::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 98D211691; Sun, 16 Oct 2016 22:31:05 +0000 (UTC) (envelope-from woodsb02@gmail.com) Received: by mail-io0-x22f.google.com with SMTP id i202so170546842ioi.2; Sun, 16 Oct 2016 15:31:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=Kk0sL9CO1GSMXQsq1pkVNqMh9o05sL8mlT6r/y+pEXY=; b=Bl3uGGRFSIgnucQmFBhcaOSNImkW6Z984NiV7hn4IPkkPkdUbLXaXBs5+mPdgQSfJc BaxqDuegGG1FZptabdw/Ca4F63fMRRnhD34h1ZejfKQRtBnM37m1F8vpE+WDk6B7zr/c csggRwc/WqFj7U7Mu5E3CJhh7InCMeLtTMV6g8QJTih214cce+xIxbyNWBinUgOgPRk3 dCmMkTnjkNVVXZu9RQ2lsN7jFEio60neuKnQXLyfuErryGI5NZzpoN+h/If1/w9SvkKw YzuasqoF1vvKHUQlIoyBtZwD8xQzlvu4nl/HcumiDAVB5x+i1mYRibEl4gp/mNTkkeyv ghTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=Kk0sL9CO1GSMXQsq1pkVNqMh9o05sL8mlT6r/y+pEXY=; b=d/A9as3N2VlzFXYd6+OmQnKJ1JZ+WeVdPISeBAOgxmfTbMBFbE9GCy3N2vbN797IRq PLVYxCY280IHk6Yuf4zpvy4cysNq2fpjy/krZ8RL4l651DTMZkdpt1wsyXZfvD0yUogL 1tl9Qt7BOPSlXi13fxPvEWOUTF+lJXZ3GrdIOTFpuPPsnaw52o6FCum27pKP/BtS9xEC DIHSXVsFCELm7WhoV8sq4xkfr2/FAlab5eB8KSY6C2jqQ1SE6o0MhKaquV3fRa7cyrvK q36FhBZYHC3XjmirJVNXkd24azHb0JvEJWblePN/nySf/BBlBdZPPduAN8yxeR0MHCzx X2/g== X-Gm-Message-State: AA6/9Rk+OmLepKOLlyjNYH+PqrUMgyXZTTgXVkizCOLOvgHwZBmvwyZRtn/B49+zYwoX1jOKFp8abAYHM7DUQg== X-Received: by 10.107.182.8 with SMTP id g8mr19441008iof.215.1476657064973; Sun, 16 Oct 2016 15:31:04 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.67.196 with HTTP; Sun, 16 Oct 2016 15:31:04 -0700 (PDT) In-Reply-To: <864m4cszoa.fsf@WorkBox.Home> References: <864m4cszoa.fsf@WorkBox.Home> From: Ben Woods Date: Mon, 17 Oct 2016 06:31:04 +0800 Message-ID: Subject: Re: help install new kernel on root-on-ZFS on 11-STABLE To: "Brandon J. Wandersee" Cc: Aryeh Friedman , FreeBSD Stable List , FreeBSD Mailing List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 22:31:05 -0000 On Monday, 17 October 2016, Brandon J. Wandersee < brandon.wandersee@gmail.com> wrote: > > Last I checked, the automated installer created a separate pool called > "bootpool," with a symlink from /boot to /bootpool. > I believe this is the case if the option for full disk encryption is selected in the installer. This is required until the boot loader is capable of recognising and accessing a geli encrypted disk. Regards, Ben -- -- From: Benjamin Woods woodsb02@gmail.com From owner-freebsd-stable@freebsd.org Sun Oct 16 23:49:21 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2FBEC14BCA for ; Sun, 16 Oct 2016 23:49:21 +0000 (UTC) (envelope-from 3ARIEWAgTBvopq-tgrn0ceeqwpvu.iqqing.eqouvcdnghtggduf.qti@gaia.bounces.google.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id D16261647 for ; Sun, 16 Oct 2016 23:49:21 +0000 (UTC) (envelope-from 3ARIEWAgTBvopq-tgrn0ceeqwpvu.iqqing.eqouvcdnghtggduf.qti@gaia.bounces.google.com) Received: by mailman.ysv.freebsd.org (Postfix) id CD9EBC14BC9; Sun, 16 Oct 2016 23:49:21 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CD410C14BC8 for ; Sun, 16 Oct 2016 23:49:21 +0000 (UTC) (envelope-from 3ARIEWAgTBvopq-tgrn0ceeqwpvu.iqqing.eqouvcdnghtggduf.qti@gaia.bounces.google.com) Received: from mail-it0-x245.google.com (mail-it0-x245.google.com [IPv6:2607:f8b0:4001:c0b::245]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A13B41646 for ; Sun, 16 Oct 2016 23:49:21 +0000 (UTC) (envelope-from 3ARIEWAgTBvopq-tgrn0ceeqwpvu.iqqing.eqouvcdnghtggduf.qti@gaia.bounces.google.com) Received: by mail-it0-x245.google.com with SMTP id 4so61465136itv.4 for ; Sun, 16 Oct 2016 16:49:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=accounts.google.com; s=20120806; h=mime-version:date:feedback-id:message-id:subject:from:to; bh=RSoGDVH4EdTpzg6Cbuq6ucJjvhQPxnS8qyXhSVBGgqs=; b=P1PN+sYQPxIxEhUHinoYIusy1tfHGasJR0UvhQV/EATXGJ6xaZ1viyKfsp1PRb1BzB SbX30zWsvSkcR1gV79QvR8j0qxe1qfWxBb93WBf2aUHEQqDkLeJlbsZGQ8mTam2imNCS CTzSZFJbCsSAUkS0MfQU4SWAg8AlWUdzrIaaruVSSAmwR/UF1rcyEiF/POzjiGSx1Nki l9ecsz7fv1/4qtm0xye0NO6JzKOLevONs6YlDdA8ohMT3xdduXgTDyg/LOuVrIx4JxyY 0uTOw1DLcV4a+KvhFkclq0dr1kBVej2yGaXyp8VZ+hza3c//4fclswJzZDJ9WtObJVz/ 7KxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:feedback-id:message-id:subject :from:to; bh=RSoGDVH4EdTpzg6Cbuq6ucJjvhQPxnS8qyXhSVBGgqs=; b=RY/n3mJBgll0C5FGI9jVqUUPvaOoUQ3Ton6izYxA9ZIYZWQWDOOjG1KDTAX6gvQSpd Jdbqt8xfrC8vOkRfeBUp/0aN/p6lbTrTh58F72YoXv3l8+Jg+ACOfPWu5dunOQ6qvffy cK6n0l+3r445a+/ZF+AZ/qvv97lowcKJYgAX8Iaj6MUARJlAke1djFB9ULpEUTS0URby ptvAr9n2RH6FAyuWtruFUW8CE8TlLbFde9TbWm72oCuAScYXxhlwo94s12YNWKrdfNXV uKum+sWFDSFxCYAS5NjWpmBYeADqtZu9voN1PCfvq9bcxziheYUDBLqBZ7GcWkVylIyP 9Z1w== X-Gm-Message-State: AA6/9RllSY8UfHc5+ly9UjnM884PYfdK2U1WAHj5jKObxHAHbyM4JgbUpwzLajO9LvXZbkZ1FVDdWuqCHOU9tG/G MIME-Version: 1.0 X-Received: by 10.36.46.134 with SMTP id i128mr2346236ita.16.1476661761063; Sun, 16 Oct 2016 16:49:21 -0700 (PDT) Date: Sun, 16 Oct 2016 23:49:09 +0000 (UTC) X-Notifications: XEAAAAFE4TLkqngCptOP4Rd9-abI X-Account-Notification-Type: 2-RECOVERY-anexp:ire-control Feedback-ID: 2-RECOVERY-anexp:ire-control:account-notifier Message-ID: <1x7PeggN1gkI75CmBzbt3Q@notifications.google.com> Subject: Your recovery email address changed From: Google To: stable@freebsd.org Content-Type: text/plain; charset=UTF-8; format=flowed; delsp=yes Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Oct 2016 23:49:22 -0000 WW91ciByZWNvdmVyeSBlbWFpbCBhZGRyZXNzIGNoYW5nZWQNCg0KDQoNCkhpIFZOU+WoseS5kOWf jjg3NjUwMi5jb23ms6jlhozpgIE4OOWFgyznvZHkuIrmnIDngavniIbnmoTmuLjmiI/lubPlj7As 6LaF5aW96LWi6ZKx5ri45oiPLA0KVGhlIHJlY292ZXJ5IGVtYWlsIGZvciB5b3VyIEdvb2dsZSBB Y2NvdW50IG1heGhpbGw1NDBAZ21haWwuY29tIHdhcw0KcmVjZW50bHkgY2hhbmdlZC4NCg0KKkRv bid0IHJlY29nbml6ZSB0aGlzIGFjdGl2aXR5PyoNClJldmlldyB5b3VyIHJlY2VudGx5IHVzZWQg ZGV2aWNlcw0KPGh0dHBzOi8vYWNjb3VudHMuZ29vZ2xlLmNvbS9BY2NvdW50Q2hvb3Nlcj9FbWFp bD1tYXhoaWxsNTQwQGdtYWlsLmNvbSZjb250aW51ZT1odHRwczovL3NlY3VyaXR5Lmdvb2dsZS5j b20vc2V0dGluZ3Mvc2VjdXJpdHkvYWN0aXZpdHkvbnQvMTQ3NjY2MTc0OTUzMD9yZm4lM0QyJTI2 cmZuYyUzRDElMjZldCUzRDQlMjZhc2FlJTNEMiUyNmFuZXhwJTNEaXJlLWNvbnRyb2w+DQpub3cu DQoNCkJlc3QsDQpUaGUgR29vZ2xlIEFjY291bnRzIHRlYW0NCg0KDQoNClRoaXMgZW1haWwgY2Fu J3QgcmVjZWl2ZSByZXBsaWVzLiBGb3IgbW9yZSBpbmZvcm1hdGlvbiwgdmlzaXQgdGhlIEdvb2ds ZQ0KQWNjb3VudHMgSGVscCBDZW50ZXIgPGh0dHBzOi8vc3VwcG9ydC5nb29nbGUuY29tL2FjY291 bnRzL2Fuc3dlci8yNDUwMjM2Pi4NCg0KDQoNCllvdSByZWNlaXZlZCB0aGlzIG1hbmRhdG9yeSBl bWFpbCBzZXJ2aWNlIGFubm91bmNlbWVudCB0byB1cGRhdGUgeW91IGFib3V0DQppbXBvcnRhbnQg Y2hhbmdlcyB0byB5b3VyIEdvb2dsZSBwcm9kdWN0IG9yIGFjY291bnQuDQrCqSAyMDE2IEdvb2ds ZSBJbmMuLCAxNjAwIEFtcGhpdGhlYXRyZSBQYXJrd2F5LCBNb3VudGFpbiBWaWV3LCBDQSA5NDA0 MywgVVNBDQo= From owner-freebsd-stable@freebsd.org Mon Oct 17 00:32:59 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 86690C14867 for ; Mon, 17 Oct 2016 00:32:59 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-vk0-x22d.google.com (mail-vk0-x22d.google.com [IPv6:2607:f8b0:400c:c05::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 345C81D31 for ; Mon, 17 Oct 2016 00:32:59 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-vk0-x22d.google.com with SMTP id q126so104615534vkd.2 for ; Sun, 16 Oct 2016 17:32:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to; bh=oYrFjYV46m3Dctj0Sd/hs18lPfJhbZKHFQmntg9Kct0=; b=sAa5F/reZGHeHSLaXKFdy0nAjoAF/R55rxY6wEXKXj4kHcImGBNPFxWChG6ZaL/14G 043aQND3BdxHvlaAW8otV/exVNAJeCHRJ8flk6a6r8w9nbooW1L9eQDSmv3EvwzxBlc/ 3OfAO2pNd4De2V+6R7bW6iZO3xnma+8Y7wWHmcAJWC6iwaJC8SregKXMATimcuCOQ9TN VVJK296PyoBbOUuI8X388x5rocJRtc8DQBaqyTXt8gXb02K2IaQGPCLw3p/9tXsaML75 KXkpzPbAA/L/v/gM483dL6e2XLmQK6g0FvWlPBGXaCv5DZ5JzKzX52cWa0HoTfgKMcZ2 keNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to; bh=oYrFjYV46m3Dctj0Sd/hs18lPfJhbZKHFQmntg9Kct0=; b=JWacufW9TgrAqoF7K43DVYdZAHh7uiViySaJJT4Ya2Xv3SgIHZP2wOwtLtOam2vUcn NlyzD9y6X1Gf4Q+mG5mgkUkzhcr/iaklw+9ZY+GkqXYNZMF4UUBgZkJlzspfrykCAdCE 4JCO5SW6BrLFJr10dyLzEwQnrYkQXfQE6gB1h89CW9X9I9lZ+hjLvOFfU+c3G64suNsX 3qFw2FeWY8eRlg4EsCZ/tlIXb9NzeCVnOUkFs/OIDVcSqAqHIKgFj9MOe3nSazU3ZquB PgggDT7B/CK1mqwEQIEsx0zuYByLU7/F1YFx/t3KlZVEPnP9pU/q2yqNF4f5kcx/giIP vmgA== X-Gm-Message-State: AA6/9RkmuO9U+3iGOZk5EqzGdnkxlj40pl1qenWvt0ZxopMlO5EE3cAY1GmgtlKJPW3Bk41QzhGNhj2icXq9yQ== X-Received: by 10.31.148.22 with SMTP id w22mr13224919vkd.87.1476664378189; Sun, 16 Oct 2016 17:32:58 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.103.118.78 with HTTP; Sun, 16 Oct 2016 17:32:57 -0700 (PDT) In-Reply-To: <20161016174540.GI1069@albert.catwhisker.org> References: <20161016162605.GG1069@albert.catwhisker.org> <20161016174540.GI1069@albert.catwhisker.org> From: Kevin Oberman Date: Sun, 16 Oct 2016 17:32:57 -0700 X-Google-Sender-Auth: LwTfD9BRYTWRYL5R5nI6_qj6bV0 Message-ID: Subject: Re: sshd whines & dies after releng/10 "freebsd-update" run To: David Wolfskill , Xin Li , FreeBSD-STABLE Mailing List , Xin LI Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 00:32:59 -0000 On Sun, Oct 16, 2016 at 10:45 AM, David Wolfskill wrote: > On Sun, Oct 16, 2016 at 10:29:00AM -0700, Xin Li wrote: > > ... > > On 10/16/16 09:26, David Wolfskill wrote: > > > And over the last year or so, it's worked pretty well: I have the > > > machine set up (as is usually my approach) to be able to boot from > > > either of a couple of slices. I use a "dump | restore" pipeline > > > to copy the / and /usr file systems from the "active" slice to the > > > "inactive" slice, adjust /etc/fstab on the inactive slice to reflect > > > reality for when it's the boot slice, then (while the file systemms > > > from the other slice are still mounted -- e.g., on /S2) run > > > "freebsd-update -b /S2 fetch install", then reboot from the > > > newly-updated slice. > > > > > > In the past, that's Just Worked. > > > > Your usage probably worked because you were lucky for a few times in the > > past. (details below) > > > > > This weekend, though, I was planning to update my other systems tfrom > > > stable/10 to stable/11, so I figured I'd try freebsd-update on this > > > machine first. > > > > > [...] > > > root@sisboombah:/tmp # `which sshd` -d > > > Undefined symbol "ssh_compat13" referenced from COPY relocation in > /usr/sbin/sshd > > > > > > Any clues? > > > > I think this is not going to work (stable/10 -> releng/10.3) due to ABI > > incompatibility in a downgrade. > > I seem to have failed to commnunicate clearly: The machine in question > does not, and has not, run "stable". It runs releng. > > At the moment (on the "old" slice), it reports: > > sisboombah(10.3-RELEASE-p7)[1] uname -a > FreeBSD sisboombah.catwhisker.org 10.3-RELEASE-p7 FreeBSD 10.3-RELEASE-p7 > #0: Thu Aug 11 18:38:15 UTC 2016 root@amd64-builder.daemonology.net: > /usr/obj/usr/src/sys/GENERIC amd64 > sisboombah(10.3-RELEASE-p7)[2] > > > Basically, freebsd-update is treating your stable/10 as a 10.3-RELEASE > > installation and will fetch only changes from 10.3-RELEASE to the latest > > patchlevel. > > I can see that... if the machine were running stable. > > > Because of a SSH vulnerability that affects 10.3, freebsd-update would > > patch libssh (shared library used by sshd and friends), however the > > change does not affect the main binary. This worked by replacing your > > existing libssh with the one shipped by freebsd-update (effectively > > downgraded the library) and that would break sshd. > > As a reality check: > sisboombah(10.3-RELEASE-p7)[4] sudo mount /S2 > Password: > sisboombah(10.3-RELEASE-p7)[5] sudo mount /S2/usr > sisboombah(10.3-RELEASE-p7)[6] ls -lT {,/S2}/usr/lib/private/libssh.so.* > -r--r--r-- 1 root wheel 634232 Oct 16 11:57:32 2016 > /S2/usr/lib/private/libssh.so.5 > -r--r--r-- 1 root wheel 569864 Jun 5 13:37:52 2016 > /usr/lib/private/libssh.so.5 > sisboombah(10.3-RELEASE-p7)[7] ls -lT {,/S2}/usr/sbin/ssh* > -r-xr-xr-x 1 root wheel 297736 Jun 5 13:38:35 2016 /S2/usr/sbin/sshd > -r-xr-xr-x 1 root wheel 297736 Jun 5 13:38:35 2016 /usr/sbin/sshd > sisboombah(10.3-RELEASE-p7)[8] > > > I think upgrade -r 10.2-RELEASE (ideally, 11.0-RELEASE though as it > > would eliminate the possibility of any potential incompatibility) would > > work because that would result in a full rewrite of all files. > > Well, I had seen reports of folks having "issues" with attempts to > use freebsd-update to get to releng/11 from systems that weren't > as up-to-date as they might be; I was actually trying to avoid a > problem.... :-} > > Peace, > david > -- > David H. Wolfskill david@catwhisker.org > Those who would murder in the name of God or prophet are blasphemous > cowards. > > See http://www.catwhisker.org/~david/publickey.gpg for my public key. > I believe sshd no longer supports ssh1 compatibility and it looks like you might still have an entry in /etc/sshd/sshd.config trying to touch v1. Check the file for any non-default entries. Compare your sshd_config with the default version in /usr/src/crypto/openssh. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-stable@freebsd.org Mon Oct 17 00:44:18 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02418C14BDF for ; Mon, 17 Oct 2016 00:44:17 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: from mail-lf0-x231.google.com (mail-lf0-x231.google.com [IPv6:2a00:1450:4010:c07::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 757EE116D for ; Mon, 17 Oct 2016 00:44:17 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: by mail-lf0-x231.google.com with SMTP id b75so258597804lfg.3 for ; Sun, 16 Oct 2016 17:44:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=+fLJ9nh7yo8Nrt7B5hIahZG0nWIzxZ8K+JtyK+ASUIg=; b=BNy1qGyC06Hj63l+7sqOvCH4hbfJxtoeyuYKDCajsVCLBXHlX1aWh7OPtdmY24fV3T gKz5VGoZnSDfpxha3EwCZvy/HJmlskBqaUs2drbsXVqhHj4bQ7aGoFR7HWrW5P1zV3ww //BxOX7vu9jVk+sbfiCHFdTcpVsE9pTsO1aqE3U/0N4LNgn+kPlcAAVX+j80QCxnv7ET lR07ju2X6OmkZAOjIWVXNy8LjM/40F1KXw+hg7rFARRE7U60L02toWJr9KksAGi5tsLc edPMnnwTgdwgaLZb5D0zL2us4DMcYQFf9foPzSAs5gjcNDHWM2RjchYhujjy0F4dPNvs JGzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=+fLJ9nh7yo8Nrt7B5hIahZG0nWIzxZ8K+JtyK+ASUIg=; b=YK683YBLNNldsZLlAdGZc/LR3+XjB62/5t8V7RiizHqnRB7Cs1/9H4thm0jkhll98s 7lYEicIT57wf7HBbNx4l3fTUOD0WqFY+VnKv0qaTmHQwPnt9RmMjtWea/ts06X7bekA+ YMoaS94g23m/DgoCDrZ0+wNEFZ8ScIPz+KDR1InoFu5gsq9/SdPI3U2oOJaS/QvdW8bE xRsX+ebQb72IyK8diQz3VNCbXGbHAA85MUJtt2v1EIoBzYNBVkBDa5CCKoIXyrvb9XbJ SkJPJoidKga28ApUPHblVyhjv4ABJh1/GhBW49GMHUojWPe3m5HkHhAvJWvgenW6oUTr cSqQ== X-Gm-Message-State: AA6/9RnLq+Ll12FnJQhoXKvw2POnk9liuLIjsy/84UwWvYb9RfM2nngTIT8dd8l7geY8CtOcc01eflQbeEJoHw== X-Received: by 10.28.103.70 with SMTP id b67mr6931249wmc.74.1476665055194; Sun, 16 Oct 2016 17:44:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.158.130 with HTTP; Sun, 16 Oct 2016 17:44:14 -0700 (PDT) From: Rostislav Krasny Date: Mon, 17 Oct 2016 03:44:14 +0300 Message-ID: Subject: I'm upset about FreeBSD To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 00:44:18 -0000 Hi, I've been using FreeBSD for many years. Not as my main operating system, though. But anyway several bugs and patches were contributed and somebody even added my name into the additional contributors list. That's pleasing but today I tried to install the FreeBSD 11.0 and I'm upset about this operating system. First of all I faced an old problem that I reported here a year ago: http://comments.gmane.org/gmane.os.freebsd.stable/96598 Completely new USB flash drive flashed by the FreeBSD-11.0-RELEASE-i386-mini-memstick.img file kills every Windows again. If I use the Rufus util to write the img file (using DD mode) the Windows dies immediately after the flashing. If I use the Win32DiskImager (suggested by the Handbook) it doesn't reinitialize the USB storage and Windows dies only if I remove and put that USB flash drive again or boot Windows when it is connected. Nothing was done to fix this nasty bug for a year. Ok, I've used the trick and rebooted Windows manually into the BIOS setup right after I flashed the USB disk by the Win32DiskImager. The FreeBSD 11.0 installation has started. I did it on my old computer with MBR disk schema and installed the FreeBSD on the second MBR slice (ada0s2). The installation finished. I changed the BIOS boot configuration back, started to reboot and quickly removed the USB flash drive (I didn't want to kill Windows again). But suddenly my computer appeared unbootable. The new MBR code, that was rewritten without any notation, can't find any OS. I tried to reboot several times. Why bsdinstall doesn't ask about changing the MBR code? Maybe I don't want to change it. Or maybe I want to install the BSD boot manager version of the MBR code (boot0cfg -B). Ok, I booted from the installation USB drive again and ran 'boot0cfg -B ada0'. My computer is bootable again, but only for Windows. If I press F1 the Windows boots properly. If I press F2 or F2 and Enter I see only hash tags. If I reboot after that I see F2 already selected. But if I wait (for F2 - FreeBSD) to the selected OS automatic boot I see the hash tags again. WTF? I've never seen such bugs with boot0 in any previous FreeBSD version. I'm upset. Someone else at my place was throwing the FreeBSD 11.0 installation media far away right after the first problem. What is going on? At least, how to finish the quest? FreeBSD is losing popularity for Linux these days. I think such a bad user experience of very basic use cases is one of the main reason for this loss. Forgive me for saying that, I'm just upset. From owner-freebsd-stable@freebsd.org Mon Oct 17 01:01:54 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 17785C12168 for ; Mon, 17 Oct 2016 01:01:54 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id F3E1B1A5F for ; Mon, 17 Oct 2016 01:01:53 +0000 (UTC) (envelope-from david@catwhisker.org) Received: by mailman.ysv.freebsd.org (Postfix) id F031AC12166; Mon, 17 Oct 2016 01:01:53 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFD8DC12164 for ; Mon, 17 Oct 2016 01:01:53 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (mx.catwhisker.org [198.144.209.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C30A71A5D for ; Mon, 17 Oct 2016 01:01:52 +0000 (UTC) (envelope-from david@catwhisker.org) Received: from albert.catwhisker.org (localhost [127.0.0.1]) by albert.catwhisker.org (8.15.2/8.15.2) with ESMTP id u9H11pEP003861; Mon, 17 Oct 2016 01:01:51 GMT (envelope-from david@albert.catwhisker.org) Received: (from david@localhost) by albert.catwhisker.org (8.15.2/8.15.2/Submit) id u9H11pRu003860; Sun, 16 Oct 2016 18:01:51 -0700 (PDT) (envelope-from david) Date: Sun, 16 Oct 2016 18:01:51 -0700 From: David Wolfskill To: Kevin Oberman Cc: stable@freebsd.org Subject: Re: sshd whines & dies after releng/10 "freebsd-update" run Message-ID: <20161017010151.GB2480@albert.catwhisker.org> Mail-Followup-To: David Wolfskill , Kevin Oberman , stable@freebsd.org References: <20161016162605.GG1069@albert.catwhisker.org> <20161016174540.GI1069@albert.catwhisker.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="l76fUT7nc3MelDdI" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.0 (2016-08-17) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 01:01:54 -0000 --l76fUT7nc3MelDdI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Oct 16, 2016 at 05:32:57PM -0700, Kevin Oberman wrote: > ... > I believe sshd no longer supports ssh1 compatibility and it looks like you > might still have an entry in /etc/sshd/sshd.config trying to touch v1. > Check the file for any non-default entries. Compare your sshd_config with > the default version in /usr/src/crypto/openssh. > .... I used to explicitly disable v1 compatibility..... The machine that's a target of the "freebsd-update" attention has no sources, so I copied sshd_config from it to /tmp on my laptop (which does): g1-252(11.0-S)[4] diff -u /S2/usr/src/crypto/openssh/sshd_config /tmp/sshd_= config --- /S2/usr/src/crypto/openssh/sshd_config 2016-03-13 04:13:31.3236900= 00 -0700 +++ /tmp/sshd_config 2016-06-05 06:37:55.000000000 -0700 @@ -1,5 +1,5 @@ -# $OpenBSD: sshd_config,v 1.98 2016/02/17 05:29:04 djm Exp $ -# $FreeBSD: stable/10/crypto/openssh/sshd_config 296781 2016-03-12 23= :53:20Z des $ +# $OpenBSD: sshd_config,v 1.93 2014/01/10 05:59:19 djm Exp $ +# $FreeBSD: stable/10/crypto/openssh/sshd_config 264692 2014-04-20 12= :46:18Z des $ =20 # This is the sshd server system-wide configuration file. See # sshd_config(5) for more information. @@ -120,7 +120,7 @@ #MaxStartups 10:30:100 #PermitTunnel no #ChrootDirectory none -#VersionAddendum FreeBSD-20160310 +#VersionAddendum FreeBSD-20140420 =20 # no default banner path #Banner none @@ -128,6 +128,18 @@ # override default of no subsystems Subsystem sftp /usr/libexec/sftp-server =20 +# Disable HPN tuning improvements. +#HPNDisabled no + +# Buffer size for HPN to non-HPN connections. +#HPNBufferSize 2048 + +# TCP receive socket buffer polling for HPN. Disable on non autotuning ke= rnels. +#TcpRcvBufPoll yes + +# Allow the use of the NONE cipher. +#NoneEnabled no + # Example of overriding settings on a per-user basis #Match User anoncvs # X11Forwarding no g1-252(11.0-S)[5]=20 On the off-chance that the VersionAddendum might be confusing at least one of us, I copied the stable/11 version of the file to the appropiate place on the freebsd-update target machine, then rebooted. Still no joy: other things work, but not ssh. Thanks for the suggestion. I'm a bit... perplexed. [The machine in question would be the last machine I have still running FreeBSD-10 -- I've migrated each of the others to stable/11.] Peace, david --=20 David H. Wolfskill david@catwhisker.org Those who would murder in the name of God or prophet are blasphemous coward= s. See http://www.catwhisker.org/~david/publickey.gpg for my public key. --l76fUT7nc3MelDdI Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJYBCL/XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRDQ0I3Q0VGOTE3QTgwMUY0MzA2NEQ3N0Ix NTM5Q0M0MEEwNDlFRTE3AAoJEBU5zECgSe4XMegH+wZaaeXpH8JF/Z/l2P22igfG 9N9VE047qoHPoa16y0hQ7nN8+6sUeVBOm4Kl8F9pdrs5PKdfhbwh4YvmoPnyFc82 ed9pNyCP4jIcXz8SEmO1WGG342UDLk1YZ7rTMlyZrzLIqvsjVXMLpukOlvsQMBR7 Pwv5QUpEtKiiDYpQavVRDWzdqpZhv1Nnj4v/6HM64MvZKl14Q+ZLpVDj+D1jdWMp Ffg1EJAFSa2w7KDw4TRafV/4D6BjvOi34ULbYK3yvXeTjR+Lorw6/oDEXEh6huqP bL2E9itKRdZxsecfwb02ibmnLVnVsu/ZlRWp3vaywlMMBjVUiH5YvAflxlcgbmQ= =gZD4 -----END PGP SIGNATURE----- --l76fUT7nc3MelDdI-- From owner-freebsd-stable@freebsd.org Mon Oct 17 04:38:38 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A40E7C0D487; Mon, 17 Oct 2016 04:38:38 +0000 (UTC) (envelope-from FreeBSD@shaneware.biz) Received: from ipmail06.adl6.internode.on.net (ipmail06.adl6.internode.on.net [150.101.137.145]) by mx1.freebsd.org (Postfix) with ESMTP id 15A321C01; Mon, 17 Oct 2016 04:38:37 +0000 (UTC) (envelope-from FreeBSD@shaneware.biz) Received: from ppp121-45-99-127.bras2.adl6.internode.on.net (HELO leader.local) ([121.45.99.127]) by ipmail06.adl6.internode.on.net with ESMTP; 17 Oct 2016 15:03:27 +1030 Subject: Re: help install new kernel on root-on-ZFS on 11-STABLE To: Ben Woods , "Brandon J. Wandersee" References: <864m4cszoa.fsf@WorkBox.Home> Cc: FreeBSD Stable List , Aryeh Friedman , FreeBSD Mailing List From: Shane Ambler Message-ID: <8764c713-1204-b780-a80d-3e0e422797bf@ShaneWare.Biz> Date: Mon, 17 Oct 2016 15:03:21 +1030 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 04:38:38 -0000 On 17/10/2016 09:01, Ben Woods wrote: > On Monday, 17 October 2016, Brandon J. Wandersee < > brandon.wandersee@gmail.com> wrote: >> >> Last I checked, the automated installer created a separate pool called >> "bootpool," with a symlink from /boot to /bootpool. >> > > I believe this is the case if the option for full disk encryption is > selected in the installer. This is required until the boot loader is > capable of recognising and accessing a geli encrypted disk. For a "plain" install, /boot is nothing more than a directory created in the root filesystem that will get created during installation. This is the same for UFS and ZFS. I haven't checked during a build but I don't think anything gets installed into /boot before installkernel so you shouldn't expect it to exist before installkernel. As mentioned you can create a separate pool for /boot so you can load the kernel from a non-encrypted filesystem that can then mount the remaining encrypted filesystems. Another alternative setup is using boot environments that would create filesystems as /sys/ROOT/ that are then mounted as the root filesystem. Like a plain system the /boot folder would still be created at the installkernel stage. -- FreeBSD - the place to B...Storing Data Shane Ambler From owner-freebsd-stable@freebsd.org Mon Oct 17 06:57:20 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0AF13C15CE1 for ; Mon, 17 Oct 2016 06:57:20 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 768C611CB for ; Mon, 17 Oct 2016 06:57:19 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:c47f:381:9c23:2301] ([IPv6:2a02:2698:27:b4d9:c47f:381:9c23:2301]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9H6vEMJ089689 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Mon, 17 Oct 2016 11:57:15 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1476687435; bh=st6An3ZiXfzH8zDaO4OMEQTTHttrvkVT3GyoPSpoIC0=; h=Subject:To:References:From:Date:In-Reply-To; b=oCkiGyGB8z2Jati1o8NKIUSW1YMLYbTIK7LvoORJPbKuTE1EL27Y/zztfVH2JLBB2 sSLvFbO6FsIRdlRp+siB5m2FsdYcLJmLhMaw8MDkQRYDKjx63bBwyiKcXccbP2NaH8 JU5PmyHh82igh0+Ca34X73cIdPDuCQn912yG/y9k= Subject: Re: I'm upset about FreeBSD To: freebsd-stable@freebsd.org References: From: "Eugene M. Zheganin" Message-ID: <8f828527-f859-8ffd-bdbd-b5db1975dff4@norma.perm.ru> Date: Mon, 17 Oct 2016 11:57:16 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 06:57:20 -0000 Hi. On 17.10.2016 5:44, Rostislav Krasny wrote: > Hi, > > I've been using FreeBSD for many years. Not as my main operating > system, though. But anyway several bugs and patches were contributed > and somebody even added my name into the additional contributors list. > That's pleasing but today I tried to install the FreeBSD 11.0 and I'm > upset about this operating system. > > First of all I faced an old problem that I reported here a year ago: > http://comments.gmane.org/gmane.os.freebsd.stable/96598 > Completely new USB flash drive flashed by the > FreeBSD-11.0-RELEASE-i386-mini-memstick.img file kills every Windows > again. If I use the Rufus util to write the img file (using DD mode) > the Windows dies immediately after the flashing. If I use the > Win32DiskImager (suggested by the Handbook) it doesn't reinitialize > the USB storage and Windows dies only if I remove and put that USB > flash drive again or boot Windows when it is connected. Nothing was > done to fix this nasty bug for a year. I saw this particular bug, and I must say - man, it's not FreeBSD, it's Rufus. So far Windows doesn't have any decent tool to write the image with. As about Rufus - somehow it does produce broken images on a USB stick (not always though), which make every Windows installation to BSOD immediately after inserting. And this continues until you reinitialize the stick boot area. My opinion on this wasn't changing regardless of the operation system: if something traps after something valid happens (like USB flash is inserted) - that's OS problem, not the problem of whoever triggered this. Especially in the case when the USB flash is inserted, containing no FS that Windows can recognize and mount ouf-of-the-box. A non-bugged OS just shouldn't trap whatever is inserted in it's USB port, because it feels like in the cheap The Net movie with Sandra Bullock. FreeBSD has many problems (and I'm upset with it too), but this just isn't it. Just because such thing never happens when you create the image using dd on just any OS that has it natively. So it's bad experience with Rufus, not with FreeBSD. P.S. By the way, win32diskimager is a total mess too. Sometimes it just does nothing instead of writing an image. I did try almost all of the free win32 tools to write image with, and didn't find any that would completely satisfy me. Rufus would be the best, if it didn't have this ridiculous bug with BSOD. Eugene. From owner-freebsd-stable@freebsd.org Mon Oct 17 07:19:18 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E3945C1371C for ; Mon, 17 Oct 2016 07:19:18 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A7FCC1035 for ; Mon, 17 Oct 2016 07:19:17 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.36] (unknown [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPA id E1BA69DCCFA; Mon, 17 Oct 2016 09:13:35 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 10.0 \(3226\)) Subject: Re: I'm upset about FreeBSD From: Borja Marcos In-Reply-To: Date: Mon, 17 Oct 2016 09:13:35 +0200 Cc: freebsd-stable@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <17B6FA9E-1FDA-4C56-8925-B975B33A3D8E@sarenet.es> References: To: Rostislav Krasny X-Mailer: Apple Mail (2.3226) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 07:19:19 -0000 > On 17 Oct 2016, at 02:44, Rostislav Krasny = wrote: >=20 > Hi, >=20 > First of all I faced an old problem that I reported here a year ago: > http://comments.gmane.org/gmane.os.freebsd.stable/96598 > Completely new USB flash drive flashed by the > FreeBSD-11.0-RELEASE-i386-mini-memstick.img file kills every Windows > again. If I use the Rufus util to write the img file (using DD mode) > the Windows dies immediately after the flashing. If I use the > Win32DiskImager (suggested by the Handbook) it doesn't reinitialize > the USB storage and Windows dies only if I remove and put that USB > flash drive again or boot Windows when it is connected. Nothing was > done to fix this nasty bug for a year. I=E2=80=99m afraid that=C2=B4s a Windows problem. And potentially a = critical one. That barfing upon USB insertion might point to a buffer overflow condition. Now that people from Microsoft are reading these lists and polishing = support for the Microsoft hypervisor, maybe they should slap some wrists in-house = (hard!). *nudge*-*nudge*-*wink*-*wink*. Borja. From owner-freebsd-stable@freebsd.org Mon Oct 17 12:11:16 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E495BC1303B for ; Mon, 17 Oct 2016 12:11:16 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: from mail-qt0-x22d.google.com (mail-qt0-x22d.google.com [IPv6:2607:f8b0:400d:c0d::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A91DB648 for ; Mon, 17 Oct 2016 12:11:16 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: by mail-qt0-x22d.google.com with SMTP id s49so115692774qta.0 for ; Mon, 17 Oct 2016 05:11:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=ixx1TDq6f5I9InRrxaWP3wRjl0U9ckyew8Tr8rkXihU=; b=BoWzVsQ4bTpxFw42h5op+66KSpggU8Xlb9tnirBQazRvkO1Hgoaf/vBKuRvEz0sMxM gzg7wWZt74XPP9X0Hiz9uBH71VZE1DE7eYaQizv4Kuad7V+B94NG2amwFi+M4HKo71K7 1Y4vWxUwISdVYHiS5I0DFbTQtSRWocf/cssV5JF0rb0xA2QB5tOOcSYrrgkogXinY/Sb hN+Y6aGsNzkmeP7ws03iUuDjYb9fp5RK97wxUpWtN6XawWreNjEwFarHkEixZWqXitHx CX8wPng5AH1bC1Tzp045hLQKzo3am4dDj3VNJTIqTIKCGlixupdSoC3MGfRti9MV5WAd ngNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=ixx1TDq6f5I9InRrxaWP3wRjl0U9ckyew8Tr8rkXihU=; b=gJIQOju/s9fClkTda9WHK1kIBCOZQh8T+7KjqlUpe3fcdsM4USNHb86wz6+SY2xXsF 26VhqAb5WrUdDrLB1gOhqmSV2Uv9TJM3LCVO+2sDm7gC1ioQU8qtvCkujLwMv4gSqwII P1PGDeK5wAqoiLaWS65qsNrINMgaSQGbgbAQefdmZDsZdCWtGQefoZ/KMWtpo0hGKVCk WuFBnKDtYxC4bCCtG7KaPyXVb5o8QcST213J3E7lbhLvzy+JB7qfnfsObEv25sIhkKCl WE3YcltbLta2cDYXRJp+f+lIFDbGLoQc07HeiRCnp4nZbitRNdpkXAeBHXohH4WFke2R sb/A== X-Gm-Message-State: AA6/9RkgvctkMx/OY+XB9eunaikno3gh18FzzVnAoM68HTA+o/im+g76ZfqUMNkpgXURpjCV67KxIoGhn4Ljbg== X-Received: by 10.28.24.80 with SMTP id 77mr8587957wmy.74.1476706275402; Mon, 17 Oct 2016 05:11:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.158.130 with HTTP; Mon, 17 Oct 2016 05:11:15 -0700 (PDT) In-Reply-To: References: From: Rostislav Krasny Date: Mon, 17 Oct 2016 15:11:15 +0300 Message-ID: Subject: Re: I'm upset about FreeBSD To: freebsd-stable@freebsd.org Cc: "Eugene M. Zheganin" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 12:11:17 -0000 On 17.10.2016 11:57:16 +0500, Eugene M. Zheganin wrote: > Hi. > > On 17.10.2016 5:44, Rostislav Krasny wrote: > > Hi, > > > > I've been using FreeBSD for many years. Not as my main operating > > > system, though. But anyway several bugs and patches were contributed > > and somebody even added my name into the additional contributors list. > > That's pleasing but today I tried to install the FreeBSD 11.0 and I'm > > > upset about this operating system. > > > > First of all I faced an old problem that I reported here a year ago: > > http://comments.gmane.org/gmane.os.freebsd.stable/96598 > > > Completely new USB flash drive flashed by the > > FreeBSD-11.0-RELEASE-i386-mini-memstick.img file kills every Windows > > again. If I use the Rufus util to write the img file (using DD mode) > > the Windows dies immediately after the flashing. If I use the > > Win32DiskImager (suggested by the Handbook) it doesn't reinitialize > > the USB storage and Windows dies only if I remove and put that USB > > flash drive again or boot Windows when it is connected. Nothing was > > done to fix this nasty bug for a year. > > I saw this particular bug, and I must say - man, it's not FreeBSD, it's Rufus. > So far Windows doesn't have any decent tool to write the image with. As > about Rufus - somehow it does produce broken images on a USB stick > (not always though), which make every Windows installation to BSOD > immediately after inserting. And this continues until you reinitialize the > stick boot area. The DD mode in Rufus and in any other flashing util that supports the DD mode (including the native dd(1) program) just writes the image file byte by byte without any change, isn't it? If you say the boot area re-initialization resolves the BSOD issue then why the boot area isn't fixed in the image file at the first place? I agree that Windows has a serious bug but why FreeBSD doesn't try to workaround it? After all people try to install FreeBSD and if the Windows bug and the FreeBSD developers stubbornness prevent them to do so they just can't and won't install FreeBSD. This is a lose-lose situation. > P.S. By the way, win32diskimager is a total mess too. Sometimes it just > does nothing instead of writing an image. I did try almost all of the free > win32 tools to write image with, and didn't find any that would completely > satisfy me. Rufus would be the best, if it didn't have this ridiculous bug with > BSOD. Did you try the native FreeBSD dd(1) program or a MinGW version of dd(1)? And what about other issues I've described in my first email? I managed to install FreeBSD 11.0 but it still unbootable. P.S. please Cc your replies to me since I don't receive this mailing list emails directly, although I'm subscribed. From owner-freebsd-stable@freebsd.org Mon Oct 17 12:31:50 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA5B6C1353A for ; Mon, 17 Oct 2016 12:31:50 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-qk0-x230.google.com (mail-qk0-x230.google.com [IPv6:2607:f8b0:400d:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F942EBF for ; Mon, 17 Oct 2016 12:31:50 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: by mail-qk0-x230.google.com with SMTP id n189so231294894qke.0 for ; Mon, 17 Oct 2016 05:31:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=10rm+AJAJbPKLZQID06mZqJjGJBpSeK4SK3wWCl12ls=; b=Hs6JZg9FUQPWhkXYNh9E3V9S+twfIBEhiHmnlPKtArgsyVBBnZMWcAl4/Iqv6YU5sC gardkJ+nAL8Zxz1YSSuMVni4bp5Rhczq9K3KHBepVLmV9zQV2djL0hp2FX3IB8+3fijO S5ZWgIIk5yWe38CpN6xi4woTW/zjRiuNnmq08Do0sLwkFMeKKslUoROAlIKULQbpG+ra ++Omk0xtRt4RW69aQxDIts0SVdWUwQ9EZ1A0BFZCKLTE3atxNVD26HPTikItQR2AmEqB U8dNjmi/PRk7XjDh2vrA2XPQ9SrFh0LdASUclgkOcVovpqyFX+tmDbWGdJeVc4rhsiRj SZAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=10rm+AJAJbPKLZQID06mZqJjGJBpSeK4SK3wWCl12ls=; b=XgcTxxuVk1XqQgCAfnjUA623V5eCR27D4jarjVrh0hwBjEugzWXZyzq+hPgLvtKGP3 0rRggTXBLwvixwc+8U2vWswu/tc5pWqxX6YnevQF99lZ5LIokIcgWuzafCYI8PzRRebF vQmYyYOer1G2ZtnwEFb5DGSpiFTGnWrIeZBn4fxH9ii1KNX2LLebCMdTISAqTd3MZlQf L1gudOir/rryvu1yzAEtZ6jmSJQ+DBMF2VozjLxA0EqTUUQc2QpDGRuSlqRfk8CHoSRS YisApgjYO27zq/NXUtWiEN7JCGbao1BKwOTE6fQrIAY+bA+6dgEDYLZ3z1HWYN6bEG/J encg== X-Gm-Message-State: AA6/9Rkh3NiqqyaoykP4DVmuXySXxpILkYCSrtoZrVa+1Ygm0VUlb/pJr/XP2gfJnA6O5XBvDldttboWiMZ2yA== X-Received: by 10.194.82.100 with SMTP id h4mr11068944wjy.183.1476707509477; Mon, 17 Oct 2016 05:31:49 -0700 (PDT) MIME-Version: 1.0 Received: by 10.195.11.137 with HTTP; Mon, 17 Oct 2016 05:31:48 -0700 (PDT) In-Reply-To: References: From: krad Date: Mon, 17 Oct 2016 13:31:48 +0100 Message-ID: Subject: Re: I'm upset about FreeBSD To: Rostislav Krasny Cc: freebsd-stable , "Eugene M. Zheganin" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 12:31:50 -0000 Does this just affect MBR layouts? If possible you might want to consider UEFI booting for both windows and other os's, It's probably safer as you dont need to plays with partitions and bootloaders. On 17 October 2016 at 13:11, Rostislav Krasny wrote: > On 17.10.2016 11:57:16 +0500, Eugene M. Zheganin wrote: > > Hi. > > > > On 17.10.2016 5:44, Rostislav Krasny wrote: > > > Hi, > > > > > > I've been using FreeBSD for many years. Not as my main operating > > > > system, though. But anyway several bugs and patches were contributed > > > and somebody even added my name into the additional contributors list. > > > That's pleasing but today I tried to install the FreeBSD 11.0 and I'm > > > > upset about this operating system. > > > > > > First of all I faced an old problem that I reported here a year ago: > > > http://comments.gmane.org/gmane.os.freebsd.stable/96598 > > > > Completely new USB flash drive flashed by the > > > FreeBSD-11.0-RELEASE-i386-mini-memstick.img file kills every Windows > > > again. If I use the Rufus util to write the img file (using DD mode) > > > the Windows dies immediately after the flashing. If I use the > > > Win32DiskImager (suggested by the Handbook) it doesn't reinitialize > > > the USB storage and Windows dies only if I remove and put that USB > > > flash drive again or boot Windows when it is connected. Nothing was > > > done to fix this nasty bug for a year. > > > > I saw this particular bug, and I must say - man, it's not FreeBSD, it's > Rufus. > > So far Windows doesn't have any decent tool to write the image with. As > > about Rufus - somehow it does produce broken images on a USB stick > > (not always though), which make every Windows installation to BSOD > > immediately after inserting. And this continues until you reinitialize > the > > stick boot area. > > The DD mode in Rufus and in any other flashing util that supports the > DD mode (including the native dd(1) program) just writes the image > file byte by byte without any change, isn't it? If you say the boot > area re-initialization resolves the BSOD issue then why the boot area > isn't fixed in the image file at the first place? I agree that Windows > has a serious bug but why FreeBSD doesn't try to workaround it? After > all people try to install FreeBSD and if the Windows bug and the > FreeBSD developers stubbornness prevent them to do so they just can't > and won't install FreeBSD. This is a lose-lose situation. > > > P.S. By the way, win32diskimager is a total mess too. Sometimes it just > > does nothing instead of writing an image. I did try almost all of the > free > > win32 tools to write image with, and didn't find any that would > completely > > satisfy me. Rufus would be the best, if it didn't have this ridiculous > bug with > > BSOD. > > Did you try the native FreeBSD dd(1) program or a MinGW version of dd(1)? > > And what about other issues I've described in my first email? I > managed to install FreeBSD 11.0 but it still unbootable. > > > P.S. please Cc your replies to me since I don't receive this mailing > list emails directly, although I'm subscribed. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Mon Oct 17 12:39:35 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44EA7C136AF for ; Mon, 17 Oct 2016 12:39:35 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: from mail-lf0-x229.google.com (mail-lf0-x229.google.com [IPv6:2a00:1450:4010:c07::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D1AD32C9 for ; Mon, 17 Oct 2016 12:39:34 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: by mail-lf0-x229.google.com with SMTP id l131so241167081lfl.2 for ; Mon, 17 Oct 2016 05:39:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=oAtYkq/pqHRRyu43nhL+3PFW5jAN2GbWxF1DLWSTrYY=; b=abbz97nrR6TCNXp1pW3WXhj6TgO3chWv9n+qfhA/qvv1Ack3dtf5IJDrjTxsQWoZyE w7IIJajRoyabfgcgt/GHF/deBY3tnyJsqvDyOwr6mu20tZgzgA+DxBLgRL/Om6PCEW0z j44046JW+YA4wUmo4kb5tR8vJigF/+nNkBk2I1a6WP7F3NI2FBtRWEhhaLZ24M9iULDp YGG97wvkKqJD2Dp9cSQHJUSnfVqZwXg2E4o7f7rnzY+/CP5OjcreE74s3Q9bAXSmD/F9 CJU539w9hmns9pCj0rHv3iTBBh1XIG3NLFZQ8xXPLThtkPbFAVX52XUCbr49O1MpVZ/L xEqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=oAtYkq/pqHRRyu43nhL+3PFW5jAN2GbWxF1DLWSTrYY=; b=ZZKLZNIIaQjOSucAz3wXH5E3jcbct1FnxxUBmPnXynyFR1NCy/AGVqi6G+x7RP/3zS zRV6bK+q/G3YfxQ7LcDZK2Qb3CbRr5Io5hfXYTHPwXhhbVVUvJZOhKVO+iRvOVI5pFxb +NxiSQ7PxzWpCG7Z+z++rYgX/rq6sFCVvljw+j5O6WD8xwdKnq31dY3/ccU791Ht/hmS 1fXkLpeo74wmUWzMQ4BFz/CNjcqctpwBF+xEtIlWPvmjlzZNVmBFs5bGY3xopt9WiBhY ibP5pZdXvlmH3fgTbg3FKD621HmVFugLL/UPl+T8THnGmNjdmdx5qPZEKEGJu+H6Cwhi sRpA== X-Gm-Message-State: AA6/9Rn7G8lb6zr5UvzSq2fG7u7oEfmt1ZSQ4IZvRnodykxMAZJfNfAP0UHRo/maG65kRKjc6P2Lgv6tMNBFBQ== X-Received: by 10.28.23.204 with SMTP id 195mr7834026wmx.101.1476707971530; Mon, 17 Oct 2016 05:39:31 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.158.130 with HTTP; Mon, 17 Oct 2016 05:39:30 -0700 (PDT) In-Reply-To: References: From: Rostislav Krasny Date: Mon, 17 Oct 2016 15:39:30 +0300 Message-ID: Subject: Re: I'm upset about FreeBSD To: krad Cc: freebsd-stable , "Eugene M. Zheganin" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 12:39:35 -0000 On Mon, Oct 17, 2016 at 3:31 PM, krad wrote: > > Does this just affect MBR layouts? If possible you might want to consider > UEFI booting for both windows and other os's, It's probably safer as you > dont need to plays with partitions and bootloaders. This is an old computer that doesn't support UEFI booting. In a past I tried older FreeBSD versions on it and I don't remember any boot issue with them. At least up to 9.X versions. From owner-freebsd-stable@freebsd.org Mon Oct 17 12:50:04 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9806FC139C4 for ; Mon, 17 Oct 2016 12:50:04 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 84C82C01 for ; Mon, 17 Oct 2016 12:50:04 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: by mailman.ysv.freebsd.org (Postfix) id 84182C139C2; Mon, 17 Oct 2016 12:50:04 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 83BD4C139C1 for ; Mon, 17 Oct 2016 12:50:04 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from smtprelay02.ispgateway.de (smtprelay02.ispgateway.de [80.67.31.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F734C00 for ; Mon, 17 Oct 2016 12:50:03 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from [78.35.169.228] (helo=fabiankeil.de) by smtprelay02.ispgateway.de with esmtpsa (TLSv1.2:AES256-GCM-SHA384:256) (Exim 4.84) (envelope-from ) id 1bw79R-0000l4-OT for stable@freebsd.org; Mon, 17 Oct 2016 14:36:21 +0200 Date: Mon, 17 Oct 2016 14:34:16 +0200 From: Fabian Keil To: Subject: Poor ZFS ARC metadata hit/miss stats after recent ZFS updates Message-ID: <20161017143416.14024482@fabiankeil.de> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; boundary="Sig_/kXs_2iSLeTV4N4W7nq0.Ihk"; protocol="application/pgp-signature" X-Df-Sender: Nzc1MDY3 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 12:50:04 -0000 --Sig_/kXs_2iSLeTV4N4W7nq0.Ihk Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable After rebasing some of my systems from r305866 to r307312 (plus local patches) I noticed that most of the ARC accesses are counted as misses now. Example: [fk@elektrobier2 ~]$ uptime 2:03PM up 1 day, 18:36, 7 users, load averages: 0.29, 0.36, 0.30 [fk@elektrobier2 ~]$ zfs-stats -E ------------------------------------------------------------------------ ZFS Subsystem Report Mon Oct 17 14:03:58 2016 ------------------------------------------------------------------------ ARC Efficiency: 3.38m Cache Hit Ratio: 12.87% 435.23k Cache Miss Ratio: 87.13% 2.95m Actual Hit Ratio: 9.55% 323.15k Data Demand Efficiency: 6.61% 863.01k CACHE HITS BY CACHE LIST: Most Recently Used: 18.97% 82.54k Most Frequently Used: 55.28% 240.60k Most Recently Used Ghost: 8.88% 38.63k Most Frequently Used Ghost: 24.84% 108.12k CACHE HITS BY DATA TYPE: Demand Data: 13.10% 57.03k Prefetch Data: 0.00% 0 Demand Metadata: 32.94% 143.36k Prefetch Metadata: 53.96% 234.85k CACHE MISSES BY DATA TYPE: Demand Data: 27.35% 805.98k Prefetch Data: 0.00% 0 Demand Metadata: 71.21% 2.10m Prefetch Metadata: 1.44% 42.48k ------------------------------------------------------------------------ I suspect that this is caused by r307265 ("MFC r305323: MFV r302991: 6950 ARC should cache compressed data") which removed a ARCSTAT_CONDSTAT() call but I haven't confirmed this yet. The system performance doesn't actually seem to be negatively affected and repeated metadata accesses that are counted as misses are still served from memory. On my freshly booted laptop I get: fk@t520 /usr/ports $for i in 1 2 3; do \ /usr/local/etc/munin/plugins/zfs-absolute-arc-hits-and-misses; \ time git status > /dev/null; \ done; \ /usr/local/etc/munin/plugins/zfs-absolute-arc-hits-and-misses; zfs_arc_hits.value 5758 zfs_arc_misses.value 275416 zfs_arc_demand_metadata_hits.value 4331 zfs_arc_demand_metadata_misses.value 270252 zfs_arc_demand_data_hits.value 304 zfs_arc_demand_data_misses.value 3345 zfs_arc_prefetch_metadata_hits.value 1103 zfs_arc_prefetch_metadata_misses.value 1489 zfs_arc_prefetch_data_hits.value 20 zfs_arc_prefetch_data_misses.value 334 real 1m23.398s user 0m0.974s sys 0m12.273s zfs_arc_hits.value 11346 zfs_arc_misses.value 389748 zfs_arc_demand_metadata_hits.value 7723 zfs_arc_demand_metadata_misses.value 381018 zfs_arc_demand_data_hits.value 400 zfs_arc_demand_data_misses.value 3412 zfs_arc_prefetch_metadata_hits.value 3202 zfs_arc_prefetch_metadata_misses.value 4885 zfs_arc_prefetch_data_hits.value 21 zfs_arc_prefetch_data_misses.value 437 real 0m1.472s user 0m0.452s sys 0m1.820s zfs_arc_hits.value 11348 zfs_arc_misses.value 428536 zfs_arc_demand_metadata_hits.value 7723 zfs_arc_demand_metadata_misses.value 419782 zfs_arc_demand_data_hits.value 400 zfs_arc_demand_data_misses.value 3436 zfs_arc_prefetch_metadata_hits.value 3204 zfs_arc_prefetch_metadata_misses.value 4885 zfs_arc_prefetch_data_hits.value 21 zfs_arc_prefetch_data_misses.value 437 real 0m1.537s user 0m0.461s sys 0m1.860s zfs_arc_hits.value 11352 zfs_arc_misses.value 467334 zfs_arc_demand_metadata_hits.value 7723 zfs_arc_demand_metadata_misses.value 458556 zfs_arc_demand_data_hits.value 400 zfs_arc_demand_data_misses.value 3460 zfs_arc_prefetch_metadata_hits.value 3208 zfs_arc_prefetch_metadata_misses.value 4885 zfs_arc_prefetch_data_hits.value 21 zfs_arc_prefetch_data_misses.value 437 Disabling ARC compression through vfs.zfs.compressed_arc_enabled does not affect the accounting issue. Can anybody reproduce this? Fabian --Sig_/kXs_2iSLeTV4N4W7nq0.Ihk Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlgExUgACgkQBYqIVf93VJ2QygCgsqtprbFDDiEzr+i0FsUTrwp/ NAQAoLO33YwEhIs0utnZCLDdfEk5J/E2 =OWpZ -----END PGP SIGNATURE----- --Sig_/kXs_2iSLeTV4N4W7nq0.Ihk-- From owner-freebsd-stable@freebsd.org Mon Oct 17 13:00:46 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A9B69C13CB1 for ; Mon, 17 Oct 2016 13:00:46 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 98E07134; Mon, 17 Oct 2016 13:00:46 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 775FFC4; Mon, 17 Oct 2016 13:00:46 +0000 (UTC) Date: Mon, 17 Oct 2016 13:00:42 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <87913082.0.1476709246031.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Build failed in Jenkins: FreeBSD_stable_10 #426 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: FAILURE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 13:00:46 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/426/---------------------= --------------------- [...truncated 105166 lines...] --- PartialInlining.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/PartialInlining.cpp -o Pa= rtialInlining.o --- all_subdir_libllvminstcombine --- --- libllvminstcombine.a --- building static llvminstcombine library ranlib -D libllvminstcombine.a --- all_subdir_libllvmlinker --- =3D=3D=3D> lib/clang/libllvmlinker (all) --- LinkModules.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= linker/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/= src/lib/clang/libllvmlinker/../../../contrib/llvm/tools/clang/include -I/bu= ilds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmlinker/../../../contr= ib/llvm/lib/Linker -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmlinker/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX = -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -= DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER = -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebs= d10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSRO= OT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -= Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmlinker/../../../contrib/llvm/lib/Linker/LinkModules.cpp -o LinkModul= es.o --- all_subdir_libllvmipo --- --- PassManagerBuilder.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/PassManagerBuilder.cpp -o= PassManagerBuilder.o --- StripDeadPrototypes.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/StripDeadPrototypes.cpp -= o StripDeadPrototypes.o --- all_subdir_libllvmcodegen --- --- MachineVerifier.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/MachineVerifier.cpp -= o MachineVerifier.o --- all_subdir_libllvmipo --- --- StripSymbols.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/StripSymbols.cpp -o Strip= Symbols.o --- DeadArgumentElimination.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/DeadArgumentElimination.c= pp -o DeadArgumentElimination.o --- all_subdir_libllvmlinker --- --- libllvmlinker.a --- building static llvmlinker library ranlib -D libllvmlinker.a --- all_subdir_libllvmmc --- =3D=3D=3D> lib/clang/libllvmmc (all) --- ELFObjectWriter.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/ELFObjectWriter.cpp -o ELFObjectWriter.o --- MCAsmBackend.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCAsmBackend.cpp -o MCAsmBackend.o --- all_subdir_libllvmmcparser --- =3D=3D=3D> lib/clang/libllvmmcparser (all) --- AsmLexer.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mcparser/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_1= 0/src/lib/clang/libllvmmcparser/../../../contrib/llvm/tools/clang/include -= I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmcparser/../../..= /contrib/llvm/lib/MC/MCParser -I. -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmmcparser/../../../contrib/llvm/../../lib/clang/include -D= LLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACR= OS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STA= TIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-u= nknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -D= DEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exception= s -fno-rtti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/s= rc/lib/clang/libllvmmcparser/../../../contrib/llvm/lib/MC/MCParser/AsmLexer= .cpp -o AsmLexer.o --- all_subdir_libllvmipo --- --- FunctionAttrs.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/FunctionAttrs.cpp -o Func= tionAttrs.o --- all_subdir_libllvmmcparser --- --- AsmParser.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mcparser/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_1= 0/src/lib/clang/libllvmmcparser/../../../contrib/llvm/tools/clang/include -= I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmcparser/../../..= /contrib/llvm/lib/MC/MCParser -I. -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmmcparser/../../../contrib/llvm/../../lib/clang/include -D= LLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACR= OS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STA= TIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-u= nknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -D= DEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exception= s -fno-rtti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/s= rc/lib/clang/libllvmmcparser/../../../contrib/llvm/lib/MC/MCParser/AsmParse= r.cpp -o AsmParser.o --- all_subdir_libllvmcodegen --- --- OcamlGC.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/OcamlGC.cpp -o OcamlG= C.o --- all_subdir_libllvmmc --- --- MCAsmInfo.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCAsmInfo.cpp -o MCAsmInfo.o --- all_subdir_libllvmcodegen --- --- OptimizePHIs.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/OptimizePHIs.cpp -o O= ptimizePHIs.o --- all_subdir_libllvmmc --- --- MCAsmInfoCOFF.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCAsmInfoCOFF.cpp -o MCAsmInfoCOFF.o --- all_subdir_libllvmipo --- --- GlobalOpt.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/GlobalOpt.cpp -o GlobalOp= t.o --- all_subdir_libllvmmc --- --- MCAsmInfoDarwin.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCAsmInfoDarwin.cpp -o MCAsmInfoDarwin.o --- all_subdir_libllvmcodegen --- --- PHIElimination.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/PHIElimination.cpp -o= PHIElimination.o --- all_subdir_libllvmmc --- --- MCAsmInfoELF.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCAsmInfoELF.cpp -o MCAsmInfoELF.o --- all_subdir_libllvmmcparser --- --- COFFAsmParser.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mcparser/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_1= 0/src/lib/clang/libllvmmcparser/../../../contrib/llvm/tools/clang/include -= I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmcparser/../../..= /contrib/llvm/lib/MC/MCParser -I. -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmmcparser/../../../contrib/llvm/../../lib/clang/include -D= LLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACR= OS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STA= TIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-u= nknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -D= DEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exception= s -fno-rtti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/s= rc/lib/clang/libllvmmcparser/../../../contrib/llvm/lib/MC/MCParser/COFFAsmP= arser.cpp -o COFFAsmParser.o --- all_subdir_libllvmmc --- --- MCAsmStreamer.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCAsmStreamer.cpp -o MCAsmStreamer.o --- all_subdir_libllvmmcparser --- --- DarwinAsmParser.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mcparser/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_1= 0/src/lib/clang/libllvmmcparser/../../../contrib/llvm/tools/clang/include -= I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmcparser/../../..= /contrib/llvm/lib/MC/MCParser -I. -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmmcparser/../../../contrib/llvm/../../lib/clang/include -D= LLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACR= OS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STA= TIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-u= nknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -D= DEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exception= s -fno-rtti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/s= rc/lib/clang/libllvmmcparser/../../../contrib/llvm/lib/MC/MCParser/DarwinAs= mParser.cpp -o DarwinAsmParser.o --- all_subdir_libllvmcodegen --- --- PHIEliminationUtils.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/PHIEliminationUtils.c= pp -o PHIEliminationUtils.o --- all_subdir_libllvmmc --- --- MCAssembler.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCAssembler.cpp -o MCAssembler.o --- all_subdir_libllvmcodegen --- --- Passes.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/Passes.cpp -o Passes.= o --- all_subdir_libllvmipo --- --- InlineAlways.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/InlineAlways.cpp -o Inlin= eAlways.o --- all_subdir_libllvmmcparser --- --- ELFAsmParser.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mcparser/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_1= 0/src/lib/clang/libllvmmcparser/../../../contrib/llvm/tools/clang/include -= I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmcparser/../../..= /contrib/llvm/lib/MC/MCParser -I. -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmmcparser/../../../contrib/llvm/../../lib/clang/include -D= LLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACR= OS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STA= TIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-u= nknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -D= DEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exception= s -fno-rtti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/s= rc/lib/clang/libllvmmcparser/../../../contrib/llvm/lib/MC/MCParser/ELFAsmPa= rser.cpp -o ELFAsmParser.o --- all_subdir_libllvmmc --- --- MCAtom.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCAtom.cpp -o MCAtom.o --- all_subdir_libllvmipo --- --- InlineSimple.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/InlineSimple.cpp -o Inlin= eSimple.o --- all_subdir_libllvmmcparser --- --- MCAsmLexer.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mcparser/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_1= 0/src/lib/clang/libllvmmcparser/../../../contrib/llvm/tools/clang/include -= I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmcparser/../../..= /contrib/llvm/lib/MC/MCParser -I. -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmmcparser/../../../contrib/llvm/../../lib/clang/include -D= LLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACR= OS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STA= TIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-u= nknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -D= DEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exception= s -fno-rtti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/s= rc/lib/clang/libllvmmcparser/../../../contrib/llvm/lib/MC/MCParser/MCAsmLex= er.cpp -o MCAsmLexer.o --- all_subdir_libllvmmc --- --- MCCodeEmitter.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCCodeEmitter.cpp -o MCCodeEmitter.o --- all_subdir_libllvmcodegen --- --- PeepholeOptimizer.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/PeepholeOptimizer.cpp= -o PeepholeOptimizer.o --- all_subdir_libllvmmc --- --- MCCodeGenInfo.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCCodeGenInfo.cpp -o MCCodeGenInfo.o --- all_subdir_libllvmmcparser --- --- MCAsmParser.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mcparser/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_1= 0/src/lib/clang/libllvmmcparser/../../../contrib/llvm/tools/clang/include -= I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmcparser/../../..= /contrib/llvm/lib/MC/MCParser -I. -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmmcparser/../../../contrib/llvm/../../lib/clang/include -D= LLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACR= OS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STA= TIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-u= nknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -D= DEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exception= s -fno-rtti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/s= rc/lib/clang/libllvmmcparser/../../../contrib/llvm/lib/MC/MCParser/MCAsmPar= ser.cpp -o MCAsmParser.o --- all_subdir_libllvmmc --- --- MCContext.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCContext.cpp -o MCContext.o --- all_subdir_libllvmipo --- --- Inliner.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/Inliner.cpp -o Inliner.o --- all_subdir_libllvmmcparser --- --- MCAsmParserExtension.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mcparser/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_1= 0/src/lib/clang/libllvmmcparser/../../../contrib/llvm/tools/clang/include -= I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmcparser/../../..= /contrib/llvm/lib/MC/MCParser -I. -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmmcparser/../../../contrib/llvm/../../lib/clang/include -D= LLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACR= OS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STA= TIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-u= nknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -D= DEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exception= s -fno-rtti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/s= rc/lib/clang/libllvmmcparser/../../../contrib/llvm/lib/MC/MCParser/MCAsmPar= serExtension.cpp -o MCAsmParserExtension.o --- MCTargetAsmParser.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mcparser/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_1= 0/src/lib/clang/libllvmmcparser/../../../contrib/llvm/tools/clang/include -= I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmcparser/../../..= /contrib/llvm/lib/MC/MCParser -I. -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmmcparser/../../../contrib/llvm/../../lib/clang/include -D= LLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACR= OS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STA= TIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-u= nknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -D= DEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exception= s -fno-rtti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/s= rc/lib/clang/libllvmmcparser/../../../contrib/llvm/lib/MC/MCParser/MCTarget= AsmParser.cpp -o MCTargetAsmParser.o --- libllvmmcparser.a --- building static llvmmcparser library ranlib -D libllvmmcparser.a --- all_subdir_libllvmipo --- --- PruneEH.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= ipo/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src= /lib/clang/libllvmipo/../../../contrib/llvm/tools/clang/include -I/builds/w= orkspace/FreeBSD_stable_10/src/lib/clang/libllvmipo/../../../contrib/llvm/l= ib/Transforms/IPO -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/l= ibllvmipo/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLL= VM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLA= NG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno= -strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.= 3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT= =3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -Wn= o-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/li= bllvmipo/../../../contrib/llvm/lib/Transforms/IPO/PruneEH.cpp -o PruneEH.o --- all_subdir_libllvmcodegen --- --- PostRASchedulerList.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/PostRASchedulerList.c= pp -o PostRASchedulerList.o --- all_subdir_libllvmmc --- --- MCDwarf.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCDwarf.cpp -o MCDwarf.o --- MCELF.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCELF.cpp -o MCELF.o --- MCELFObjectTargetWriter.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCELFObjectTargetWriter.cpp -o MCELFObjectTargetWriter.o --- all_subdir_libllvmipo --- --- libllvmipo.a --- building static llvmipo library ranlib -D libllvmipo.a --- all_subdir_libllvmobjcarcopts --- =3D=3D=3D> lib/clang/libllvmobjcarcopts (all) --- ObjCARC.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= objcarcopts/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stabl= e_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/tools/clang/inc= lude -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts= /../../../contrib/llvm/lib/Transforms/ObjCARC -I. -I/builds/workspace/FreeB= SD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/../../l= ib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D_= _STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER= -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unkno= wn-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protec= tor -fno-exceptions -fno-rtti -Wno-c++11-extensions -c /builds/workspace/= FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/li= b/Transforms/ObjCARC/ObjCARC.cpp -o ObjCARC.o --- all_subdir_libllvmobject --- =3D=3D=3D> lib/clang/libllvmobject (all) --- Archive.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= object/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/= src/lib/clang/libllvmobject/../../../contrib/llvm/tools/clang/include -I/bu= ilds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobject/../../../contr= ib/llvm/lib/Object -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX = -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -= DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER = -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebs= d10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSRO= OT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -= Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/lib/Object/Archive.cpp -o Archive.o --- all_subdir_libllvmmc --- --- MCELFStreamer.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCELFStreamer.cpp -o MCELFStreamer.o --- all_subdir_libllvmobject --- --- Binary.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= object/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/= src/lib/clang/libllvmobject/../../../contrib/llvm/tools/clang/include -I/bu= ilds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobject/../../../contr= ib/llvm/lib/Object -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX = -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -= DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER = -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebs= d10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSRO= OT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -= Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/lib/Object/Binary.cpp -o Binary.o --- all_subdir_libllvmcodegen --- --- ProcessImplicitDefs.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/ProcessImplicitDefs.c= pp -o ProcessImplicitDefs.o --- all_subdir_libllvmobjcarcopts --- --- ObjCARCOpts.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= objcarcopts/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stabl= e_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/tools/clang/inc= lude -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts= /../../../contrib/llvm/lib/Transforms/ObjCARC -I. -I/builds/workspace/FreeB= SD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/../../l= ib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D_= _STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER= -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unkno= wn-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protec= tor -fno-exceptions -fno-rtti -Wno-c++11-extensions -c /builds/workspace/= FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/li= b/Transforms/ObjCARC/ObjCARCOpts.cpp -o ObjCARCOpts.o --- all_subdir_libllvmobject --- --- COFFObjectFile.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= object/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/= src/lib/clang/libllvmobject/../../../contrib/llvm/tools/clang/include -I/bu= ilds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobject/../../../contr= ib/llvm/lib/Object -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX = -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -= DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER = -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebs= d10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSRO= OT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -= Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/lib/Object/COFFObjectFile.cpp -o COFFOb= jectFile.o --- all_subdir_libllvmmc --- --- MCExpr.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCExpr.cpp -o MCExpr.o --- all_subdir_libllvmcodegen --- --- PrologEpilogInserter.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/PrologEpilogInserter.= cpp -o PrologEpilogInserter.o --- all_subdir_libllvmobject --- --- COFFYAML.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= object/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/= src/lib/clang/libllvmobject/../../../contrib/llvm/tools/clang/include -I/bu= ilds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobject/../../../contr= ib/llvm/lib/Object -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX = -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -= DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER = -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebs= d10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSRO= OT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -= Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/lib/Object/COFFYAML.cpp -o COFFYAML.o --- all_subdir_libllvmmc --- --- MCExternalSymbolizer.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCExternalSymbolizer.cpp -o MCExternalSymbolizer.o --- all_subdir_libllvmobject --- --- ELF.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= object/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/= src/lib/clang/libllvmobject/../../../contrib/llvm/tools/clang/include -I/bu= ilds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobject/../../../contr= ib/llvm/lib/Object -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX = -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -= DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER = -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebs= d10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSRO= OT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -= Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/lib/Object/ELF.cpp -o ELF.o --- all_subdir_libllvmmc --- --- MCFunction.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCFunction.cpp -o MCFunction.o --- all_subdir_libllvmobject --- --- ELFObjectFile.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= object/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/= src/lib/clang/libllvmobject/../../../contrib/llvm/tools/clang/include -I/bu= ilds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobject/../../../contr= ib/llvm/lib/Object -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX = -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -= DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER = -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebs= d10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSRO= OT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -= Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/lib/Object/ELFObjectFile.cpp -o ELFObje= ctFile.o --- all_subdir_libllvmobjcarcopts --- --- ObjCARCExpand.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= objcarcopts/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stabl= e_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/tools/clang/inc= lude -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts= /../../../contrib/llvm/lib/Transforms/ObjCARC -I. -I/builds/workspace/FreeB= SD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/../../l= ib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D_= _STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER= -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unkno= wn-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protec= tor -fno-exceptions -fno-rtti -Wno-c++11-extensions -c /builds/workspace/= FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/li= b/Transforms/ObjCARC/ObjCARCExpand.cpp -o ObjCARCExpand.o --- all_subdir_libllvmmc --- --- MCInst.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCInst.cpp -o MCInst.o --- all_subdir_libllvmcodegen --- --- PseudoSourceValue.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/PseudoSourceValue.cpp= -o PseudoSourceValue.o --- all_subdir_libllvmmc --- --- MCInstPrinter.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCInstPrinter.cpp -o MCInstPrinter.o --- all_subdir_libllvmcodegen --- --- RegAllocBase.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/RegAllocBase.cpp -o R= egAllocBase.o --- all_subdir_libllvmobjcarcopts --- --- ObjCARCAPElim.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= objcarcopts/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stabl= e_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/tools/clang/inc= lude -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts= /../../../contrib/llvm/lib/Transforms/ObjCARC -I. -I/builds/workspace/FreeB= SD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/../../l= ib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D_= _STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER= -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unkno= wn-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protec= tor -fno-exceptions -fno-rtti -Wno-c++11-extensions -c /builds/workspace/= FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/li= b/Transforms/ObjCARC/ObjCARCAPElim.cpp -o ObjCARCAPElim.o --- all_subdir_libllvmmc --- --- MCInstrAnalysis.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCInstrAnalysis.cpp -o MCInstrAnalysis.o --- MCLabel.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCLabel.cpp -o MCLabel.o --- all_subdir_libllvmobject --- --- ELFYAML.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= object/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/= src/lib/clang/libllvmobject/../../../contrib/llvm/tools/clang/include -I/bu= ilds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobject/../../../contr= ib/llvm/lib/Object -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX = -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -= DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER = -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebs= d10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSRO= OT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -= Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/lib/Object/ELFYAML.cpp -o ELFYAML.o --- all_subdir_libllvmmc --- --- MCMachOStreamer.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCMachOStreamer.cpp -o MCMachOStreamer.o --- all_subdir_libllvmcodegen --- --- RegAllocBasic.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= codegen/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10= /src/lib/clang/libllvmcodegen/../../../contrib/llvm/tools/clang/include -I/= builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmcodegen/../../../co= ntrib/llvm/lib/CodeGen -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/cl= ang/libllvmcodegen/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_= UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDE= BUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANAL= YZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-f= reebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_= SYSROOT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-r= tti -Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/c= lang/libllvmcodegen/../../../contrib/llvm/lib/CodeGen/RegAllocBasic.cpp -o = RegAllocBasic.o --- all_subdir_libllvmobjcarcopts --- --- ObjCARCAliasAnalysis.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= objcarcopts/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stabl= e_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/tools/clang/inc= lude -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts= /../../../contrib/llvm/lib/Transforms/ObjCARC -I. -I/builds/workspace/FreeB= SD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/../../l= ib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D_= _STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER= -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unkno= wn-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protec= tor -fno-exceptions -fno-rtti -Wno-c++11-extensions -c /builds/workspace/= FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/li= b/Transforms/ObjCARC/ObjCARCAliasAnalysis.cpp -o ObjCARCAliasAnalysis.o Resuming build at Mon Oct 17 13:00:36 GMT 2016 after Jenkins restart Waiting to resume part of FreeBSD_stable_10 #426: ??? Ready to run at Mon Oct 17 13:00:37 GMT 2016 --- all_subdir_libllvmobject --- --- Error.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= object/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/= src/lib/clang/libllvmobject/../../../contrib/llvm/tools/clang/include -I/bu= ilds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobject/../../../contr= ib/llvm/lib/Object -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX = -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -= DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER = -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebs= d10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSRO= OT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -= Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/lib/Object/Error.cpp -o Error.o --- all_subdir_libllvmmc --- --- MCMachObjectTargetWriter.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= mc/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/src/= lib/clang/libllvmmc/../../../contrib/llvm/tools/clang/include -I/builds/wor= kspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../contrib/llvm/lib/= MC -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../= ../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D= __STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT = -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing= -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-a= rguments -fstack-protector -fno-exceptions -fno-rtti -Wno-c++11-extensions= -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmmc/../../../c= ontrib/llvm/lib/MC/MCMachObjectTargetWriter.cpp -o MCMachObjectTargetWriter= .o --- all_subdir_libllvmobjcarcopts --- --- ObjCARCContract.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= objcarcopts/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stabl= e_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/tools/clang/inc= lude -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts= /../../../contrib/llvm/lib/Transforms/ObjCARC -I. -I/builds/workspace/FreeB= SD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/../../l= ib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D_= _STDC_CONSTANT_MACROS -DNDEBUG -DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER= -DCLANG_ENABLE_STATIC_ANALYZER -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_= TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unkno= wn-freebsd10.3\" -DDEFAULT_SYSROOT=3D\"\" -Qunused-arguments -fstack-protec= tor -fno-exceptions -fno-rtti -Wno-c++11-extensions -c /builds/workspace/= FreeBSD_stable_10/src/lib/clang/libllvmobjcarcopts/../../../contrib/llvm/li= b/Transforms/ObjCARC/ObjCARCContract.cpp -o ObjCARCContract.o --- all_subdir_libllvmobject --- --- MachOObjectFile.o --- c++ -O2 -pipe -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/libllvm= object/../../../contrib/llvm/include -I/builds/workspace/FreeBSD_stable_10/= src/lib/clang/libllvmobject/../../../contrib/llvm/tools/clang/include -I/bu= ilds/workspace/FreeBSD_stable_10/src/lib/clang/libllvmobject/../../../contr= ib/llvm/lib/Object -I. -I/builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX = -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -= DCLANG_ENABLE_ARCMT -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER = -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebs= d10.3\" -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSRO= OT=3D\"\" -Qunused-arguments -fstack-protector -fno-exceptions -fno-rtti -= Wno-c++11-extensions -c /builds/workspace/FreeBSD_stable_10/src/lib/clang/= libllvmobject/../../../contrib/llvm/lib/Object/MachOObjectFile.cpp -o MachO= ObjectFile.o [Pipeline] } [Pipeline] // stage [Pipeline] } [Pipeline] // withEnv [Pipeline] } [Pipeline] // dir [Pipeline] } [Pipeline] // node [Pipeline] node Running on master in /usr/local/jenkins/workspace/FreeBSD_stable_10 [Pipeline] { [Pipeline] step From owner-freebsd-stable@freebsd.org Mon Oct 17 13:43:03 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 04FA9C12D6B for ; Mon, 17 Oct 2016 13:43:03 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from mail.mimar.rs (mail1.mimar.rs [193.53.106.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AEA35F5A for ; Mon, 17 Oct 2016 13:43:02 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from mail1.mimar.rs (localhost [127.0.1.128]) by mail.mimar.rs (Postfix) with ESMTP id DF09B45BA209 for ; Mon, 17 Oct 2016 15:37:11 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mimar.rs; h= content-transfer-encoding:content-type:content-type:mime-version :x-mailer:organization:message-id:subject:subject:from:from:date :date:received:received; s=mimar-0901; t=1476711429; x= 1478525830; bh=LNK/HABEYy9ywXU2vIdvo2jzhyZiqSsHzfD96XxouCc=; b=k k3AJPnFA+7KyzXEftoo8fQlhDx7UAJuK18lxlx+BpcywjsvIsoWKTYJCqv5UmHDW 5ZR5pu87NVEVO4wa8gS/95dlONc71sBlRP0ls6K1iYWDXuVyu2WD7HXJgkNBZl0K qr+yKVIZfCIzAlaZ4vn3FhCDocAZY/shvpu4pbReoY= X-Virus-Scanned: amavisd-new at mimar.rs Received: from mail.mimar.rs ([127.0.1.128]) by mail1.mimar.rs (amavis.mimar.rs [127.0.1.128]) (amavisd-new, port 10026) with LMTP id 3T10ZSyIYs5o for ; Mon, 17 Oct 2016 15:37:09 +0200 (CEST) Received: from efreet.mimar.rs (93-87-131-33.dynamic.isp.telekom.rs [93.87.131.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: marko.cupac@mimar.rs) by mail.mimar.rs (Postfix) with ESMTPSA id 2461545BA10E for ; Mon, 17 Oct 2016 15:37:09 +0200 (CEST) Date: Mon, 17 Oct 2016 15:37:08 +0200 From: Marko =?UTF-8?B?Q3VwYcSH?= To: freebsd-stable@freebsd.org Subject: moving ezjail-based jails from 10.3 host to 11.0 host Message-ID: <20161017153708.411492e5@efreet.mimar.rs> Organization: Mimar X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 13:43:03 -0000 Hi, I have 10.3 host which runs a dozen or so ezjail-based jails. I have installed another 11.0 host, and I'd like to move jails to it. Can I just archive jails on 10.3, scp them to 11.0, and re-create them there by restoring from archive (-a switch)? Are there any additional actions I should perform? Thank you in advance, --=20 Before enlightenment - chop wood, draw water. After enlightenment - chop wood, draw water. Marko Cupa=C4=87 https://www.mimar.rs/ From owner-freebsd-stable@freebsd.org Mon Oct 17 14:05:02 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E24BAC14462 for ; Mon, 17 Oct 2016 14:05:02 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: from mail-qt0-x230.google.com (mail-qt0-x230.google.com [IPv6:2607:f8b0:400d:c0d::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C91BF8 for ; Mon, 17 Oct 2016 14:05:02 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: by mail-qt0-x230.google.com with SMTP id f6so125063841qtd.2 for ; Mon, 17 Oct 2016 07:05:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=tM1YM4ZrqNLnX0u7VCF5teUL5d4/6Z9EztxLYC0kB4A=; b=Yi6KpNrNgvdg6N6jpg0LPWva1c+sQSD685j9HZbL+EdCzxptPD4/ilc0VtSweubWDK Ktvft3zvspSO6hsEEVh1Z4gLFgl5vkYNIctno7kv5ScXgWHNzjjC0E2AYKg1p+j1vhxd PaKuTG3kfuc9Q9c5MVIQ2BQUR/fQy7ZDbDoFC33IzSHpOccJHGt2WhufG7zXp89kXyfX yVg73wy0H9HNHiDjWqRSSr0ukF4kkeG3tC/YL1k3J48G93W9KPYJO7XCPerVEWDjwtYA Irn5/dwSkgwgTVD7JTQ9bgoU8TFae8xfWIR8dRsB0AVuIfmXrJRX7/nPfdEB/kpyHGNn m/TA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=tM1YM4ZrqNLnX0u7VCF5teUL5d4/6Z9EztxLYC0kB4A=; b=GsTgTxKIsfG64lRuiYZrAVNR2kEaP2JXBZrIWDkmpjPLjiGktIcBh+qAun+jcqz0Dd CKiUm+qU8rpek3Zu0JeWDOqqsEQUm007dujOidgJdf7Zu4HJtJcs3Y1G0xSBuYcKGH9i j05AEHPr7g4JXYFZ2hzXiI5qXT1xOTeWHa+Yz7SW8CjnaxZ6tXWw6Oo43ZD6OcpztDHr 2R8ONRSuCPXlh4YZiJLRMMGSjWCZznVOpz1U86gPv5RIhBHtIzrCkIYG5mnq9k9R5ZsR tCtX7lOKZf+TT4PJNZG14nz7SB3uJZRRclb6pShdgLTkM33zXja8J8o0NgU8Z5PU8fuS eF1Q== X-Gm-Message-State: AA6/9RlDt1X3vBJsxUCmHvmYs00Y7nyTo8sXwUI2y0uIR6MB+OoTQEzF6Q36YFMpFCI9LkNYVIo6mOdxFf+kpg== X-Received: by 10.194.201.36 with SMTP id jx4mr11279931wjc.38.1476713101695; Mon, 17 Oct 2016 07:05:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.158.130 with HTTP; Mon, 17 Oct 2016 07:05:00 -0700 (PDT) In-Reply-To: References: From: Rostislav Krasny Date: Mon, 17 Oct 2016 17:05:00 +0300 Message-ID: Subject: Re: I'm upset about FreeBSD To: krad Cc: freebsd-stable , "Eugene M. Zheganin" Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 14:05:03 -0000 On Mon, Oct 17, 2016 at 3:39 PM, Rostislav Krasny wrote: > On Mon, Oct 17, 2016 at 3:31 PM, krad wrote: >> >> Does this just affect MBR layouts? If possible you might want to consider >> UEFI booting for both windows and other os's, It's probably safer as you >> dont need to plays with partitions and bootloaders. > > This is an old computer that doesn't support UEFI booting. In a past I > tried older FreeBSD versions on it and I don't remember any boot issue > with them. At least up to 9.X versions. Ok. I dropped the FreeBSD ada0s2 slice and rewrote the MBR code by MbrFix util from www.sysint.no/mbrfix Then I tried to install FreeBSD 11.0 again. Previously I used the manual partitioning and now I used the guided UFS partitioning. This time FreeBSD was installed properly without any boot issue. FreeBSD was booting straight away. After that I ran "boot0cfg -B ada0" and the boot0 boot manager is working properly again: both F1 for Windows and F2 for FreeBSD. It's much better than in my first try. There probably is some bug in the bsdinstall(8) when the manual partitioning is used instead of the guided one. From owner-freebsd-stable@freebsd.org Mon Oct 17 14:05:07 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1D09C14465 for ; Mon, 17 Oct 2016 14:05:07 +0000 (UTC) (envelope-from lists@peter.de.com) Received: from elsa.gfuzz.de (elsa.gfuzz.de [78.46.164.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 87318FB for ; Mon, 17 Oct 2016 14:05:07 +0000 (UTC) (envelope-from lists@peter.de.com) Received: from localhost (localhost [127.0.0.1]) by elsa.gfuzz.de (Postfix) with ESMTP id 28BDD10017A; Mon, 17 Oct 2016 15:55:10 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at elsa.gfuzz.de Received: from elsa.gfuzz.de ([127.0.0.1]) by localhost (localhost [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F6LJ4IsrqStY; Mon, 17 Oct 2016 15:55:09 +0200 (CEST) Received: from mail.opdns.de (ipbcc19187.dynamic.kabel-deutschland.de [188.193.145.135]) (Authenticated sender: oliver@gfuzz.de) by elsa.gfuzz.de (Postfix) with ESMTPSA id 3C2D21000AB; Mon, 17 Oct 2016 15:55:09 +0200 (CEST) Date: Mon, 17 Oct 2016 15:55:07 +0200 From: Oliver Peter To: Marko =?utf-8?B?Q3VwYcSH?= Cc: freebsd-stable@freebsd.org Subject: Re: moving ezjail-based jails from 10.3 host to 11.0 host Message-ID: <20161017135507.GA10396@mail.opdns.de> References: <20161017153708.411492e5@efreet.mimar.rs> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <20161017153708.411492e5@efreet.mimar.rs> X-Operating-System: Linux 4.4.16-1-pve x86_64 User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 14:05:07 -0000 On Mon, Oct 17, 2016 at 03:37:08PM +0200, Marko Cupa=C4=87 wrote: > I have 10.3 host which runs a dozen or so ezjail-based jails. I have > installed another 11.0 host, and I'd like to move jails to it. I would switch them to iocage+zfs, ezjail is sooo 90s. :) Have a look at the documentation: http://iocage.readthedocs.io/en/latest/basic-use.html All jail settings are stored within ZFS properties so an upcoming migration would only need a zfs send | zfs receive. > Can I just archive jails on 10.3, scp them to 11.0, and re-create them > there by restoring from archive (-a switch)? Further I would recommend to use rsync -av instead of scp. --=20 Oliver PETER oliver@gfuzz.de 0x456D688F From owner-freebsd-stable@freebsd.org Mon Oct 17 14:18:04 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27C26C148E9 for ; Mon, 17 Oct 2016 14:18:04 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE754BA9 for ; Mon, 17 Oct 2016 14:18:03 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 8EDE4303184 for ; Mon, 17 Oct 2016 09:17:56 -0500 (CDT) To: freebsd-stable@freebsd.org From: Karl Denninger Subject: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE Message-ID: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> Date: Mon, 17 Oct 2016 09:17:54 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms020504020101080608050301" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 14:18:04 -0000 This is a cryptographically signed message in MIME format. --------------ms020504020101080608050301 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable This is a situation I've had happen before, and reported -- it appeared to be a kernel stack overflow, and it has gotten materially worse on 11.0-STABLE. The issue occurs after some period of time (normally a week or so.) The system has a mirrored pair of large drives used for backup purposes to which ZFS snapshots are written using a script that iterates over the system. The panic /only /happens when the root filesystem is being sent, and it appears that the panic itself is being triggered by an I/O pattern on the /backup /drive -- not the source drives. Zpool scrubs on the source are clean; I am going to run one now on the backup, but in the past that has been clean as well. I now have a *repeatable* panic in that if I attempt a "zfs list -rt all backup" on the backup volume I get the below panic. A "zfs list" does*__*not panic the system. The operating theory previously (after digging through the passed structures in the dump) was that the ZFS system was attempting to issue TRIMs on a device that can't do them before the ZFS system realizes this and stops asking (the backup volume is comprised of spinning rust) but the appearance of the panic now on the volume when I simply do a "zfs list -rt all backup" appears to negate that theory since no writes are performed by that operation, and thus no TRIM calls should be issued. I can leave the backup volume in the state that causes this for a short period of time in an attempt to find and fix this. NewFS.denninger.net dumped core - see /var/crash/vmcore.1 Mon Oct 17 09:02:33 CDT 2016 FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 r307318M: Fri Oct 14 09:23:46 CDT 2016 =20 karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 panic: double fault GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you = are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for detail= s. This GDB was configured as "amd64-marcel-freebsd"... Unread portion of the kernel message buffer: Fatal double fault rip =3D 0xffffffff8220d9ec rsp =3D 0xfffffe066821f000 rbp =3D 0xfffffe066821f020 cpuid =3D 6; apic id =3D 14 panic: double fault cpuid =3D 6 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0649d78e30 vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 panic() at panic+0x43/frame 0xfffffe0649d78f10 dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30 Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 --- trap 0x17, rip =3D 0xffffffff8220d9ec, rsp =3D 0xfffffe066821f000, rb= p =3D 0xfffffe066821f020 --- avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame 0xfffffe066821f530 vdev_queue_io_done() at vdev_queue_io_done+0x83/frame 0xfffffe066821f570 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f650 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f6e0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f7c0 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f850 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f930 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f9c0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821faa0 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fb30 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fc10 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fca0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fd80 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fe10 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fef0 zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821ff80 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220060 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682200f0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682201d0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220260 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220340 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682203d0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682204b0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220540 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220620 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682206b0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220790 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220820 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220900 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220990 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220a70 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220b00 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220be0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220c70 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220d50 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220de0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220ec0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220f50 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221030 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682210c0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682211a0 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221230 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221310 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682213a0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221480 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221510 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682215f0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221680 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221760 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682217f0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682218d0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221960 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221a40 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221ad0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221bb0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221c40 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221d20 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221db0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221e90 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221f20 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222000 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222090 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222170 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222200 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682222e0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222370 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222450 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682224e0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682225c0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222650 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222730 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682227c0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682228a0 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222930 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960 zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222a10 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222aa0 zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0 zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame 0xfffffe0668222b80 taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame 0xfffffe0668222bb0 fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- Uptime: 4m29s Dumping 1497 out of 24531 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. done. Loaded symbols for /boot/kernel/geom_eli.ko Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/aesni.ko...Reading symbols from /usr/lib/debug//boot/kernel/aesni.ko.debug...done. done. Loaded symbols for /boot/kernel/aesni.ko Reading symbols from /boot/kernel/uhid.ko...Reading symbols from /usr/lib/debug//boot/kernel/uhid.ko.debug...done. done. Loaded symbols for /boot/kernel/uhid.ko Reading symbols from /boot/kernel/ums.ko...Reading symbols from /usr/lib/debug//boot/kernel/ums.ko.debug...done. done. Loaded symbols for /boot/kernel/ums.ko Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. done. Loaded symbols for /boot/kernel/ipfw.ko Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. done. Loaded symbols for /boot/kernel/ipfw_nat.ko Reading symbols from /boot/kernel/libalias.ko...Reading symbols from /usr/lib/debug//boot/kernel/libalias.ko.debug...done. done. Loaded symbols for /boot/kernel/libalias.ko Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols from /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. done. Loaded symbols for /boot/kernel/daemon_saver.ko #0 doadump (textdump=3D1) at pcpu.h:221 221 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=3D1) at pcpu.h:221 #1 0xffffffff80ae9b6e in kern_reboot (howto=3D260) at /usr/src/sys/kern/kern_shutdown.c:366 #2 0xffffffff80aea13b in vpanic (fmt=3D, ap=3D) at /usr/src/sys/kern/kern_shutdown.c:759 #3 0xffffffff80ae9f73 in panic (fmt=3D0x0) at /usr/src/sys/kern/kern_shutdown.c:690 #4 0xffffffff80fd18f2 in dblfault_handler (frame=3D= ) at /usr/src/sys/amd64/amd64/trap.c:868 #5 0xffffffff80fb406c in Xdblfault () at /usr/src/sys/amd64/amd64/exception.S:292 #6 0xffffffff8220d9ec in avl_rotation (tree=3D0xfffff8004dc54510, node=3D0xfffff8056e45cdd0, balance=3D2) at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 #7 0xffffffff8220de58 in avl_remove (tree=3D, data=3D) at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:439 #9 0xffffffff822a2123 in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:907 #10 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e3f33d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #11 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #12 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e3f33d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #13 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #14 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #15 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e578000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #16 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #17 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e578000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #18 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #19 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #20 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ec3d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #21 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #22 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ec3d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #23 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #24 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #25 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e48a000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #26 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #27 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e48a000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #28 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #29 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #30 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e57ab88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #31 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #32 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e57ab88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #33 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #34 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #35 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69a3000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #36 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #37 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69a3000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #38 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #39 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #40 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5c8000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #41 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #42 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5c8000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #43 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #44 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #45 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23f9000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #46 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #47 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23f9000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #48 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #49 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #50 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5147b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #51 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #52 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5147b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #53 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #54 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #55 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e977000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #56 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #57 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e977000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #58 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #59 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #60 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90b3d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #61 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #62 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90b3d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #63 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #64 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #65 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b1160b88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #66 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #67 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b1160b88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #68 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #69 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #70 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4f53d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #71 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #72 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4f53d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #73 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #74 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #75 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8ebb88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #76 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #77 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8ebb88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #78 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #79 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #80 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8d9b88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #81 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #82 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8d9b88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #83 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #84 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #85 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21a03d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #86 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #87 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21a03d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #88 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #89 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #90 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e6903d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #91 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #92 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e6903d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #93 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #94 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #95 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a22dc7b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #96 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #97 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a22dc7b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #98 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #99 0xffffffff822a216d in vdev_queue_io_done (zio=3D= ) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #100 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5fb7b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #101 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #102 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5fb7b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #103 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #104 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #105 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ae3d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #106 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #107 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ae3d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #108 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #109 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #110 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a2131b88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #111 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #112 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a2131b88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #113 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #114 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #115 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e80b7b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #116 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #117 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e80b7b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #118 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #119 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #120 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b14b9000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #121 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #122 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b14b9000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #123 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #124 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #125 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21fa3d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #126 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #127 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21fa3d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #128 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #129 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #130 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5333d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #131 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #132 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5333d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #133 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #134 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #135 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90ab88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #136 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #137 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90ab88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #138 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #139 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #140 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4047b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #141 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #142 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4047b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #143 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #144 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #145 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e7083d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #146 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #147 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e7083d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #148 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #149 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #150 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23917b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #151 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #152 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23917b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #153 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #154 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #155 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e636000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #156 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #157 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e636000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #158 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #159 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #160 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5afb88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #161 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #162 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5afb88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #163 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #164 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #165 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69523d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #166 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #167 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69523d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #168 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #169 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #170 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e47c000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #171 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #172 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e47c000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #173 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #174 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #175 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b17b23d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #176 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #177 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b17b23d8) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #178 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #179 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #180 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d367b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #181 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #182 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a6d367b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #183 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #184 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #185 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e99fb88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #186 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #187 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e99fb88) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #188 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #189 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #190 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5227b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #191 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #192 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5227b0) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= #193 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #194 0xffffffff822a216d in vdev_queue_io_done (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 #195 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d3c000) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= #196 0xffffffff822bbefd in zio_execute (zio=3D) at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= #197 0xffffffff80b4895a in taskqueue_run_locked (queue=3D) at /usr/src/sys/kern/subr_taskqueue.c:454 #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=3D) at /usr/src/sys/kern/subr_taskqueue.c:724 #199 0xffffffff80a9f255 in fork_exit ( callout=3D0xffffffff80b49a70 , arg=3D0xfffff8057caa72a0, frame=3D0xfffffe0668222c00) at /usr/src/sys/kern/kern_fork.c:1040 #200 0xffffffff80fb44ae in fork_trampoline () at /usr/src/sys/amd64/amd64/exception.S:611 #201 0x0000000000000000 in ?? () Current language: auto; currently minimal (kgdb) --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms020504020101080608050301 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEwMTcxNDE3NTRaME8GCSqGSIb3DQEJBDFCBEBs owjsHsm1VcxRjJVMpRsb10xXGsHlMObTuzawp9lj4wmJmuVNu0RL4R7AfwttKTJQj1qOh5bJ O6LRNIoT6khcMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIADlUn3EV3 cWZ4ZeXjbn63SwoIPlXRyPO2XUGADSSzqBiSCb+EvzI04dIKaJ1RAwABd9Vi5l6VaAeXktuv 8j7LOe/gkqs6vBhmJsHaf9cA1jCsRQj/GLEBy238SpdM925WLtDkJ4YPaBEuLwyxGBWx+x1a PQMGG6HFh5D8c3Bb0t/gYNjWvvRYA099TCj2k5/ylbjjEdwymgM8ka2DDpYctw6fD/jEKz7O H5FrjiRYWW6HazH/u5Y9UzqrNiY20lN2AqXtVs3v8VTXFQnUXXNAub1bBkg2k/tsoODcjakm Rhrkf51a8L7M5FVacJlVEC2BZCX/dq73WxKqgL1EuhXeLyoxsCPYkqZqqeJlkpSa1n+7j9Ab xjG9GwUdbgXnGxeFGjXmJKAT5qrN4nyy9iSVLuoMF5xujS/8qecyblRkuvLR4BUm/dzDPF8n NdkoCs98aK8llt4CxEA3IpGYIkTC9gHWrDSvhrC8uHz2Y9e8P3qFmPvCON3h3SXXN2AQiooZ B5A5gDV2W4F23Vv1hIrPFaNNj5SUJM7uSuUYENAf3q8YaFg0PWSbXVyv8QT5yXxpWrh2qMQZ g6+GXrBeYO1JbUzJh/c10gS/N414SOvoQTm7NXFLAbTbr/y0cyRP5MMXrrjXo0Us05eO2KEL og9TougdNm0JCWPQ/QdTIBUFIbAAAAAAAAA= --------------ms020504020101080608050301-- From owner-freebsd-stable@freebsd.org Mon Oct 17 15:03:00 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6000FC15AEC for ; Mon, 17 Oct 2016 15:03:00 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [IPv6:2001:418:3fd::f7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2AB26E4B for ; Mon, 17 Oct 2016 15:03:00 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [10.100.0.31] (haymarket.m5p.com [10.100.0.31]) by mailhost.m5p.com (8.14.5/8.14.5) with ESMTP id u9HF2qtj046339 for ; Mon, 17 Oct 2016 11:02:57 -0400 (EDT) (envelope-from george+freebsd@m5p.com) Subject: Re: Clandestine USB SD card slot References: <832db721-d03a-a21e-11ed-701190140d17@yahoo.com> <2bf09be6-e2ad-2d38-ffd8-d80b45d4c3d8@m5p.com> <82d2acb3-12cd-678e-c90a-21b9d62b5721@m5p.com> From: George Mitchell To: FreeBSD Stable Mailing List Message-ID: Date: Mon, 17 Oct 2016 11:02:52 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.73 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (mailhost.m5p.com [10.100.0.247]); Mon, 17 Oct 2016 11:02:58 -0400 (EDT) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 15:03:00 -0000 On 10/16/16 17:40, George Mitchell wrote: > On 10/16/16 14:16, Warner Losh wrote: >> On Sun, Oct 16, 2016 at 12:08 PM, Warren Block wrote: >>> On Sun, 16 Oct 2016, George Mitchell wrote: >>> >>>>> So not only is it (apparently) recognized, but the sdhci_pci driver >>>>> attached to it! But inserting or removing a card shows no activity. >>>>> What's my next step? -- George >>> >>> >>> Is a device created for the empty reader? It's worth trying to force a >>> retaste of that device with 'true > /dev/daX' after the card is inserted. >> >> Don't look for da anything. Look for mmcsd something. The sdhci_pci >> driver provides disks that are mmcsdX. Looks like card change >> interrupts aren't happening, or there's something else making the >> driver unhappy with the SDHCI controller though... >> >> Warner >> [...] > > No /dev/mm*; no log output on card insertion/removal even with > sysctl hw.sdhci.debug=1. Other sysctl info: > > sysctl -a | grep sdhci > device sdhci > hw.sdhci.enable_msi: 1 > hw.sdhci.debug: 1 > dev.sdhci_pci.0.%parent: pci0 > dev.sdhci_pci.0.%pnpinfo: vendor=0x1022 device=0x7813 subvendor=0x1025 > subdevice=0x0865 class=0x080501 > dev.sdhci_pci.0.%location: pci0:0:20:7 > dev.sdhci_pci.0.%driver: sdhci_pci > dev.sdhci_pci.0.%desc: Generic SD HCI > dev.sdhci_pci.%parent: > > -- George > After setting hw.sdhci.debug=1 and hw.mmc.debug=1 in /etc/sysctl.conf and doing a verbose boot, then inserting and removing an SD card, all I get in "dmesg | egrep mmc\|sdhci" is: sdhci_pci0: mem 0xf0c6c000-0xf0c6c0ff irq 16 at device 20.7 on pci0 sdhci_pci0: 1 slot(s) allocated sdhci_pci0: mem 0xf0c6c000-0xf0c6c0ff irq 16 at device 20.7 on pci0 sdhci_pci0-slot0: 50MHz 8bits 3.3V DMA sdhci_pci0-slot0: ============== REGISTER DUMP ============== sdhci_pci0-slot0: Sys addr: 0x00000000 | Version: 0x00001001 sdhci_pci0-slot0: Blk size: 0x00000000 | Blk cnt: 0x00000000 sdhci_pci0-slot0: Argument: 0x00000000 | Trn mode: 0x00000000 sdhci_pci0-slot0: Present: 0x01f20000 | Host ctl: 0x00000000 sdhci_pci0-slot0: Power: 0x00000000 | Blk gap: 0x00000000 sdhci_pci0-slot0: Wake-up: 0x00000000 | Clock: 0x00000000 sdhci_pci0-slot0: Timeout: 0x00000000 | Int stat: 0x00000000 sdhci_pci0-slot0: Int enab: 0x01ff00fb | Sig enab: 0x01ff00fb sdhci_pci0-slot0: AC12 err: 0x00000000 | Slot int: 0x000000ff sdhci_pci0-slot0: Caps: 0x21de32b2 | Max curr: 0x00c80064 sdhci_pci0-slot0: =========================================== sdhci_pci0: 1 slot(s) allocated (Same for "egrep mmc\|sdhci /var/log/messages".) "pciconf -lv" suggests this is a: sdhci_pci0@pci0:0:20:7: class=0x080501 card=0x08651025 chip=0x78131022 rev=0x01 hdr=0x00 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'FCH SD Flash Controller' class = base peripheral subclass = SD host controller Are there some quirks I should define for this controller? -- George From owner-freebsd-stable@freebsd.org Mon Oct 17 15:04:40 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1B63C15BFB for ; Mon, 17 Oct 2016 15:04:40 +0000 (UTC) (envelope-from herbert@mailbox.org) Received: from mx2.mailbox.org (mx2.mailbox.org [80.241.60.215]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.mailbox.org", Issuer "SwissSign Server Silver CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B295AFDB for ; Mon, 17 Oct 2016 15:04:40 +0000 (UTC) (envelope-from herbert@mailbox.org) Received: from smtp1.mailbox.org (smtp1.mailbox.org [80.241.60.240]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx2.mailbox.org (Postfix) with ESMTPS id 4E9FF43790 for ; Mon, 17 Oct 2016 17:04:32 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mailbox.org; h= content-transfer-encoding:content-type:content-type:mime-version :references:in-reply-to:subject:subject:from:from:message-id :date:date:received; s=mail20150812; t=1476716668; bh=mjOkfhtBLI m9NN3VZoUuj+3iWuSdysfnReXvBKDWQGE=; b=tuLb0IDXZ2yQoeZhTmC9Fssqnk Ps7Q8ZDRSvvw2KRkMvXqBD3Ba6IDUt6PLaeH4PMT90yUYGNFtVkVLhvHt+juNzQ/ ++pkIjkYPN2k7z1ivSEevUpwMDsoWR/aghtpZMZARM9cLnnnItmGUGaMRyHZhxcF 1FWrfkDNoGp/1k7YANOiikSZMXOhsg8cnjBTxiNArOQOONsD6AyYyxyvGxVoyXhb Gv6VevDDBVJjh79BUoz4isMyZRL+hQlaGpFBysdTHKO1+XyXCDO0Q7WvuCxeoz7d Jc+u1cVwbF6UC9neWYRviQpX83b0NM2ANSLwRmADGSGU5W9ht6ZHkuV2HD1w== X-Virus-Scanned: amavisd-new at heinlein-support.de Received: from smtp1.mailbox.org ([80.241.60.240]) by gerste.heinlein-support.de (gerste.heinlein-support.de [91.198.250.173]) (amavisd-new, port 10030) with ESMTP id Dw3B7NMYqYe5 for ; Mon, 17 Oct 2016 17:04:28 +0200 (CEST) Date: Mon, 17 Oct 2016 17:04:04 +0200 Message-ID: <86h98buie3.wl-herbert@mailbox.org> From: "Herbert J. Skuhra" To: freebsd-stable@freebsd.org Subject: Re: moving ezjail-based jails from 10.3 host to 11.0 host In-Reply-To: <20161017135507.GA10396@mail.opdns.de> References: <20161017153708.411492e5@efreet.mimar.rs> <20161017135507.GA10396@mail.opdns.de> MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 15:04:41 -0000 Oliver Peter skrev: > = > On Mon, Oct 17, 2016 at 03:37:08PM +0200, Marko Cupa=E6 wrote: >> I have 10.3 host which runs a dozen or so ezjail-based jails. I have= >> installed another 11.0 host, and I'd like to move jails to it. > = > I would switch them to iocage+zfs, ezjail is sooo 90s. :) > Have a look at the documentation: > http://iocage.readthedocs.io/en/latest/basic-use.html The github page says: **No longer supported. iocage is being rewritten in a different language. -- Herbert From owner-freebsd-stable@freebsd.org Mon Oct 17 15:05:49 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 779AFC15CC1 for ; Mon, 17 Oct 2016 15:05:49 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from mailhost.m5p.com (mailhost.m5p.com [IPv6:2001:418:3fd::f7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1805216E for ; Mon, 17 Oct 2016 15:05:48 +0000 (UTC) (envelope-from george+freebsd@m5p.com) Received: from [10.100.0.31] (haymarket.m5p.com [10.100.0.31]) by mailhost.m5p.com (8.14.5/8.14.5) with ESMTP id u9HF5f4E046377 for ; Mon, 17 Oct 2016 11:05:47 -0400 (EDT) (envelope-from george+freebsd@m5p.com) Subject: Re: Clandestine USB SD card slot To: FreeBSD Stable Mailing List References: <832db721-d03a-a21e-11ed-701190140d17@yahoo.com> <2bf09be6-e2ad-2d38-ffd8-d80b45d4c3d8@m5p.com> <82d2acb3-12cd-678e-c90a-21b9d62b5721@m5p.com> From: George Mitchell Message-ID: Date: Mon, 17 Oct 2016 11:05:41 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.73 X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (mailhost.m5p.com [10.100.0.247]); Mon, 17 Oct 2016 11:05:47 -0400 (EDT) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 15:05:49 -0000 On 10/17/16 11:02, George Mitchell wrote: > [...] > After setting hw.sdhci.debug=1 and hw.mmc.debug=1 in /etc/sysctl.conf > and doing a verbose boot, then inserting and removing an SD card, all > I get in "dmesg | egrep mmc\|sdhci" is: > > sdhci_pci0: mem 0xf0c6c000-0xf0c6c0ff irq 16 at device > 20.7 on pci0 > sdhci_pci0: 1 slot(s) allocated > sdhci_pci0: mem 0xf0c6c000-0xf0c6c0ff irq 16 at device > 20.7 on pci0 > sdhci_pci0-slot0: 50MHz 8bits 3.3V DMA > sdhci_pci0-slot0: ============== REGISTER DUMP ============== > sdhci_pci0-slot0: Sys addr: 0x00000000 | Version: 0x00001001 > sdhci_pci0-slot0: Blk size: 0x00000000 | Blk cnt: 0x00000000 > sdhci_pci0-slot0: Argument: 0x00000000 | Trn mode: 0x00000000 > sdhci_pci0-slot0: Present: 0x01f20000 | Host ctl: 0x00000000 > sdhci_pci0-slot0: Power: 0x00000000 | Blk gap: 0x00000000 > sdhci_pci0-slot0: Wake-up: 0x00000000 | Clock: 0x00000000 > sdhci_pci0-slot0: Timeout: 0x00000000 | Int stat: 0x00000000 > sdhci_pci0-slot0: Int enab: 0x01ff00fb | Sig enab: 0x01ff00fb > sdhci_pci0-slot0: AC12 err: 0x00000000 | Slot int: 0x000000ff > sdhci_pci0-slot0: Caps: 0x21de32b2 | Max curr: 0x00c80064 > sdhci_pci0-slot0: =========================================== > sdhci_pci0: 1 slot(s) allocated > > (Same for "egrep mmc\|sdhci /var/log/messages".) > > "pciconf -lv" suggests this is a: > sdhci_pci0@pci0:0:20:7: class=0x080501 card=0x08651025 chip=0x78131022 > rev=0x01 hdr=0x00 > vendor = 'Advanced Micro Devices, Inc. [AMD]' > device = 'FCH SD Flash Controller' > class = base peripheral > subclass = SD host controller > > Are there some quirks I should define for this controller? -- George > For those coming in late: FreeBSD 10.3-RELEASE-p7 #0: Thu Aug 11 18:38:15 UTC 2016 -- George From owner-freebsd-stable@freebsd.org Mon Oct 17 15:30:00 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A127AC15430 for ; Mon, 17 Oct 2016 15:30:00 +0000 (UTC) (envelope-from zkolic@sbb.rs) Received: from mproxy21.sbb.rs (mproxy21.sbb.rs [89.216.2.106]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (Client CN "smtp.sbb.rs", Issuer "COMODO RSA Domain Validation Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 02B4C14F for ; Mon, 17 Oct 2016 15:29:59 +0000 (UTC) (envelope-from zkolic@sbb.rs) Received: from knossos ([87.116.189.12]) by mproxy21.sbb.rs (8.15.2/8.15.2) with ESMTP id u9HEuV7b022066; Mon, 17 Oct 2016 16:56:31 +0200 X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.97.6 at SBB mail Received: from localhost (knossos [local]) by knossos (OpenSMTPD) with ESMTPA id 6a111cc1; Mon, 17 Oct 2016 16:56:31 +0200 (CEST) Date: Mon, 17 Oct 2016 16:56:31 +0200 From: Zoran Kolic To: imp@bsdimp.com Cc: freebsd-stable@freebsd.org Subject: Re: update from 9.3 to 11.0 Message-ID: <20161017145631.GA58857@knossos> References: <20161016141328.GA57825@knossos> <2E339839-8BD3-4B96-ACE3-41FFAD6F8484@mail.sermon-archive.info> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2E339839-8BD3-4B96-ACE3-41FFAD6F8484@mail.sermon-archive.info> X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mproxy21.sbb.rs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 15:30:00 -0000 > Index: head/usr.sbin/freebsd-update/freebsd-update.sh > =================================================================== > --- head/usr.sbin/freebsd-update/freebsd-update.sh (revision 279900) > +++ head/usr.sbin/freebsd-update/freebsd-update.sh (revision 279901) > @@ -1231,7 +1231,7 @@ fetch_metadata_sanity () { > # Some aliases to save space later: ${P} is a character which can > # appear in a path; ${M} is the four numeric metadata fields; and > # ${H} is a sha256 hash. > - P="[-+./:=%@_[~[:alnum:]]" > + P="[-+./:=,%@_[~[:alnum:]]" > M="[0-9]+\|[0-9]+\|[0-9]+\|[0-9]+" > H="[0-9a-f]{64}" Sorry for late reply, quite busy this days. This "comma" makes the change in the line. I hope it would be enough to upgrade streight to 11.0. I assume there are steps to change etc files and compiler also. Whenever I upgarade, I have to solve nvidia driver version needed for it, and some other smaller tasks. On laptop, it goes like compiling x things for intel 2000. Not an easy job, and unnerving too. Best regards Zoran From owner-freebsd-stable@freebsd.org Mon Oct 17 15:43:33 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07A67C159F2 for ; Mon, 17 Oct 2016 15:43:33 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from mail.mimar.rs (mail1.mimar.rs [193.53.106.128]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B49F2C51 for ; Mon, 17 Oct 2016 15:43:32 +0000 (UTC) (envelope-from marko.cupac@mimar.rs) Received: from mail1.mimar.rs (localhost [127.0.1.128]) by mail.mimar.rs (Postfix) with ESMTP id BA65445BA209; Mon, 17 Oct 2016 17:43:28 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=mimar.rs; h= content-transfer-encoding:content-type:content-type:mime-version :x-mailer:organization:references:in-reply-to:message-id:subject :subject:from:from:date:date:received:received; s=mimar-0901; t= 1476719006; x=1478533407; bh=vex8hGg3QRe/l0cTBlM5mUwgw+37RDjc4Be R4zu3WJ8=; b=xTH/A/sU65mPq6CVAJ/hs4GlVwA/zZsCv9LoboTwsBOOqzoe2Jn XrZe+vhvyW1zbjIctMwMr8s4aTezD36zziHhbcX2y9wIicZgIE/3UPqrcU7Uf2fd qEPtzwOBlgeLrIm99y3cqq8aYESmmwhiz8lcjqnNmNSaMS2s8GuWEtP8= X-Virus-Scanned: amavisd-new at mimar.rs Received: from mail.mimar.rs ([127.0.1.128]) by mail1.mimar.rs (amavis.mimar.rs [127.0.1.128]) (amavisd-new, port 10026) with LMTP id EGS3StcANycd; Mon, 17 Oct 2016 17:43:26 +0200 (CEST) Received: from efreet.mimar.rs (93-87-131-33.dynamic.isp.telekom.rs [93.87.131.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: marko.cupac@mimar.rs) by mail.mimar.rs (Postfix) with ESMTPSA id D72F545BA10E; Mon, 17 Oct 2016 17:43:25 +0200 (CEST) Date: Mon, 17 Oct 2016 17:43:25 +0200 From: Marko =?UTF-8?B?Q3VwYcSH?= To: Oliver Peter Cc: freebsd-stable@freebsd.org Subject: Re: moving ezjail-based jails from 10.3 host to 11.0 host Message-ID: <20161017174325.49386b19@efreet.mimar.rs> In-Reply-To: <20161017135507.GA10396@mail.opdns.de> References: <20161017153708.411492e5@efreet.mimar.rs> <20161017135507.GA10396@mail.opdns.de> Organization: Mimar X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 15:43:33 -0000 On Mon, 17 Oct 2016 15:55:07 +0200 Oliver Peter wrote: > On Mon, Oct 17, 2016 at 03:37:08PM +0200, Marko Cupa=C4=87 wrote: > > I have 10.3 host which runs a dozen or so ezjail-based jails. I have > > installed another 11.0 host, and I'd like to move jails to it. =20 >=20 > I would switch them to iocage+zfs, ezjail is sooo 90s. :) > Have a look at the documentation: > http://iocage.readthedocs.io/en/latest/basic-use.html >=20 > All jail settings are stored within ZFS properties so an upcoming > migration would only need a zfs send | zfs receive. >=20 > > Can I just archive jails on 10.3, scp them to 11.0, and re-create > > them there by restoring from archive (-a switch)? =20 >=20 > Further I would recommend to use rsync -av instead of scp. Oliver, I do appreciate you took the time to respond to my question. However, when I asked how to move my ezjail-based jails, I meant exactly that. I did not ask which is the best jail management system (for my use case I'm completely fine with ezjail), or about the good and bad things about ZFS (I have hardware RAID controller which can't do JBOD on this server), or advantages of rsync over scp (doesn't make much difference for one-time transfer of single .tar.gz file over LAN). Now, to give answer to my own question: everything works fine, after - of course - reinstalling all the packages with `pkg-static upgrade -f'. Regards, --=20 Before enlightenment - chop wood, draw water. After enlightenment - chop wood, draw water. Marko Cupa=C4=87 https://www.mimar.rs/ From owner-freebsd-stable@freebsd.org Mon Oct 17 16:02:16 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D089C15EDA for ; Mon, 17 Oct 2016 16:02:16 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CB341C4D for ; Mon, 17 Oct 2016 16:02:15 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id BEE0030365D for ; Mon, 17 Oct 2016 11:02:12 -0500 (CDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: freebsd-stable@freebsd.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> From: Karl Denninger Message-ID: <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> Date: Mon, 17 Oct 2016 11:02:10 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms090805080502070508080105" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 16:02:16 -0000 This is a cryptographically signed message in MIME format. --------------ms090805080502070508080105 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Update from my test system: Setting vfs.zfs.vdev_trim_max_active to 10 (from default 64) does *not* stop the panics. Setting vfs.zfs.vdev.trim.enabled =3D 0 (which requires a reboot) DOES stop the panics. I am going to run a scrub on the pack, but I suspect the pack itself (now that I can actually mount it without the machine blowing up!) is fin= e. THIS (OBVIOUSLY) NEEDS ATTENTION! On 10/17/2016 09:17, Karl Denninger wrote: > This is a situation I've had happen before, and reported -- it appeared= > to be a kernel stack overflow, and it has gotten materially worse on > 11.0-STABLE. > > The issue occurs after some period of time (normally a week or so.) Th= e > system has a mirrored pair of large drives used for backup purposes to > which ZFS snapshots are written using a script that iterates over the > system. > > The panic /only /happens when the root filesystem is being sent, and it= > appears that the panic itself is being triggered by an I/O pattern on > the /backup /drive -- not the source drives. Zpool scrubs on the sourc= e > are clean; I am going to run one now on the backup, but in the past tha= t > has been clean as well. > > I now have a *repeatable* panic in that if I attempt a "zfs list -rt al= l > backup" on the backup volume I get the below panic. A "zfs list" > does*__*not panic the system. > > The operating theory previously (after digging through the passed > structures in the dump) was that the ZFS system was attempting to issue= > TRIMs on a device that can't do them before the ZFS system realizes thi= s > and stops asking (the backup volume is comprised of spinning rust) but > the appearance of the panic now on the volume when I simply do a "zfs > list -rt all backup" appears to negate that theory since no writes are > performed by that operation, and thus no TRIM calls should be issued. > > I can leave the backup volume in the state that causes this for a short= > period of time in an attempt to find and fix this. > > > NewFS.denninger.net dumped core - see /var/crash/vmcore.1 > > Mon Oct 17 09:02:33 CDT 2016 > > FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 > r307318M: Fri Oct 14 09:23:46 CDT 2016 =20 > karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 > > panic: double fault > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and yo= u are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for deta= ils. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > Fatal double fault > rip =3D 0xffffffff8220d9ec > rsp =3D 0xfffffe066821f000 > rbp =3D 0xfffffe066821f020 > cpuid =3D 6; apic id =3D 14 > panic: double fault > cpuid =3D 6 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe0649d78e30 > vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 > panic() at panic+0x43/frame 0xfffffe0649d78f10 > dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30 > Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 > --- trap 0x17, rip =3D 0xffffffff8220d9ec, rsp =3D 0xfffffe066821f000, = rbp =3D > 0xfffffe066821f020 --- > avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 > avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 > vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame > 0xfffffe066821f530 > vdev_queue_io_done() at vdev_queue_io_done+0x83/frame 0xfffffe066821f57= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f650= > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f6e= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f7c0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f85= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f930= > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f9c= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821faa0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fb3= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fc10= > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fca= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fd80= > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fe1= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fef0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821ff8= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220060= > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682200f= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682201d0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822026= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220340= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682203d= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682204b0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822054= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220620= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682206b= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220790= > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822082= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220900= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822099= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220a70= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220b0= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220be0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220c7= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220d50= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220de= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220ec0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220f5= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221030= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682210c= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682211a0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822123= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221310= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682213a= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221480= > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822151= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682215f0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822168= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221760= > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682217f= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682218d0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822196= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221a40= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221ad= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221bb0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221c4= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221d20= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221db= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221e90= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221f2= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222000= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822209= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222170= > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822220= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682222e0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822237= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222450= > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682224e= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682225c0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822265= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222730= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682227c= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682228a0= > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066822293= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222a10= > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222aa= 0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 > taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame > 0xfffffe0668222b80 > taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame > 0xfffffe0668222bb0 > fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 > --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- > Uptime: 4m29s > Dumping 1497 out of 24531 > MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/zfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from= > /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. > done. > Loaded symbols for /boot/kernel/opensolaris.ko > Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from > /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. > done. > Loaded symbols for /boot/kernel/geom_eli.ko > Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from= > /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. > done. > Loaded symbols for /boot/kernel/geom_mirror.ko > Reading symbols from /boot/kernel/aesni.ko...Reading symbols from > /usr/lib/debug//boot/kernel/aesni.ko.debug...done. > done. > Loaded symbols for /boot/kernel/aesni.ko > Reading symbols from /boot/kernel/uhid.ko...Reading symbols from > /usr/lib/debug//boot/kernel/uhid.ko.debug...done. > done. > Loaded symbols for /boot/kernel/uhid.ko > Reading symbols from /boot/kernel/ums.ko...Reading symbols from > /usr/lib/debug//boot/kernel/ums.ko.debug...done. > done. > Loaded symbols for /boot/kernel/ums.ko > Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from > /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. > done. > Loaded symbols for /boot/kernel/ipfw.ko > Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from > /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. > done. > Loaded symbols for /boot/kernel/ipfw_nat.ko > Reading symbols from /boot/kernel/libalias.ko...Reading symbols from > /usr/lib/debug//boot/kernel/libalias.ko.debug...done. > done. > Loaded symbols for /boot/kernel/libalias.ko > Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols fro= m > /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. > done. > Loaded symbols for /boot/kernel/daemon_saver.ko > #0 doadump (textdump=3D1) at pcpu.h:221 > 221 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) #0 doadump (textdump=3D1) at pcpu.h:221 > #1 0xffffffff80ae9b6e in kern_reboot (howto=3D260) > at /usr/src/sys/kern/kern_shutdown.c:366 > #2 0xffffffff80aea13b in vpanic (fmt=3D, > ap=3D) at /usr/src/sys/kern/kern_shutdown.c:75= 9 > #3 0xffffffff80ae9f73 in panic (fmt=3D0x0) > at /usr/src/sys/kern/kern_shutdown.c:690 > #4 0xffffffff80fd18f2 in dblfault_handler (frame=3D) > at /usr/src/sys/amd64/amd64/trap.c:868 > #5 0xffffffff80fb406c in Xdblfault () > at /usr/src/sys/amd64/amd64/exception.S:292 > #6 0xffffffff8220d9ec in avl_rotation (tree=3D0xfffff8004dc54510, > node=3D0xfffff8056e45cdd0, balance=3D2) > at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 > #7 0xffffffff8220de58 in avl_remove (tree=3D, > data=3D) > at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 > #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:43= 9 > #9 0xffffffff822a2123 in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:90= 7 > #10 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e3f33d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #11 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #12 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e3f33d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #13 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #14 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #15 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e578000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #16 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #17 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e578000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #18 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #19 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #20 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ec3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #21 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #22 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ec3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #23 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #24 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #25 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e48a000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #26 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #27 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e48a000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #28 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #29 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #30 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e57ab88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #31 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #32 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e57ab88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #33 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #34 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #35 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69a3000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #36 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #37 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69a3000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #38 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #39 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #40 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5c8000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #41 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #42 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5c8000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #43 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #44 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #45 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23f9000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #46 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #47 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23f9000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #48 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #49 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #50 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5147b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #51 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #52 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5147b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #53 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #54 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #55 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e977000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #56 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #57 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e977000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #58 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #59 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #60 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90b3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #61 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #62 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90b3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #63 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #64 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #65 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b1160b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #66 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #67 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b1160b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #68 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #69 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #70 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4f53d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #71 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #72 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4f53d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #73 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #74 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #75 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8ebb88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #76 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #77 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8ebb88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #78 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #79 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #80 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8d9b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #81 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #82 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8d9b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #83 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #84 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #85 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21a03d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #86 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #87 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21a03d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #88 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #89 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #90 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e6903d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #91 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #92 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e6903d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #93 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #94 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #95 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a22dc7b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #96 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #97 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a22dc7b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #98 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #99 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #100 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5fb7b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #101 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #102 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5fb7b0)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #103 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #104 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #105 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ae3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #106 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #107 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ae3d8)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #108 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #109 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #110 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a2131b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #111 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #112 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a2131b88)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #113 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #114 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #115 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e80b7b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #116 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #117 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e80b7b0)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #118 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #119 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #120 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b14b9000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #121 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #122 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b14b9000)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #123 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #124 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #125 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21fa3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #126 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #127 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21fa3d8)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #128 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #129 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #130 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5333d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #131 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #132 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5333d8)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #133 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #134 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #135 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90ab88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #136 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #137 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90ab88)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #138 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #139 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #140 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4047b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #141 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #142 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4047b0)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #143 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #144 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #145 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e7083d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #146 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #147 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e7083d8)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #148 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #149 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #150 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23917b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #151 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #152 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23917b0)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #153 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #154 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #155 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e636000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #156 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #157 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e636000)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #158 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #159 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #160 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5afb88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #161 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #162 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5afb88)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #163 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #164 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #165 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69523d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #166 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #167 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69523d8)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #168 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #169 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #170 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e47c000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #171 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #172 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e47c000)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #173 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #174 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #175 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b17b23d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #176 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #177 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b17b23d8)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #178 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #179 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #180 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d367b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #181 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #182 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a6d367b0)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #183 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #184 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #185 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e99fb88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #186 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #187 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e99fb88)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #188 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #189 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #190 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5227b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #191 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #192 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5227b0)= > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 12 > #193 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #194 0xffffffff822a216d in vdev_queue_io_done (zio=3D) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:91= 3 > #195 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d3c000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:31= 37 > #196 0xffffffff822bbefd in zio_execute (zio=3D) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:16= 51 > #197 0xffffffff80b4895a in taskqueue_run_locked (queue=3D out>) > at /usr/src/sys/kern/subr_taskqueue.c:454 > #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=3D) > at /usr/src/sys/kern/subr_taskqueue.c:724 > #199 0xffffffff80a9f255 in fork_exit ( > callout=3D0xffffffff80b49a70 , > arg=3D0xfffff8057caa72a0, frame=3D0xfffffe0668222c00) > at /usr/src/sys/kern/kern_fork.c:1040 > #200 0xffffffff80fb44ae in fork_trampoline () > at /usr/src/sys/amd64/amd64/exception.S:611 > #201 0x0000000000000000 in ?? () > Current language: auto; currently minimal > (kgdb) > > --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms090805080502070508080105 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEwMTcxNjAyMTBaME8GCSqGSIb3DQEJBDFCBEBu 1dvvxNl8w5wVJn1OQk8dVZRZxlDaPQqkSrFaNICZZ5K3d4EE8sxzrzqWdSAuGFajua9O56wF VW5mRGK0uYa9MGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAJ/Ga2khE yO1WAOja0vqp/gyZ0faxzQwp4J6JMxp3N8cHwyXd8F906ZsoqAqxJPWh2pS0vapSWmO7Mvtf zQ7eNwRr5H1LboId046HHZaGGjptP7J7LPcP9PM35f/xobk0PzWTOi25+6/DseTNFmlwg8rX M7j2vZNvPZGHAT3HwCIHIht9Zook7tOobDGgmvjwEHaTE0eHFB5wHHaZ0czT/3fKMXV7x3gd m2+rMWteNzd2xIaaXksGZiqSTlHtoK7I7bUMfJpYx3cdOf9+Vop+4CKEWpTwuPBz596Jmi8N 4LsGAtJXznbC65FzaIiZ5bA+P6fUroaG+n8qoY6raE9hHUqj+Blc2yz41m2p3D7O4V7drbbb AEb/KH3cpwpjWXoLMpzqOHpoJSqEp8+n1eYKIzAlJ1vEMfkWp3arwrt/b5hxIP5oVfVzcRON Rm6J8LzpNzybHNvcPgz90VKWpekQOG5Js2GSWdfBbDNmWUjpS7c9jobyK7Uq1Epc2oaJopKd 88DcHzrQ23e4d56Tq1gO8MGutXayayavAUyOdcPfJRgIdjtwhMn+q1WPjmCaum2PctxFs9TO Hf5Y1IyWshCntX5ELxdnZiHomJrHrifjk/Cab+P+megCQYJTtmbfhayv6Keui+HCMTqur7W5 cV0XF7ya7rvz8uUTZMPMI0OL1NwAAAAAAAA= --------------ms090805080502070508080105-- From owner-freebsd-stable@freebsd.org Mon Oct 17 16:14:40 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CDB20C15224 for ; Mon, 17 Oct 2016 16:14:40 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9109726C for ; Mon, 17 Oct 2016 16:14:40 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 95BB428453; Mon, 17 Oct 2016 18:14:32 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id D985B28431; Mon, 17 Oct 2016 18:14:31 +0200 (CEST) Subject: Re: moving ezjail-based jails from 10.3 host to 11.0 host To: =?UTF-8?Q?Marko_Cupa=c4=87?= , Oliver Peter Cc: freebsd-stable@freebsd.org References: <20161017153708.411492e5@efreet.mimar.rs> <20161017135507.GA10396@mail.opdns.de> <20161017174325.49386b19@efreet.mimar.rs> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <5804F8E7.6020505@quip.cz> Date: Mon, 17 Oct 2016 18:14:31 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 In-Reply-To: <20161017174325.49386b19@efreet.mimar.rs> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 16:14:40 -0000 Marko Cupać wrote on 2016/10/17 17:43: [...] > Now, to give answer to my own question: everything works fine, after - > of course - reinstalling all the packages with `pkg-static upgrade -f'. Reinstalling all packages? So you didn't just move jails from one host to another but you upgraded them to 11.0 also? Because I had feeling that you want just to move them to another machine. And if you just move them (completely with shared basejail) you don't need to reinstall packages. As Oliver said you can use rsync to transfer them without need to compress + scp + uncompress. It is much faster transfer than making archives. Miroslav Lachman From owner-freebsd-stable@freebsd.org Mon Oct 17 16:52:35 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6AEC7C15ECE for ; Mon, 17 Oct 2016 16:52:35 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 5CEDFD55; Mon, 17 Oct 2016 16:52:35 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 8B83DC9; Mon, 17 Oct 2016 16:52:35 +0000 (UTC) Date: Mon, 17 Oct 2016 16:52:35 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <1714359220.1.1476723155509.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <87913082.0.1476709246031.JavaMail.jenkins@jenkins-9.freebsd.org> References: <87913082.0.1476709246031.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is unstable: FreeBSD_stable_10 #427 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 16:52:35 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/427/ From owner-freebsd-stable@freebsd.org Mon Oct 17 17:30:46 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F36F4C15B6A for ; Mon, 17 Oct 2016 17:30:46 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from mail1.yamagi.org (yugo.yamagi.org [212.48.122.103]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BCD00974 for ; Mon, 17 Oct 2016 17:30:45 +0000 (UTC) (envelope-from lists@yamagi.org) Received: from p5b0be342.dip0.t-ipconnect.de ([91.11.227.66] helo=pine.home.yamagi.org) by mail1.yamagi.org with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bwB9B-000ChX-Aw for freebsd-stable@freebsd.org; Mon, 17 Oct 2016 18:52:25 +0200 Date: Mon, 17 Oct 2016 18:52:15 +0200 From: Yamagi Burmeister To: freebsd-stable@freebsd.org Subject: Re: I'm upset about FreeBSD Message-Id: <20161017185215.97528d6cf6377058e4760785@yamagi.org> In-Reply-To: References: X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 17:30:47 -0000 On Mon, 17 Oct 2016 03:44:14 +0300 Rostislav Krasny wrote: > First of all I faced an old problem that I reported here a year ago: > http://comments.gmane.org/gmane.os.freebsd.stable/96598 > Completely new USB flash drive flashed by the > FreeBSD-11.0-RELEASE-i386-mini-memstick.img file kills every Windows > again. If I use the Rufus util to write the img file (using DD mode) > the Windows dies immediately after the flashing. If I use the > Win32DiskImager (suggested by the Handbook) it doesn't reinitialize > the USB storage and Windows dies only if I remove and put that USB > flash drive again or boot Windows when it is connected. Nothing was > done to fix this nasty bug for a year. As was already said in the other answers this is a bug in Windows. Particulary in the partition parser. partmgr.sys (running in kernel mode) crashes while parsing the FreeBSD installation images GPT setup. This may be a variant of the bug known as "Kindle is crashing Win 10": http://answers.microsoft.com/en-us/windows/forum/windows_10-performance/plugging-in-kindle-is-crashing-windows-10-after/5db0d867-0822-4512-919e-3d7786353f95?page=1 That bug was patched on september 13 and I'm unable to reproduce the crash on a fully patched Win 10 VM. But there's no patch for Win 7, even with all patches applied my Win 7 VM is still crashing as soon as the FreeBSD installation image is connected. I did some debugging and I'm pretty sure that the problem is not the pmbr used for classic BIOS boot but the GPT itself. But my knowledge of GPT and especially Windows internals is limit. So maybe someone with more insight can look into this. Or even better: Complain to Microsoft. Even if the GPT is invalid it should crash the kernel. Regards, Yamagi -- Homepage: www.yamagi.org XMPP: yamagi@yamagi.org GnuPG/GPG: 0xEFBCCBCB From owner-freebsd-stable@freebsd.org Mon Oct 17 17:43:50 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A280C15F2E for ; Mon, 17 Oct 2016 17:43:50 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x232.google.com (mail-it0-x232.google.com [IPv6:2607:f8b0:4001:c0b::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E059418F for ; Mon, 17 Oct 2016 17:43:49 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x232.google.com with SMTP id k64so14614608itb.0 for ; Mon, 17 Oct 2016 10:43:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=9gmnl5HQwpXRVzcYvHRAngHUnGcFwXS1LRHxp0KQ6og=; b=USsxevC49T28lqAH973ryEr72ngynburfKWjCxS2e/DtFju9PNCE89V1XCkDEGem7Y SgzIU1OYI1OUpirvYt7PKB8jDmUgvLRunGZIyC9YCh0Ad7BZ7eDnofg39Y7+Bby5/ZK2 91ECZljC9ODH/wbLK9woMSGPfbCLG4Myw6FOpATz7WLMGlKvVYCvL3g3KQlXIFboDYZK IZ0MUbehmf3AXv+UN4D6aJrmMCS8mXRMHvbHXuu0C2QfvT5jXCPUn2hdWDtU26gJ0kxn o8r5gEe0Ff/a6WqOiHtBk4sgz65Ib7q1Kdmw2q47XUESd44tiiQFY77fib2doVYTrmG3 Holw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=9gmnl5HQwpXRVzcYvHRAngHUnGcFwXS1LRHxp0KQ6og=; b=aIewr8FNnaRgvQsyy8Gp6KMxy90+mhz7p36DvvII4PdAtGuuRFe7AKIsUcdKSK6Ir0 a+GGzH8PLnICrMU8B2NF3jtd3tYFZcEO+1jvKd+46XMLdPtD4ajtwwnY7/QAyS7MZHCp Je4SZd0fvaC3rslwTzuSU8yZ894THfIK0UHGk7PGH7GcWLJUccksAh73mC8bb0cfGlFm DcnLQrQReuwM48oK4Mj3v2ZYo03ixwWDiAToOuOhH0LeQPKccqaRZi5Fy7nrNmVRsnfa 4uSpDhlOCwaF6xVxap3ectnYTGwp1ETxZ2e+deRQIS6BaAlx7YDkZwlqQJuu1X4Tw5UR /B4w== X-Gm-Message-State: AA6/9RklqbW3IGohoGGMxUc71Kql3F5PhhYKjhTWbITW0IuqZz8waE9Cu1Cy4yob9U8l6MrszRN5L+a+1PIvoA== X-Received: by 10.36.43.82 with SMTP id h79mr10847611ita.60.1476726228732; Mon, 17 Oct 2016 10:43:48 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.77.85 with HTTP; Mon, 17 Oct 2016 10:43:48 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> From: Warner Losh Date: Mon, 17 Oct 2016 11:43:48 -0600 X-Google-Sender-Auth: 0ucEVX2eL2ZuQP1F2D-4A9oqjSU Message-ID: Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: Karl Denninger Cc: FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 17:43:50 -0000 what's your underlying media? Warner On Mon, Oct 17, 2016 at 10:02 AM, Karl Denninger wrote: > Update from my test system: > > Setting vfs.zfs.vdev_trim_max_active to 10 (from default 64) does *not* > stop the panics. > > Setting vfs.zfs.vdev.trim.enabled = 0 (which requires a reboot) DOES > stop the panics. > > I am going to run a scrub on the pack, but I suspect the pack itself > (now that I can actually mount it without the machine blowing up!) is fine. > > THIS (OBVIOUSLY) NEEDS ATTENTION! > > On 10/17/2016 09:17, Karl Denninger wrote: >> This is a situation I've had happen before, and reported -- it appeared >> to be a kernel stack overflow, and it has gotten materially worse on >> 11.0-STABLE. >> >> The issue occurs after some period of time (normally a week or so.) The >> system has a mirrored pair of large drives used for backup purposes to >> which ZFS snapshots are written using a script that iterates over the >> system. >> >> The panic /only /happens when the root filesystem is being sent, and it >> appears that the panic itself is being triggered by an I/O pattern on >> the /backup /drive -- not the source drives. Zpool scrubs on the source >> are clean; I am going to run one now on the backup, but in the past that >> has been clean as well. >> >> I now have a *repeatable* panic in that if I attempt a "zfs list -rt all >> backup" on the backup volume I get the below panic. A "zfs list" >> does*__*not panic the system. >> >> The operating theory previously (after digging through the passed >> structures in the dump) was that the ZFS system was attempting to issue >> TRIMs on a device that can't do them before the ZFS system realizes this >> and stops asking (the backup volume is comprised of spinning rust) but >> the appearance of the panic now on the volume when I simply do a "zfs >> list -rt all backup" appears to negate that theory since no writes are >> performed by that operation, and thus no TRIM calls should be issued. >> >> I can leave the backup volume in the state that causes this for a short >> period of time in an attempt to find and fix this. >> >> >> NewFS.denninger.net dumped core - see /var/crash/vmcore.1 >> >> Mon Oct 17 09:02:33 CDT 2016 >> >> FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 >> r307318M: Fri Oct 14 09:23:46 CDT 2016 >> karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 >> >> panic: double fault >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> >> Fatal double fault >> rip = 0xffffffff8220d9ec >> rsp = 0xfffffe066821f000 >> rbp = 0xfffffe066821f020 >> cpuid = 6; apic id = 14 >> panic: double fault >> cpuid = 6 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >> 0xfffffe0649d78e30 >> vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 >> panic() at panic+0x43/frame 0xfffffe0649d78f10 >> dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30 >> Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 >> --- trap 0x17, rip = 0xffffffff8220d9ec, rsp = 0xfffffe066821f000, rbp = >> 0xfffffe066821f020 --- >> avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 >> avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 >> vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame >> 0xfffffe066821f530 >> vdev_queue_io_done() at vdev_queue_io_done+0x83/frame 0xfffffe066821f570 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f650 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f6e0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f7c0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f850 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f930 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f9c0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821faa0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fb30 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fc10 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fca0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fd80 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fe10 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fef0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821ff80 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220060 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682200f0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682201d0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220260 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220340 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682203d0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682204b0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220540 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220620 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682206b0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220790 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220820 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220900 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220990 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220a70 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220b00 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220be0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220c70 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220d50 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220de0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220ec0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220f50 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221030 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682210c0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682211a0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221230 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221310 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682213a0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221480 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221510 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682215f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221680 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221760 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682217f0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682218d0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221960 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221a40 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221ad0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221bb0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221c40 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221d20 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221db0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221e90 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221f20 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222000 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222090 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222170 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222200 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682222e0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222370 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222450 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682224e0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682225c0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222650 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222730 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682227c0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682228a0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222930 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222a10 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222aa0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 >> taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame >> 0xfffffe0668222b80 >> taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame >> 0xfffffe0668222bb0 >> fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 >> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >> Uptime: 4m29s >> Dumping 1497 out of 24531 >> MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% >> >> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/zfs.ko >> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/opensolaris.ko >> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/geom_eli.ko >> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/geom_mirror.ko >> Reading symbols from /boot/kernel/aesni.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/aesni.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/aesni.ko >> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/uhid.ko >> Reading symbols from /boot/kernel/ums.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/ums.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/ums.ko >> Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/ipfw.ko >> Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/ipfw_nat.ko >> Reading symbols from /boot/kernel/libalias.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/libalias.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/libalias.ko >> Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/daemon_saver.ko >> #0 doadump (textdump=1) at pcpu.h:221 >> 221 pcpu.h: No such file or directory. >> in pcpu.h >> (kgdb) #0 doadump (textdump=1) at pcpu.h:221 >> #1 0xffffffff80ae9b6e in kern_reboot (howto=260) >> at /usr/src/sys/kern/kern_shutdown.c:366 >> #2 0xffffffff80aea13b in vpanic (fmt=, >> ap=) at /usr/src/sys/kern/kern_shutdown.c:759 >> #3 0xffffffff80ae9f73 in panic (fmt=0x0) >> at /usr/src/sys/kern/kern_shutdown.c:690 >> #4 0xffffffff80fd18f2 in dblfault_handler (frame=) >> at /usr/src/sys/amd64/amd64/trap.c:868 >> #5 0xffffffff80fb406c in Xdblfault () >> at /usr/src/sys/amd64/amd64/exception.S:292 >> #6 0xffffffff8220d9ec in avl_rotation (tree=0xfffff8004dc54510, >> node=0xfffff8056e45cdd0, balance=2) >> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 >> #7 0xffffffff8220de58 in avl_remove (tree=, >> data=) >> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 >> #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:439 >> #9 0xffffffff822a2123 in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:907 >> #10 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e3f33d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #11 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #12 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e3f33d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #13 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #14 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #15 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e578000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #16 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #17 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e578000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #18 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #19 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #20 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ec3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #21 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #22 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ec3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #23 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #24 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #25 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e48a000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #26 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #27 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e48a000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #28 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #29 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #30 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e57ab88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #31 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #32 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e57ab88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #33 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #34 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #35 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69a3000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #36 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #37 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69a3000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #38 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #39 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #40 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5c8000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #41 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #42 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5c8000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #43 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #44 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #45 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23f9000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #46 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #47 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23f9000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #48 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #49 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #50 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5147b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #51 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #52 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5147b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #53 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #54 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #55 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e977000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #56 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #57 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e977000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #58 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #59 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #60 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90b3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #61 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #62 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90b3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #63 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #64 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #65 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b1160b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #66 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #67 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b1160b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #68 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #69 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #70 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4f53d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #71 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #72 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4f53d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #73 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #74 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #75 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8ebb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #76 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #77 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8ebb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #78 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #79 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #80 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8d9b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #81 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #82 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8d9b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #83 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #84 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #85 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21a03d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #86 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #87 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21a03d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #88 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #89 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #90 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e6903d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #91 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #92 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e6903d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #93 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #94 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #95 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a22dc7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #96 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #97 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a22dc7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #98 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #99 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #100 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5fb7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #101 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #102 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5fb7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #103 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #104 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #105 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ae3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #106 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #107 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ae3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #108 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #109 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #110 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a2131b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #111 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #112 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a2131b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #113 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #114 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #115 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e80b7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #116 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #117 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e80b7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #118 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #119 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #120 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b14b9000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #121 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #122 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b14b9000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #123 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #124 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #125 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21fa3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #126 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #127 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21fa3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #128 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #129 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #130 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5333d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #131 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #132 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5333d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #133 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #134 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #135 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90ab88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #136 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #137 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90ab88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #138 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #139 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #140 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4047b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #141 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #142 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4047b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #143 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #144 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #145 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e7083d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #146 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #147 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e7083d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #148 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #149 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #150 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23917b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #151 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #152 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23917b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #153 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #154 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #155 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e636000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #156 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #157 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e636000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #158 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #159 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #160 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5afb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #161 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #162 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5afb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #163 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #164 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #165 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69523d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #166 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #167 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69523d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #168 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #169 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #170 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e47c000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #171 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #172 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e47c000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #173 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #174 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #175 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b17b23d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #176 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #177 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b17b23d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #178 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #179 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #180 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d367b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #181 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #182 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a6d367b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #183 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #184 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #185 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e99fb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #186 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #187 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e99fb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #188 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #189 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #190 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5227b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #191 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #192 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5227b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #193 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #194 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #195 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d3c000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #196 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #197 0xffffffff80b4895a in taskqueue_run_locked (queue=> out>) >> at /usr/src/sys/kern/subr_taskqueue.c:454 >> #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=) >> at /usr/src/sys/kern/subr_taskqueue.c:724 >> #199 0xffffffff80a9f255 in fork_exit ( >> callout=0xffffffff80b49a70 , >> arg=0xfffff8057caa72a0, frame=0xfffffe0668222c00) >> at /usr/src/sys/kern/kern_fork.c:1040 >> #200 0xffffffff80fb44ae in fork_trampoline () >> at /usr/src/sys/amd64/amd64/exception.S:611 >> #201 0x0000000000000000 in ?? () >> Current language: auto; currently minimal >> (kgdb) >> >> > > -- > Karl Denninger > karl@denninger.net > /The Market Ticker/ > /[S/MIME encrypted email preferred]/ From owner-freebsd-stable@freebsd.org Mon Oct 17 18:08:45 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62C0AC15C66 for ; Mon, 17 Oct 2016 18:08:45 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E95E58FE for ; Mon, 17 Oct 2016 18:08:44 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 014FD303B66 for ; Mon, 17 Oct 2016 13:08:42 -0500 (CDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> To: FreeBSD-STABLE Mailing List From: Karl Denninger Message-ID: <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> Date: Mon, 17 Oct 2016 13:08:40 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms090900030904000100020607" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 18:08:45 -0000 This is a cryptographically signed message in MIME format. --------------ms090900030904000100020607 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable The target (and devices that trigger this) are a pair of 4Gb 7200RPM SATA rotating rust drives (zmirror) with each provider geli-encrypted (that is, the actual devices used for the pool create are the .eli's) The machine generating the problem has both rotating rust devices *and* SSDs, so I can't simply shut TRIM off system-wide and call it a day as TRIM itself is heavily-used; both the boot/root pools and a Postgresql database pool are on SSDs, while several terabytes of lesser-used data is on a pool of Raidz2 that is made up of spinning rust. vfs.zfs.trim.max_interval: 1 vfs.zfs.trim.timeout: 30 vfs.zfs.trim.txg_delay: 32 vfs.zfs.trim.enabled: 1 vfs.zfs.vdev.trim_max_pending: 10000 vfs.zfs.vdev.trim_max_active: 64 vfs.zfs.vdev.trim_min_active: 1 vfs.zfs.vdev.trim_on_init: 1 kstat.zfs.misc.zio_trim.failed: 0 kstat.zfs.misc.zio_trim.unsupported: 1080 kstat.zfs.misc.zio_trim.success: 573768 kstat.zfs.misc.zio_trim.bytes: 28964282368 The machine in question has been up for ~3 hours now since the last panic, so obviously TRIM is being heavily used... The issue, once the problem has been created, is *portable* and it is not being caused by the SSD source drives. That is, once the machine panics if I remove the two disks that form the backup pool, physically move them to my sandbox machine, geli attach the drives and import the pool within seconds the second machine will panic in the identical fashion. It's possible (but have not proved) that if I were to reboot enough times the filesystem would eventually reach consistency with the removed snapshots all gone and the panics would stop, but I got a half-dozen of them sequentially this morning on my test machine so I'm not at all sure how many more I'd need to allow to run, or whether *any* of the removals committed before the panic (if not then the cycle of reboot/attach/panic would never end....) :-) Reducing trim_max_active (to 10, a quite-drastic reduction) did not stop the panics. What appears to be happening is that the removal of the datasets in question on a reasonably newly-imported pool, whether it occurs by the incremental zfs recv -Fudv or by zfs destroy -r from the command line, generates a large number of TRIM requests which are of course rejected by the providers as spinning rust does not support them. However the attempt to queue them generates a stack overflow and double-fault panic as a result, and since once the command is issued the filesystem now has the deletes pending and the consistent state is in fact with them gone, any attempt to reattach the drives with TRIM enabled can result in an immediate additional panic. I tried to work around this in my backup script by creating and then destroying a file on the backup volume, then sleeping for a few seconds before the backup actually commenced, in the hope that this would (1) trigger a TRIM attempt and (2) lead the system to recognize that the target volume cannot support TRIM and thus stop trying to do so (and thus not lead to the storm that exhausts the stack and panic.) That approach, however (see below), failed to prevent the problem. # # Now try to trigger TRIM so that we don't have a storm of them # echo "Attempting to disable TRIM on spinning rust...." mount -t zfs backup/no-trim /mnt dd if=3D/dev/random of=3D/mnt/kill-trim bs=3D128k count=3D2 echo "Performed 2 writes" sleep 2 rm /mnt/kill-trim echo "Performed delete of written file" sleep 5 umount /mnt echo "Unmounted temporary filesystem" sleep 2 echo "TRIM disable theoretically done...." On 10/17/2016 12:43, Warner Losh wrote: > what's your underlying media? > > Warner > > > On Mon, Oct 17, 2016 at 10:02 AM, Karl Denninger w= rote: >> Update from my test system: >> >> Setting vfs.zfs.vdev_trim_max_active to 10 (from default 64) does *not= * >> stop the panics. >> >> Setting vfs.zfs.vdev.trim.enabled =3D 0 (which requires a reboot) DOES= >> stop the panics. >> >> I am going to run a scrub on the pack, but I suspect the pack itself >> (now that I can actually mount it without the machine blowing up!) is = fine. >> >> THIS (OBVIOUSLY) NEEDS ATTENTION! >> >> On 10/17/2016 09:17, Karl Denninger wrote: >>> This is a situation I've had happen before, and reported -- it appear= ed >>> to be a kernel stack overflow, and it has gotten materially worse on >>> 11.0-STABLE. >>> >>> The issue occurs after some period of time (normally a week or so.) = The >>> system has a mirrored pair of large drives used for backup purposes t= o >>> which ZFS snapshots are written using a script that iterates over the= >>> system. >>> >>> The panic /only /happens when the root filesystem is being sent, and = it >>> appears that the panic itself is being triggered by an I/O pattern on= >>> the /backup /drive -- not the source drives. Zpool scrubs on the sou= rce >>> are clean; I am going to run one now on the backup, but in the past t= hat >>> has been clean as well. >>> >>> I now have a *repeatable* panic in that if I attempt a "zfs list -rt = all >>> backup" on the backup volume I get the below panic. A "zfs list" >>> does*__*not panic the system. >>> >>> The operating theory previously (after digging through the passed >>> structures in the dump) was that the ZFS system was attempting to iss= ue >>> TRIMs on a device that can't do them before the ZFS system realizes t= his >>> and stops asking (the backup volume is comprised of spinning rust) bu= t >>> the appearance of the panic now on the volume when I simply do a "zfs= >>> list -rt all backup" appears to negate that theory since no writes ar= e >>> performed by that operation, and thus no TRIM calls should be issued.= >>> >>> I can leave the backup volume in the state that causes this for a sho= rt >>> period of time in an attempt to find and fix this. >>> >>> >>> NewFS.denninger.net dumped core - see /var/crash/vmcore.1 >>> >>> Mon Oct 17 09:02:33 CDT 2016 >>> >>> FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 >>> r307318M: Fri Oct 14 09:23:46 CDT 2016 >>> karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 >>> >>> panic: double fault >>> >>> GNU gdb 6.1.1 [FreeBSD] >>> Copyright 2004 Free Software Foundation, Inc. >>> GDB is free software, covered by the GNU General Public License, and = you are >>> welcome to change it and/or distribute copies of it under certain >>> conditions. >>> Type "show copying" to see the conditions. >>> There is absolutely no warranty for GDB. Type "show warranty" for de= tails. >>> This GDB was configured as "amd64-marcel-freebsd"... >>> >>> Unread portion of the kernel message buffer: >>> >>> Fatal double fault >>> rip =3D 0xffffffff8220d9ec >>> rsp =3D 0xfffffe066821f000 >>> rbp =3D 0xfffffe066821f020 >>> cpuid =3D 6; apic id =3D 14 >>> panic: double fault >>> cpuid =3D 6 >>> KDB: stack backtrace: >>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >>> 0xfffffe0649d78e30 >>> vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 >>> panic() at panic+0x43/frame 0xfffffe0649d78f10 >>> dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30 >>> Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 >>> --- trap 0x17, rip =3D 0xffffffff8220d9ec, rsp =3D 0xfffffe066821f000= , rbp =3D >>> 0xfffffe066821f020 --- >>> avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 >>> avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 >>> vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame >>> 0xfffffe066821f530 >>> vdev_queue_io_done() at vdev_queue_io_done+0x83/frame 0xfffffe066821f= 570 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f6= 50 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f= 6e0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f7= c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f= 850 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f9= 30 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f= 9c0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fa= a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f= b30 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fc= 10 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f= ca0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fd= 80 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f= e10 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fe= f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f= f80 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682200= 60 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= 0f0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682201= d0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= 260 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682203= 40 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= 3d0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682204= b0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= 540 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682206= 20 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= 6b0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682207= 90 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= 820 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682209= 00 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= 990 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220a= 70 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= b00 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220b= e0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= c70 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220d= 50 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= de0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220e= c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220= f50 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682210= 30 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= 0c0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682211= a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= 230 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682213= 10 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= 3a0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682214= 80 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= 510 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682215= f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= 680 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682217= 60 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= 7f0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682218= d0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= 960 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221a= 40 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= ad0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221b= b0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= c40 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221d= 20 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= db0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221e= 90 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221= f20 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682220= 00 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222= 090 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682221= 70 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222= 200 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682222= e0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222= 370 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682224= 50 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222= 4e0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682225= c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222= 650 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682227= 30 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222= 7c0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682228= a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222= 930 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222a= 10 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222= aa0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 >>> taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame >>> 0xfffffe0668222b80 >>> taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame >>> 0xfffffe0668222bb0 >>> fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 >>> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >>> Uptime: 4m29s >>> Dumping 1497 out of 24531 >>> MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% >>> >>> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/zfs.ko >>> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols fr= om >>> /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/opensolaris.ko >>> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/geom_eli.ko >>> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols fr= om >>> /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/geom_mirror.ko >>> Reading symbols from /boot/kernel/aesni.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/aesni.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/aesni.ko >>> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/uhid.ko >>> Reading symbols from /boot/kernel/ums.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/ums.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/ums.ko >>> Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/ipfw.ko >>> Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/ipfw_nat.ko >>> Reading symbols from /boot/kernel/libalias.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/libalias.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/libalias.ko >>> Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols f= rom >>> /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/daemon_saver.ko >>> #0 doadump (textdump=3D1) at pcpu.h:221 >>> 221 pcpu.h: No such file or directory. >>> in pcpu.h >>> (kgdb) #0 doadump (textdump=3D1) at pcpu.h:221 >>> #1 0xffffffff80ae9b6e in kern_reboot (howto=3D260) >>> at /usr/src/sys/kern/kern_shutdown.c:366 >>> #2 0xffffffff80aea13b in vpanic (fmt=3D, >>> ap=3D) at /usr/src/sys/kern/kern_shutdown.c:= 759 >>> #3 0xffffffff80ae9f73 in panic (fmt=3D0x0) >>> at /usr/src/sys/kern/kern_shutdown.c:690 >>> #4 0xffffffff80fd18f2 in dblfault_handler (frame=3D) >>> at /usr/src/sys/amd64/amd64/trap.c:868 >>> #5 0xffffffff80fb406c in Xdblfault () >>> at /usr/src/sys/amd64/amd64/exception.S:292 >>> #6 0xffffffff8220d9ec in avl_rotation (tree=3D0xfffff8004dc54510, >>> node=3D0xfffff8056e45cdd0, balance=3D2) >>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 >>> #7 0xffffffff8220de58 in avl_remove (tree=3D, >>> data=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 >>> #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 439 >>> #9 0xffffffff822a2123 in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 907 >>> #10 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e3f33d8)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #11 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #12 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e3f33d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #13 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #14 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #15 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e578000)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #16 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #17 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e578000= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #18 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #19 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #20 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ec3d8)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #21 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #22 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ec3d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #23 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #24 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #25 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e48a000)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #26 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #27 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e48a000= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #28 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #29 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #30 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e57ab88)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #31 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #32 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e57ab88= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #33 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #34 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #35 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69a3000)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #36 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #37 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69a3000= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #38 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #39 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #40 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5c8000)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #41 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #42 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5c8000= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #43 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #44 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #45 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23f9000)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #46 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #47 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23f9000= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #48 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #49 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #50 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5147b0)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #51 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #52 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5147b0= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #53 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #54 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #55 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e977000)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #56 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #57 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e977000= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #58 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #59 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #60 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90b3d8)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #61 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #62 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90b3d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #63 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #64 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #65 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b1160b88)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #66 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #67 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b1160b88= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #68 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #69 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #70 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4f53d8)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #71 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #72 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4f53d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #73 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #74 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #75 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8ebb88)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #76 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #77 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8ebb88= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #78 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #79 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #80 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8d9b88)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #81 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #82 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8d9b88= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #83 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #84 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #85 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21a03d8)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #86 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #87 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21a03d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #88 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #89 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #90 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e6903d8)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #91 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #92 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e6903d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #93 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #94 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #95 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a22dc7b0)= >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #96 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #97 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a22dc7b0= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #98 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #99 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #100 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5fb7b0= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #101 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #102 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5fb7b= 0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #103 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #104 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #105 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ae3d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #106 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #107 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ae3d= 8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #108 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #109 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #110 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a2131b88= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #111 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #112 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a2131b8= 8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #113 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #114 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #115 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e80b7b0= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #116 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #117 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e80b7b= 0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #118 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #119 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #120 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b14b9000= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #121 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #122 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b14b900= 0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #123 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #124 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #125 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21fa3d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #126 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #127 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21fa3d= 8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #128 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #129 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #130 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5333d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #131 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #132 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5333d= 8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #133 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #134 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #135 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90ab88= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #136 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #137 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90ab8= 8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #138 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #139 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #140 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4047b0= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #141 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #142 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4047b= 0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #143 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #144 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #145 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e7083d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #146 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #147 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e7083d= 8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #148 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #149 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #150 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23917b0= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #151 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #152 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23917b= 0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #153 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #154 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #155 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e636000= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #156 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #157 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e63600= 0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #158 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #159 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #160 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5afb88= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #161 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #162 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5afb8= 8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #163 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #164 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #165 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69523d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #166 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #167 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69523d= 8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #168 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #169 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #170 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e47c000= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #171 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #172 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e47c00= 0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #173 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #174 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #175 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b17b23d8= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #176 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #177 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b17b23d= 8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #178 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #179 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #180 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d367b0= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #181 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #182 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a6d367b= 0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #183 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #184 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #185 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e99fb88= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #186 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #187 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e99fb8= 8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #188 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #189 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #190 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5227b0= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #191 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #192 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5227b= 0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3112 >>> #193 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #194 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:= 913 >>> #195 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d3c000= ) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 3137 >>> #196 0xffffffff822bbefd in zio_execute (zio=3D) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:= 1651 >>> #197 0xffffffff80b4895a in taskqueue_run_locked (queue=3D>> out>) >>> at /usr/src/sys/kern/subr_taskqueue.c:454 >>> #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=3D) >>> at /usr/src/sys/kern/subr_taskqueue.c:724 >>> #199 0xffffffff80a9f255 in fork_exit ( >>> callout=3D0xffffffff80b49a70 , >>> arg=3D0xfffff8057caa72a0, frame=3D0xfffffe0668222c00) >>> at /usr/src/sys/kern/kern_fork.c:1040 >>> #200 0xffffffff80fb44ae in fork_trampoline () >>> at /usr/src/sys/amd64/amd64/exception.S:611 >>> #201 0x0000000000000000 in ?? () >>> Current language: auto; currently minimal >>> (kgdb) >>> >>> >> -- >> Karl Denninger >> karl@denninger.net >> /The Market Ticker/ >> /[S/MIME encrypted email preferred]/ --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms090900030904000100020607 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEwMTcxODA4NDBaME8GCSqGSIb3DQEJBDFCBEBt NG0at3W7rzXELJhd1+h0bFAjin4DKITneKa1eW5ZZn4Lwu5VJO43WZ4nKqDKMVpl4AS9eBJA zdDT9f+XD3FEMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAdSlvGYOL bRAL+b4UNX2MJ4SQDAVKcxjQPS6d4x7pb1fqa/MsuImSfpJCoBBG1EcRTaDbJgdEdDWvzJs0 ZmPO8kK/vWYQwGMG17xurCjGBwZk2od6kJAgfjNxmdwffmxJee3BfRmEfhJqsk23cdOT/I64 /vZilR8rfDhyL1Rwm2aE/D4dqdoIH+XE9C0d7Z9ukKmN9RiM6Q5oZKNkIJD3ltc10kEqMr4q hOiXeQ4g9atM3+ii5Y33mvCs8i+16v8jVNckfuC+5f/q0LUOzVsBQg9fu2gP6Fa5hODTbdxz OvLzPpqzDdQW0sM89JlWnpY1+onZDq1FzsO5MTpaoNhMDmsrb+GF/PPUhhq5C65OjnOoZQ6C tXOuq6EtT5iGyjyAI45hm1Bjp35du3Xu8OC0yKd7mxbNTYxCF2w8i5mxeo9A9kC1JWQe3MK6 LGz/xkAVIvMN5N0WPz5WVN8nxFWzjkw3HYuk3ji6M9FqrBeUDf7NXl6I8K85MRxY+ParTgCu S26KqPGOMhlIyTLj962LtHz1hDfSQ7eH1sdBVxqCSl4quaqncIMC/wgmW7a/ZsyEScL3lrDO +SjfGMlCYEhGa7eoo7VVgvTrVQjlEzShzxKxUXK5ntT3Db29cDyhro8FC0cHeIsEgJp0rE6l a9dFR8BgLPoPJrAIrQo4yUrwS00AAAAAAAA= --------------ms090900030904000100020607-- From owner-freebsd-stable@freebsd.org Mon Oct 17 18:53:54 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 676BAC1686B for ; Mon, 17 Oct 2016 18:53:54 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-qk0-x22c.google.com (mail-qk0-x22c.google.com [IPv6:2607:f8b0:400d:c09::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 11355EF for ; Mon, 17 Oct 2016 18:53:53 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-qk0-x22c.google.com with SMTP id n189so251849013qke.0 for ; Mon, 17 Oct 2016 11:53:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=UqBnWTn5b/h+oFp0MlfZHFev7fHLJeEuhWNVixLRjEA=; b=fSE3afXY8RcgkqcWJi80BZsW9R9/4e92gO+jz3hGcKNyITCI/plFYWYjGkD2XUcFPw Z67Zw+eJxpBKuG/ngRCa/Ps8kfpJifwlBP2dw7MYxcp23CBmOzrIVeWM12nLH5wLTU9l 4rL/lLKTrdNsPghZCFL4KxPGAEl2CmvXPYJCkWg694dMwkhvMMW1BDSjP0WZ+5Kshl/U 7a/IzS9QN2w8PgIqgJGlo0EkVlo/EEAeFjcCEZctwdecDB1h+6MAjvYOLnUUdzGrO+H0 20JVQQoBQSIxVlIyjzKSGuek/RrqwgOHCa1WZaux8tHwXKEJmPIMWqS+F+4ks4pLiz43 RoMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=UqBnWTn5b/h+oFp0MlfZHFev7fHLJeEuhWNVixLRjEA=; b=aSJISsl8AA1WxXE0uPF11mm12v7achEQtqWeXE2mSow2ju8l5VjWUsSs7vEK4oZUgG 2bUu06QCxXU6+sUxQPaE67zflorU1u5ChkPUp3efeg5XN2J7BV0fMIP3RHj3H4+ZF05b 4AOVrriy8UDg22NiBZKVxJ1pTjzeq0YljP7vb5VALl41q7xe5XnsgQpEnBW27OGhnWpD I2wkVjuD1xZtvBmPsbgABPM0MfyxguWEuf8iHWi8uph6cvOEzJ6U2FOYKKIkoO7hJWmC 89zkgwxsu4OvrktTdqD6sRMc0q7jkV3AwBDZWE9lu4UFVIQbwjlAB7uLqb84rh9Bfw4d u8uA== X-Gm-Message-State: AA6/9Rminr6joC9PTzc4EnIHpQ6TfXQIk/sVizlnjkZI3juk1XiPy2yAi8zXtAonU6omn8sn X-Received: by 10.194.68.106 with SMTP id v10mr12629311wjt.149.1476730432451; Mon, 17 Oct 2016 11:53:52 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id wh3sm55196965wjb.49.2016.10.17.11.53.51 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Oct 2016 11:53:51 -0700 (PDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: freebsd-stable@freebsd.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> From: Steven Hartland Message-ID: <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> Date: Mon, 17 Oct 2016 19:54:17 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 18:53:54 -0000 You're hitting stack exhaustion, have you tried increasing the kernel stack pages? It can be changed from /boot/loader.conf kern.kstack_pages="6" Default on amd64 is 4 IIRC On 17/10/2016 19:08, Karl Denninger wrote: > The target (and devices that trigger this) are a pair of 4Gb 7200RPM > SATA rotating rust drives (zmirror) with each provider geli-encrypted > (that is, the actual devices used for the pool create are the .eli's) > > The machine generating the problem has both rotating rust devices *and* > SSDs, so I can't simply shut TRIM off system-wide and call it a day as > TRIM itself is heavily-used; both the boot/root pools and a Postgresql > database pool are on SSDs, while several terabytes of lesser-used data > is on a pool of Raidz2 that is made up of spinning rust. snip... > > NewFS.denninger.net dumped core - see /var/crash/vmcore.1 > > Mon Oct 17 09:02:33 CDT 2016 > > FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 > r307318M: Fri Oct 14 09:23:46 CDT 2016 > karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 > > panic: double fault > > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "amd64-marcel-freebsd"... > > Unread portion of the kernel message buffer: > > Fatal double fault > rip = 0xffffffff8220d9ec > rsp = 0xfffffe066821f000 > rbp = 0xfffffe066821f020 > cpuid = 6; apic id = 14 > panic: double fault > cpuid = 6 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe0649d78e30 > vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 > panic() at panic+0x43/frame 0xfffffe0649d78f10 > dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30 > Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 > --- trap 0x17, rip = 0xffffffff8220d9ec, rsp = 0xfffffe066821f000, rbp = > 0xfffffe066821f020 --- > avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 > avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 > vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame > 0xfffffe066821f530 > vdev_queue_io_done() at vdev_queue_io_done+0x83/frame 0xfffffe066821f570 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f650 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f6e0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f7c0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f850 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f930 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f9c0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821faa0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fb30 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fc10 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fca0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fd80 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fe10 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fef0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821ff80 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220060 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682200f0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682201d0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220260 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220340 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682203d0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682204b0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220540 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220620 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682206b0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220790 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220820 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220900 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220990 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220a70 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220b00 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220be0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220c70 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220d50 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220de0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220ec0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220f50 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221030 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682210c0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682211a0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221230 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221310 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682213a0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221480 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221510 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682215f0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221680 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221760 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682217f0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682218d0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221960 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221a40 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221ad0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221bb0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221c40 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221d20 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221db0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221e90 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221f20 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222000 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222090 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222170 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222200 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682222e0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222370 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222450 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682224e0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682225c0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222650 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222730 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682227c0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682228a0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222930 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960 > zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 > zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222a10 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 > vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222aa0 > zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0 > zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 > taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame > 0xfffffe0668222b80 > taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame > 0xfffffe0668222bb0 > fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > Uptime: 4m29s > Dumping 1497 out of 24531 > MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/zfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from > /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. > done. > Loaded symbols for /boot/kernel/opensolaris.ko > Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from > /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. > done. > Loaded symbols for /boot/kernel/geom_eli.ko > Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from > /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. > done. > Loaded symbols for /boot/kernel/geom_mirror.ko > Reading symbols from /boot/kernel/aesni.ko...Reading symbols from > /usr/lib/debug//boot/kernel/aesni.ko.debug...done. > done. > Loaded symbols for /boot/kernel/aesni.ko > Reading symbols from /boot/kernel/uhid.ko...Reading symbols from > /usr/lib/debug//boot/kernel/uhid.ko.debug...done. > done. > Loaded symbols for /boot/kernel/uhid.ko > Reading symbols from /boot/kernel/ums.ko...Reading symbols from > /usr/lib/debug//boot/kernel/ums.ko.debug...done. > done. > Loaded symbols for /boot/kernel/ums.ko > Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from > /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. > done. > Loaded symbols for /boot/kernel/ipfw.ko > Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from > /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. > done. > Loaded symbols for /boot/kernel/ipfw_nat.ko > Reading symbols from /boot/kernel/libalias.ko...Reading symbols from > /usr/lib/debug//boot/kernel/libalias.ko.debug...done. > done. > Loaded symbols for /boot/kernel/libalias.ko > Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols from > /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. > done. > Loaded symbols for /boot/kernel/daemon_saver.ko > #0 doadump (textdump=1) at pcpu.h:221 > 221 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) #0 doadump (textdump=1) at pcpu.h:221 > #1 0xffffffff80ae9b6e in kern_reboot (howto=260) > at /usr/src/sys/kern/kern_shutdown.c:366 > #2 0xffffffff80aea13b in vpanic (fmt=, > ap=) at /usr/src/sys/kern/kern_shutdown.c:759 > #3 0xffffffff80ae9f73 in panic (fmt=0x0) > at /usr/src/sys/kern/kern_shutdown.c:690 > #4 0xffffffff80fd18f2 in dblfault_handler (frame=) > at /usr/src/sys/amd64/amd64/trap.c:868 > #5 0xffffffff80fb406c in Xdblfault () > at /usr/src/sys/amd64/amd64/exception.S:292 > #6 0xffffffff8220d9ec in avl_rotation (tree=0xfffff8004dc54510, > node=0xfffff8056e45cdd0, balance=2) > at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 > #7 0xffffffff8220de58 in avl_remove (tree=, > data=) > at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 > #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:439 > #9 0xffffffff822a2123 in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:907 > #10 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e3f33d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #11 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #12 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e3f33d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #13 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #14 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #15 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e578000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #16 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #17 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e578000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #18 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #19 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #20 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ec3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #21 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #22 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ec3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #23 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #24 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #25 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e48a000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #26 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #27 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e48a000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #28 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #29 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #30 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e57ab88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #31 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #32 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e57ab88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #33 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #34 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #35 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69a3000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #36 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #37 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69a3000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #38 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #39 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #40 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5c8000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #41 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #42 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5c8000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #43 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #44 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #45 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23f9000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #46 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #47 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23f9000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #48 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #49 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #50 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5147b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #51 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #52 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5147b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #53 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #54 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #55 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e977000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #56 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #57 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e977000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #58 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #59 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #60 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90b3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #61 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #62 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90b3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #63 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #64 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #65 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b1160b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #66 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #67 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b1160b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #68 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #69 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #70 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4f53d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #71 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #72 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4f53d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #73 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #74 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #75 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8ebb88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #76 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #77 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8ebb88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #78 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #79 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #80 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8d9b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #81 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #82 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8d9b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #83 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #84 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #85 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21a03d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #86 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #87 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21a03d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #88 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #89 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #90 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e6903d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #91 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #92 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e6903d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #93 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #94 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #95 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a22dc7b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #96 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #97 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a22dc7b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #98 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #99 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #100 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5fb7b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #101 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #102 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5fb7b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #103 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #104 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #105 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ae3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #106 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #107 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ae3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #108 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #109 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #110 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a2131b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #111 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #112 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a2131b88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #113 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #114 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #115 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e80b7b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #116 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #117 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e80b7b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #118 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #119 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #120 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b14b9000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #121 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #122 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b14b9000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #123 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #124 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #125 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21fa3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #126 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #127 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21fa3d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #128 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #129 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #130 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5333d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #131 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #132 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5333d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #133 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #134 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #135 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90ab88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #136 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #137 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90ab88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #138 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #139 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #140 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4047b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #141 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #142 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4047b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #143 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #144 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #145 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e7083d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #146 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #147 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e7083d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #148 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #149 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #150 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23917b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #151 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #152 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23917b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #153 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #154 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #155 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e636000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #156 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #157 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e636000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #158 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #159 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #160 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5afb88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #161 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #162 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5afb88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #163 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #164 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #165 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69523d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #166 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #167 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69523d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #168 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #169 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #170 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e47c000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #171 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #172 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e47c000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #173 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #174 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #175 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b17b23d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #176 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #177 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b17b23d8) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #178 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #179 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #180 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d367b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #181 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #182 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a6d367b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #183 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #184 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #185 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e99fb88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #186 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #187 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e99fb88) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #188 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #189 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #190 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5227b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #191 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #192 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5227b0) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 > #193 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #194 0xffffffff822a216d in vdev_queue_io_done (zio=) > at > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 > #195 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d3c000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 > #196 0xffffffff822bbefd in zio_execute (zio=) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 > #197 0xffffffff80b4895a in taskqueue_run_locked (queue= out>) > at /usr/src/sys/kern/subr_taskqueue.c:454 > #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=) > at /usr/src/sys/kern/subr_taskqueue.c:724 > #199 0xffffffff80a9f255 in fork_exit ( > callout=0xffffffff80b49a70 , > arg=0xfffff8057caa72a0, frame=0xfffffe0668222c00) > at /usr/src/sys/kern/kern_fork.c:1040 > #200 0xffffffff80fb44ae in fork_trampoline () > at /usr/src/sys/amd64/amd64/exception.S:611 > #201 0x0000000000000000 in ?? () > Current language: auto; currently minimal > (kgdb) > > >>> -- >>> Karl Denninger >>> karl@denninger.net >>> /The Market Ticker/ >>> /[S/MIME encrypted email preferred]/ From owner-freebsd-stable@freebsd.org Mon Oct 17 18:58:52 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE847C169A2 for ; Mon, 17 Oct 2016 18:58:52 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 548E92C9 for ; Mon, 17 Oct 2016 18:58:51 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id B8341303D3E for ; Mon, 17 Oct 2016 13:58:50 -0500 (CDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: freebsd-stable@freebsd.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> From: Karl Denninger Message-ID: <76551fd6-0565-ee6c-b0f2-7d472ad6a4b3@denninger.net> Date: Mon, 17 Oct 2016 13:58:48 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms080809080201000509080103" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 18:58:53 -0000 This is a cryptographically signed message in MIME format. --------------ms080809080201000509080103 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable I can certainly attempt setting that higher.... but is that not just hiding the problem rather than addressing it? On 10/17/2016 13:54, Steven Hartland wrote: > You're hitting stack exhaustion, have you tried increasing the kernel > stack pages? > It can be changed from /boot/loader.conf > kern.kstack_pages=3D"6" > > Default on amd64 is 4 IIRC > > On 17/10/2016 19:08, Karl Denninger wrote: >> The target (and devices that trigger this) are a pair of 4Gb 7200RPM >> SATA rotating rust drives (zmirror) with each provider geli-encrypted >> (that is, the actual devices used for the pool create are the .eli's) >> >> The machine generating the problem has both rotating rust devices *and= * >> SSDs, so I can't simply shut TRIM off system-wide and call it a day as= >> TRIM itself is heavily-used; both the boot/root pools and a Postgresql= >> database pool are on SSDs, while several terabytes of lesser-used data= >> is on a pool of Raidz2 that is made up of spinning rust. > snip... >> >> NewFS.denninger.net dumped core - see /var/crash/vmcore.1 >> >> Mon Oct 17 09:02:33 CDT 2016 >> >> FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 >> r307318M: Fri Oct 14 09:23:46 CDT 2016 >> karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 >> >> panic: double fault >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and >> you are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for >> details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> >> Fatal double fault >> rip =3D 0xffffffff8220d9ec >> rsp =3D 0xfffffe066821f000 >> rbp =3D 0xfffffe066821f020 >> cpuid =3D 6; apic id =3D 14 >> panic: double fault >> cpuid =3D 6 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >> 0xfffffe0649d78e30 >> vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 >> panic() at panic+0x43/frame 0xfffffe0649d78f10 >> dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30 >> Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 >> --- trap 0x17, rip =3D 0xffffffff8220d9ec, rsp =3D 0xfffffe066821f000,= rbp =3D >> 0xfffffe066821f020 --- >> avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 >> avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 >> vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame >> 0xfffffe066821f530 >> vdev_queue_io_done() at vdev_queue_io_done+0x83/frame 0xfffffe066821f5= 70 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f65= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f6= e0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f7c= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f8= 50 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f93= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f9= c0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821faa= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fb= 30 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fc1= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fc= a0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fd8= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fe= 10 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fef= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821ff= 80 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822006= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682200= f0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682201d= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682202= 60 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822034= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682203= d0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682204b= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682205= 40 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822062= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682206= b0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822079= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682208= 20 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822090= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682209= 90 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220a7= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220b= 00 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220be= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220c= 70 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220d5= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220d= e0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220ec= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220f= 50 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822103= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682210= c0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682211a= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682212= 30 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822131= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682213= a0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822148= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682215= 10 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682215f= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682216= 80 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822176= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682217= f0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682218d= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682219= 60 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221a4= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221a= d0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221bb= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221c= 40 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221d2= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221d= b0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221e9= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221f= 20 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822200= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682220= 90 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822217= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682222= 00 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682222e= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682223= 70 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822245= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682224= e0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682225c= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682226= 50 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066822273= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682227= c0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682228a= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682229= 30 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222a1= 0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222a= a0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 >> taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame >> 0xfffffe0668222b80 >> taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame >> 0xfffffe0668222bb0 >> fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 >> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >> Uptime: 4m29s >> Dumping 1497 out of 24531 >> MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% >> >> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/zfs.ko >> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols fro= m >> /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/opensolaris.ko >> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/geom_eli.ko >> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols fro= m >> /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/geom_mirror.ko >> Reading symbols from /boot/kernel/aesni.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/aesni.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/aesni.ko >> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/uhid.ko >> Reading symbols from /boot/kernel/ums.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/ums.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/ums.ko >> Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/ipfw.ko >> Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/ipfw_nat.ko >> Reading symbols from /boot/kernel/libalias.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/libalias.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/libalias.ko >> Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols fr= om >> /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/daemon_saver.ko >> #0 doadump (textdump=3D1) at pcpu.h:221 >> 221 pcpu.h: No such file or directory. >> in pcpu.h >> (kgdb) #0 doadump (textdump=3D1) at pcpu.h:221 >> #1 0xffffffff80ae9b6e in kern_reboot (howto=3D260) >> at /usr/src/sys/kern/kern_shutdown.c:366 >> #2 0xffffffff80aea13b in vpanic (fmt=3D, >> ap=3D) at /usr/src/sys/kern/kern_shutdown.c:= 759 >> #3 0xffffffff80ae9f73 in panic (fmt=3D0x0) >> at /usr/src/sys/kern/kern_shutdown.c:690 >> #4 0xffffffff80fd18f2 in dblfault_handler (frame=3D) >> at /usr/src/sys/amd64/amd64/trap.c:868 >> #5 0xffffffff80fb406c in Xdblfault () >> at /usr/src/sys/amd64/amd64/exception.S:292 >> #6 0xffffffff8220d9ec in avl_rotation (tree=3D0xfffff8004dc54510, >> node=3D0xfffff8056e45cdd0, balance=3D2) >> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 >> #7 0xffffffff8220de58 in avl_remove (tree=3D, >> data=3D) >> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 >> #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:4= 39 >> #9 0xffffffff822a2123 in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 07 >> #10 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e3f33d8) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #11 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #12 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e3f33d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #13 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #14 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #15 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e578000) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #16 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #17 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e578000)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #18 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #19 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #20 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ec3d8) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #21 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #22 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ec3d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #23 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #24 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #25 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e48a000) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #26 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #27 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e48a000)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #28 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #29 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #30 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e57ab88) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #31 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #32 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e57ab88)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #33 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #34 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #35 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69a3000) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #36 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #37 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69a3000)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #38 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #39 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #40 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5c8000) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #41 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #42 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5c8000)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #43 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #44 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #45 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23f9000) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #46 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #47 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23f9000)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #48 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #49 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #50 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5147b0) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #51 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #52 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5147b0)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #53 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #54 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #55 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e977000) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #56 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #57 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e977000)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #58 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #59 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #60 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90b3d8) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #61 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #62 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90b3d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #63 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #64 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #65 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b1160b88) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #66 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #67 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b1160b88)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #68 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #69 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #70 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4f53d8) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #71 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #72 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4f53d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #73 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #74 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #75 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8ebb88) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #76 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #77 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8ebb88)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #78 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #79 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #80 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8d9b88) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #81 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #82 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8d9b88)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #83 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #84 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #85 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21a03d8) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #86 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #87 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21a03d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #88 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #89 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #90 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e6903d8) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #91 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #92 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e6903d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #93 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #94 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #95 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a22dc7b0) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #96 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #97 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a22dc7b0)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #98 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #99 0xffffffff822a216d in vdev_queue_io_done (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #100 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5fb7b0)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #101 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #102 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5fb7b0= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #103 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #104 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #105 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ae3d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #106 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #107 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ae3d8= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #108 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #109 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #110 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a2131b88)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #111 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #112 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a2131b88= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #113 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #114 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #115 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e80b7b0)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #116 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #117 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e80b7b0= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #118 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #119 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #120 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b14b9000)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #121 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #122 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b14b9000= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #123 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #124 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #125 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21fa3d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #126 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #127 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21fa3d8= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #128 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #129 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #130 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5333d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #131 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #132 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5333d8= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #133 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #134 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #135 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90ab88)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #136 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #137 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90ab88= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #138 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #139 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #140 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4047b0)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #141 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #142 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4047b0= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #143 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #144 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #145 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e7083d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #146 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #147 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e7083d8= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #148 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #149 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #150 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23917b0)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #151 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #152 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23917b0= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #153 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #154 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #155 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e636000)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #156 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #157 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e636000= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #158 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #159 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #160 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5afb88)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #161 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #162 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5afb88= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #163 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #164 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #165 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69523d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #166 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #167 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69523d8= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #168 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #169 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #170 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e47c000)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #171 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #172 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e47c000= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #173 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #174 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #175 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b17b23d8)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #176 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #177 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b17b23d8= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #178 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #179 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #180 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d367b0)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #181 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #182 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a6d367b0= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #183 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #184 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #185 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e99fb88)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #186 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #187 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e99fb88= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #188 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #189 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #190 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5227b0)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #191 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #192 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5227b0= ) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #193 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #194 0xffffffff822a216d in vdev_queue_io_done (zio=3D> out>) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:9= 13 >> #195 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d3c000)= >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #196 0xffffffff822bbefd in zio_execute (zio=3D) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #197 0xffffffff80b4895a in taskqueue_run_locked (queue=3D> out>) >> at /usr/src/sys/kern/subr_taskqueue.c:454 >> #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=3D> optimized out>) >> at /usr/src/sys/kern/subr_taskqueue.c:724 >> #199 0xffffffff80a9f255 in fork_exit ( >> callout=3D0xffffffff80b49a70 , >> arg=3D0xfffff8057caa72a0, frame=3D0xfffffe0668222c00) >> at /usr/src/sys/kern/kern_fork.c:1040 >> #200 0xffffffff80fb44ae in fork_trampoline () >> at /usr/src/sys/amd64/amd64/exception.S:611 >> #201 0x0000000000000000 in ?? () >> Current language: auto; currently minimal >> (kgdb) >> >> >>>> --=20 >>>> Karl Denninger >>>> karl@denninger.net >>>> /The Market Ticker/ >>>> /[S/MIME encrypted email preferred]/ > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g" --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms080809080201000509080103 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEwMTcxODU4NDhaME8GCSqGSIb3DQEJBDFCBEA2 2rc7UeAf+gDugkKl61hZmMT3zoQt2fmr34/4fwRNKNgQVyLB/5k7dE6+Ymy3lvYA+fYkxYSa Wx4BsuEA/rKrMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAnOcy+jTD /5AaQNvcxWJcwIxkGOPeq+1NG4HZqfOrbKDMVw8PGKyuxwi23xbW/a5nrOwGdB/SHKpbwQ3r Qx3Jzxi6cfhRJDs5gYVJjBxKJy6xUUAich3HUzHhbnL5T8zh3FVcz1tmmk0w8wOuWQCD7IQA frfgmNWimjCABeNhuWTuBZc6CMQccadLAIksJKsqY/ePa4ok5sl65/w1sidc1VYyVSDSxmwa ratdhgjWTwdRuYumr6/H3uI8EqS+5KFBlh1vKqa9ZHwRnjUJ2pCmwAr7/4CC1IEqcoLQ+mX0 1nRA7TGkYkUxcct7ZB5/DpXMOAbelwOLv/LfhMUiOjGygaLkZ6BgnoNzTjiT3NGEgJhn6UNf 2SWyClpXk34VyQqDxpQiMRHfaJlkh/cTy0PcsXItgnEvsbFTDXRnx7dlhTZTNFtKS2+xOyGL G5yQfxfVcHTPa6PxJfOnARznMA6rQmgtosb9e03ic1PU8NwLxTc5Y4bT+Qip0634cWW7tOU2 jL8cTEva/WmdiStvFmYI/1HDjoVd8yZbrQ+zzFQDndlv/CRb5dulLh67FtOR93h8O3Fqgiut DqLVpyPN7Klq4FVwowP6QuWPzMs+DpluwD2PSsolJsCuuyFUYiGnLDFlz//SzjPlRnFc7i2A 4XndKNo8+Z+1FDu6i5RwxvKvvX0AAAAAAAA= --------------ms080809080201000509080103-- From owner-freebsd-stable@freebsd.org Mon Oct 17 19:48:15 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D15A3AF41F7 for ; Mon, 17 Oct 2016 19:48:15 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id C38A9BBE; Mon, 17 Oct 2016 19:48:15 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id EEC05CD; Mon, 17 Oct 2016 19:48:15 +0000 (UTC) Date: Mon, 17 Oct 2016 19:48:15 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <1652629877.2.1476733695915.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <1714359220.1.1476723155509.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1714359220.1.1476723155509.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #428 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 19:48:15 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/428/ From owner-freebsd-stable@freebsd.org Mon Oct 17 19:53:55 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9F8AEAF45BE for ; Mon, 17 Oct 2016 19:53:55 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 5282B20A for ; Mon, 17 Oct 2016 19:53:53 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id WAA21127; Mon, 17 Oct 2016 22:53:51 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1bwDyp-000Pnx-LP; Mon, 17 Oct 2016 22:53:51 +0300 Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: Steven Hartland , freebsd-stable@FreeBSD.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> From: Andriy Gapon Message-ID: Date: Mon, 17 Oct 2016 22:52:55 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 19:53:55 -0000 On 17/10/2016 21:54, Steven Hartland wrote: > You're hitting stack exhaustion, have you tried increasing the kernel stack pages? > It can be changed from /boot/loader.conf > kern.kstack_pages="6" > > Default on amd64 is 4 IIRC Steve, perhaps you can think of a more proper fix? :-) https://lists.freebsd.org/pipermail/freebsd-stable/2016-July/085047.html > On 17/10/2016 19:08, Karl Denninger wrote: >> The target (and devices that trigger this) are a pair of 4Gb 7200RPM >> SATA rotating rust drives (zmirror) with each provider geli-encrypted >> (that is, the actual devices used for the pool create are the .eli's) >> >> The machine generating the problem has both rotating rust devices *and* >> SSDs, so I can't simply shut TRIM off system-wide and call it a day as >> TRIM itself is heavily-used; both the boot/root pools and a Postgresql >> database pool are on SSDs, while several terabytes of lesser-used data >> is on a pool of Raidz2 that is made up of spinning rust. > snip... >> >> NewFS.denninger.net dumped core - see /var/crash/vmcore.1 >> >> Mon Oct 17 09:02:33 CDT 2016 >> >> FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 >> r307318M: Fri Oct 14 09:23:46 CDT 2016 >> karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 >> >> panic: double fault >> >> GNU gdb 6.1.1 [FreeBSD] >> Copyright 2004 Free Software Foundation, Inc. >> GDB is free software, covered by the GNU General Public License, and you are >> welcome to change it and/or distribute copies of it under certain >> conditions. >> Type "show copying" to see the conditions. >> There is absolutely no warranty for GDB. Type "show warranty" for details. >> This GDB was configured as "amd64-marcel-freebsd"... >> >> Unread portion of the kernel message buffer: >> >> Fatal double fault >> rip = 0xffffffff8220d9ec >> rsp = 0xfffffe066821f000 >> rbp = 0xfffffe066821f020 >> cpuid = 6; apic id = 14 >> panic: double fault >> cpuid = 6 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >> 0xfffffe0649d78e30 >> vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 >> panic() at panic+0x43/frame 0xfffffe0649d78f10 >> dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30 >> Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 >> --- trap 0x17, rip = 0xffffffff8220d9ec, rsp = 0xfffffe066821f000, rbp = >> 0xfffffe066821f020 --- >> avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 >> avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 >> vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame >> 0xfffffe066821f530 >> vdev_queue_io_done() at vdev_queue_io_done+0x83/frame 0xfffffe066821f570 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f650 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f6e0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f7c0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f850 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f930 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f9c0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821faa0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fb30 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fc10 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fca0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fd80 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fe10 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fef0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821ff80 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220060 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682200f0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682201d0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220260 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220340 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682203d0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682204b0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220540 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220620 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682206b0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220790 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220820 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220900 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220990 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220a70 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220b00 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220be0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220c70 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220d50 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220de0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220ec0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220f50 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221030 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682210c0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682211a0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221230 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221310 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682213a0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221480 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221510 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682215f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221680 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221760 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682217f0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682218d0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221960 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221a40 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221ad0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221bb0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221c40 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221d20 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221db0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221e90 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221f20 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222000 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222090 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222170 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222200 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682222e0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222370 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222450 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682224e0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682225c0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222650 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222730 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682227c0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682228a0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222930 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 >> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222a10 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 >> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222aa0 >> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0 >> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 >> taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame >> 0xfffffe0668222b80 >> taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame >> 0xfffffe0668222bb0 >> fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 >> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >> Uptime: 4m29s >> Dumping 1497 out of 24531 >> MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% >> >> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/zfs.ko >> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/opensolaris.ko >> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/geom_eli.ko >> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/geom_mirror.ko >> Reading symbols from /boot/kernel/aesni.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/aesni.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/aesni.ko >> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/uhid.ko >> Reading symbols from /boot/kernel/ums.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/ums.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/ums.ko >> Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/ipfw.ko >> Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/ipfw_nat.ko >> Reading symbols from /boot/kernel/libalias.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/libalias.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/libalias.ko >> Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols from >> /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. >> done. >> Loaded symbols for /boot/kernel/daemon_saver.ko >> #0 doadump (textdump=1) at pcpu.h:221 >> 221 pcpu.h: No such file or directory. >> in pcpu.h >> (kgdb) #0 doadump (textdump=1) at pcpu.h:221 >> #1 0xffffffff80ae9b6e in kern_reboot (howto=260) >> at /usr/src/sys/kern/kern_shutdown.c:366 >> #2 0xffffffff80aea13b in vpanic (fmt=, >> ap=) at /usr/src/sys/kern/kern_shutdown.c:759 >> #3 0xffffffff80ae9f73 in panic (fmt=0x0) >> at /usr/src/sys/kern/kern_shutdown.c:690 >> #4 0xffffffff80fd18f2 in dblfault_handler (frame=) >> at /usr/src/sys/amd64/amd64/trap.c:868 >> #5 0xffffffff80fb406c in Xdblfault () >> at /usr/src/sys/amd64/amd64/exception.S:292 >> #6 0xffffffff8220d9ec in avl_rotation (tree=0xfffff8004dc54510, >> node=0xfffff8056e45cdd0, balance=2) >> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 >> #7 0xffffffff8220de58 in avl_remove (tree=, >> data=) >> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 >> #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:439 >> #9 0xffffffff822a2123 in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:907 >> #10 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e3f33d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #11 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #12 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e3f33d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #13 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #14 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #15 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e578000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #16 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #17 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e578000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #18 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #19 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #20 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ec3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #21 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #22 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ec3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #23 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #24 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #25 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e48a000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #26 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #27 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e48a000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #28 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #29 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #30 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e57ab88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #31 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #32 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e57ab88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #33 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #34 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #35 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69a3000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #36 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #37 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69a3000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #38 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #39 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #40 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5c8000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #41 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #42 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5c8000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #43 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #44 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #45 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23f9000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #46 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #47 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23f9000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #48 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #49 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #50 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5147b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #51 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #52 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5147b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #53 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #54 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #55 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e977000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #56 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #57 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e977000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #58 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #59 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #60 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90b3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #61 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #62 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90b3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #63 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #64 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #65 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b1160b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #66 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #67 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b1160b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #68 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #69 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #70 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4f53d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #71 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #72 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4f53d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #73 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #74 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #75 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8ebb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #76 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #77 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8ebb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #78 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #79 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #80 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8d9b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #81 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #82 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8d9b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #83 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #84 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #85 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21a03d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #86 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #87 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21a03d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #88 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #89 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #90 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e6903d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #91 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #92 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e6903d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #93 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #94 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #95 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a22dc7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #96 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #97 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a22dc7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #98 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #99 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #100 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5fb7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #101 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #102 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5fb7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #103 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #104 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #105 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ae3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #106 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #107 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ae3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #108 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #109 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #110 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a2131b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #111 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #112 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a2131b88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #113 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #114 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #115 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e80b7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #116 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #117 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e80b7b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #118 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #119 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #120 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b14b9000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #121 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #122 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b14b9000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #123 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #124 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #125 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21fa3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #126 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #127 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21fa3d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #128 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #129 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #130 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5333d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #131 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #132 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5333d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #133 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #134 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #135 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90ab88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #136 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #137 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90ab88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #138 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #139 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #140 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4047b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #141 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #142 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4047b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #143 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #144 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #145 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e7083d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #146 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #147 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e7083d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #148 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #149 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #150 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23917b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #151 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #152 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23917b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #153 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #154 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #155 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e636000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #156 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #157 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e636000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #158 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #159 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #160 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5afb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #161 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #162 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5afb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #163 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #164 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #165 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69523d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #166 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #167 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69523d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #168 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #169 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #170 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e47c000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #171 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #172 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e47c000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #173 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #174 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #175 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b17b23d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #176 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #177 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b17b23d8) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #178 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #179 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #180 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d367b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #181 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #182 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a6d367b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #183 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #184 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #185 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e99fb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #186 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #187 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e99fb88) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #188 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #189 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #190 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5227b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #191 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #192 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5227b0) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >> #193 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #194 0xffffffff822a216d in vdev_queue_io_done (zio=) >> at >> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >> #195 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d3c000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >> #196 0xffffffff822bbefd in zio_execute (zio=) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >> #197 0xffffffff80b4895a in taskqueue_run_locked (queue=> out>) >> at /usr/src/sys/kern/subr_taskqueue.c:454 >> #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=) >> at /usr/src/sys/kern/subr_taskqueue.c:724 >> #199 0xffffffff80a9f255 in fork_exit ( >> callout=0xffffffff80b49a70 , >> arg=0xfffff8057caa72a0, frame=0xfffffe0668222c00) >> at /usr/src/sys/kern/kern_fork.c:1040 >> #200 0xffffffff80fb44ae in fork_trampoline () >> at /usr/src/sys/amd64/amd64/exception.S:611 >> #201 0x0000000000000000 in ?? () >> Current language: auto; currently minimal >> (kgdb) >> >> >>>> -- >>>> Karl Denninger >>>> karl@denninger.net >>>> /The Market Ticker/ >>>> /[S/MIME encrypted email preferred]/ > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > -- Andriy Gapon From owner-freebsd-stable@freebsd.org Mon Oct 17 20:16:38 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64E78AF4DBC for ; Mon, 17 Oct 2016 20:16:38 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-lf0-x234.google.com (mail-lf0-x234.google.com [IPv6:2a00:1450:4010:c07::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 709BE1BF for ; Mon, 17 Oct 2016 20:16:37 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-lf0-x234.google.com with SMTP id b75so306191039lfg.3 for ; Mon, 17 Oct 2016 13:16:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=KYNGngg2mkmVAmKqQ51ubtDcvZ3BVHtkfGBu4/WOrGo=; b=TlEDgF/HqOlZYJlcxbaLSE/YHSoUa9A1IRsmj1fbialC2JuISZ57LXEjtbUf0Iy0yy fgLbPtn5UN0rHJCU3j20XKNWdPUhUHnQxuuzIYuLdEEHF6/1nxz8n8khx8RZvjzO+pSD UdVGSTA7PvONjaSuLrbFGT7UhVC+XCn+/PAhp+oU1y5JnzqqyCj9AXQHWPLdTTqXNVpM Tf7MHh9gpkOdfwQgubNGCrcVCX0hf+0k0XfdGV31EKO+QwQNmJDdaCqVTGktZkuNzpRh ujtM2k1ynmSjLYkBs0J9owcqH5V23BOwPEw2JYRM/F915XZY+oZqkyqM8Gi5VbyEDrnP Qqxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=KYNGngg2mkmVAmKqQ51ubtDcvZ3BVHtkfGBu4/WOrGo=; b=PosNeHY5qwfUUFJ3VAPGCqLCdWyBJL8kHYgmWzqEZkMAUbhuRZyW587ZsugMdIKLr5 zSds9tmufFU7KE0pfjkFc9ki2wjLHuGGubaAErYMbJFrwTGPlOk0ZGBKVjp4vQdTdff9 /d0lJw8TelT4p7NJ5B5lkx8+yvK5g1CtRN3NWWH8JD7rC9ay+v+eDJoibzes/GieN5u+ 1GNHB3yRy3Pr44uTZs94vTHoOXHhVuN0RlA+gSj2EHtHSPldfQuI8m4jUxsPrLf1ZU/4 9XdtDIpNigFTQfabXlyvkgKe8ZICtVVR+5JPVinw8+xzxbgPIxPZB42dKDmQh2H40iea TWOg== X-Gm-Message-State: AA6/9RkjSqI0PY3w1dVUvXDt9EjswgezXwY68jYH/rRd+dLiksV29CyGpLDwcJjQRrR+q7Oj X-Received: by 10.28.37.70 with SMTP id l67mr3042wml.44.1476735393826; Mon, 17 Oct 2016 13:16:33 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id a5sm20396598wjv.6.2016.10.17.13.16.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Oct 2016 13:16:32 -0700 (PDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: freebsd-stable@freebsd.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> <76551fd6-0565-ee6c-b0f2-7d472ad6a4b3@denninger.net> From: Steven Hartland Message-ID: <25ff3a3e-77a9-063b-e491-8d10a06e6ae2@multiplay.co.uk> Date: Mon, 17 Oct 2016 21:16:59 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <76551fd6-0565-ee6c-b0f2-7d472ad6a4b3@denninger.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 20:16:38 -0000 Be good to confirm its not an infinite loop by giving it a good bump first. On 17/10/2016 19:58, Karl Denninger wrote: > I can certainly attempt setting that higher.... but is that not just > hiding the problem rather than addressing it? > > > On 10/17/2016 13:54, Steven Hartland wrote: >> You're hitting stack exhaustion, have you tried increasing the kernel >> stack pages? >> It can be changed from /boot/loader.conf >> kern.kstack_pages="6" >> >> Default on amd64 is 4 IIRC >> >> On 17/10/2016 19:08, Karl Denninger wrote: >>> The target (and devices that trigger this) are a pair of 4Gb 7200RPM >>> SATA rotating rust drives (zmirror) with each provider geli-encrypted >>> (that is, the actual devices used for the pool create are the .eli's) >>> >>> The machine generating the problem has both rotating rust devices *and* >>> SSDs, so I can't simply shut TRIM off system-wide and call it a day as >>> TRIM itself is heavily-used; both the boot/root pools and a Postgresql >>> database pool are on SSDs, while several terabytes of lesser-used data >>> is on a pool of Raidz2 that is made up of spinning rust. >> snip... >>> NewFS.denninger.net dumped core - see /var/crash/vmcore.1 >>> >>> Mon Oct 17 09:02:33 CDT 2016 >>> >>> FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 >>> r307318M: Fri Oct 14 09:23:46 CDT 2016 >>> karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 >>> >>> panic: double fault >>> >>> GNU gdb 6.1.1 [FreeBSD] >>> Copyright 2004 Free Software Foundation, Inc. >>> GDB is free software, covered by the GNU General Public License, and >>> you are >>> welcome to change it and/or distribute copies of it under certain >>> conditions. >>> Type "show copying" to see the conditions. >>> There is absolutely no warranty for GDB. Type "show warranty" for >>> details. >>> This GDB was configured as "amd64-marcel-freebsd"... >>> >>> Unread portion of the kernel message buffer: >>> >>> Fatal double fault >>> rip = 0xffffffff8220d9ec >>> rsp = 0xfffffe066821f000 >>> rbp = 0xfffffe066821f020 >>> cpuid = 6; apic id = 14 >>> panic: double fault >>> cpuid = 6 >>> KDB: stack backtrace: >>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >>> 0xfffffe0649d78e30 >>> vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 >>> panic() at panic+0x43/frame 0xfffffe0649d78f10 >>> dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30 >>> Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 >>> --- trap 0x17, rip = 0xffffffff8220d9ec, rsp = 0xfffffe066821f000, rbp = >>> 0xfffffe066821f020 --- >>> avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 >>> avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 >>> vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame >>> 0xfffffe066821f530 >>> vdev_queue_io_done() at vdev_queue_io_done+0x83/frame 0xfffffe066821f570 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f650 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f6e0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f7c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f850 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f930 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f9c0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821faa0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fb30 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fc10 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fca0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fd80 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fe10 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fef0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821ff80 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220060 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682200f0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682201d0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220260 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220340 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682203d0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682204b0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220540 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220620 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682206b0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220790 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220820 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220900 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220990 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220a70 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220b00 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220be0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220c70 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220d50 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220de0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220ec0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220f50 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221030 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682210c0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682211a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221230 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221310 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682213a0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221480 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221510 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682215f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221680 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221760 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682217f0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682218d0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221960 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221a40 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221ad0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221bb0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221c40 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221d20 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221db0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221e90 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221f20 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222000 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222090 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222170 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222200 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682222e0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222370 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222450 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682224e0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682225c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222650 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222730 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682227c0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682228a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222930 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222a10 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222aa0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 >>> taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame >>> 0xfffffe0668222b80 >>> taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame >>> 0xfffffe0668222bb0 >>> fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 >>> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >>> Uptime: 4m29s >>> Dumping 1497 out of 24531 >>> MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% >>> >>> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/zfs.ko >>> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/opensolaris.ko >>> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/geom_eli.ko >>> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/geom_mirror.ko >>> Reading symbols from /boot/kernel/aesni.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/aesni.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/aesni.ko >>> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/uhid.ko >>> Reading symbols from /boot/kernel/ums.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/ums.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/ums.ko >>> Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/ipfw.ko >>> Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/ipfw_nat.ko >>> Reading symbols from /boot/kernel/libalias.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/libalias.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/libalias.ko >>> Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/daemon_saver.ko >>> #0 doadump (textdump=1) at pcpu.h:221 >>> 221 pcpu.h: No such file or directory. >>> in pcpu.h >>> (kgdb) #0 doadump (textdump=1) at pcpu.h:221 >>> #1 0xffffffff80ae9b6e in kern_reboot (howto=260) >>> at /usr/src/sys/kern/kern_shutdown.c:366 >>> #2 0xffffffff80aea13b in vpanic (fmt=, >>> ap=) at /usr/src/sys/kern/kern_shutdown.c:759 >>> #3 0xffffffff80ae9f73 in panic (fmt=0x0) >>> at /usr/src/sys/kern/kern_shutdown.c:690 >>> #4 0xffffffff80fd18f2 in dblfault_handler (frame=) >>> at /usr/src/sys/amd64/amd64/trap.c:868 >>> #5 0xffffffff80fb406c in Xdblfault () >>> at /usr/src/sys/amd64/amd64/exception.S:292 >>> #6 0xffffffff8220d9ec in avl_rotation (tree=0xfffff8004dc54510, >>> node=0xfffff8056e45cdd0, balance=2) >>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 >>> #7 0xffffffff8220de58 in avl_remove (tree=, >>> data=) >>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 >>> #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:439 >>> #9 0xffffffff822a2123 in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:907 >>> #10 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e3f33d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #11 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #12 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e3f33d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #13 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #14 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #15 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e578000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #16 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #17 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e578000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #18 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #19 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #20 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ec3d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #21 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #22 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ec3d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #23 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #24 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #25 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e48a000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #26 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #27 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e48a000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #28 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #29 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #30 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e57ab88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #31 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #32 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e57ab88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #33 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #34 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #35 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69a3000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #36 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #37 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69a3000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #38 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #39 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #40 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5c8000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #41 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #42 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5c8000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #43 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #44 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #45 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23f9000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #46 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #47 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23f9000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #48 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #49 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #50 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5147b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #51 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #52 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5147b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #53 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #54 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #55 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e977000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #56 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #57 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e977000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #58 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #59 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #60 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90b3d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #61 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #62 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90b3d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #63 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #64 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #65 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b1160b88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #66 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #67 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b1160b88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #68 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #69 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #70 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4f53d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #71 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #72 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4f53d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #73 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #74 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #75 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8ebb88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #76 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #77 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8ebb88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #78 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #79 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #80 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8d9b88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #81 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #82 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8d9b88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #83 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #84 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #85 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21a03d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #86 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #87 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21a03d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #88 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #89 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #90 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e6903d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #91 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #92 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e6903d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #93 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #94 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #95 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a22dc7b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #96 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #97 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a22dc7b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #98 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #99 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #100 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5fb7b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #101 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #102 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5fb7b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #103 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #104 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #105 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ae3d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #106 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #107 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ae3d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #108 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #109 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #110 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a2131b88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #111 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #112 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a2131b88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #113 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #114 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #115 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e80b7b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #116 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #117 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e80b7b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #118 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #119 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #120 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b14b9000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #121 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #122 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b14b9000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #123 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #124 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #125 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21fa3d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #126 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #127 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21fa3d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #128 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #129 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #130 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5333d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #131 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #132 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5333d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #133 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #134 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #135 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90ab88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #136 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #137 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90ab88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #138 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #139 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #140 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4047b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #141 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #142 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4047b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #143 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #144 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #145 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e7083d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #146 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #147 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e7083d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #148 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #149 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #150 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23917b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #151 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #152 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23917b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #153 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #154 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #155 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e636000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #156 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #157 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e636000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #158 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #159 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #160 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5afb88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #161 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #162 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5afb88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #163 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #164 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #165 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69523d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #166 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #167 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69523d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #168 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #169 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #170 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e47c000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #171 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #172 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e47c000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #173 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #174 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #175 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b17b23d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #176 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #177 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b17b23d8) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #178 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #179 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #180 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d367b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #181 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #182 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a6d367b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #183 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #184 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #185 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e99fb88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #186 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #187 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e99fb88) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #188 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #189 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #190 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5227b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #191 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #192 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5227b0) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #193 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #194 0xffffffff822a216d in vdev_queue_io_done (zio=>> out>) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #195 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d3c000) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #196 0xffffffff822bbefd in zio_execute (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #197 0xffffffff80b4895a in taskqueue_run_locked (queue=>> out>) >>> at /usr/src/sys/kern/subr_taskqueue.c:454 >>> #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=>> optimized out>) >>> at /usr/src/sys/kern/subr_taskqueue.c:724 >>> #199 0xffffffff80a9f255 in fork_exit ( >>> callout=0xffffffff80b49a70 , >>> arg=0xfffff8057caa72a0, frame=0xfffffe0668222c00) >>> at /usr/src/sys/kern/kern_fork.c:1040 >>> #200 0xffffffff80fb44ae in fork_trampoline () >>> at /usr/src/sys/amd64/amd64/exception.S:611 >>> #201 0x0000000000000000 in ?? () >>> Current language: auto; currently minimal >>> (kgdb) >>> >>> >>>>> -- >>>>> Karl Denninger >>>>> karl@denninger.net >>>>> /The Market Ticker/ >>>>> /[S/MIME encrypted email preferred]/ >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Mon Oct 17 20:22:49 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0148DAF4FEF for ; Mon, 17 Oct 2016 20:22:49 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 83F44831 for ; Mon, 17 Oct 2016 20:22:48 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 1EEBC303058 for ; Mon, 17 Oct 2016 15:22:46 -0500 (CDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: freebsd-stable@freebsd.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> <76551fd6-0565-ee6c-b0f2-7d472ad6a4b3@denninger.net> <25ff3a3e-77a9-063b-e491-8d10a06e6ae2@multiplay.co.uk> From: Karl Denninger Message-ID: <26e092b2-17c6-8744-5035-d0853d733870@denninger.net> Date: Mon, 17 Oct 2016 15:22:44 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <25ff3a3e-77a9-063b-e491-8d10a06e6ae2@multiplay.co.uk> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms020209010902040701010306" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 20:22:49 -0000 This is a cryptographically signed message in MIME format. --------------ms020209010902040701010306 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Since I cleared it (by setting TRIM off on the test machine, rebooting, importing the pool and noting that it did not panic -- pulled drives, re-inserted into the production machine and ran backup routine -- all was normal) it may be a while before I see it again (a week or so is usua= l.) It appears to be related to entropy in the filesystem that comes up as "eligible" to be removed from the backup volume, which (not surprisingly) tends to happen a few days after I do a new world build or something similar (the daily and/or periodic snapshots roll off at about that point.) I don't happen to have a spare pair of high-performance SSDs I can stick in the sandbox machine in an attempt to force the condition to assert itself in test, unfortunately. I *am* concerned that it's not "simple" stack exhaustion because setting the max outstanding TRIMs on a per-vdev basis down quite-dramatically did *not* prevent it from happening -- and if it was simply stack depth related I would have expected that to put a stop to it. On 10/17/2016 15:16, Steven Hartland wrote: > Be good to confirm its not an infinite loop by giving it a good bump > first. > > On 17/10/2016 19:58, Karl Denninger wrote: >> I can certainly attempt setting that higher.... but is that not just >> hiding the problem rather than addressing it? >> >> >> On 10/17/2016 13:54, Steven Hartland wrote: >>> You're hitting stack exhaustion, have you tried increasing the kernel= >>> stack pages? >>> It can be changed from /boot/loader.conf >>> kern.kstack_pages=3D"6" >>> >>> Default on amd64 is 4 IIRC >>> >>> On 17/10/2016 19:08, Karl Denninger wrote: >>>> The target (and devices that trigger this) are a pair of 4Gb 7200RPM= >>>> SATA rotating rust drives (zmirror) with each provider geli-encrypte= d >>>> (that is, the actual devices used for the pool create are the .eli's= ) >>>> >>>> The machine generating the problem has both rotating rust devices >>>> *and* >>>> SSDs, so I can't simply shut TRIM off system-wide and call it a day = as >>>> TRIM itself is heavily-used; both the boot/root pools and a Postgres= ql >>>> database pool are on SSDs, while several terabytes of lesser-used da= ta >>>> is on a pool of Raidz2 that is made up of spinning rust. >>> snip... >>>> NewFS.denninger.net dumped core - see /var/crash/vmcore.1 >>>> >>>> Mon Oct 17 09:02:33 CDT 2016 >>>> >>>> FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 >>>> r307318M: Fri Oct 14 09:23:46 CDT 2016 >>>> karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 >>>> >>>> panic: double fault >>>> >>>> GNU gdb 6.1.1 [FreeBSD] >>>> Copyright 2004 Free Software Foundation, Inc. >>>> GDB is free software, covered by the GNU General Public License, and= >>>> you are >>>> welcome to change it and/or distribute copies of it under certain >>>> conditions. >>>> Type "show copying" to see the conditions. >>>> There is absolutely no warranty for GDB. Type "show warranty" for >>>> details. >>>> This GDB was configured as "amd64-marcel-freebsd"... >>>> >>>> Unread portion of the kernel message buffer: >>>> >>>> Fatal double fault >>>> rip =3D 0xffffffff8220d9ec >>>> rsp =3D 0xfffffe066821f000 >>>> rbp =3D 0xfffffe066821f020 >>>> cpuid =3D 6; apic id =3D 14 >>>> panic: double fault >>>> cpuid =3D 6 >>>> KDB: stack backtrace: >>>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >>>> 0xfffffe0649d78e30 >>>> vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 >>>> panic() at panic+0x43/frame 0xfffffe0649d78f10 >>>> dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30= >>>> Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 >>>> --- trap 0x17, rip =3D 0xffffffff8220d9ec, rsp =3D 0xfffffe066821f00= 0, >>>> rbp =3D >>>> 0xfffffe066821f020 --- >>>> avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 >>>> avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 >>>> vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame >>>> 0xfffffe066821f530 >>>> vdev_queue_io_done() at vdev_queue_io_done+0x83/frame >>>> 0xfffffe066821f570 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe066821f650 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe066821f6e0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe066821f7c0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe066821f850 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe066821f930 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe066821f9c0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe066821faa0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe066821fb30 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe066821fc10 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe066821fca0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe066821fd80 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe066821fe10 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe066821fef0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe066821ff80 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668220060 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe06682200f0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe06682201d0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668220260 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668220340 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe06682203d0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe06682204b0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668220540 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668220620 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe06682206b0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668220790 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668220820 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668220900 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668220990 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668220a70 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668220b00 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668220be0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668220c70 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668220d50 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668220de0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668220ec0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668220f50 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668221030 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe06682210c0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe06682211a0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668221230 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668221310 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe06682213a0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668221480 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668221510 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe06682215f0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668221680 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668221760 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe06682217f0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe06682218d0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668221960 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668221a40 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668221ad0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668221bb0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668221c40 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668221d20 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668221db0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668221e90 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668221f20 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668222000 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668222090 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668222170 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668222200 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe06682222e0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668222370 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668222450 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe06682224e0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe06682225c0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668222650 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668222730 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe06682227c0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe06682228a0 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668222930 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 >>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>> 0xfffffe0668222a10 >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 >>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>> 0xfffffe0668222aa0 >>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0= >>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 >>>> taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame >>>> 0xfffffe0668222b80 >>>> taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame >>>> 0xfffffe0668222bb0 >>>> fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 >>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 >>>> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >>>> Uptime: 4m29s >>>> Dumping 1497 out of 24531 >>>> MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% >>>> >>>> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >>>> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/zfs.ko >>>> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols >>>> from >>>> /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/opensolaris.ko >>>> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from= >>>> /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/geom_eli.ko >>>> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols >>>> from >>>> /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/geom_mirror.ko >>>> Reading symbols from /boot/kernel/aesni.ko...Reading symbols from >>>> /usr/lib/debug//boot/kernel/aesni.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/aesni.ko >>>> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from >>>> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/uhid.ko >>>> Reading symbols from /boot/kernel/ums.ko...Reading symbols from >>>> /usr/lib/debug//boot/kernel/ums.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/ums.ko >>>> Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from >>>> /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/ipfw.ko >>>> Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from= >>>> /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/ipfw_nat.ko >>>> Reading symbols from /boot/kernel/libalias.ko...Reading symbols from= >>>> /usr/lib/debug//boot/kernel/libalias.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/libalias.ko >>>> Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols >>>> from >>>> /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. >>>> done. >>>> Loaded symbols for /boot/kernel/daemon_saver.ko >>>> #0 doadump (textdump=3D1) at pcpu.h:221 >>>> 221 pcpu.h: No such file or directory. >>>> in pcpu.h >>>> (kgdb) #0 doadump (textdump=3D1) at pcpu.h:221 >>>> #1 0xffffffff80ae9b6e in kern_reboot (howto=3D260) >>>> at /usr/src/sys/kern/kern_shutdown.c:366 >>>> #2 0xffffffff80aea13b in vpanic (fmt=3D, >>>> ap=3D) at >>>> /usr/src/sys/kern/kern_shutdown.c:759 >>>> #3 0xffffffff80ae9f73 in panic (fmt=3D0x0) >>>> at /usr/src/sys/kern/kern_shutdown.c:690 >>>> #4 0xffffffff80fd18f2 in dblfault_handler (frame=3D>>> out>) >>>> at /usr/src/sys/amd64/amd64/trap.c:868 >>>> #5 0xffffffff80fb406c in Xdblfault () >>>> at /usr/src/sys/amd64/amd64/exception.S:292 >>>> #6 0xffffffff8220d9ec in avl_rotation (tree=3D0xfffff8004dc54510, >>>> node=3D0xfffff8056e45cdd0, balance=3D2) >>>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 >>>> #7 0xffffffff8220de58 in avl_remove (tree=3D, >>>> data=3D) >>>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 >>>> #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :439 >>>> >>>> #9 0xffffffff822a2123 in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :907 >>>> >>>> #10 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e3f33d8= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #11 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #12 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e3f33d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #13 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #14 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #15 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e578000= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #16 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #17 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e57800= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #18 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #19 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #20 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ec3d8= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #21 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #22 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ec3d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #23 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #24 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #25 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e48a000= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #26 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #27 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e48a00= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #28 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #29 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #30 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e57ab88= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #31 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #32 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e57ab8= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #33 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #34 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #35 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69a3000= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #36 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #37 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69a300= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #38 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #39 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #40 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5c8000= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #41 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #42 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5c800= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #43 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #44 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #45 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23f9000= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #46 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #47 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23f900= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #48 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #49 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #50 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5147b0= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #51 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #52 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5147b= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #53 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #54 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #55 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e977000= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #56 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #57 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e97700= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #58 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #59 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #60 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90b3d8= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #61 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #62 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90b3d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #63 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #64 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #65 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b1160b88= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #66 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #67 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b1160b8= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #68 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #69 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #70 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4f53d8= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #71 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #72 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4f53d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #73 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #74 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #75 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8ebb88= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #76 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #77 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8ebb8= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #78 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #79 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #80 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8d9b88= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #81 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #82 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8d9b8= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #83 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #84 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #85 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21a03d8= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #86 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #87 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21a03d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #88 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #89 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #90 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e6903d8= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #91 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #92 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e6903d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #93 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #94 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #95 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a22dc7b0= ) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #96 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #97 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a22dc7b= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #98 0xffffffff822bbefd in zio_execute (zio=3D) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #99 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #100 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5fb7b= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #101 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #102 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5fb7= b0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #103 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #104 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #105 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ae3d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #106 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #107 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ae3= d8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #108 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #109 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #110 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a2131b8= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #111 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #112 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a2131b= 88) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #113 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #114 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #115 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e80b7b= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #116 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #117 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e80b7= b0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #118 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #119 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #120 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b14b900= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #121 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #122 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b14b90= 00) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #123 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #124 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #125 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21fa3d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #126 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #127 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21fa3= d8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #128 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #129 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #130 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5333d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #131 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #132 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5333= d8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #133 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #134 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #135 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90ab8= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #136 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #137 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90ab= 88) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #138 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #139 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #140 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4047b= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #141 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #142 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4047= b0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #143 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #144 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #145 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e7083d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #146 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #147 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e7083= d8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #148 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #149 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #150 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23917b= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #151 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #152 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23917= b0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #153 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #154 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #155 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e63600= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #156 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #157 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e6360= 00) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #158 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #159 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #160 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5afb8= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #161 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #162 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5afb= 88) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #163 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #164 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #165 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69523d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #166 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #167 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69523= d8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #168 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #169 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #170 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e47c00= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #171 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #172 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e47c0= 00) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #173 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #174 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #175 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b17b23d= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #176 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #177 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b17b23= d8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #178 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #179 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #180 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d367b= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #181 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #182 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a6d367= b0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #183 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #184 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #185 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e99fb8= 8) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #186 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #187 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e99fb= 88) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #188 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #189 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #190 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5227b= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #191 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #192 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5227= b0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>> #193 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #194 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>> out>) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c= :913 >>>> >>>> #195 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d3c00= 0) >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>> #196 0xffffffff822bbefd in zio_execute (zio=3D)= >>>> at >>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>> #197 0xffffffff80b4895a in taskqueue_run_locked (queue=3D>>> optimized >>>> out>) >>>> at /usr/src/sys/kern/subr_taskqueue.c:454 >>>> #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=3D>>> optimized out>) >>>> at /usr/src/sys/kern/subr_taskqueue.c:724 >>>> #199 0xffffffff80a9f255 in fork_exit ( >>>> callout=3D0xffffffff80b49a70 , >>>> arg=3D0xfffff8057caa72a0, frame=3D0xfffffe0668222c00) >>>> at /usr/src/sys/kern/kern_fork.c:1040 >>>> #200 0xffffffff80fb44ae in fork_trampoline () >>>> at /usr/src/sys/amd64/amd64/exception.S:611 >>>> #201 0x0000000000000000 in ?? () >>>> Current language: auto; currently minimal >>>> (kgdb) >>>> >>>> >>>>>> --=20 >>>>>> Karl Denninger >>>>>> karl@denninger.net >>>>>> /The Market Ticker/ >>>>>> /[S/MIME encrypted email preferred]/ >>> _______________________________________________ >>> freebsd-stable@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>> To unsubscribe, send any mail to >>> "freebsd-stable-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g" --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms020209010902040701010306 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEwMTcyMDIyNDRaME8GCSqGSIb3DQEJBDFCBEBD SxPkkVliNyJn5MJyl2y0Dd6m5ZhzAGpkqyjAImE47rnEdCi0PrUs4RmErcgrsRegUIapZoy0 PspD3M7ml3alMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAiXOqnbPF eFhqqO1oWvvvGhgRNUTxfwz33KiFTwW974YH40XzSAoMmgUVyq0GADp87h18R2snGkkWfa0a 24K7tVHAeGhoQvvMFLsaVeaSj9+ajSvP//4/Tl1u+8qPLynjyCmVkub3ORZCjFbgrE70vklm 57M1ylMjvFZQkOPU2U3UlQytiCrJbs8XVVdJfser0ckNkk81ZMq8ubZ/Ojx+8+dFK4Vn8p1E uXaKewjpylySMBqXF7U3Aua8mnMdOTYUSlqdAvM6ooL9NPcwdIfaCckAgIhY0PJvprIAJcmw 7q4oqCqyOXhZfGiS1fKpyMjD2x9C4crk5mTo4RtHnGw9tA5lRo3e/cYLwQPvp7ii1NlmGRHk CwWt4h9l8XAIX4G16CwHRFJtQ46ECjU/JGk3NBUQlPdLeGYeZUTKnOjc2X2nJfsNszlZPbHq rGF35En9MfhgmGb2W6DIei9fXcxgib+Xgt9J1YgPFQqMMHgesl5DW7hGpFx8MJuqaDyZBDv5 ZqzDN38DE4SOlSXYtn2eJYdpDt5mMM9cNNNUGRl3Lbc3J9DaIWucAAZCvI5uA4N1tKuYbaOe wOIu49GLec4SS+8KIsxKgrOp8ZOxboqRcq6odTMCO/7R1jAQP7tOFI+qXHG4Pm0CTPUNW7Rv fEpdbICwKGvtGRuCnX9WZnBvu30AAAAAAAA= --------------ms020209010902040701010306-- From owner-freebsd-stable@freebsd.org Mon Oct 17 21:40:14 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E323C163B1 for ; Mon, 17 Oct 2016 21:40:14 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-qk0-x229.google.com (mail-qk0-x229.google.com [IPv6:2607:f8b0:400d:c09::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA89374A for ; Mon, 17 Oct 2016 21:40:13 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-qk0-x229.google.com with SMTP id o68so306414440qkf.3 for ; Mon, 17 Oct 2016 14:40:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=04HHzCjli9/Nfb0+5yD57hYCwtRVlE8hibMim4W4Mcs=; b=oBJR66Ha2qrvxW6iK9paO5ynKaOtTkQit87Q2Vv3iq5clkEXQq+nkkAMgdI+5tkpwa vbo8417H3QO3GDefm14GfzwtypYPju7SNhN6pRKOwa0eIDxhJVGG5edgOV4F6HyodX2r QwA69xMR2RzEE7AU82qRwxRd5Ro1aVpCrx0KekMxqAfXtCWqj3S2ID2slExqKNgIob8Q zxW65ckfnyca0QMA2mnnozfBGM4aXCbSkd8h7meKVfnZKU8O5jZTZK0uhQd34H1kRKui FP3zM8cntybPCfrW6Mo9kzrxn8Egu+hjgvTeTvTOU+HojArlL9ZXrLx8jXUb7Ep+ljVC HMYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=04HHzCjli9/Nfb0+5yD57hYCwtRVlE8hibMim4W4Mcs=; b=dzEHLLn5/0DmsIEMsDfAUgapyCCFMj+pbU5h8fe+zpmKWNADFHNDJyMHGyKXRl8Utu zQfbnpn9sg/NXwEl+SrS2cAj0VjJXMp8/AxZHuU/lzWzYMFLR2KHn1cAOnwdvz5141az 3aqN9DW3iimyWa8QQkF1cntQAnFX5qsjEj/hRYfYYS7RIePdGYQjkZN/XMQcV+RLoUuc Yo4rXrWCmfPLwHoBjnXU59CY230CCj5kPFB7on/RiEp6mZCLi1/YBruuDiLFsVuqANCA 1V5RpeeRskS5voVOzYlK+VwJahybkaTtg59G00YR0SA9vRVFQlUe7qcFJAmMIQ0glBcC l58Q== X-Gm-Message-State: AA6/9Rlz1yja5M9JUDABY7S77haZyKY7Vy/mGY40T45mkdyNI6KKq91hON7139sdV9kW9A58 X-Received: by 10.194.103.41 with SMTP id ft9mr12356173wjb.8.1476740412246; Mon, 17 Oct 2016 14:40:12 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id w1sm5752607wje.36.2016.10.17.14.40.11 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Oct 2016 14:40:11 -0700 (PDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: freebsd-stable@freebsd.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> <76551fd6-0565-ee6c-b0f2-7d472ad6a4b3@denninger.net> <25ff3a3e-77a9-063b-e491-8d10a06e6ae2@multiplay.co.uk> <26e092b2-17c6-8744-5035-d0853d733870@denninger.net> From: Steven Hartland Message-ID: Date: Mon, 17 Oct 2016 22:40:37 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <26e092b2-17c6-8744-5035-d0853d733870@denninger.net> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 21:40:14 -0000 Setting those values will only effect what's queued to the device not what's actually outstanding. On 17/10/2016 21:22, Karl Denninger wrote: > Since I cleared it (by setting TRIM off on the test machine, rebooting, > importing the pool and noting that it did not panic -- pulled drives, > re-inserted into the production machine and ran backup routine -- all > was normal) it may be a while before I see it again (a week or so is usual.) > > It appears to be related to entropy in the filesystem that comes up as > "eligible" to be removed from the backup volume, which (not > surprisingly) tends to happen a few days after I do a new world build or > something similar (the daily and/or periodic snapshots roll off at about > that point.) > > I don't happen to have a spare pair of high-performance SSDs I can stick > in the sandbox machine in an attempt to force the condition to assert > itself in test, unfortunately. > > I *am* concerned that it's not "simple" stack exhaustion because setting > the max outstanding TRIMs on a per-vdev basis down quite-dramatically > did *not* prevent it from happening -- and if it was simply stack depth > related I would have expected that to put a stop to it. > > On 10/17/2016 15:16, Steven Hartland wrote: >> Be good to confirm its not an infinite loop by giving it a good bump >> first. >> >> On 17/10/2016 19:58, Karl Denninger wrote: >>> I can certainly attempt setting that higher.... but is that not just >>> hiding the problem rather than addressing it? >>> >>> >>> On 10/17/2016 13:54, Steven Hartland wrote: >>>> You're hitting stack exhaustion, have you tried increasing the kernel >>>> stack pages? >>>> It can be changed from /boot/loader.conf >>>> kern.kstack_pages="6" >>>> >>>> Default on amd64 is 4 IIRC >>>> >>>> On 17/10/2016 19:08, Karl Denninger wrote: >>>>> The target (and devices that trigger this) are a pair of 4Gb 7200RPM >>>>> SATA rotating rust drives (zmirror) with each provider geli-encrypted >>>>> (that is, the actual devices used for the pool create are the .eli's) >>>>> >>>>> The machine generating the problem has both rotating rust devices >>>>> *and* >>>>> SSDs, so I can't simply shut TRIM off system-wide and call it a day as >>>>> TRIM itself is heavily-used; both the boot/root pools and a Postgresql >>>>> database pool are on SSDs, while several terabytes of lesser-used data >>>>> is on a pool of Raidz2 that is made up of spinning rust. >>>> snip... >>>>> NewFS.denninger.net dumped core - see /var/crash/vmcore.1 >>>>> >>>>> Mon Oct 17 09:02:33 CDT 2016 >>>>> >>>>> FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 >>>>> r307318M: Fri Oct 14 09:23:46 CDT 2016 >>>>> karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 >>>>> >>>>> panic: double fault >>>>> >>>>> GNU gdb 6.1.1 [FreeBSD] >>>>> Copyright 2004 Free Software Foundation, Inc. >>>>> GDB is free software, covered by the GNU General Public License, and >>>>> you are >>>>> welcome to change it and/or distribute copies of it under certain >>>>> conditions. >>>>> Type "show copying" to see the conditions. >>>>> There is absolutely no warranty for GDB. Type "show warranty" for >>>>> details. >>>>> This GDB was configured as "amd64-marcel-freebsd"... >>>>> >>>>> Unread portion of the kernel message buffer: >>>>> >>>>> Fatal double fault >>>>> rip = 0xffffffff8220d9ec >>>>> rsp = 0xfffffe066821f000 >>>>> rbp = 0xfffffe066821f020 >>>>> cpuid = 6; apic id = 14 >>>>> panic: double fault >>>>> cpuid = 6 >>>>> KDB: stack backtrace: >>>>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >>>>> 0xfffffe0649d78e30 >>>>> vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 >>>>> panic() at panic+0x43/frame 0xfffffe0649d78f10 >>>>> dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30 >>>>> Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 >>>>> --- trap 0x17, rip = 0xffffffff8220d9ec, rsp = 0xfffffe066821f000, >>>>> rbp = >>>>> 0xfffffe066821f020 --- >>>>> avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 >>>>> avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 >>>>> vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame >>>>> 0xfffffe066821f530 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0x83/frame >>>>> 0xfffffe066821f570 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe066821f650 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe066821f6e0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe066821f7c0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe066821f850 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe066821f930 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe066821f9c0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe066821faa0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe066821fb30 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe066821fc10 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe066821fca0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe066821fd80 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe066821fe10 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe066821fef0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe066821ff80 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668220060 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe06682200f0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe06682201d0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668220260 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668220340 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe06682203d0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe06682204b0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668220540 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668220620 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe06682206b0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668220790 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668220820 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668220900 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668220990 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668220a70 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668220b00 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668220be0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668220c70 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668220d50 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668220de0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668220ec0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668220f50 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668221030 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe06682210c0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe06682211a0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668221230 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668221310 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe06682213a0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668221480 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668221510 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe06682215f0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668221680 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668221760 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe06682217f0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe06682218d0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668221960 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668221a40 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668221ad0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668221bb0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668221c40 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668221d20 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668221db0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668221e90 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668221f20 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668222000 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668222090 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668222170 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668222200 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe06682222e0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668222370 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668222450 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe06682224e0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe06682225c0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668222650 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668222730 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe06682227c0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe06682228a0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668222930 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 >>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>> 0xfffffe0668222a10 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 >>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>> 0xfffffe0668222aa0 >>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0 >>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 >>>>> taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame >>>>> 0xfffffe0668222b80 >>>>> taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame >>>>> 0xfffffe0668222bb0 >>>>> fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 >>>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 >>>>> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >>>>> Uptime: 4m29s >>>>> Dumping 1497 out of 24531 >>>>> MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% >>>>> >>>>> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >>>>> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/zfs.ko >>>>> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols >>>>> from >>>>> /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/opensolaris.ko >>>>> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from >>>>> /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/geom_eli.ko >>>>> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols >>>>> from >>>>> /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/geom_mirror.ko >>>>> Reading symbols from /boot/kernel/aesni.ko...Reading symbols from >>>>> /usr/lib/debug//boot/kernel/aesni.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/aesni.ko >>>>> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from >>>>> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/uhid.ko >>>>> Reading symbols from /boot/kernel/ums.ko...Reading symbols from >>>>> /usr/lib/debug//boot/kernel/ums.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/ums.ko >>>>> Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from >>>>> /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/ipfw.ko >>>>> Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from >>>>> /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/ipfw_nat.ko >>>>> Reading symbols from /boot/kernel/libalias.ko...Reading symbols from >>>>> /usr/lib/debug//boot/kernel/libalias.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/libalias.ko >>>>> Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols >>>>> from >>>>> /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. >>>>> done. >>>>> Loaded symbols for /boot/kernel/daemon_saver.ko >>>>> #0 doadump (textdump=1) at pcpu.h:221 >>>>> 221 pcpu.h: No such file or directory. >>>>> in pcpu.h >>>>> (kgdb) #0 doadump (textdump=1) at pcpu.h:221 >>>>> #1 0xffffffff80ae9b6e in kern_reboot (howto=260) >>>>> at /usr/src/sys/kern/kern_shutdown.c:366 >>>>> #2 0xffffffff80aea13b in vpanic (fmt=, >>>>> ap=) at >>>>> /usr/src/sys/kern/kern_shutdown.c:759 >>>>> #3 0xffffffff80ae9f73 in panic (fmt=0x0) >>>>> at /usr/src/sys/kern/kern_shutdown.c:690 >>>>> #4 0xffffffff80fd18f2 in dblfault_handler (frame=>>>> out>) >>>>> at /usr/src/sys/amd64/amd64/trap.c:868 >>>>> #5 0xffffffff80fb406c in Xdblfault () >>>>> at /usr/src/sys/amd64/amd64/exception.S:292 >>>>> #6 0xffffffff8220d9ec in avl_rotation (tree=0xfffff8004dc54510, >>>>> node=0xfffff8056e45cdd0, balance=2) >>>>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 >>>>> #7 0xffffffff8220de58 in avl_remove (tree=, >>>>> data=) >>>>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 >>>>> #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:439 >>>>> >>>>> #9 0xffffffff822a2123 in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:907 >>>>> >>>>> #10 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e3f33d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #11 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #12 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e3f33d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #13 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #14 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #15 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e578000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #16 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #17 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e578000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #18 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #19 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #20 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ec3d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #21 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #22 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ec3d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #23 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #24 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #25 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e48a000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #26 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #27 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e48a000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #28 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #29 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #30 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e57ab88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #31 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #32 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e57ab88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #33 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #34 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #35 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69a3000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #36 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #37 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69a3000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #38 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #39 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #40 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5c8000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #41 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #42 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5c8000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #43 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #44 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #45 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23f9000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #46 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #47 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23f9000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #48 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #49 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #50 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5147b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #51 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #52 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5147b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #53 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #54 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #55 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e977000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #56 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #57 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e977000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #58 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #59 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #60 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90b3d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #61 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #62 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90b3d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #63 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #64 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #65 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b1160b88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #66 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #67 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b1160b88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #68 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #69 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #70 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4f53d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #71 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #72 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4f53d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #73 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #74 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #75 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8ebb88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #76 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #77 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8ebb88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #78 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #79 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #80 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8d9b88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #81 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #82 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8d9b88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #83 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #84 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #85 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21a03d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #86 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #87 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21a03d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #88 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #89 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #90 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e6903d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #91 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #92 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e6903d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #93 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #94 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #95 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a22dc7b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #96 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #97 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a22dc7b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #98 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #99 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #100 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5fb7b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #101 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #102 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5fb7b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #103 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #104 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #105 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ae3d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #106 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #107 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ae3d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #108 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #109 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #110 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a2131b88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #111 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #112 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a2131b88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #113 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #114 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #115 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e80b7b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #116 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #117 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e80b7b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #118 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #119 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #120 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b14b9000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #121 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #122 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b14b9000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #123 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #124 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #125 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21fa3d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #126 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #127 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21fa3d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #128 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #129 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #130 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5333d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #131 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #132 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5333d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #133 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #134 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #135 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90ab88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #136 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #137 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90ab88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #138 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #139 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #140 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4047b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #141 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #142 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4047b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #143 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #144 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #145 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e7083d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #146 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #147 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e7083d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #148 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #149 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #150 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23917b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #151 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #152 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23917b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #153 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #154 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #155 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e636000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #156 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #157 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e636000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #158 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #159 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #160 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5afb88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #161 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #162 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5afb88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #163 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #164 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #165 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69523d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #166 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #167 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69523d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #168 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #169 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #170 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e47c000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #171 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #172 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e47c000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #173 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #174 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #175 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b17b23d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #176 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #177 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b17b23d8) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #178 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #179 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #180 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d367b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #181 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #182 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a6d367b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #183 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #184 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #185 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e99fb88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #186 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #187 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e99fb88) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #188 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #189 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #190 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5227b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #191 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #192 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5227b0) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>>>> #193 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #194 0xffffffff822a216d in vdev_queue_io_done (zio=>>>> out>) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>>>> >>>>> #195 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d3c000) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>>>> #196 0xffffffff822bbefd in zio_execute (zio=) >>>>> at >>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>>>> #197 0xffffffff80b4895a in taskqueue_run_locked (queue=>>>> optimized >>>>> out>) >>>>> at /usr/src/sys/kern/subr_taskqueue.c:454 >>>>> #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=>>>> optimized out>) >>>>> at /usr/src/sys/kern/subr_taskqueue.c:724 >>>>> #199 0xffffffff80a9f255 in fork_exit ( >>>>> callout=0xffffffff80b49a70 , >>>>> arg=0xfffff8057caa72a0, frame=0xfffffe0668222c00) >>>>> at /usr/src/sys/kern/kern_fork.c:1040 >>>>> #200 0xffffffff80fb44ae in fork_trampoline () >>>>> at /usr/src/sys/amd64/amd64/exception.S:611 >>>>> #201 0x0000000000000000 in ?? () >>>>> Current language: auto; currently minimal >>>>> (kgdb) >>>>> >>>>> >>>>>>> -- >>>>>>> Karl Denninger >>>>>>> karl@denninger.net >>>>>>> /The Market Ticker/ >>>>>>> /[S/MIME encrypted email preferred]/ >>>> _______________________________________________ >>>> freebsd-stable@freebsd.org mailing list >>>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>> To unsubscribe, send any mail to >>>> "freebsd-stable-unsubscribe@freebsd.org" >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Mon Oct 17 21:42:44 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0F2ABC16539 for ; Mon, 17 Oct 2016 21:42:44 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-lf0-x232.google.com (mail-lf0-x232.google.com [IPv6:2a00:1450:4010:c07::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D377AF5 for ; Mon, 17 Oct 2016 21:42:43 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-lf0-x232.google.com with SMTP id b81so305592871lfe.1 for ; Mon, 17 Oct 2016 14:42:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=fGeP9uIaf3fBB2QjlfBx/wWuhFifXuyLmza6hls5vbc=; b=x9u986PqGk13nDb6GDwX2g+JTgM0zjYqecwC0Pr84coElciAzBcPc0Ji1U73cYgL/t YMzxOzwZdyNFrncpl3aKzhkyWGUEJBM/ueK1aNPBzhWcKqsxbEb6zxHzd4bVXbMpeRfk ha1P14zxeOMZ0tlWFWpV4PH/XCjONFIhKGQqzK0X2vfWjbtIHMGbdaRzl1QhjKyV9Acv p8wOwUzbJw8A1nMV+EoB/ud0xW38McXN8BCxb8ytEWFcYXbQsxUETDquJhc0O9R3kagI va7//eyAS5UQG9X86444djZVNbrBrf6T1q9fgBuaI9HVZlqN1vvPpe/S6Yk7o3zXKvgR +Dbw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=fGeP9uIaf3fBB2QjlfBx/wWuhFifXuyLmza6hls5vbc=; b=SzioJvls5WBvl9pxKd+u4fuKXMz4PMh7OU8msAbgI6fTnbjSfprab0OAf8BP+2jF0L C0/VG1DlCH54RmO1xcPOMsvTqQnUB3/J+UcIG0cQUcDXMswjhZkjdmniEDjuGHI5Jeic R+yGKVOycEC9R08AmRtwxsFuqqM49ymE+rbjibtm3DVwAc/I8e3Ef551auW55vmGj1QJ ltPtpXxi24z7chSQPOkULhJe2gcdAq8YZk74/BOL8pLoUm+NleYnOwH8X84OES6Q+aAR rdlkdAyh3fpmN96FAfBXTrjxK+wh3IkN4Wxv/lNhGZWlR56gAqBqTJNHpi4VMtaO82Z3 7xfw== X-Gm-Message-State: AA6/9RlsSKISKEIqy1/k/w+edEn3K0Hsc15Q4wBHvwOdJZDN3AMXjPsbmpfhXWWBn/f6MWVD X-Received: by 10.28.98.67 with SMTP id w64mr4224946wmb.50.1476740560383; Mon, 17 Oct 2016 14:42:40 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id wn5sm14739314wjb.42.2016.10.17.14.42.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Oct 2016 14:42:39 -0700 (PDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: Andriy Gapon , freebsd-stable@FreeBSD.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> From: Steven Hartland Message-ID: Date: Mon, 17 Oct 2016 22:43:05 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 21:42:44 -0000 On 17/10/2016 20:52, Andriy Gapon wrote: > On 17/10/2016 21:54, Steven Hartland wrote: >> You're hitting stack exhaustion, have you tried increasing the kernel stack pages? >> It can be changed from /boot/loader.conf >> kern.kstack_pages="6" >> >> Default on amd64 is 4 IIRC > Steve, > > perhaps you can think of a more proper fix? :-) > https://lists.freebsd.org/pipermail/freebsd-stable/2016-July/085047.html Yes need to find some time to have a look at it, but given how rare this is and with TRIM being re-implemented upstream in a totally different manor I'm reticent to spend any real time on it. >> On 17/10/2016 19:08, Karl Denninger wrote: >>> The target (and devices that trigger this) are a pair of 4Gb 7200RPM >>> SATA rotating rust drives (zmirror) with each provider geli-encrypted >>> (that is, the actual devices used for the pool create are the .eli's) >>> >>> The machine generating the problem has both rotating rust devices *and* >>> SSDs, so I can't simply shut TRIM off system-wide and call it a day as >>> TRIM itself is heavily-used; both the boot/root pools and a Postgresql >>> database pool are on SSDs, while several terabytes of lesser-used data >>> is on a pool of Raidz2 that is made up of spinning rust. >> snip... >>> NewFS.denninger.net dumped core - see /var/crash/vmcore.1 >>> >>> Mon Oct 17 09:02:33 CDT 2016 >>> >>> FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 >>> r307318M: Fri Oct 14 09:23:46 CDT 2016 >>> karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 >>> >>> panic: double fault >>> >>> GNU gdb 6.1.1 [FreeBSD] >>> Copyright 2004 Free Software Foundation, Inc. >>> GDB is free software, covered by the GNU General Public License, and you are >>> welcome to change it and/or distribute copies of it under certain >>> conditions. >>> Type "show copying" to see the conditions. >>> There is absolutely no warranty for GDB. Type "show warranty" for details. >>> This GDB was configured as "amd64-marcel-freebsd"... >>> >>> Unread portion of the kernel message buffer: >>> >>> Fatal double fault >>> rip = 0xffffffff8220d9ec >>> rsp = 0xfffffe066821f000 >>> rbp = 0xfffffe066821f020 >>> cpuid = 6; apic id = 14 >>> panic: double fault >>> cpuid = 6 >>> KDB: stack backtrace: >>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >>> 0xfffffe0649d78e30 >>> vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 >>> panic() at panic+0x43/frame 0xfffffe0649d78f10 >>> dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f30 >>> Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 >>> --- trap 0x17, rip = 0xffffffff8220d9ec, rsp = 0xfffffe066821f000, rbp = >>> 0xfffffe066821f020 --- >>> avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 >>> avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 >>> vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame >>> 0xfffffe066821f530 >>> vdev_queue_io_done() at vdev_queue_io_done+0x83/frame 0xfffffe066821f570 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f650 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f6e0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f710 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f7c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f850 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f880 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821f930 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821f9c0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821faa0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fb30 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb60 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fc10 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fca0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fcd0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fd80 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821fe10 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe40 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe066821fef0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe066821ff80 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ffb0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220060 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682200f0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220120 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682201d0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220260 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220290 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220340 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682203d0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220400 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682204b0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220540 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220570 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220620 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682206b0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206e0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220790 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220820 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220850 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220900 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220990 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220a70 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220b00 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b30 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220be0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220c70 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220ca0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220d50 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220de0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e10 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668220ec0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668220f50 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f80 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221030 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682210c0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682211a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221230 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221260 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221310 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682213a0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213d0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221480 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221510 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221540 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682215f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221680 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216b0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221760 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682217f0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221820 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682218d0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221960 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221990 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221a40 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221ad0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b00 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221bb0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221c40 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c70 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221d20 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221db0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221de0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668221e90 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668221f20 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f50 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222000 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222090 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222170 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222200 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222230 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682222e0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222370 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222450 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682224e0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222510 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682225c0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222650 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222680 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222730 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe06682227c0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227f0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe06682228a0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222930 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222960 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 >>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame 0xfffffe0668222a10 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 >>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame 0xfffffe0668222aa0 >>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222ad0 >>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 >>> taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame >>> 0xfffffe0668222b80 >>> taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame >>> 0xfffffe0668222bb0 >>> fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 >>> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >>> Uptime: 4m29s >>> Dumping 1497 out of 24531 >>> MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% >>> >>> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/zfs.ko >>> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/opensolaris.ko >>> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/geom_eli.ko >>> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/geom_mirror.ko >>> Reading symbols from /boot/kernel/aesni.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/aesni.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/aesni.ko >>> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/uhid.ko >>> Reading symbols from /boot/kernel/ums.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/ums.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/ums.ko >>> Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/ipfw.ko >>> Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/ipfw_nat.ko >>> Reading symbols from /boot/kernel/libalias.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/libalias.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/libalias.ko >>> Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbols from >>> /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. >>> done. >>> Loaded symbols for /boot/kernel/daemon_saver.ko >>> #0 doadump (textdump=1) at pcpu.h:221 >>> 221 pcpu.h: No such file or directory. >>> in pcpu.h >>> (kgdb) #0 doadump (textdump=1) at pcpu.h:221 >>> #1 0xffffffff80ae9b6e in kern_reboot (howto=260) >>> at /usr/src/sys/kern/kern_shutdown.c:366 >>> #2 0xffffffff80aea13b in vpanic (fmt=, >>> ap=) at /usr/src/sys/kern/kern_shutdown.c:759 >>> #3 0xffffffff80ae9f73 in panic (fmt=0x0) >>> at /usr/src/sys/kern/kern_shutdown.c:690 >>> #4 0xffffffff80fd18f2 in dblfault_handler (frame=) >>> at /usr/src/sys/amd64/amd64/trap.c:868 >>> #5 0xffffffff80fb406c in Xdblfault () >>> at /usr/src/sys/amd64/amd64/exception.S:292 >>> #6 0xffffffff8220d9ec in avl_rotation (tree=0xfffff8004dc54510, >>> node=0xfffff8056e45cdd0, balance=2) >>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:304 >>> #7 0xffffffff8220de58 in avl_remove (tree=, >>> data=) >>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:818 >>> #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:439 >>> #9 0xffffffff822a2123 in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:907 >>> #10 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e3f33d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #11 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #12 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e3f33d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #13 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #14 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #15 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e578000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #16 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #17 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e578000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #18 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #19 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #20 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ec3d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #21 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #22 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ec3d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #23 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #24 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #25 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e48a000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #26 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #27 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e48a000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #28 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #29 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #30 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e57ab88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #31 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #32 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e57ab88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #33 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #34 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #35 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69a3000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #36 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #37 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69a3000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #38 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #39 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #40 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5c8000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #41 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #42 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5c8000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #43 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #44 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #45 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23f9000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #46 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #47 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23f9000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #48 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #49 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #50 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5147b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #51 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #52 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5147b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #53 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #54 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #55 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e977000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #56 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #57 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e977000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #58 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #59 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #60 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90b3d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #61 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #62 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90b3d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #63 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #64 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #65 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b1160b88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #66 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #67 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b1160b88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #68 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #69 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #70 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4f53d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #71 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #72 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4f53d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #73 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #74 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #75 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8ebb88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #76 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #77 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8ebb88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #78 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #79 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #80 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e8d9b88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #81 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #82 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e8d9b88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #83 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #84 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #85 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21a03d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #86 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #87 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21a03d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #88 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #89 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #90 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e6903d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #91 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #92 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e6903d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #93 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #94 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #95 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a22dc7b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #96 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #97 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a22dc7b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #98 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #99 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #100 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5fb7b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #101 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #102 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5fb7b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #103 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #104 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #105 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5ae3d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #106 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #107 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5ae3d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #108 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #109 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #110 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a2131b88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #111 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #112 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a2131b88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #113 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #114 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #115 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e80b7b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #116 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #117 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e80b7b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #118 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #119 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #120 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b14b9000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #121 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #122 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b14b9000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #123 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #124 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #125 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a21fa3d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #126 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #127 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a21fa3d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #128 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #129 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #130 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5333d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #131 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #132 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5333d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #133 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #134 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #135 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e90ab88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #136 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #137 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e90ab88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #138 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #139 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #140 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e4047b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #141 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #142 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e4047b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #143 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #144 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #145 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e7083d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #146 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #147 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e7083d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #148 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #149 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #150 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff802a23917b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #151 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #152 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff802a23917b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #153 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #154 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #155 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e636000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #156 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #157 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e636000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #158 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #159 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #160 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5afb88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #161 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #162 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5afb88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #163 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #164 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #165 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a69523d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #166 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #167 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a69523d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #168 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #169 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #170 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e47c000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #171 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #172 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e47c000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #173 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #174 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #175 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800b17b23d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #176 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #177 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800b17b23d8) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #178 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #179 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #180 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d367b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #181 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #182 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff800a6d367b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #183 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #184 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #185 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e99fb88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #186 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #187 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e99fb88) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #188 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #189 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #190 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff8056e5227b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #191 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #192 0xffffffff822bf75d in zio_vdev_io_start (zio=0xfffff8056e5227b0) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112 >>> #193 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #194 0xffffffff822a216d in vdev_queue_io_done (zio=) >>> at >>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:913 >>> #195 0xffffffff822bf919 in zio_vdev_io_done (zio=0xfffff800a6d3c000) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137 >>> #196 0xffffffff822bbefd in zio_execute (zio=) >>> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651 >>> #197 0xffffffff80b4895a in taskqueue_run_locked (queue=>> out>) >>> at /usr/src/sys/kern/subr_taskqueue.c:454 >>> #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=) >>> at /usr/src/sys/kern/subr_taskqueue.c:724 >>> #199 0xffffffff80a9f255 in fork_exit ( >>> callout=0xffffffff80b49a70 , >>> arg=0xfffff8057caa72a0, frame=0xfffffe0668222c00) >>> at /usr/src/sys/kern/kern_fork.c:1040 >>> #200 0xffffffff80fb44ae in fork_trampoline () >>> at /usr/src/sys/amd64/amd64/exception.S:611 >>> #201 0x0000000000000000 in ?? () >>> Current language: auto; currently minimal >>> (kgdb) >>> >>> >>>>> -- >>>>> Karl Denninger >>>>> karl@denninger.net >>>>> /The Market Ticker/ >>>>> /[S/MIME encrypted email preferred]/ >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" >> > From owner-freebsd-stable@freebsd.org Mon Oct 17 21:50:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41EF2C16749 for ; Mon, 17 Oct 2016 21:50:05 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 98BD2AF for ; Mon, 17 Oct 2016 21:50:04 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 31D5F303394 for ; Mon, 17 Oct 2016 16:50:02 -0500 (CDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: freebsd-stable@freebsd.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> <76551fd6-0565-ee6c-b0f2-7d472ad6a4b3@denninger.net> <25ff3a3e-77a9-063b-e491-8d10a06e6ae2@multiplay.co.uk> <26e092b2-17c6-8744-5035-d0853d733870@denninger.net> From: Karl Denninger Message-ID: Date: Mon, 17 Oct 2016 16:50:00 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms090702050402030004080907" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 21:50:05 -0000 This is a cryptographically signed message in MIME format. --------------ms090702050402030004080907 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable I will make some effort on the sandbox machine to see if I can come up with a way to replicate this. I do have plenty of spare larger drives laying around that used to be in service and were obsolesced due to capacity -- but what I don't know if whether the system will misbehave if the source is all spinning rust. In other words: 1. Root filesystem is mirrored spinning rust (production is mirrored SSDs= ) 2. Backup is mirrored spinning rust (of approx the same size) 3. Set up auto-snapshot exactly as the production system has now (which the sandbox is NOT since I don't care about incremental recovery on that machine; it's a sandbox!) 4. Run a bunch of build-somethings (e.g. buildworlds, cross-build for the Pi2s I have here, etc) to generate a LOT of filesystem entropy across lots of snapshots. 5. Back that up. 6. Export the backup pool. 7. Re-import it and "zfs destroy -r" the backup filesystem. That is what got me in a reboot loop after the *first* panic; I was simply going to destroy the backup filesystem and re-run the backup, but as soon as I issued that zfs destroy the machine panic'd and as soon as I re-attached it after a reboot it panic'd again. Repeat until I set trim=3D0. But... if I CAN replicate it that still shouldn't be happening, and the system should *certainly* survive attempting to TRIM on a vdev that doesn't support TRIMs, even if the removal is for a large amount of space and/or files on the target, without blowing up. BTW I bet it isn't that rare -- if you're taking timed snapshots on an active filesystem (with lots of entropy) and then make the mistake of trying to remove those snapshots (as is the case with a zfs destroy -r or a zfs recv of an incremental copy that attempts to sync against a source) on a pool that has been imported before the system realizes that TRIM is unavailable on those vdevs. Noting this: Yes need to find some time to have a look at it, but given how rare this is and with TRIM being re-implemented upstream in a totally different manor I'm reticent to spend any real time on it. What's in-process in this regard, if you happen to have a reference? On 10/17/2016 16:40, Steven Hartland wrote: > Setting those values will only effect what's queued to the device not > what's actually outstanding. > > On 17/10/2016 21:22, Karl Denninger wrote: >> Since I cleared it (by setting TRIM off on the test machine, rebooting= , >> importing the pool and noting that it did not panic -- pulled drives, >> re-inserted into the production machine and ran backup routine -- all >> was normal) it may be a while before I see it again (a week or so is >> usual.) >> >> It appears to be related to entropy in the filesystem that comes up as= >> "eligible" to be removed from the backup volume, which (not >> surprisingly) tends to happen a few days after I do a new world build = or >> something similar (the daily and/or periodic snapshots roll off at abo= ut >> that point.) >> >> I don't happen to have a spare pair of high-performance SSDs I can sti= ck >> in the sandbox machine in an attempt to force the condition to assert >> itself in test, unfortunately. >> >> I *am* concerned that it's not "simple" stack exhaustion because setti= ng >> the max outstanding TRIMs on a per-vdev basis down quite-dramatically >> did *not* prevent it from happening -- and if it was simply stack dept= h >> related I would have expected that to put a stop to it. >> >> On 10/17/2016 15:16, Steven Hartland wrote: >>> Be good to confirm its not an infinite loop by giving it a good bump >>> first. >>> >>> On 17/10/2016 19:58, Karl Denninger wrote: >>>> I can certainly attempt setting that higher.... but is that not just= >>>> hiding the problem rather than addressing it? >>>> >>>> >>>> On 10/17/2016 13:54, Steven Hartland wrote: >>>>> You're hitting stack exhaustion, have you tried increasing the kern= el >>>>> stack pages? >>>>> It can be changed from /boot/loader.conf >>>>> kern.kstack_pages=3D"6" >>>>> >>>>> Default on amd64 is 4 IIRC >>>>> >>>>> On 17/10/2016 19:08, Karl Denninger wrote: >>>>>> The target (and devices that trigger this) are a pair of 4Gb 7200R= PM >>>>>> SATA rotating rust drives (zmirror) with each provider >>>>>> geli-encrypted >>>>>> (that is, the actual devices used for the pool create are the >>>>>> .eli's) >>>>>> >>>>>> The machine generating the problem has both rotating rust devices >>>>>> *and* >>>>>> SSDs, so I can't simply shut TRIM off system-wide and call it a >>>>>> day as >>>>>> TRIM itself is heavily-used; both the boot/root pools and a >>>>>> Postgresql >>>>>> database pool are on SSDs, while several terabytes of lesser-used >>>>>> data >>>>>> is on a pool of Raidz2 that is made up of spinning rust. >>>>> snip... >>>>>> NewFS.denninger.net dumped core - see /var/crash/vmcore.1 >>>>>> >>>>>> Mon Oct 17 09:02:33 CDT 2016 >>>>>> >>>>>> FreeBSD NewFS.denninger.net 11.0-STABLE FreeBSD 11.0-STABLE #13 >>>>>> r307318M: Fri Oct 14 09:23:46 CDT 2016 >>>>>> karl@NewFS.denninger.net:/usr/obj/usr/src/sys/KSD-SMP amd64 >>>>>> >>>>>> panic: double fault >>>>>> >>>>>> GNU gdb 6.1.1 [FreeBSD] >>>>>> Copyright 2004 Free Software Foundation, Inc. >>>>>> GDB is free software, covered by the GNU General Public License, a= nd >>>>>> you are >>>>>> welcome to change it and/or distribute copies of it under certain >>>>>> conditions. >>>>>> Type "show copying" to see the conditions. >>>>>> There is absolutely no warranty for GDB. Type "show warranty" for= >>>>>> details. >>>>>> This GDB was configured as "amd64-marcel-freebsd"... >>>>>> >>>>>> Unread portion of the kernel message buffer: >>>>>> >>>>>> Fatal double fault >>>>>> rip =3D 0xffffffff8220d9ec >>>>>> rsp =3D 0xfffffe066821f000 >>>>>> rbp =3D 0xfffffe066821f020 >>>>>> cpuid =3D 6; apic id =3D 14 >>>>>> panic: double fault >>>>>> cpuid =3D 6 >>>>>> KDB: stack backtrace: >>>>>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >>>>>> 0xfffffe0649d78e30 >>>>>> vpanic() at vpanic+0x182/frame 0xfffffe0649d78eb0 >>>>>> panic() at panic+0x43/frame 0xfffffe0649d78f10 >>>>>> dblfault_handler() at dblfault_handler+0xa2/frame 0xfffffe0649d78f= 30 >>>>>> Xdblfault() at Xdblfault+0xac/frame 0xfffffe0649d78f30 >>>>>> --- trap 0x17, rip =3D 0xffffffff8220d9ec, rsp =3D 0xfffffe066821f= 000, >>>>>> rbp =3D >>>>>> 0xfffffe066821f020 --- >>>>>> avl_rotation() at avl_rotation+0xc/frame 0xfffffe066821f020 >>>>>> avl_remove() at avl_remove+0x1c8/frame 0xfffffe066821f070 >>>>>> vdev_queue_io_to_issue() at vdev_queue_io_to_issue+0x87f/frame >>>>>> 0xfffffe066821f530 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0x83/frame >>>>>> 0xfffffe066821f570 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f5= a0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f5f0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe066821f650 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f6a0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe066821f6e0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f7= 10 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f760 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe066821f7c0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f810 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe066821f850 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f8= 80 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f8d0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe066821f930 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821f980 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe066821f9c0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821f9= f0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fa40 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe066821faa0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821faf0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe066821fb30 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fb= 60 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fbb0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe066821fc10 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fc60 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe066821fca0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fc= d0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fd20 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe066821fd80 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fdd0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe066821fe10 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821fe= 40 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821fe90 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe066821fef0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe066821ff40 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe066821ff80 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe066821ff= b0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220000 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668220060 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682200b0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe06682200f0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682201= 20 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220170 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe06682201d0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220220 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668220260 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682202= 90 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682202e0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668220340 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220390 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe06682203d0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682204= 00 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220450 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe06682204b0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220500 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668220540 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682205= 70 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682205c0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668220620 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220670 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe06682206b0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682206= e0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220730 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668220790 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682207e0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668220820 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682208= 50 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682208a0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668220900 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220950 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668220990 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682209= c0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220a10 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668220a70 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220ac0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668220b00 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220b= 30 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220b80 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668220be0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220c30 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668220c70 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220c= a0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220cf0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668220d50 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220da0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668220de0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220e= 10 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220e60 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668220ec0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220f10 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668220f50 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668220f= 80 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668220fd0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668221030 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221080 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe06682210c0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682210= f0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221140 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe06682211a0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682211f0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668221230 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682212= 60 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682212b0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668221310 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221360 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe06682213a0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682213= d0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221420 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668221480 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682214d0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668221510 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682215= 40 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221590 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe06682215f0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221640 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668221680 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682216= b0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221700 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668221760 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682217b0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe06682217f0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682218= 20 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221870 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe06682218d0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221920 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668221960 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682219= 90 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682219e0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668221a40 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221a90 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668221ad0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221b= 00 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221b50 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668221bb0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221c00 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668221c40 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221c= 70 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221cc0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668221d20 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221d70 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668221db0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221d= e0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221e30 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668221e90 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221ee0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668221f20 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668221f= 50 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668221fa0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668222000 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222050 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668222090 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682220= c0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222110 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668222170 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682221c0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668222200 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682222= 30 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222280 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe06682222e0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222330 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668222370 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682223= a0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682223f0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668222450 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682224a0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe06682224e0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682225= 10 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222560 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe06682225c0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222610 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668222650 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682226= 80 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682226d0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668222730 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222780 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe06682227c0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682227= f0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222840 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe06682228a0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682228f0 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668222930 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe06682229= 60 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe06682229b0 >>>>>> zio_vdev_io_start() at zio_vdev_io_start+0x34d/frame >>>>>> 0xfffffe0668222a10 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222a60 >>>>>> vdev_queue_io_done() at vdev_queue_io_done+0xcd/frame >>>>>> 0xfffffe0668222aa0 >>>>>> zio_vdev_io_done() at zio_vdev_io_done+0xd9/frame 0xfffffe0668222a= d0 >>>>>> zio_execute() at zio_execute+0x23d/frame 0xfffffe0668222b20 >>>>>> taskqueue_run_locked() at taskqueue_run_locked+0x14a/frame >>>>>> 0xfffffe0668222b80 >>>>>> taskqueue_thread_loop() at taskqueue_thread_loop+0xe8/frame >>>>>> 0xfffffe0668222bb0 >>>>>> fork_exit() at fork_exit+0x85/frame 0xfffffe0668222bf0 >>>>>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0668222bf0 >>>>>> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >>>>>> Uptime: 4m29s >>>>>> Dumping 1497 out of 24531 >>>>>> MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% >>>>>> >>>>>> Reading symbols from /boot/kernel/zfs.ko...Reading symbols from >>>>>> /usr/lib/debug//boot/kernel/zfs.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/zfs.ko >>>>>> Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols= >>>>>> from >>>>>> /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/opensolaris.ko >>>>>> Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols fr= om >>>>>> /usr/lib/debug//boot/kernel/geom_eli.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/geom_eli.ko >>>>>> Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols= >>>>>> from >>>>>> /usr/lib/debug//boot/kernel/geom_mirror.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/geom_mirror.ko >>>>>> Reading symbols from /boot/kernel/aesni.ko...Reading symbols from >>>>>> /usr/lib/debug//boot/kernel/aesni.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/aesni.ko >>>>>> Reading symbols from /boot/kernel/uhid.ko...Reading symbols from >>>>>> /usr/lib/debug//boot/kernel/uhid.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/uhid.ko >>>>>> Reading symbols from /boot/kernel/ums.ko...Reading symbols from >>>>>> /usr/lib/debug//boot/kernel/ums.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/ums.ko >>>>>> Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from >>>>>> /usr/lib/debug//boot/kernel/ipfw.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/ipfw.ko >>>>>> Reading symbols from /boot/kernel/ipfw_nat.ko...Reading symbols fr= om >>>>>> /usr/lib/debug//boot/kernel/ipfw_nat.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/ipfw_nat.ko >>>>>> Reading symbols from /boot/kernel/libalias.ko...Reading symbols fr= om >>>>>> /usr/lib/debug//boot/kernel/libalias.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/libalias.ko >>>>>> Reading symbols from /boot/kernel/daemon_saver.ko...Reading symbol= s >>>>>> from >>>>>> /usr/lib/debug//boot/kernel/daemon_saver.ko.debug...done. >>>>>> done. >>>>>> Loaded symbols for /boot/kernel/daemon_saver.ko >>>>>> #0 doadump (textdump=3D1) at pcpu.h:221 >>>>>> 221 pcpu.h: No such file or directory. >>>>>> in pcpu.h >>>>>> (kgdb) #0 doadump (textdump=3D1) at pcpu.h:221 >>>>>> #1 0xffffffff80ae9b6e in kern_reboot (howto=3D260) >>>>>> at /usr/src/sys/kern/kern_shutdown.c:366 >>>>>> #2 0xffffffff80aea13b in vpanic (fmt=3D, >>>>>> ap=3D) at >>>>>> /usr/src/sys/kern/kern_shutdown.c:759 >>>>>> #3 0xffffffff80ae9f73 in panic (fmt=3D0x0) >>>>>> at /usr/src/sys/kern/kern_shutdown.c:690 >>>>>> #4 0xffffffff80fd18f2 in dblfault_handler (frame=3D>>>>> out>) >>>>>> at /usr/src/sys/amd64/amd64/trap.c:868 >>>>>> #5 0xffffffff80fb406c in Xdblfault () >>>>>> at /usr/src/sys/amd64/amd64/exception.S:292 >>>>>> #6 0xffffffff8220d9ec in avl_rotation (tree=3D0xfffff8004dc54510,= >>>>>> node=3D0xfffff8056e45cdd0, balance=3D2) >>>>>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:3= 04 >>>>>> #7 0xffffffff8220de58 in avl_remove (tree=3D= , >>>>>> data=3D) >>>>>> at /usr/src/sys/cddl/contrib/opensolaris/common/avl/avl.c:8= 18 >>>>>> #8 0xffffffff822a1eff in vdev_queue_io_to_issue (vq=3D>>>>> optimized >>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:439 >>>>>> >>>>>> >>>>>> #9 0xffffffff822a2123 in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:907 >>>>>> >>>>>> >>>>>> #10 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e3f33= d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #11 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #12 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e3f3= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #13 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #14 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #15 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5780= 00) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #16 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #17 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e578= 000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #18 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #19 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #20 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ec3= d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #21 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #22 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5ec= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #23 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #24 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #25 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e48a0= 00) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #26 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #27 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e48a= 000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #28 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #29 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #30 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e57ab= 88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #31 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #32 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e57a= b88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #33 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #34 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #35 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a69a30= 00) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #36 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #37 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800a69a3= 000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #38 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #39 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #40 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5c80= 00) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #41 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #42 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e5c8= 000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #43 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #44 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #45 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a23f90= 00) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #46 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #47 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a23f9= 000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #48 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #49 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #50 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5147= b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #51 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #52 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e514= 7b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #53 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #54 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #55 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e9770= 00) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #56 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #57 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e977= 000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #58 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #59 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #60 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90b3= d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #61 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #62 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e90b= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #63 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #64 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #65 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b1160b= 88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #66 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #67 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff800b1160= b88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #68 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #69 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #70 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e4f53= d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #71 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #72 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e4f5= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #73 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #74 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #75 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8ebb= 88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #76 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #77 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8eb= b88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #78 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #79 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #80 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e8d9b= 88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #81 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #82 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e8d9= b88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #83 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #84 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #85 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21a03= d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #86 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #87 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a21a0= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #88 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #89 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #90 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e6903= d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #91 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #92 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff8056e690= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #93 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #94 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #95 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a22dc7= b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #96 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #97 0xffffffff822bf75d in zio_vdev_io_start (zio=3D0xfffff802a22dc= 7b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #98 0xffffffff822bbefd in zio_execute (zio=3D= ) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #99 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #100 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5fb= 7b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #101 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #102 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e5fb7b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #103 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #104 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #105 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5ae= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #106 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #107 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e5ae3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #108 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #109 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #110 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a2131= b88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #111 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #112 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff802a2131b88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #113 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #114 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #115 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e80b= 7b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #116 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #117 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e80b7b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #118 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #119 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #120 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b14b9= 000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #121 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #122 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff800b14b9000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #123 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #124 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #125 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a21fa= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #126 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #127 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff802a21fa3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #128 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #129 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #130 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e533= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #131 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #132 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e5333d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #133 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #134 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #135 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e90a= b88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #136 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #137 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e90ab88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #138 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #139 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #140 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e404= 7b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #141 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #142 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e4047b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #143 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #144 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #145 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e708= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #146 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #147 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e7083d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #148 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #149 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #150 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff802a2391= 7b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #151 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #152 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff802a23917b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #153 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #154 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #155 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e636= 000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #156 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #157 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e636000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #158 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #159 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #160 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e5af= b88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #161 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #162 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e5afb88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #163 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #164 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #165 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6952= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #166 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #167 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff800a69523d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #168 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #169 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #170 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e47c= 000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #171 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #172 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e47c000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #173 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #174 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #175 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800b17b2= 3d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #176 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #177 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff800b17b23d8) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #178 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #179 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #180 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d36= 7b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #181 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #182 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff800a6d367b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #183 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #184 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #185 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e99f= b88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #186 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #187 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e99fb88) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #188 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #189 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #190 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff8056e522= 7b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #191 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #192 0xffffffff822bf75d in zio_vdev_io_start >>>>>> (zio=3D0xfffff8056e5227b0) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3112= >>>>>> #193 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #194 0xffffffff822a216d in vdev_queue_io_done (zio=3D>>>>> out>) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue= =2Ec:913 >>>>>> >>>>>> >>>>>> #195 0xffffffff822bf919 in zio_vdev_io_done (zio=3D0xfffff800a6d3c= 000) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:3137= >>>>>> #196 0xffffffff822bbefd in zio_execute (zio=3D) >>>>>> at >>>>>> /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1651= >>>>>> #197 0xffffffff80b4895a in taskqueue_run_locked (queue=3D>>>>> optimized >>>>>> out>) >>>>>> at /usr/src/sys/kern/subr_taskqueue.c:454 >>>>>> #198 0xffffffff80b49b58 in taskqueue_thread_loop (arg=3D>>>>> optimized out>) >>>>>> at /usr/src/sys/kern/subr_taskqueue.c:724 >>>>>> #199 0xffffffff80a9f255 in fork_exit ( >>>>>> callout=3D0xffffffff80b49a70 , >>>>>> arg=3D0xfffff8057caa72a0, frame=3D0xfffffe0668222c00) >>>>>> at /usr/src/sys/kern/kern_fork.c:1040 >>>>>> #200 0xffffffff80fb44ae in fork_trampoline () >>>>>> at /usr/src/sys/amd64/amd64/exception.S:611 >>>>>> #201 0x0000000000000000 in ?? () >>>>>> Current language: auto; currently minimal >>>>>> (kgdb) >>>>>> >>>>>> >>>>>>>> --=20 >>>>>>>> Karl Denninger >>>>>>>> karl@denninger.net >>>>>>>> /The Market Ticker/ >>>>>>>> /[S/MIME encrypted email preferred]/ >>>>> _______________________________________________ >>>>> freebsd-stable@freebsd.org mailing list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>>>> To unsubscribe, send any mail to >>>>> "freebsd-stable-unsubscribe@freebsd.org" >>> _______________________________________________ >>> freebsd-stable@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >>> To unsubscribe, send any mail to >>> "freebsd-stable-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g" --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms090702050402030004080907 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEwMTcyMTUwMDBaME8GCSqGSIb3DQEJBDFCBEDD 2raSvQ0MtHDlD4uLkgruGKirfquUn7LlpM/qO97f/qJNrx333Lof5XtbxK67zfk+J2xkVx1m OXPtobCMsrNLMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAZtlArLyl 91aIQgIxdD5WePp0kHB5nE7IpzE9Hu1sXIzzALqsIma1yktQodOh0Aql/FjOS8Y777fP2nbl WYPknpLKq6q6Ixfrj0Di8tAMKas4x8XXQH8jT0nRpVqmfWlL6TgyKkSFp68wCnUmoewOMGx8 SyD1L0UjH10WORA0uQ11m2YdqW8onHVF5kAxfl09vMVWzNgGLiLUbfUbx4eZPqLTAaVtvMfC Ar7NKgjueZe2p1DkzyBnipjo3pe1sfGolc5+Zleo4Jf/82Q7zxPYp8flIvWDRxGIRJk2LYsu U7SxNkW/pK+/H9oX4pHZ6nYxK6dwdIqzd0EoFMPF6A3i5U4i8O390RH5v7cvDLv8MgvONOpy 3AKJ9w+qx8sNb8+ZtxfjNu2N+2u3xunV0AiVqc//87HuiUpqzlUXL0nqMka3Ca79CsYi6rI7 WADXiV0hWix3L3KnD0+dg8pGPm5TRAaJzjXu0ckht+1J8orHiwYqCFalhCe/mEmeKgiHhRA5 idTcXsAald/PSYovs8pOEkqPycyV+7obKUFYyhB51kOhhENncvTpv5YLcMd24gB04Fq/101b AOZWjCbM44o1rBiNy1vGfE0rjYH8QhPEe/SbYv3qETlC1f3+CMEiLTnFZmvI1P5iFCoxtzfz g1mXteoG4MaGeuplvXeLlp/gP1EAAAAAAAA= --------------ms090702050402030004080907-- From owner-freebsd-stable@freebsd.org Mon Oct 17 23:31:57 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBC0EC16866 for ; Mon, 17 Oct 2016 23:31:57 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-lf0-x22f.google.com (mail-lf0-x22f.google.com [IPv6:2a00:1450:4010:c07::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6170E9A0 for ; Mon, 17 Oct 2016 23:31:57 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-lf0-x22f.google.com with SMTP id x79so320908701lff.0 for ; Mon, 17 Oct 2016 16:31:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=voXqfP7frQ1xhQfT5If+ur5L0D6MfyvWeQPwdwj7hE0=; b=W+8Y0B9Tac/OjptU7d/qiIWlBRL1GhXyPbeNN66MucRqbtgJoDlkf2ZTUsRRRAg7oH ohxucvMhS8DJb9Qy+xWivHP1NWMScYEmO7GeFidj0yx1FNbSoOi0AQEQ+KLQOjwUxyDc o/wt20TDPeLR/h3Yvpd67t7ftllQYkV8laB3lhgrWK01806lzjhS+wuCFQmsD1ek1WBg uZbGTRSLR0lVtCxahf7BCr20MWTpxRmOmCCKDmf1CmB6bkEcu08T3JnKKo3QQ/FBO3Hj saxJdHCBF5/c/HY29vAb1bQCQn5cG/PiRhHNWWTR0KUOp3upmVsDH2PSjeD+mH1iWt2C JhJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=voXqfP7frQ1xhQfT5If+ur5L0D6MfyvWeQPwdwj7hE0=; b=l0XXMHsZ8buYLwGvQ3bnnA3LNZeLYDoWUhPKtRADMNnUZPZRU0pQd5JcoxA6lXdMyG VDCC8np6gKW7obtSQlSgzXxub9LfZIi3HnhV3yMhwD2D0HaTprQ4YfPotwznKOShBPfJ vSiXbf7O/hzTLZQwjCQ52EVWjGIG8a8qWGJNE0Z/BPsdhm3m/zpmAsIJs/3mO61kyoLp /CaVzJ4ru+xrHpoGGaFzhA+vtXJ997wxQRwYw1f41Pl9Hj06O8SHXpLmY8QazOcVP8z6 QoKLgZc7fNLmoqIZZXLn+RH+xFqiOFrBH87ZrSKnHfWT41S6hKgywsDonVKhuK7mTvr8 Dr6w== X-Gm-Message-State: AA6/9RkdJDeGYW74ShDxkJy4jU3ufX8/YHObpxWUmfixwXU0wvuBBf+46SSfsBUyh0RZmDTJ X-Received: by 10.28.54.71 with SMTP id d68mr9336426wma.63.1476747114966; Mon, 17 Oct 2016 16:31:54 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id j1sm46812914wjl.21.2016.10.17.16.31.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 17 Oct 2016 16:31:54 -0700 (PDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: freebsd-stable@freebsd.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> <76551fd6-0565-ee6c-b0f2-7d472ad6a4b3@denninger.net> <25ff3a3e-77a9-063b-e491-8d10a06e6ae2@multiplay.co.uk> <26e092b2-17c6-8744-5035-d0853d733870@denninger.net> From: Steven Hartland Message-ID: <4d4909b7-c44b-996e-90e1-ca446e8e4813@multiplay.co.uk> Date: Tue, 18 Oct 2016 00:32:20 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 17 Oct 2016 23:31:57 -0000 On 17/10/2016 22:50, Karl Denninger wrote: > I will make some effort on the sandbox machine to see if I can come up > with a way to replicate this. I do have plenty of spare larger drives > laying around that used to be in service and were obsolesced due to > capacity -- but what I don't know if whether the system will misbehave > if the source is all spinning rust. > > In other words: > > 1. Root filesystem is mirrored spinning rust (production is mirrored SSDs) > > 2. Backup is mirrored spinning rust (of approx the same size) > > 3. Set up auto-snapshot exactly as the production system has now (which > the sandbox is NOT since I don't care about incremental recovery on that > machine; it's a sandbox!) > > 4. Run a bunch of build-somethings (e.g. buildworlds, cross-build for > the Pi2s I have here, etc) to generate a LOT of filesystem entropy > across lots of snapshots. > > 5. Back that up. > > 6. Export the backup pool. > > 7. Re-import it and "zfs destroy -r" the backup filesystem. > > That is what got me in a reboot loop after the *first* panic; I was > simply going to destroy the backup filesystem and re-run the backup, but > as soon as I issued that zfs destroy the machine panic'd and as soon as > I re-attached it after a reboot it panic'd again. Repeat until I set > trim=0. > > But... if I CAN replicate it that still shouldn't be happening, and the > system should *certainly* survive attempting to TRIM on a vdev that > doesn't support TRIMs, even if the removal is for a large amount of > space and/or files on the target, without blowing up. > > BTW I bet it isn't that rare -- if you're taking timed snapshots on an > active filesystem (with lots of entropy) and then make the mistake of > trying to remove those snapshots (as is the case with a zfs destroy -r > or a zfs recv of an incremental copy that attempts to sync against a > source) on a pool that has been imported before the system realizes that > TRIM is unavailable on those vdevs. > > Noting this: > > Yes need to find some time to have a look at it, but given how rare > this is and with TRIM being re-implemented upstream in a totally > different manor I'm reticent to spend any real time on it. > > What's in-process in this regard, if you happen to have a reference? Looks like it may be still in review: https://reviews.csiden.org/r/263/ From owner-freebsd-stable@freebsd.org Tue Oct 18 01:47:11 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 487C0C153E2 for ; Tue, 18 Oct 2016 01:47:11 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 381407DF; Tue, 18 Oct 2016 01:47:11 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 7D15AD4; Tue, 18 Oct 2016 01:47:11 +0000 (UTC) Date: Tue, 18 Oct 2016 01:47:11 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <676828102.3.1476755231436.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <1652629877.2.1476733695915.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1652629877.2.1476733695915.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #429 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 01:47:11 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/429/ From owner-freebsd-stable@freebsd.org Tue Oct 18 06:20:45 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9E309C16DFF for ; Tue, 18 Oct 2016 06:20:45 +0000 (UTC) (envelope-from kiri@kx.openedu.org) Received: from kx.openedu.org (flets-sg1027.kamome.or.jp [202.216.24.27]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 454CD194 for ; Tue, 18 Oct 2016 06:20:43 +0000 (UTC) (envelope-from kiri@kx.openedu.org) Received: from kx.openedu.org (kx.openedu.org [103.15.187.19]) by kx.openedu.org (8.14.5/8.14.5) with ESMTP id u9I61ERr072100; Tue, 18 Oct 2016 15:01:15 +0900 (JST) (envelope-from kiri@kx.openedu.org) Message-Id: <201610180601.u9I61ERr072100@kx.openedu.org> Date: Tue, 18 Oct 2016 15:01:14 +0900 From: KIRIYAMA Kazuhiko To: Marko =?ISO-8859-2?Q?Cupa=E6?= Cc: Oliver Peter , freebsd-stable@freebsd.org Subject: Re: moving ezjail-based jails from 10.3 host to 11.0 host In-Reply-To: <20161017174325.49386b19@efreet.mimar.rs> References: <20161017153708.411492e5@efreet.mimar.rs> <20161017135507.GA10396@mail.opdns.de> <20161017174325.49386b19@efreet.mimar.rs> User-Agent: Wanderlust/2.14.0 (Africa) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-8859-4?Q?Goj=F2?=) APEL/10.6 MULE XEmacs/21.4 (patch 22) (Instant Classic) (amd64--freebsd) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 06:20:45 -0000 At Mon, 17 Oct 2016 17:43:25 +0200, Marko Cupa=E6 wrote: >=20 > On Mon, 17 Oct 2016 15:55:07 +0200 > Oliver Peter wrote: >=20 > > On Mon, Oct 17, 2016 at 03:37:08PM +0200, Marko Cupa=E6 wrote: > > > I have 10.3 host which runs a dozen or so ezjail-based jails. I have > > > installed another 11.0 host, and I'd like to move jails to it. =20 > >=20 > > I would switch them to iocage+zfs, ezjail is sooo 90s. :) > > Have a look at the documentation: > > http://iocage.readthedocs.io/en/latest/basic-use.html > >=20 > > All jail settings are stored within ZFS properties so an upcoming > > migration would only need a zfs send | zfs receive. > >=20 > > > Can I just archive jails on 10.3, scp them to 11.0, and re-create > > > them there by restoring from archive (-a switch)? =20 > >=20 > > Further I would recommend to use rsync -av instead of scp. >=20 > Oliver, >=20 > I do appreciate you took the time to respond to my question. However, > when I asked how to move my ezjail-based jails, I meant exactly that. > I did not ask which is the best jail management system (for my use case > I'm completely fine with ezjail), or about the good and bad things > about ZFS (I have hardware RAID controller which can't do JBOD on this > server), or advantages of rsync over scp (doesn't make much difference > for one-time transfer of single .tar.gz file over LAN). >=20 > Now, to give answer to my own question: everything works fine, after - > of course - reinstalling all the packages with `pkg-static upgrade -f'. Also I've been same worry about this. But ezjail also worked in 12.0-CURRENT and almost everything stuffs used in that except below: (1) `jail_JID_devfs_ruleset' must be specified by number but name like export jail_desktop_devfs_ruleset=3D11 (2) Additional parameters shoule be specified by `jail_JID_parameters' with semicolon separated list like export jail_desktop_parameters=3D"allow.kmem; allow.sysvipc;" (3) If you through ICMP packets, set jail_JID_socket_unixiproute_only=3D"NO" But now basically FreeBSD does not supprt variable based specification like ezjail but jail.conf configuration. So jail spit warning whenever start/restart: root@kx:~ # service ezjail restart mail Stopping jails: mail.tfc. Starting jails: mail.tfc. /etc/rc.d/jail: WARNING: Per-jail configuration via jail_* variables is ob= solete. Please consider migrating to /etc/jail.conf. root@kx:~ #=20 >=20 > Regards, > --=20 > Before enlightenment - chop wood, draw water. > After enlightenment - chop wood, draw water. >=20 > Marko Cupa=E6 > https://www.mimar.rs/ > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" --- KIRIYAMA Kazuhiko From owner-freebsd-stable@freebsd.org Tue Oct 18 07:10:59 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 39CBCC16E72 for ; Tue, 18 Oct 2016 07:10:59 +0000 (UTC) (envelope-from alonsoschaich@fastmail.fm) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 007E1CCE for ; Tue, 18 Oct 2016 07:10:58 +0000 (UTC) (envelope-from alonsoschaich@fastmail.fm) Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.nyi.internal (Postfix) with ESMTP id 006E320871; Tue, 18 Oct 2016 03:10:55 -0400 (EDT) Received: from frontend2 ([10.202.2.161]) by compute4.internal (MEProxy); Tue, 18 Oct 2016 03:10:56 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=fastmail.fm; h=cc :content-type:date:from:in-reply-to:message-id:mime-version :references:subject:to:x-sasl-enc:x-sasl-enc; s=mesmtp; bh=GrS7V /QF/5yUpPSiAZseD7TzsTk=; b=TKUyXmleMmnnBesI7hc1K1IxC1G7QvFbowEki Ut3hI329jn7s3dyVA1a2o+AdjZ5tJkz+AXC0ayiOoVE5zfPMaBCp7fyAjdkKGE4d a88eS3KeFwloANAztQC8G//CITXv5kuH76jYGvn1shscSFRqwYf7l76v3qTQoP2p n9xJ1E= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-sasl-enc :x-sasl-enc; s=smtpout; bh=GrS7V/QF/5yUpPSiAZseD7TzsTk=; b=n89ys datxcGhccGFJmodehHNNxuGClHruxHn/W5gOMMZfrA8ESRxpdPpHyOf6MhOpCFk7 AepjvL5llaONj4Jq55N73oEiAvAb7lhRhDXrgL24gWRe8WzCxVXWuDNS9Ek/8Z19 /HvhyBYtUVsaHrqnaPrvxcb5YkZkpfmsh0v7dA= X-Sasl-enc: JG8OgqIKohsdBLA1caKlqMSpzDjqwG2t+DLD0DX9aJ3Y 1476774655 Received: from sunstream.divine.net (hsi-kbw-046-005-018-127.hsi8.kabel-badenwuerttemberg.de [46.5.18.127]) by mail.messagingengine.com (Postfix) with ESMTPA id 385ADCC07B; Tue, 18 Oct 2016 03:10:55 -0400 (EDT) Date: Tue, 18 Oct 2016 09:10:42 +0200 From: Schaich Alonso To: Kevin Oberman Cc: Dmitry Luhtionov , FreeBSD-STABLE Mailing List Subject: Re: make buildwotrld can not find zlib,h Message-Id: <20161018091042.dbc86eadd7c15f1e7fc4b5e1@fastmail.fm> In-Reply-To: References: X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) Mime-Version: 1.0 Content-Type: multipart/signed; protocol="application/pgp-signature"; micalg="PGP-SHA512"; boundary="Signature=_Tue__18_Oct_2016_09_10_42_+0200_IXPCeqz1GtpNB+lF" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 07:10:59 -0000 --Signature=_Tue__18_Oct_2016_09_10_42_+0200_IXPCeqz1GtpNB+lF Content-Type: text/plain; charset=US-ASCII Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, 14 Oct 2016 11:06:55 -0700 Kevin Oberman wrote: > On Fri, Oct 14, 2016 at 6:10 AM, Dmitry Luhtionov > wrote: >=20 > > c++ -O2 -pipe > > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include > > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/tools/clang/ > > include > > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support -= I. > > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/../../lib/cla= ng/ > > include > > -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS > > -D__STDC_CONSTANT_MACROS -DNDEBUG -fno-strict-aliasing > > -DLLVM_DEFAULT_TARGET_TRIPLE=3D\"x86_64-unknown-freebsd11.0\" > > -DLLVM_HOST_TRIPLE=3D\"x86_64-unknown-freebsd11.0\" > > -DDEFAULT_SYSROOT=3D\"/usr/obj/usr/src/tmp\" -MD -MF.depend.Compression= .o > > -MTCompression.o -Qunused-arguments > > -I/usr/obj/usr/src/tmp/legacy/usr/include -std=3Dc++11 -fno-exceptions > > -fno-rtti -stdlib=3Dlibc++ -Wno-c++11-extensions -c > > /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/ > > Compression.cpp > > -o Compression.o > > /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/ > > Compression.cpp:21:10: > > fatal error: 'zlib.h' file not found > > #include > > ^ > > 1 error generated. > > >=20 > Very odd. /usr/include/zlib.h is and long has been a standard component of > FreeBSD and should be present on your system. Can you confirm its absence? > Anything that could be in /etc/src.conf that might trigger this? (I can't > see anything obvious, but src.conf(5) is very long.) >=20 > I'm also not sure whether, at this point in the build, you should be using > the system's include files or those in /usr/obj/usr/src/tmp/usr/include/ = or > /usr/src/lib/libz/zlib.h, which is what should be copied to > /usr/obj/usr/src/tmp/usr/include. Normally the system's files are not used > during the build. >=20 > Have you tried completely removing /usr/obj (rm -r /usr/obj/*) before > starting the build with -DNO_CLEAN? > -- > Kevin Oberman, Part time kid herder and retired Network Engineer > E-mail: rkoberman@gmail.com > PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" Hi, I'm having an issue with libarchive that might or might not be related: > [...] > (cd /usr/src/lib/libarchive/tests && DEPENDFILE=3D.depend.libarchive_tes= t NO_SUBDIR=3D1 make -f /usr/src/lib/libarchive/tests/Makefile _RECURSING_= PROGS=3Dt PROG=3Dlibarchive_test ) > cc -O2 -pipe -I/usr/src/lib/libarchive -I/usr/obj/usr/src/lib/libarchive/= tests -I/usr/src/contrib/libarchive/libarchive -I/usr/src/contrib/libarchiv= e/test_utils -DHAVE_LIBLZMA=3D1 -DHAVE_LZMA_H=3D1 -g -MD -MF.depend.liba= rchive_test.main.o -MTmain.o -std=3Dgnu99 -fstack-protector-strong -Qunu= sed-arguments -c /usr/src/contrib/libarchive/libarchive/test/main.c -o mai= n.o > /usr/src/contrib/libarchive/libarchive/test/main.c:2400:10: fatal error: = 'list.h' file not found > #include "list.h" > ^ > 1 error generated. > *** Error code 1 >=20 > Stop. > make[1]: stopped in /usr/src/lib/libarchive/tests > *** Error code 1 > [...] list.h is perfectly there, but it's in /usr/src/lib/libarchive/tests rather= than /usr/obj/usr/src/lib/libarchive/tests (the second -I is wrong), so ei= ther some copy-to-builddir target was skipped or the build process should i= nclude the source path instead of the build path. I removed anything from /usr/obj/ and also commented-away anything I had in= make.conf and src.conf prior to attempting to compile it (11-STABLE r30755= 0). Alonso --Signature=_Tue__18_Oct_2016_09_10_42_+0200_IXPCeqz1GtpNB+lF Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAEBCgBmBQJYBcrzXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRGRUQ1N0JDOERFQjk5NEQ5QjUyQzBBMzVG RjhGNkI2RDBBQUNGQzY3AAoJEP+Pa20KrPxn0IEIAIVfYip87Nuedr4VdY/fQrqv q94r1Ncfpl0hSIB50dZdyQLBgLcuvIdNF01rBp1G8dOe5IIUUMW3b76nLd1rzBK9 tLvRY9JKS9grOGRA6H+cXulHukMG1R73aKwL59K4C4om/X66+qa60b1oKmYQ9Jgb lpALPDYg8HJEZV0HXibTbPjr2zXVrXXiCkskxpfl80Nibtzmgwa/cjouskcDjqC5 hhqRVarJz78ZqP0geln1zTaRzUNLnubWEIaIZSoTRZwIFTM05Gb2ewN2WAfXoUZR ctWr47MISkLpMUjOkFSkd7XaKtq4wJy/GPF4bxZ1eG6aX5jsfiwz6DqRKZaJK+g= =dkGY -----END PGP SIGNATURE----- --Signature=_Tue__18_Oct_2016_09_10_42_+0200_IXPCeqz1GtpNB+lF-- From owner-freebsd-stable@freebsd.org Tue Oct 18 10:00:19 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 33147C1700D for ; Tue, 18 Oct 2016 10:00:19 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: from mail-qk0-x22b.google.com (mail-qk0-x22b.google.com [IPv6:2607:f8b0:400d:c09::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E1C771000 for ; Tue, 18 Oct 2016 10:00:18 +0000 (UTC) (envelope-from kraduk@gmail.com) Received: by mail-qk0-x22b.google.com with SMTP id z190so281710028qkc.2 for ; Tue, 18 Oct 2016 03:00:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=xUjBTpVkBMFJeoYLLHLfwFwC13lLI6zGRpB+BSOy3Ao=; b=0xvsZdp4wocl1oxlPwN5sQppQxNyY6xrLVStFuh4koYWFsacqc/Ou7xakiTOnzzb+u tOviSVz0G6MfbPLpbzGHb1h7A5Om09vkbFLJXxfVKzjW78f0JFRSZmJMcr3M3IOOJszx nrV5t67Sd55vrbROvMunZrnpamdctUPVraZY2x03ox7GI/3UPBCAFLctkQWjBNNSF0HU fTvvU8AtHIEbgqCDr6TsCU5NMBBzAzkaW86RgTPYSp+tjJh+5k/WBN+csnyk5j57gjRb QgiZerOUEXEC5Z6TjGePjzWHBrtvSKmxr/+aj0EnEvbor+UPYsitmz6CpL/Q4g9xWOwW J9aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=xUjBTpVkBMFJeoYLLHLfwFwC13lLI6zGRpB+BSOy3Ao=; b=dgnGmg6vI+m7RVbcH26SNWjkExum6qADoAZ10bnOhoKB6cjf8AU0CLE5HLIwa7Reod AwhbH52MGF3NAG5coC6Ewif/l6VqVcJpOrbPpEvOUf0s0ddo+1JL+5DntwfWBCzoI18a E/uljVBprobuA1GJhd/nfk42ffROQPVIk0UxnhwoJvBHKN0A3ymvp+gWagu2PpInpqOh 5hCGJpMBTPIDB5Qe14q4R/YkRCBXpRCd+l/OQOTKauvvLP3OEtbBdhwF5L4SFwRsCmqg uRV4dkmjUcowV3bfH/SaW+3SL0ClugAKsi53Lrz8fGBNUCQUwq2RwrP+OPH8vTIm7oRz S8Rw== X-Gm-Message-State: AA6/9RmRoQOsq03SPEZJEuwYk2GKmAsIIw7vhLxJ2dq6V25hlCRNlNIk2ZhcjG5vZmT3r9VcDDk8opdcfYqz0g== X-Received: by 10.194.82.100 with SMTP id h4mr13930162wjy.183.1476784818075; Tue, 18 Oct 2016 03:00:18 -0700 (PDT) MIME-Version: 1.0 Received: by 10.195.11.137 with HTTP; Tue, 18 Oct 2016 03:00:17 -0700 (PDT) In-Reply-To: <20161017185215.97528d6cf6377058e4760785@yamagi.org> References: <20161017185215.97528d6cf6377058e4760785@yamagi.org> From: krad Date: Tue, 18 Oct 2016 11:00:17 +0100 Message-ID: Subject: Re: I'm upset about FreeBSD To: Yamagi Burmeister Cc: freebsd-stable Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 10:00:19 -0000 NOTE: you can only boot from gpt layout in uefi mode in windows On 17 October 2016 at 17:52, Yamagi Burmeister wrote: > On Mon, 17 Oct 2016 03:44:14 +0300 > Rostislav Krasny wrote: > > > First of all I faced an old problem that I reported here a year ago: > > http://comments.gmane.org/gmane.os.freebsd.stable/96598 > > Completely new USB flash drive flashed by the > > FreeBSD-11.0-RELEASE-i386-mini-memstick.img file kills every Windows > > again. If I use the Rufus util to write the img file (using DD mode) > > the Windows dies immediately after the flashing. If I use the > > Win32DiskImager (suggested by the Handbook) it doesn't reinitialize > > the USB storage and Windows dies only if I remove and put that USB > > flash drive again or boot Windows when it is connected. Nothing was > > done to fix this nasty bug for a year. > > As was already said in the other answers this is a bug in Windows. > Particulary in the partition parser. partmgr.sys (running in kernel > mode) crashes while parsing the FreeBSD installation images GPT > setup. This may be a variant of the bug known as "Kindle is crashing > Win 10": > > http://answers.microsoft.com/en-us/windows/forum/windows_ > 10-performance/plugging-in-kindle-is-crashing-windows-10- > after/5db0d867-0822-4512-919e-3d7786353f95?page=1 > > That bug was patched on september 13 and I'm unable to reproduce the > crash on a fully patched Win 10 VM. But there's no patch for Win 7, > even with all patches applied my Win 7 VM is still crashing as soon > as the FreeBSD installation image is connected. > > I did some debugging and I'm pretty sure that the problem is not the > pmbr used for classic BIOS boot but the GPT itself. But my knowledge > of GPT and especially Windows internals is limit. So maybe someone > with more insight can look into this. > > Or even better: Complain to Microsoft. Even if the GPT is invalid it > should crash the kernel. > > Regards, > Yamagi > > -- > Homepage: www.yamagi.org > XMPP: yamagi@yamagi.org > GnuPG/GPG: 0xEFBCCBCB > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" > From owner-freebsd-stable@freebsd.org Tue Oct 18 10:53:11 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46D2EC17EBF for ; Tue, 18 Oct 2016 10:53:11 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 8F4DC1EBA for ; Tue, 18 Oct 2016 10:53:10 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA24377; Tue, 18 Oct 2016 13:53:03 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1bwS10-0000i2-WD; Tue, 18 Oct 2016 13:53:03 +0300 Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: Steven Hartland , freebsd-stable@FreeBSD.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> From: Andriy Gapon Message-ID: Date: Tue, 18 Oct 2016 13:52:06 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 10:53:11 -0000 On 18/10/2016 00:43, Steven Hartland wrote: > On 17/10/2016 20:52, Andriy Gapon wrote: >> On 17/10/2016 21:54, Steven Hartland wrote: >>> You're hitting stack exhaustion, have you tried increasing the kernel stack pages? >>> It can be changed from /boot/loader.conf >>> kern.kstack_pages="6" >>> >>> Default on amd64 is 4 IIRC >> Steve, >> >> perhaps you can think of a more proper fix? :-) >> https://lists.freebsd.org/pipermail/freebsd-stable/2016-July/085047.html > Yes need to find some time to have a look at it, but given how rare this is and > with TRIM being re-implemented upstream in a totally different manor I'm > reticent to spend any real time on it. Fair enough. Especially given that there is only a single affected system reported so far. -- Andriy Gapon From owner-freebsd-stable@freebsd.org Tue Oct 18 11:11:38 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D00BC16715 for ; Tue, 18 Oct 2016 11:11:38 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C0632BF6 for ; Tue, 18 Oct 2016 11:11:36 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id u9IAvTu1093814; Tue, 18 Oct 2016 21:57:30 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Tue, 18 Oct 2016 21:57:29 +1100 (EST) From: Ian Smith To: Yamagi Burmeister cc: freebsd-stable@freebsd.org Subject: Re: I'm upset about FreeBSD In-Reply-To: <20161017185215.97528d6cf6377058e4760785@yamagi.org> Message-ID: <20161018212750.J6806@sola.nimnet.asn.au> References: <20161017185215.97528d6cf6377058e4760785@yamagi.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 11:11:38 -0000 On Mon, 17 Oct 2016 18:52:15 +0200, Yamagi Burmeister wrote: > On Mon, 17 Oct 2016 03:44:14 +0300 > Rostislav Krasny wrote: > > > First of all I faced an old problem that I reported here a year ago: > > http://comments.gmane.org/gmane.os.freebsd.stable/96598 > > Completely new USB flash drive flashed by the > > FreeBSD-11.0-RELEASE-i386-mini-memstick.img file kills every Windows > > again. If I use the Rufus util to write the img file (using DD mode) > > the Windows dies immediately after the flashing. If I use the > > Win32DiskImager (suggested by the Handbook) it doesn't reinitialize > > the USB storage and Windows dies only if I remove and put that USB > > flash drive again or boot Windows when it is connected. Nothing was > > done to fix this nasty bug for a year. > > As was already said in the other answers this is a bug in Windows. > Particulary in the partition parser. partmgr.sys (running in kernel > mode) crashes while parsing the FreeBSD installation images GPT > setup. This may be a variant of the bug known as "Kindle is crashing > Win 10": > > http://answers.microsoft.com/en-us/windows/forum/windows_10-performance/plugging-in-kindle-is-crashing-windows-10-after/5db0d867-0822-4512-919e-3d7786353f95?page=1 It's interesting that people primarily oriented to Windows tend to say 'X crashes my Windows' rather than 'my Windows crashes when X happens', so your far more cluey approach is refreshing in this regard .. > That bug was patched on september 13 and I'm unable to reproduce the > crash on a fully patched Win 10 VM. But there's no patch for Win 7, > even with all patches applied my Win 7 VM is still crashing as soon > as the FreeBSD installation image is connected. Amazing; what we'd call a kernel panic due to merely inserting a device. > I did some debugging and I'm pretty sure that the problem is not the > pmbr used for classic BIOS boot but the GPT itself. But my knowledge > of GPT and especially Windows internals is limit. So maybe someone > with more insight can look into this. If FreeBSD GPT images (and Kindle readers) can trigger this, so could a theoretically unlimited combination of data on block 2 of USB media; modifying FreeBSD to fix a Windows bug should be out of the question. > Or even better: Complain to Microsoft. Even if the GPT is invalid it > should crash the kernel. Well, exactly so, given s/should/should not/ .. and they'll have at least three images to test against: 10.3 (i386 only) and 11.0 (amd64 and i386) apart from the Kindles; should be a clue or two in there .. cheers, Ian From owner-freebsd-stable@freebsd.org Tue Oct 18 13:48:02 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF54DC16BF7 for ; Tue, 18 Oct 2016 13:48:02 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id A1987CE3; Tue, 18 Oct 2016 13:48:02 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id B96E2EB; Tue, 18 Oct 2016 13:48:02 +0000 (UTC) Date: Tue, 18 Oct 2016 13:48:02 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <769840592.4.1476798482414.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <676828102.3.1476755231436.JavaMail.jenkins@jenkins-9.freebsd.org> References: <676828102.3.1476755231436.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #430 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 13:48:02 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/430/ From owner-freebsd-stable@freebsd.org Tue Oct 18 16:56:56 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4D469C176B7 for ; Tue, 18 Oct 2016 16:56:56 +0000 (UTC) (envelope-from lexag@mail.ru) Received: from fallback2.mail.ru (fallback2.mail.ru [94.100.179.22]) by mx1.freebsd.org (Postfix) with ESMTP id 9C432CE6 for ; Tue, 18 Oct 2016 16:56:55 +0000 (UTC) (envelope-from lexag@mail.ru) Received: from f338.i.mail.ru (f338.i.mail.ru [217.69.140.234]) by fallback2.mail.ru (mPOP.Fallback_MX) with ESMTP id 50267786926B for ; Tue, 18 Oct 2016 18:12:18 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail2; h=Content-Type:Message-ID:Reply-To:Date:MIME-Version:Subject:To:From; bh=1G5dAkv7iJ5hPhq0KavnRx0HDcgcSfYnnOHr1tVmb9Q=; b=dhUkEgIiRp+HENXL9qHcTfgkY7ai7+32LEzq9x62Yx6aq4FFqODejyWKoQoFM1rPYCX2f/BKsniaL1ok+P1PMycQZDQli7MhwVX4oek7eS9pUqT6NThkfkIE65SgrqVGypXYBn3Bu7pkZGMTZ82QU98veH4jElplIXa+JWSeF2g=; Received: from [213.187.111.94] (ident=mail) by f338.i.mail.ru with local (envelope-from ) id 1bwW3k-0003oS-Hl for freebsd-stable@freebsd.org; Tue, 18 Oct 2016 18:12:09 +0300 Received: from [213.187.111.94] by e.mail.ru with HTTP; Tue, 18 Oct 2016 18:12:08 +0300 From: =?UTF-8?B?bGV4YSBndXNrb3Y=?= To: freebsd-stable@freebsd.org Subject: =?UTF-8?B?VlQgbm90IHNob3dpbmcgY3lyaWxsaWMgY2hhcmFjdGVycyBpbiB0ZXh0IG1v?= =?UTF-8?B?ZGU=?= MIME-Version: 1.0 X-Mailer: Mail.Ru Mailer 1.0 X-Originating-IP: [213.187.111.94] Date: Tue, 18 Oct 2016 18:12:08 +0300 Reply-To: =?UTF-8?B?bGV4YSBndXNrb3Y=?= X-Priority: 3 (Normal) Message-ID: <1476803528.432765648@f338.i.mail.ru> Authentication-Results: f338.i.mail.ru; auth=pass smtp.auth=lexag@mail.ru smtp.mailfrom=lexag@mail.ru X-95568C8E: 26815A15162540332840C59422DF1D5F199E47233F92776887959EFD9D048BA7C7B69D01972835159BA536E4D1503A3B83FAB1837865CABA9C49F0EC2FE72FF488BC8E08FBB947E2DD6CF3BC0931E2E7DC486A557FBD259A02A25C8A631CD08384B4DC3511CB792F87959EFD9D048BA7738997BD70C1643704FFD32E6C7FAF34CE37F99B8AE8A85678EF4415B693251108DB9C88D2580D436E9615857AD7BE2F041048F065DAD97587959EFD9D048BA7A6179E2BF3FAF700ED987E2E712D560CC2D72B3CA76BF4E8A96DC724E108FEB999C0AA0556ECC8503EF93E8B40729F5C0D2E5A50F6A53CEF96BCE47A2080CCB6D39B5F961A0A8A782DAB524B949EFA8F29437302FA48944F469F567EAD7748DB3FC41F08C0C8FB509D9BB2523D0BFC3FE5B5E3802068D22AF6F6E4D60496E20696B05DB3096C55BA779FC875B80AA582A7C70784DAB7959F5C6EB231107BA8A6E1AB025DA3DA201A7866D6147AF826D8AF1F0888C9C5A67FE914202507CE4B5C17D763367B87B9D56513A8A76CE2E0D9B1AAA4F897E61CB4 X-Mailru-Sender: 87FAA2DFF06D159D7201E5A609FE2955BBE9DE7F168ECE996F4DDB4294C5EA749BC7D83B045CA8D23555984C5E13135E X-Mras: OK X-Spam: undefined Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 16:56:56 -0000 CkhlbGxvIGV2ZXJ5b25lLiAKCkNhbm5vdCBmaWd1cmUgb3V0IGlmIHZ0IGNvbnNvbGUgZHJpdmVy IHN1cHBvcnRzIG5vbi1hbnNpIGNoYXJhY3RlcnMgd2hlbiBzdGFydGVkIGluIHRleHRtb2RlLiBU aGUgZm9udCBpbiBzeXMvZGV2L3Z0L3Z0X2ZvbnRfZGVmYXVsdC5jIHNlZW0gdG8gaW5jbHVkZSBj eXJpbGxpYyBnbHlwaHMsIGJ1dCBlaXRoZXIgaXQgaXMgbm90IGJlaW5nIHVzZWQgaW4gdGV4dG1v ZGUgb3Igc29tZXRoaW5nIGVsc2UgaXMgYnJva2VuIHNpbmNlIG9ubHkgJz8nIGFyZSBkaXNwbGF5 ZWQgaW5zdGVhZCBvZiBwcm9wZXIgc3ltYm9scy4gVGhlIExBTkcvTENfQUxMIGRvZXNudCBhZmZl Y3QgYW55dGhpbmcuIFRoZSBwcm9ibGVtIG9jY3VycyBvbiAxMS4wLVJFTEVBU0UgYW5kIDEwLjMt UkVMRUFTRS4KVGV4dG1vZGUgaXMgcmVxdWlyZWQgc2luY2UgdnQgcGVyZm9ybWFuY2UgaW4gZ3Jh cGhpY2FsIG1vZGUgaXMgdmVyeSBwb29yIG9uIEh5cGVyViBhbmQgdGV4dG1vZGUgc2VlbXMgdG8g YmUgdGhlIG9ubHkgb3B0aW9uLgoKLS0gCkFsZXhleSBHdXNrb3Y= From owner-freebsd-stable@freebsd.org Tue Oct 18 19:50:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 908AAC17BA8 for ; Tue, 18 Oct 2016 19:50:05 +0000 (UTC) (envelope-from lexag@mail.ru) Received: from fallback5.mail.ru (fallback5.mail.ru [94.100.181.253]) by mx1.freebsd.org (Postfix) with ESMTP id 193F3850 for ; Tue, 18 Oct 2016 19:50:04 +0000 (UTC) (envelope-from lexag@mail.ru) Received: from f390.i.mail.ru (f390.i.mail.ru [185.5.136.61]) by fallback5.mail.ru (mPOP.Fallback_MX) with ESMTP id 987931439DD98 for ; Tue, 18 Oct 2016 22:29:00 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mail.ru; s=mail2; h=Content-Type:Message-ID:Reply-To:Date:MIME-Version:Subject:To:From; bh=DJAe5t9vOqtGt23cq6rhkpA1zXkkV0kBtrgTPgG+I7k=; b=cxm987ttF5oeVeP0N20uRfA+NTSTCO4723vjiOzf/Xu1ntFMGmqb/hbGyn8nWwqsN/IEcZ/7HZRQSnalPjI9tmT3miTDTetMYHj09OvfC8H9tOQR9cdfWZVVCkuhdQR87lDj+c12bMJ7X4khNlw+/j22rbiyoAifJeHrdAWPXwE=; Received: from [93.158.230.18] (ident=mail) by f390.i.mail.ru with local (envelope-from ) id 1bwa4B-0005CG-AH for freebsd-stable@freebsd.org; Tue, 18 Oct 2016 22:28:51 +0300 Received: from [93.158.230.18] by e.mail.ru with HTTP; Tue, 18 Oct 2016 22:28:51 +0300 From: =?UTF-8?B?bGV4YSBndXNrb3Y=?= To: =?UTF-8?B?ZnJlZWJzZC1zdGFibGU=?= Subject: =?UTF-8?B?VlQgbm90IHNob3dpbmcgY3lyaWxsaWMgY2hhcmFjdGVycyBpbiB0ZXh0IG1v?= =?UTF-8?B?ZGU=?= MIME-Version: 1.0 X-Mailer: Mail.Ru Mailer 1.0 X-Originating-IP: [93.158.230.18] Date: Tue, 18 Oct 2016 22:28:51 +0300 Reply-To: =?UTF-8?B?bGV4YSBndXNrb3Y=?= X-Priority: 3 (Normal) Message-ID: <1476818931.550937396@f390.i.mail.ru> Authentication-Results: f390.i.mail.ru; auth=pass smtp.auth=lexag@mail.ru smtp.mailfrom=lexag@mail.ru X-95568C8E: 26815A15162540332840C59422DF1D5F199E47233F92776887959EFD9D048BA7C7B69D01972835159BA536E4D1503A3B83FAB1837865CABA9C49F0EC2FE72FF488BC8E08FBB947E2DD6CF3BC0931E2E7DC486A557FBD259A02A25C8A631CD08384B4DC3511CB792F87959EFD9D048BA7738997BD70C1643704FFD32E6C7FAF34CE37F99B8AE8A85678EF4415B693251108DB9C88D2580D436E9615857AD7BE2F041048F065DAD97587959EFD9D048BA7A6179E2BF3FAF700ED987E2E712D560CC2D72B3CA76BF4E87D170E26BD450E1A342FE349B32151FF4AD746BF6A4EA6A1B8833B1B61982963EBBBCE6AB234454C8D2D1705B2D56CB02DC166148FC3169C9998397E634EAA944CDF4B1AE75809857D36B96BAD0BE187A68DE7DDD73E0E47980C0D23384327162D21F4324733A0FF567FBB3DF345EA932033455394237A825F8AD4E0F03E6AD1039CDCA4B6D2CCBBB0EBC87731A1751F62F6352410174B4DF70F574B69D9994DB6E48079B5802D0E88BD4503FCCD991D2B93235B59A1CD7375BEA1197805FBCA X-Mailru-Sender: 61E5675DBDC349EACF4C34259A117AB75FFC5D124B0CA4D37E9167C5634BDBA5FC0F0DE836F95A0DADAB024B4846C063 X-Mras: OK X-Spam: undefined Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 19:50:05 -0000 CkhlbGxvIGV2ZXJ5b25lLsKgCgpDYW5ub3QgZmlndXJlIG91dCBpZiB2dCBjb25zb2xlIGRyaXZl ciBzdXBwb3J0cyBub24tYW5zaSBjaGFyYWN0ZXJzIHdoZW4gc3RhcnRlZCBpbiB0ZXh0bW9kZS4g VGhlIGZvbnQgaW4gc3lzL2Rldi92dC92dF9mb250X2RlZmF1bHQuYyBzZWVtIHRvIGluY2x1ZGUg Y3lyaWxsaWMgZ2x5cGhzLCBidXQgZWl0aGVyIGl0IGlzIG5vdCBiZWluZyB1c2VkIGluIHRleHRt b2RlIG9yIHNvbWV0aGluZyBlbHNlIGlzIGJyb2tlbiBzaW5jZSBvbmx5ICc/JyBhcmUgZGlzcGxh eWVkIGluc3RlYWQgb2YgcHJvcGVyIHN5bWJvbHMuIFRoZSBMQU5HL0xDX0FMTCBkb2VzbnQgYWZm ZWN0IGFueXRoaW5nLiBUaGUgcHJvYmxlbSBvY2N1cnMgb24gMTEuMC1SRUxFQVNFIGFuZCAxMC4z LVJFTEVBU0UuClRleHRtb2RlIGlzIHJlcXVpcmVkIHNpbmNlIHZ0IHBlcmZvcm1hbmNlIGluIGdy YXBoaWNhbCBtb2RlIGlzIHZlcnkgcG9vciBvbiBIeXBlclYgYW5kIHRleHRtb2RlIHNlZW1zIHRv IGJlIHRoZSBvbmx5IG9wdGlvbi4KCihzb3JyeSBhYm91dCB0aGUgcHJldmlvdXMgbWVzc3kgbWVz c2FnZSBpbiB0aGlzIG1haWxpbmcgbGlzdCwgZGlkbnQgZXhwZWN0IHByb2JsZW1zIHdpdGggaHRt bCBlbWFpbHMgOykKLS3CoApBbGV4ZXkgR3Vza292Cg== From owner-freebsd-stable@freebsd.org Tue Oct 18 20:55:49 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69631C16F85 for ; Tue, 18 Oct 2016 20:55:49 +0000 (UTC) (envelope-from karl@denninger.net) Received: from mail.denninger.net (denninger.net [70.169.168.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 29B517D3 for ; Tue, 18 Oct 2016 20:55:48 +0000 (UTC) (envelope-from karl@denninger.net) Received: from [192.168.1.40] (Karl-Desktop.Denninger.net [192.168.1.40]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.denninger.net (Postfix) with ESMTPSA id 02686303E57 for ; Tue, 18 Oct 2016 15:55:40 -0500 (CDT) Subject: Re: Repeatable panic on ZFS filesystem (used for backups); 11.0-STABLE To: freebsd-stable@freebsd.org References: <3d4f25c9-a262-a373-ec7e-755325f8810b@denninger.net> <9adecd24-6659-0da5-5c05-d0d3957a2cb3@denninger.net> <0f58b11f-0bca-bc08-6f90-4e6e530f9956@denninger.net> <43a67287-f4f8-5d3e-6c5e-b3599c6adb4d@multiplay.co.uk> <76551fd6-0565-ee6c-b0f2-7d472ad6a4b3@denninger.net> <25ff3a3e-77a9-063b-e491-8d10a06e6ae2@multiplay.co.uk> <26e092b2-17c6-8744-5035-d0853d733870@denninger.net> <4d4909b7-c44b-996e-90e1-ca446e8e4813@multiplay.co.uk> From: Karl Denninger Message-ID: <1fefed03-6062-50f9-be97-d693e25a64c9@denninger.net> Date: Tue, 18 Oct 2016 15:55:37 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <4d4909b7-c44b-996e-90e1-ca446e8e4813@multiplay.co.uk> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms040206080209020402030205" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Oct 2016 20:55:49 -0000 This is a cryptographically signed message in MIME format. --------------ms040206080209020402030205 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 10/17/2016 18:32, Steven Hartland wrote: > > > On 17/10/2016 22:50, Karl Denninger wrote: >> I will make some effort on the sandbox machine to see if I can come up= >> with a way to replicate this. I do have plenty of spare larger drives= >> laying around that used to be in service and were obsolesced due to >> capacity -- but what I don't know if whether the system will misbehave= >> if the source is all spinning rust. >> >> In other words: >> >> 1. Root filesystem is mirrored spinning rust (production is mirrored >> SSDs) >> >> 2. Backup is mirrored spinning rust (of approx the same size) >> >> 3. Set up auto-snapshot exactly as the production system has now (whic= h >> the sandbox is NOT since I don't care about incremental recovery on th= at >> machine; it's a sandbox!) >> >> 4. Run a bunch of build-somethings (e.g. buildworlds, cross-build for >> the Pi2s I have here, etc) to generate a LOT of filesystem entropy >> across lots of snapshots. >> >> 5. Back that up. >> >> 6. Export the backup pool. >> >> 7. Re-import it and "zfs destroy -r" the backup filesystem. >> >> That is what got me in a reboot loop after the *first* panic; I was >> simply going to destroy the backup filesystem and re-run the backup, b= ut >> as soon as I issued that zfs destroy the machine panic'd and as soon a= s >> I re-attached it after a reboot it panic'd again. Repeat until I set >> trim=3D0. >> >> But... if I CAN replicate it that still shouldn't be happening, and th= e >> system should *certainly* survive attempting to TRIM on a vdev that >> doesn't support TRIMs, even if the removal is for a large amount of >> space and/or files on the target, without blowing up. >> >> BTW I bet it isn't that rare -- if you're taking timed snapshots on an= >> active filesystem (with lots of entropy) and then make the mistake of >> trying to remove those snapshots (as is the case with a zfs destroy -r= >> or a zfs recv of an incremental copy that attempts to sync against a >> source) on a pool that has been imported before the system realizes th= at >> TRIM is unavailable on those vdevs. >> >> Noting this: >> >> Yes need to find some time to have a look at it, but given how ra= re >> this is and with TRIM being re-implemented upstream in a totally >> different manor I'm reticent to spend any real time on it. >> >> What's in-process in this regard, if you happen to have a reference? > Looks like it may be still in review: https://reviews.csiden.org/r/263/= > > Initial attempts to provoke the panic has failed on the sandbox machine -- it appears that I need a materially-fragmented backup volume (which makes sense, as that would greatly increase the number of TRIM's queued.)= Running a bunch of builds with snapshots taken between generates a metric ton of entropy in the filesystem, but it appears that the number of TRIMs actually issued when you bulk-remove them (with zfs destroy -r) is small enough to not cause it -- probably because the system issues one per area of freed disk, and since there is no interleaving with other (non-removed) data that number is "reasonable" since there's little fragmentation of that free space. The TRIMs *are* attempted, and they *do* fail, however..... I'm running with the 6 pages of kstack now on the production machine, and we'll see if I get another panic... --=20 Karl Denninger karl@denninger.net /The Market Ticker/ /[S/MIME encrypted email preferred]/ --------------ms040206080209020402030205 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Bl8wggZbMIIEQ6ADAgECAgEpMA0GCSqGSIb3DQEBCwUAMIGQMQswCQYDVQQGEwJVUzEQMA4G A1UECBMHRmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3Rl bXMgTExDMRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhND dWRhIFN5c3RlbXMgTExDIENBMB4XDTE1MDQyMTAyMjE1OVoXDTIwMDQxOTAyMjE1OVowWjEL MAkGA1UEBhMCVVMxEDAOBgNVBAgTB0Zsb3JpZGExGTAXBgNVBAoTEEN1ZGEgU3lzdGVtcyBM TEMxHjAcBgNVBAMTFUthcmwgRGVubmluZ2VyIChPQ1NQKTCCAiIwDQYJKoZIhvcNAQEBBQAD ggIPADCCAgoCggIBALmEWPhAdphrWd4K5VTvE5pxL3blRQPyGF3ApjUjgtavqU1Y8pbI3Byg XDj2/Uz9Si8XVj/kNbKEjkRh5SsNvx3Fc0oQ1uVjyCq7zC/kctF7yLzQbvWnU4grAPZ3IuAp 3/fFxIVaXpxEdKmyZAVDhk9az+IgHH43rdJRIMzxJ5vqQMb+n2EjadVqiGPbtG9aZEImlq7f IYDTnKyToi23PAnkPwwT+q1IkI2DTvf2jzWrhLR5DTX0fUYC0nxlHWbjgpiapyJWtR7K2YQO aevQb/3vN9gSojT2h+cBem7QIj6U69rEYcEDvPyCMXEV9VcXdcmW42LSRsPvZcBHFkWAJqMZ Myiz4kumaP+s+cIDaXitR/szoqDKGSHM4CPAZV9Yh8asvxQL5uDxz5wvLPgS5yS8K/o7zDR5 vNkMCyfYQuR6PAJxVOk5Arqvj9lfP3JSVapwbr01CoWDBkpuJlKfpQIEeC/pcCBKknllbMYq yHBO2TipLyO5Ocd1nhN/nOsO+C+j31lQHfOMRZaPQykXVPWG5BbhWT7ttX4vy5hOW6yJgeT/ o3apynlp1cEavkQRS8uJHoQszF6KIrQMID/JfySWvVQ4ksnfzwB2lRomrdrwnQ4eG/HBS+0l eozwOJNDIBlAP+hLe8A5oWZgooIIK/SulUAsfI6Sgd8dTZTTYmlhAgMBAAGjgfQwgfEwNwYI KwYBBQUHAQEEKzApMCcGCCsGAQUFBzABhhtodHRwOi8vY3VkYXN5c3RlbXMubmV0Ojg4ODgw CQYDVR0TBAIwADARBglghkgBhvhCAQEEBAMCBaAwCwYDVR0PBAQDAgXgMCwGCWCGSAGG+EIB DQQfFh1PcGVuU1NMIEdlbmVyYXRlZCBDZXJ0aWZpY2F0ZTAdBgNVHQ4EFgQUxRyULenJaFwX RtT79aNmIB/u5VkwHwYDVR0jBBgwFoAUJHGbnYV9/N3dvbDKkpQDofrTbTUwHQYDVR0RBBYw FIESa2FybEBkZW5uaW5nZXIubmV0MA0GCSqGSIb3DQEBCwUAA4ICAQBPf3cYtmKowmGIYsm6 eBinJu7QVWvxi1vqnBz3KE+HapqoIZS8/PolB/hwiY0UAE1RsjBJ7yEjihVRwummSBvkoOyf G30uPn4yg4vbJkR9lTz8d21fPshWETa6DBh2jx2Qf13LZpr3Pj2fTtlu6xMYKzg7cSDgd2bO sJGH/rcvva9Spkx5Vfq0RyOrYph9boshRN3D4tbWgBAcX9POdXCVfJONDxhfBuPHsJ6vEmPb An+XL5Yl26XYFPiODQ+Qbk44Ot1kt9s7oS3dVUrh92Qv0G3J3DF+Vt6C15nED+f+bk4gScu+ JHT7RjEmfa18GT8DcT//D1zEke1Ymhb41JH+GyZchDRWtjxsS5OBFMzrju7d264zJUFtX7iJ 3xvpKN7VcZKNtB6dLShj3v/XDsQVQWXmR/1YKWZ93C3LpRs2Y5nYdn6gEOpL/WfQFThtfnat HNc7fNs5vjotaYpBl5H8+VCautKbGOs219uQbhGZLYTv6okuKcY8W+4EJEtK0xB08vqr9Jd0 FS9MGjQE++GWo+5eQxFt6nUENHbVYnsr6bYPQsZH0CRNycgTG9MwY/UIXOf4W034UpR82TBG 1LiMsYfb8ahQJhs3wdf1nzipIjRwoZKT1vGXh/cj3gwSr64GfenURBxaFZA5O1acOZUjPrRT n3ci4McYW/0WVVA3lDGCBRMwggUPAgEBMIGWMIGQMQswCQYDVQQGEwJVUzEQMA4GA1UECBMH RmxvcmlkYTESMBAGA1UEBxMJTmljZXZpbGxlMRkwFwYDVQQKExBDdWRhIFN5c3RlbXMgTExD MRwwGgYDVQQDExNDdWRhIFN5c3RlbXMgTExDIENBMSIwIAYJKoZIhvcNAQkBFhNDdWRhIFN5 c3RlbXMgTExDIENBAgEpMA0GCWCGSAFlAwQCAwUAoIICTTAYBgkqhkiG9w0BCQMxCwYJKoZI hvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNjEwMTgyMDU1MzdaME8GCSqGSIb3DQEJBDFCBEBI z9dDmtaCkAeNlMzW4yZMrJchdvA25n5Tei2nlg+EevLD+Q9tZ+zjqMFRYrYr8nGf9TBBFC6c 0ZMVfCBAj3pQMGwGCSqGSIb3DQEJDzFfMF0wCwYJYIZIAWUDBAEqMAsGCWCGSAFlAwQBAjAK BggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcNAwICAUAwBwYFKw4DAgcwDQYI KoZIhvcNAwICASgwgacGCSsGAQQBgjcQBDGBmTCBljCBkDELMAkGA1UEBhMCVVMxEDAOBgNV BAgTB0Zsb3JpZGExEjAQBgNVBAcTCU5pY2V2aWxsZTEZMBcGA1UEChMQQ3VkYSBTeXN0ZW1z IExMQzEcMBoGA1UEAxMTQ3VkYSBTeXN0ZW1zIExMQyBDQTEiMCAGCSqGSIb3DQEJARYTQ3Vk YSBTeXN0ZW1zIExMQyBDQQIBKTCBqQYLKoZIhvcNAQkQAgsxgZmggZYwgZAxCzAJBgNVBAYT AlVTMRAwDgYDVQQIEwdGbG9yaWRhMRIwEAYDVQQHEwlOaWNldmlsbGUxGTAXBgNVBAoTEEN1 ZGEgU3lzdGVtcyBMTEMxHDAaBgNVBAMTE0N1ZGEgU3lzdGVtcyBMTEMgQ0ExIjAgBgkqhkiG 9w0BCQEWE0N1ZGEgU3lzdGVtcyBMTEMgQ0ECASkwDQYJKoZIhvcNAQEBBQAEggIAn2rgnnNi EZHYytdfyH2xDEN7jIIP9erFeFT61zryxkKUDWsB6zj2hUL0109YN9YOQMfJR0qPPA68W7Np duVddYyZ07Xhcc9l/tYERrUTa9WMmHMDidA3H3tdkruRbZDJ6fWaWUyIcqsTRfoSG5uavzzM U4wS20ouzMXBHLa+/CiJ0+h7qsqjRVdE5pkFJcAQ8hxUsP/wCDB92rEnGdwzXPWrcaOIxIBk cFlPJ7o6TcdlT62Hx7QxF1Ncq1iSz4YDAS49sens04khYP9ZsDAYFlpasc9YzQeQ9ttoQNE4 yGiDhLnf5JMqPCPqGMrWT9sLIvVTRrZdXZEkAEhOirIpA7GVQfqZa3f7jpwVk6Vv6xZj1Y44 BYsG93fAdmsIfl+fEp2D//6s1u/Y+1NcfjQVq6CikpRrAORUX7Ec93OjJieT69vJo80KkdYI ZOQyUu0FkBYGdmd+b0EaJBUJ+D3BZN4R01mMYowjsLSDqYPAB+jyiNSZdzNLAoen6k2pBJ66 qVYaZQz3rXmRfeKdC1fQBISti7jfKngTZe4/n166tplqHc2Ww+mYFWflMKBgdtgtKvUIQe4h 6+kKN0hPnlWwFHGOPHDvthRxiYV1hZ6MOhC7iA/NOdR2YJP2r0kLJj9hLuPjpt5yzLohSLpn wjG/3tuiJtEytNyVNZ+SHo0NTiQAAAAAAAA= --------------ms040206080209020402030205-- From owner-freebsd-stable@freebsd.org Wed Oct 19 03:50:56 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78FB9C18F3B for ; Wed, 19 Oct 2016 03:50:56 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-vk0-x234.google.com (mail-vk0-x234.google.com [IPv6:2607:f8b0:400c:c05::234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 240BC2EB for ; Wed, 19 Oct 2016 03:50:56 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-vk0-x234.google.com with SMTP id b186so16454896vkb.1 for ; Tue, 18 Oct 2016 20:50:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=gpF7aYOAF0WWVkded6zJ1YlD8ZUtnmzmM5vIBSefW/Q=; b=GmVMnAQ0hbY8Thkv2+UdeAVr1WsHPLKVk7GwE3cvjBHlFUnJtwrLYQcKLlESfF55EY UIJm1sFslpUMq7gTpJ7B36YRLtjkF66toXpHry/iyBRrA+6blzDzwMk5Fw3vTS2tiu/L bkSMLgFZ8JBtJrfBliyp+mGGjL5se5MJl4YhkRDfXESGCTGngTvbmwYySqpIHg3L8cli rGaNPhp/lcdNlziUHKoFOeyXUwNFN56U1t6fBvKZ3lgAT1pnO+nuoabxpeGVy8XPGuQo UGqCSfqpd8qKFVRCfEKyrkmqO6MpYu4w75cKMvU0hfJp/L56+mJCDGU+lXn2/7TzEhty gT2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=gpF7aYOAF0WWVkded6zJ1YlD8ZUtnmzmM5vIBSefW/Q=; b=e3X6dn6o6sejc5NQrWaJM0ICYR9agWjY3qhzZ2X12wwDy5HLn/wC14nvlPlfnVgOub hubatphpONug0ckbWmaN83WNDLlo1HQV/ALyPrrsmidUDV858DnJVvbx1o0/OjB8SaBb 6LLN2CQStMzHCuRRoSCsxm7V1/1WjpACK0jnMK9wyEpI9t+AxOssT1zIOqI5+1bED9vO Jwqv0VbGWq9XCTh/Y+NoxWhOvx6N5L4X6qzkrsxzm23FcMqt3s72l8PhWaPh7Q795CSF JS6x2CfNZH0BymXciQg9pFl2Z1Vo53UenL/TB5ypsDR9sOjGeKQwGhQ8HXGt+7wHttOA ELbw== X-Gm-Message-State: AA6/9RnODxUt9EPEZQo/J9jzQT+/l2aevbYn0VN1a8+5g/Zc4WJ+N1qQYubSW+Lpejxw7ldufdQqi/JZFlsSaw== X-Received: by 10.31.229.133 with SMTP id c127mr3902007vkh.153.1476849055223; Tue, 18 Oct 2016 20:50:55 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.103.118.78 with HTTP; Tue, 18 Oct 2016 20:50:54 -0700 (PDT) In-Reply-To: <20161018091042.dbc86eadd7c15f1e7fc4b5e1@fastmail.fm> References: <20161018091042.dbc86eadd7c15f1e7fc4b5e1@fastmail.fm> From: Kevin Oberman Date: Tue, 18 Oct 2016 20:50:54 -0700 X-Google-Sender-Auth: PlNxH97xPlYWjXyoKqVz1a_CCZY Message-ID: Subject: Re: make buildwotrld can not find zlib,h To: Schaich Alonso Cc: Dmitry Luhtionov , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 03:50:56 -0000 On Tue, Oct 18, 2016 at 12:10 AM, Schaich Alonso wrote: > On Fri, 14 Oct 2016 11:06:55 -0700 > Kevin Oberman wrote: > > > On Fri, Oct 14, 2016 at 6:10 AM, Dmitry Luhtionov < > dmitryluhtionov@gmail.com > > > wrote: > > > > > c++ -O2 -pipe > > > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include > > > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/tools/clang/ > > > include > > > -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support > -I. > > > -I/usr/src/lib/clang/libllvmsupport/../../../ > contrib/llvm/../../lib/clang/ > > > include > > > -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS > > > -D__STDC_CONSTANT_MACROS -DNDEBUG -fno-strict-aliasing > > > -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd11.0\" > > > -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd11.0\" > > > -DDEFAULT_SYSROOT=\"/usr/obj/usr/src/tmp\" -MD > -MF.depend.Compression.o > > > -MTCompression.o -Qunused-arguments > > > -I/usr/obj/usr/src/tmp/legacy/usr/include -std=c++11 -fno-exceptions > > > -fno-rtti -stdlib=libc++ -Wno-c++11-extensions -c > > > /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/ > > > Compression.cpp > > > -o Compression.o > > > /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/ > > > Compression.cpp:21:10: > > > fatal error: 'zlib.h' file not found > > > #include > > > ^ > > > 1 error generated. > > > > > > > Very odd. /usr/include/zlib.h is and long has been a standard component > of > > FreeBSD and should be present on your system. Can you confirm its > absence? > > Anything that could be in /etc/src.conf that might trigger this? (I can't > > see anything obvious, but src.conf(5) is very long.) > > > > I'm also not sure whether, at this point in the build, you should be > using > > the system's include files or those in /usr/obj/usr/src/tmp/usr/include/ > or > > /usr/src/lib/libz/zlib.h, which is what should be copied to > > /usr/obj/usr/src/tmp/usr/include. Normally the system's files are not > used > > during the build. > > > > Have you tried completely removing /usr/obj (rm -r /usr/obj/*) before > > starting the build with -DNO_CLEAN? > > -- > > Kevin Oberman, Part time kid herder and retired Network Engineer > > E-mail: rkoberman@gmail.com > > PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > > > -- > Kevin Oberman, Part time kid herder and retired Network Engineer > E-mail: rkoberman@gmail.com > PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 > > Hi, > > I'm having an issue with libarchive that might or might not be related: > > > [...] > > (cd /usr/src/lib/libarchive/tests && DEPENDFILE=.depend.libarchive_test > NO_SUBDIR=1 make -f /usr/src/lib/libarchive/tests/Makefile > _RECURSING_PROGS=t PROG=libarchive_test ) > > cc -O2 -pipe -I/usr/src/lib/libarchive -I/usr/obj/usr/src/lib/libarchive/tests > -I/usr/src/contrib/libarchive/libarchive -I/usr/src/contrib/libarchive/test_utils > -DHAVE_LIBLZMA=1 -DHAVE_LZMA_H=1 -g -MD -MF.depend.libarchive_test.main.o > -MTmain.o -std=gnu99 -fstack-protector-strong -Qunused-arguments -c > /usr/src/contrib/libarchive/libarchive/test/main.c -o main.o > > /usr/src/contrib/libarchive/libarchive/test/main.c:2400:10: fatal > error: 'list.h' file not found > > #include "list.h" > > ^ > > 1 error generated. > > *** Error code 1 > > > > Stop. > > make[1]: stopped in /usr/src/lib/libarchive/tests > > *** Error code 1 > > [...] > > list.h is perfectly there, but it's in /usr/src/lib/libarchive/tests > rather than /usr/obj/usr/src/lib/libarchive/tests (the second -I is > wrong), so either some copy-to-builddir target was skipped or the build > process should include the source path instead of the build path. > > I removed anything from /usr/obj/ and also commented-away anything I had > in make.conf and src.conf prior to attempting to compile it (11-STABLE > r307550). > > Alonso > You need to look at the output. One pass of the install should copy the new .h files from /usr/src to /usr/obj/usr/src. Looks like one or more things did not get copied. Do you have an /etc/src.conf file? Are any parts of the system being excluded there? What about /etc/make.conf? Since others have not reported this, it seems likely that there is something you have configured at some time. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-stable@freebsd.org Wed Oct 19 04:48:34 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 57E97C15D4D for ; Wed, 19 Oct 2016 04:48:34 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 4A1A2B21; Wed, 19 Oct 2016 04:48:34 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 6DB0F10F; Wed, 19 Oct 2016 04:48:34 +0000 (UTC) Date: Wed, 19 Oct 2016 04:48:34 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <1319508584.7.1476852514221.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <769840592.4.1476798482414.JavaMail.jenkins@jenkins-9.freebsd.org> References: <769840592.4.1476798482414.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #431 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 04:48:34 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/431/ From owner-freebsd-stable@freebsd.org Wed Oct 19 06:06:35 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 75D96C181BD for ; Wed, 19 Oct 2016 06:06:35 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 56F90A9A for ; Wed, 19 Oct 2016 06:06:35 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: by mailman.ysv.freebsd.org (Postfix) id 538CDC181BC; Wed, 19 Oct 2016 06:06:35 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51822C181BB for ; Wed, 19 Oct 2016 06:06:35 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from pv36p08im-ztdg07111401.me.com (pv36p08im-ztdg07111401.me.com [17.142.193.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2B4FDA99 for ; Wed, 19 Oct 2016 06:06:35 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from process-dkim-sign-daemon.pv36p08im-ztdg07111401.me.com by pv36p08im-ztdg07111401.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) id <0OFA00N003BEX500@pv36p08im-ztdg07111401.me.com> for stable@FreeBSD.org; Wed, 19 Oct 2016 05:06:13 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mac.com; s=4d515a; t=1476853572; bh=HWAQsWiizeyRE1cE5ptCUGI52Jd8ZCbayGeMtqWT1OE=; h=From:Content-type:MIME-version:Subject:Message-id:Date:To; b=fPFiy8PPa+TxTJv4J1Q700RMx3XRibbhmiVVwMp2KceitwGvLh8CAOklXR/25kxtj 43O7aMc7+ORRKG3aiY4M1H5xEU6I0o9HsjadL/lzVAaqK6E85Dqt79JEyBLORf/3MN dAK8erchlzeEwLPb9pE3DAGQqDC43NRwH55DNX0tjtsTsPeZyPJV1b6830roHs7LPn y/0yLza/I5FHlBK6xVkEuOXH6ndSB/cniXQgjOLf14qOK38lDggfgyejXOS/e4yLwP FFg0OsKSR8PWep/nFLwWNPP0YOPS+UhlrDIhHBMYVL4s5PR8/dFd/MF9VAy6HMHxCa PvQ92o45RCgqw== Received: from jonathanhaack.com (c-73-98-2-193.hsd1.nm.comcast.net [73.98.2.193]) by pv36p08im-ztdg07111401.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) with ESMTPSA id <0OFA008KB3IA7E40@pv36p08im-ztdg07111401.me.com> for stable@FreeBSD.org; Wed, 19 Oct 2016 05:06:12 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-10-19_02:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1034 suspectscore=4 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1603290000 definitions=main-1610190091 From: Jonathan Haack MIME-version: 1.0 (Mac OS X Mail 10.0 \(3226\)) Subject: 11.0 Install Question Message-id: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> Date: Tue, 18 Oct 2016 23:06:10 -0600 To: stable@FreeBSD.org X-Mailer: Apple Mail (2.3226) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 06:06:35 -0000 I am running 10.1 fully update with=20 freebsd-update fetch freebsd-update install then, every time I do=20 freebsd-update -r upgrade 11.0-RELEASE It says it failed an integrity check and cowardly refuses =E2=80=A6 I = have followed all forum advise and upgraded pkg, rebooted ran everything = again and again =E2=80=A6 cowardly fails each time any way to force / and or re-download and do over? Sincerely, Coach Haack DP Coordinator, Mandela International Magnet School Board of Directors, First Serve NM President, NMCTM www.jonathanhaack.com "It is better to have tried and failed than to have succeeded at doing = nothing."=20 Confidentiality Notice: This email, including all attachments is for = the sole use of the intended recipient or recipients and may contain = confidential information. Any unauthorized review, use, disclosure, or = distribution, is prohibited unless specifically provided under the New = Mexico Inspection of Public Records Act. If you are not the intended = recipient, please contact the sender and destroy all copies of this = message. From owner-freebsd-stable@freebsd.org Wed Oct 19 06:33:24 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7607C189FA for ; Wed, 19 Oct 2016 06:33:24 +0000 (UTC) (envelope-from Holger.Kipp@alogis.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id BF610B14 for ; Wed, 19 Oct 2016 06:33:24 +0000 (UTC) (envelope-from Holger.Kipp@alogis.com) Received: by mailman.ysv.freebsd.org (Postfix) id BED09C189F9; Wed, 19 Oct 2016 06:33:24 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE7C2C189F8 for ; Wed, 19 Oct 2016 06:33:24 +0000 (UTC) (envelope-from Holger.Kipp@alogis.com) Received: from alogis.com (www2.alogis.com [212.184.102.7]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 679A5B12 for ; Wed, 19 Oct 2016 06:33:23 +0000 (UTC) (envelope-from Holger.Kipp@alogis.com) Received: from msx3.exchange.alogis.com (msxcn2.exchange.alogis.com [10.1.1.22]) by alogis.com (8.13.4/8.13.1) with ESMTP id u9J6XENd088085; Wed, 19 Oct 2016 08:33:14 +0200 (CEST) (envelope-from Holger.Kipp@alogis.com) Received: from MSXCN4.exchange.alogis.com (10.1.1.18) by msxcn2.exchange.alogis.com (10.1.1.22) with Microsoft SMTP Server (TLS) id 14.3.279.2; Wed, 19 Oct 2016 08:33:14 +0200 Received: from MSXCN2.exchange.alogis.com ([fe80::11b6:f5c4:b8ee:4a89]) by MSXCN4 ([10.1.1.18]) with mapi id 14.03.0319.002; Wed, 19 Oct 2016 08:33:14 +0200 From: Holger Kipp To: Jonathan Haack CC: "stable@freebsd.org" Subject: Re: 11.0 Install Question Thread-Topic: 11.0 Install Question Thread-Index: AQHSKc8smg63RYV9NUOVuYErdO+/f6CvUZ6O Date: Wed, 19 Oct 2016 06:32:52 +0000 Message-ID: References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> In-Reply-To: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> Accept-Language: de-DE, en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: MIME-Version: 1.0 Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 06:33:24 -0000 Dear Jonathan, On 19 Oct 2016, at 08:08, Jonathan Haack > wrote: I am running 10.1 fully update with freebsd-update fetch freebsd-update install then, every time I do freebsd-update -r upgrade 11.0-RELEASE It says it failed an integrity check and cowardly refuses =85 I have follow= ed all forum advise and upgraded pkg, rebooted ran everything again and aga= in =85 cowardly fails each time According to https://www.freebsd.org/releases/11.0R/installation.html this should work. However to give any help or advise you might want to post the exact output = and integrity check information on this list first, as well as your current= FreeBSD Version (freebsd-version -ku) Just in case - you have already backed up all your valuable data and curren= t config-files? Best regards, Holger any way to force / and or re-download and do over? Sincerely, Coach Haack DP Coordinator, Mandela International Magnet School Board of Directors, First Serve NM President, NMCTM www.jonathanhaack.com "It is better to have tried and failed than to have succeeded at doing noth= ing." From owner-freebsd-stable@freebsd.org Wed Oct 19 06:50:00 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B8D71C18E7B for ; Wed, 19 Oct 2016 06:50:00 +0000 (UTC) (envelope-from christopherhall.hsw@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 95FD58D9 for ; Wed, 19 Oct 2016 06:50:00 +0000 (UTC) (envelope-from christopherhall.hsw@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 9288BC18E7A; Wed, 19 Oct 2016 06:50:00 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 922A4C18E79 for ; Wed, 19 Oct 2016 06:50:00 +0000 (UTC) (envelope-from christopherhall.hsw@gmail.com) Received: from mail-pf0-x242.google.com (mail-pf0-x242.google.com [IPv6:2607:f8b0:400e:c00::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 606C78D8 for ; Wed, 19 Oct 2016 06:50:00 +0000 (UTC) (envelope-from christopherhall.hsw@gmail.com) Received: by mail-pf0-x242.google.com with SMTP id s8so1594270pfj.2 for ; Tue, 18 Oct 2016 23:50:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=t538d5vU984HLUESULr3UnfaMThf/Cljwf5i9xX9VOA=; b=TubH+gxutGfjx00FixSeBuipJ4IzNyuf2s+tDcM4O5LGcXuSA0XVOJAUz+wp+GcR5N UvQO2usUUeffKfgga5qQ2VJg4qDUsW8jt54BXtDyH9FgoBnJR1jYpBxcwB9gJK73ATU+ Nv5lZdhe4tS5SUU1KDbS49UEevW3hYdXgyHAJKnn9Kfl3kyOcfeNlqLVTKcRlUaj0ZKx GaRJ0BZyoA1P8N5AUfO1ORp0F9TVyX3B3qxWLmuy5dd3qCggp12iefbtfnqS6kBAH/pO C/i1KQrfJMZ4+pWHC8PAYQUlRJnv17whISuvkRZGUmvI9umD65u5nHmgP2ZodUUMxVpb WduA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=t538d5vU984HLUESULr3UnfaMThf/Cljwf5i9xX9VOA=; b=iAPlcqSNpyRTTc4392+hFg/MhCcKne14OG1rskidlga24YIFQa/Hre9SwkwbommfTJ gBjiCQhF46ycvULT8r4egvAzJF+hzaS/uhlwJaK8XxyPqzmDpli8gHL6A1QGkDx1VJ81 br74Up8oRTBWl3AAV+uvt0zkyHK6fpMXw26yZfpayhjqx9MbETL4KBAArbjQthiQ69Ly QqVX1KiDX1730f6yThg30Jt1gvS+M4GBJzfYBp/P0/qDD9NTMFghmxZV8KvhOV2RORa6 mtkUaWKNWXRwM292TAFOMGT+mGC7qSkm1CAyl7FHooqbdBq+gNHdwTcE8sQX7rqW3lvD 52yw== X-Gm-Message-State: AA6/9RkoPrNmn5p/d5tgLNC3HMvx51ds+ioG9nod3d2ZjGj7QnNRcYMfktnBMMz9BvLbpw== X-Received: by 10.98.101.71 with SMTP id z68mr8208940pfb.13.1476859800033; Tue, 18 Oct 2016 23:50:00 -0700 (PDT) Received: from localhost ([2001:b030:2314:200:4649:583d:1:120]) by smtp.gmail.com with ESMTPSA id y125sm60604920pfb.16.2016.10.18.23.49.59 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 18 Oct 2016 23:49:59 -0700 (PDT) Date: Wed, 19 Oct 2016 14:49:53 +0800 From: Christopher Hall To: Jonathan Haack Cc: stable@FreeBSD.org Subject: Re: 11.0 Install Question Message-ID: <20161019144953.34e63e30@gmail.com> In-Reply-To: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> X-Mailer: Claws Mail 3.14.0 (GTK+ 2.24.29; amd64-portbld-freebsd10.3) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 06:50:00 -0000 Hello Jonathan, On Tue, 18 Oct 2016 23:06:10 -0600, Jonathan Haack wrote: > I am running 10.1 fully update with=20 >=20 > freebsd-update fetch > freebsd-update install >=20 > then, every time I do=20 >=20 > freebsd-update -r upgrade 11.0-RELEASE >=20 > It says it failed an integrity check and cowardly refuses =E2=80=A6 I have > followed all forum advise and upgraded pkg, rebooted ran everything > again and again =E2=80=A6 cowardly fails each time >=20 > any way to force / and or re-download and do over? I just noticed a line that appears to zero out bspatch in the procedure on:=20 https://www.freebsd.org/releases/11.0R/announce.html Have you tried that? >=20 >=20 > Sincerely, >=20 > Coach Haack --=20 Best Regards. Christopher Hall. From owner-freebsd-stable@freebsd.org Wed Oct 19 06:58:18 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3CD79C162E7 for ; Wed, 19 Oct 2016 06:58:18 +0000 (UTC) (envelope-from ml@netfence.it) Received: from smtp206.alice.it (smtp206.alice.it [82.57.200.102]) by mx1.freebsd.org (Postfix) with ESMTP id CC7C2F3D for ; Wed, 19 Oct 2016 06:58:17 +0000 (UTC) (envelope-from ml@netfence.it) Received: from soth.ventu (79.46.7.147) by smtp206.alice.it (8.6.060.28) (authenticated as acanedi@alice.it) id 57FB3880018073BD for freebsd-stable@freebsd.org; Wed, 19 Oct 2016 08:52:26 +0200 Received: from alamar.ventu (alamar.local.netfence.it [10.1.2.18]) by soth.ventu (8.15.2/8.15.2) with ESMTP id u9J6qOeC067100 for ; Wed, 19 Oct 2016 08:52:25 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: soth.ventu: Host alamar.local.netfence.it [10.1.2.18] claimed to be alamar.ventu From: Andrea Venturoli Subject: Nightly disk-related panic since upgrade to 10.3 To: "freebsd-stable@freebsd.org" Message-ID: Date: Wed, 19 Oct 2016 08:52:24 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 06:58:18 -0000 Hello. Last week I upgraded a 9.3/amd64 box to 10.3: since then, it crashed and rebooted at least once every night. The only exception was on Friday, when it locked without rebooting: it still answered ping request and logins through HTTP would half work; I'm under the impression that the disk subsystem was hung, so ICMP would work since it does no I/O and HTTP too worked as far as no disk access was required. Today I was able to get a couple of (almost identical) dumps: > cpuid = 1 > KDB: stack backtrace: > #0 0xffffffff804ee170 at kdb_backtrace+0x60 > #1 0xffffffff804b4576 at vpanic+0x126 > #2 0xffffffff804b4443 at panic+0x43 > #3 0xffffffff8068fd2a at softdep_deallocate_dependencies+0x6a > #4 0xffffffff805394b5 at brelse+0x145 > #5 0xffffffff8053793c at bufwrite+0x3c > #6 0xffffffff806ae20f at ffs_write+0x3df > #7 0xffffffff8076d519 at VOP_WRITE_APV+0x149 > #8 0xffffffff806ec7c9 at vnode_pager_generic_putpages+0x2a9 > #9 0xffffffff8076f3b7 at VOP_PUTPAGES_APV+0xa7 > #10 0xffffffff806ea6f5 at vnode_pager_putpages+0xc5 > #11 0xffffffff806e17f8 at vm_pageout_flush+0xc8 > #12 0xffffffff806db432 at vm_object_page_collect_flush+0x182 > #13 0xffffffff806db1cd at vm_object_page_clean+0x13d > #14 0xffffffff806dadbe at vm_object_terminate+0x8e > #15 0xffffffff806eac60 at vnode_destroy_vobject+0x90 > #16 0xffffffff806b4232 at ufs_reclaim+0x22 > #17 0xffffffff8076e5c7 at VOP_RECLAIM_APV+0xa7 Has anyone any better insight on what might be going on? The disks are all connected to a SAS RAID adapter running on mfi; I don't think it might be an hardware issue, since it has worked perfectly for years until I did the upgrade; also mfiutil says everything is ok and nothing mfi-related is in the logs. Some ideas come to mind about which I might use a second opinion: _ soft-update is broken: that would really surprise me, since I've been using that for years on this and several other boxes (10.3 too); _ snapshot creation/deletion is causing this: again I'm using that almost anywhere, so I don't think this might be the cause alone; besides, I've been able to do some dumps without trouble and I don't think anything was messing with snapshots at the time of the last two panics; _ mfi driver is broken on 10.3: this is more reasonable to me, since this is the only machine I have it on and it's the only case where I get this panics. I found https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=183618, but I get no "g_vfs_done()..." messages. Any other hint? I'd really like to find out what's going on, I'll appreciate any help and I'm willing to provide any useful info. On the other hand, this is a production server, so I have to solve this really soon. Some idea comes to mind, like disabling softupdate (knowing which file system was having trouble would help here; is there any way to know?), trying to enable journaling, upgrading to 10-STABLE, build a kernel with INVARIANTS/WITNESS/etc..., but I'd appreciate a second opinion before I start shooting in the dark. bye & Thanks av. From owner-freebsd-stable@freebsd.org Wed Oct 19 07:02:39 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5516DC16629 for ; Wed, 19 Oct 2016 07:02:39 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 357147A4 for ; Wed, 19 Oct 2016 07:02:39 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: by mailman.ysv.freebsd.org (Postfix) id 34BACC16628; Wed, 19 Oct 2016 07:02:39 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34553C16627 for ; Wed, 19 Oct 2016 07:02:39 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from pv36p08im-ztdg07111401.me.com (pv36p08im-ztdg07111401.me.com [17.142.193.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 13FD77A3 for ; Wed, 19 Oct 2016 07:02:39 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from process-dkim-sign-daemon.pv36p08im-ztdg07111401.me.com by pv36p08im-ztdg07111401.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) id <0OFA00C008VC9000@pv36p08im-ztdg07111401.me.com> for stable@FreeBSD.org; Wed, 19 Oct 2016 07:02:26 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mac.com; s=4d515a; t=1476860546; bh=yW1KIsMvfl2BpuNCkVJlgNDsg6KcI9F+a91J0kbcx3Q=; h=Content-type:MIME-version:Subject:From:Date:Message-id:To; b=lEsj34XWtdxuFDxepgcxk5enXq7LD6Tb8bovgGoXog74MpY2n52nvxW1EqkjlnEa2 vmRhfFiy3QnIWbgHz5qvwqLKodqS1BDjNX3H5i+LG3fdkG926FDhxwKiEzzIgCFIng gJJNCv6A84FSjQ9kXNEVT0kIWiAGYB0ptkjQs0CH4/G5+cj8pqtmmm3JZYSbxt3dR6 PoZiwK7dVNnf+Z5Wp+sOwpPxY1tzp2m+dee2yY9v8hv9H2wTx5EDBQN02Rkpl4ydvM 2ToViUmzDAgT6tLTtXuRO5T1hwYB34DTno+hTp/WKYr05cZ4543yPEIQp2RQt0uF3X OXEKARTS0Bahg== Received: from [192.168.1.41] (c-73-98-2-193.hsd1.nm.comcast.net [73.98.2.193]) by pv36p08im-ztdg07111401.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) with ESMTPSA id <0OFA00NUM8W0NZ40@pv36p08im-ztdg07111401.me.com>; Wed, 19 Oct 2016 07:02:25 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-10-19_02:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1034 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1603290000 definitions=main-1610190126 MIME-version: 1.0 (1.0) Subject: Re: 11.0 Install Question From: Jonathan Haack X-Mailer: iPhone Mail (14A456) In-reply-to: <20161019144953.34e63e30@gmail.com> Date: Wed, 19 Oct 2016 01:02:23 -0600 Cc: stable@FreeBSD.org Message-id: <460C5629-2B6B-4474-97D7-9E82458F43B5@mac.com> References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> To: Christopher Hall Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 07:02:39 -0000 Hey guys ... I got impatient after so many tries that I decided to step to 1= 0.3 first ... that went perfectly and now it is completed the kernel phase o= f 11.0 ... not sure why it wouldn't hop to 11 straight from 10.1 ... I found= some people with same forum and I assure you I had latest patches because i= t was a clean install production machine built off a brand new .iso ... so? = Oh well sorry can't help debug more since mine is now updating ... thanks f= or the help guys Sincerely, Coach Haack DP Coordinator, Mandela International Magnet School Board of Directors, First Serve NM President, NMCTM www.jonathanhaack.com "It is better to have tried and failed than to have succeeded at doing nothi= ng."=20 Confidentiality Notice: This email, including all attachments is for the so= le use of the intended recipient or recipients and may contain confidential i= nformation. Any unauthorized review, use, disclosure, or distribution, is p= rohibited unless specifically provided under the New Mexico Inspection of Pu= blic Records Act. If you are not the intended recipient, please contact the= sender and destroy all copies of this message. > On Oct 19, 2016, at 12:49 AM, Christopher Hall wrote: >=20 > Hello Jonathan, >=20 > On Tue, 18 Oct 2016 23:06:10 -0600, Jonathan Haack > wrote: >=20 >> I am running 10.1 fully update with=20 >>=20 >> freebsd-update fetch >> freebsd-update install >>=20 >> then, every time I do=20 >>=20 >> freebsd-update -r upgrade 11.0-RELEASE >>=20 >> It says it failed an integrity check and cowardly refuses =E2=80=A6 I hav= e >> followed all forum advise and upgraded pkg, rebooted ran everything >> again and again =E2=80=A6 cowardly fails each time >>=20 >> any way to force / and or re-download and do over? >=20 > I just noticed a line that appears to zero out bspatch in the procedure > on:=20 > https://www.freebsd.org/releases/11.0R/announce.html > Have you tried that? >=20 >>=20 >>=20 >> Sincerely, >>=20 >> Coach Haack >=20 > --=20 > Best Regards. > Christopher Hall. From owner-freebsd-stable@freebsd.org Wed Oct 19 08:09:16 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E226EC181E2 for ; Wed, 19 Oct 2016 08:09:16 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id C28EAC74 for ; Wed, 19 Oct 2016 08:09:16 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: by mailman.ysv.freebsd.org (Postfix) id C1E7DC181E1; Wed, 19 Oct 2016 08:09:16 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C1846C181E0 for ; Wed, 19 Oct 2016 08:09:16 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from pv36p08im-ztdg07111401.me.com (pv36p08im-ztdg07111401.me.com [17.142.193.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A4495C73 for ; Wed, 19 Oct 2016 08:09:16 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from process-dkim-sign-daemon.pv36p08im-ztdg07111401.me.com by pv36p08im-ztdg07111401.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) id <0OFA00900BT1AD00@pv36p08im-ztdg07111401.me.com> for stable@FreeBSD.org; Wed, 19 Oct 2016 08:09:13 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mac.com; s=4d515a; t=1476864553; bh=j0cnXOyz9ajxjEZTBSQm6bGOI8g80VF3BpAquB2JnfY=; h=Content-type:MIME-version:Subject:From:Date:Message-id:To; b=D3e7i7/WDnCTLcYZW/QxqOkylVD42DTRiskHRahCXNKbYyPazLDsJCFKZweKNuGU5 5eYt2IlWCbpxDJ4Wtlyv3XH20DZDil27PtXeQPT17pSSglf3ruIvDWkuNsk55Ye5id ZiNyR9MT5nDSet38yQmxbUGgZDktP3nRHu2cqgFYiIZgcSlhMZTsugAZesXl1hQaAE cIcknY8EwO/1uBauOsAIphjUJdRoHHoG0VAhYIm0VVa7wyJPiZPaOA5cX1vmM26gME Mp++bdEnC0gatYRPBM48V8i+Y/DaE1AQxmc3n6xA+n+3LGm8900tlTHDbYWC9e5Ryu FHSez36rEa4LA== Received: from [192.168.1.41] (c-73-98-2-193.hsd1.nm.comcast.net [73.98.2.193]) by pv36p08im-ztdg07111401.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) with ESMTPSA id <0OFA006FKBZ94J20@pv36p08im-ztdg07111401.me.com>; Wed, 19 Oct 2016 08:09:13 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-10-19_07:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1034 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1603290000 definitions=main-1610190147 MIME-version: 1.0 (1.0) Subject: Re: 11.0 Install Question From: Jonathan Haack X-Mailer: iPhone Mail (14A456) In-reply-to: <20161019144953.34e63e30@gmail.com> Date: Wed, 19 Oct 2016 02:09:08 -0600 Cc: stable@FreeBSD.org Message-id: References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> To: Christopher Hall Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 08:09:17 -0000 Awe geez ... pkg won't work ... says "shared object "libssl.so.7" not found,= required by "pkg" Sincerely, Coach Haack DP Coordinator, Mandela International Magnet School Board of Directors, First Serve NM President, NMCTM www.jonathanhaack.com "It is better to have tried and failed than to have succeeded at doing nothi= ng."=20 Confidentiality Notice: This email, including all attachments is for the so= le use of the intended recipient or recipients and may contain confidential i= nformation. Any unauthorized review, use, disclosure, or distribution, is p= rohibited unless specifically provided under the New Mexico Inspection of Pu= blic Records Act. If you are not the intended recipient, please contact the= sender and destroy all copies of this message. > On Oct 19, 2016, at 12:49 AM, Christopher Hall wrote: >=20 > Hello Jonathan, >=20 > On Tue, 18 Oct 2016 23:06:10 -0600, Jonathan Haack > wrote: >=20 >> I am running 10.1 fully update with=20 >>=20 >> freebsd-update fetch >> freebsd-update install >>=20 >> then, every time I do=20 >>=20 >> freebsd-update -r upgrade 11.0-RELEASE >>=20 >> It says it failed an integrity check and cowardly refuses =E2=80=A6 I hav= e >> followed all forum advise and upgraded pkg, rebooted ran everything >> again and again =E2=80=A6 cowardly fails each time >>=20 >> any way to force / and or re-download and do over? >=20 > I just noticed a line that appears to zero out bspatch in the procedure > on:=20 > https://www.freebsd.org/releases/11.0R/announce.html > Have you tried that? >=20 >>=20 >>=20 >> Sincerely, >>=20 >> Coach Haack >=20 > --=20 > Best Regards. > Christopher Hall. From owner-freebsd-stable@freebsd.org Wed Oct 19 08:10:21 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 60643C1828C for ; Wed, 19 Oct 2016 08:10:21 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 3DFB3D9B for ; Wed, 19 Oct 2016 08:10:21 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mailman.ysv.freebsd.org (Postfix) id 3D5E3C18288; Wed, 19 Oct 2016 08:10:21 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D0C9C18287 for ; Wed, 19 Oct 2016 08:10:21 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-io0-x236.google.com (mail-io0-x236.google.com [IPv6:2607:f8b0:4001:c06::236]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0648CD99 for ; Wed, 19 Oct 2016 08:10:21 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-io0-x236.google.com with SMTP id r30so23838389ioi.1 for ; Wed, 19 Oct 2016 01:10:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=sXFfLQT+TW8eB1MuLoD5YwOzJTLEMrZazcyabFuOUn8=; b=zxfJPeziQ9qV4gbDkbN9cUPDKCKzUz9zpVecgAQ4qsUwvNwkwON8jYlfQSlUeT4nT5 dvvub1l/UOfogcBIOhiO3rethfVYunlRjmk7fXrfOzfhzSt9QzV7uuHjE0yHQeguNV/7 OYvWhhO2MM4VFsyLFtHiR3Nr3Hx5jDUnz6Wq4+10KKiqtgko2trLq0i/98o+8+W+SFRX pGTtRGm1uJlnnqvcWm9//3TH36hWbSF9cnXf4mWwP/uVqGJ3wlbAi9ir9VOeI88Ea6K1 QkWHZnD6vhHDIzfh0JaCbGujBOe9ynPuvrntz831p9Th9xsf0Lmm2U34XNPjqtOxJbGW tqGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=sXFfLQT+TW8eB1MuLoD5YwOzJTLEMrZazcyabFuOUn8=; b=jcTMwaN6UHK4pic+IhHxWciwpOADHTFZy/DtgFaaxb7dUprCyz9v2htAujTQaUCV6x JJQ4l9SHDtGBpA65bFNk9SkGzKK9jOYsULU17aNt2kbJnrJc2x5xJrG0UCxfE+eCbGKz LV/Q1U1t4YyBgOMsZ53XAmKL9Lpn6NE4xU5roBzulSP4jWhxCbFZwtPo9g8kmFHWuQRW iae0XFbsDCdxVJMSbgoFgZWO/Q03lxXzP6P07KxCS74AXx20ur6UH5w/mH0LkOvNjHcP ZZnwwxLDluuW1qSn9R8stnvZAwgbrTuUHSSwoLMN7ttIfhr+tScoNY4W/BfQ/0jot8Pw VSYw== X-Gm-Message-State: AA6/9RkVj6GlU5KoGeyGX8l/M56OYvv08yzkuTh6PuFlqW268LG5LBf+5nuja6+EDgvvPaqfzuegmnnTMuGhrw== X-Received: by 10.107.139.79 with SMTP id n76mr5849284iod.166.1476864620396; Wed, 19 Oct 2016 01:10:20 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.77.85 with HTTP; Wed, 19 Oct 2016 01:10:19 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> From: Warner Losh Date: Wed, 19 Oct 2016 02:10:19 -0600 X-Google-Sender-Auth: OSzW9J9YMU6JluOoHRkvojIeOds Message-ID: Subject: Re: 11.0 Install Question To: Jonathan Haack Cc: Christopher Hall , stable@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 08:10:21 -0000 Try pkg-static. On Wed, Oct 19, 2016 at 2:09 AM, Jonathan Haack wrote: > Awe geez ... pkg won't work ... says "shared object "libssl.so.7" not fou= nd, required by "pkg" > > > > > > > > > Sincerely, > > Coach Haack > > DP Coordinator, Mandela International Magnet School > Board of Directors, First Serve NM > President, NMCTM > > www.jonathanhaack.com > > "It is better to have tried and failed than to have succeeded at doing no= thing." > > Confidentiality Notice: This email, including all attachments is for the= sole use of the intended recipient or recipients and may contain confident= ial information. Any unauthorized review, use, disclosure, or distribution= , is prohibited unless specifically provided under the New Mexico Inspectio= n of Public Records Act. If you are not the intended recipient, please con= tact the sender and destroy all copies of this message. > >> On Oct 19, 2016, at 12:49 AM, Christopher Hall wrote: >> >> Hello Jonathan, >> >> On Tue, 18 Oct 2016 23:06:10 -0600, Jonathan Haack >> wrote: >> >>> I am running 10.1 fully update with >>> >>> freebsd-update fetch >>> freebsd-update install >>> >>> then, every time I do >>> >>> freebsd-update -r upgrade 11.0-RELEASE >>> >>> It says it failed an integrity check and cowardly refuses =E2=80=A6 I h= ave >>> followed all forum advise and upgraded pkg, rebooted ran everything >>> again and again =E2=80=A6 cowardly fails each time >>> >>> any way to force / and or re-download and do over? >> >> I just noticed a line that appears to zero out bspatch in the procedure >> on: >> https://www.freebsd.org/releases/11.0R/announce.html >> Have you tried that? >> >>> >>> >>> Sincerely, >>> >>> Coach Haack >> >> -- >> Best Regards. >> Christopher Hall. > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed Oct 19 08:30:59 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34321C18A66 for ; Wed, 19 Oct 2016 08:30:59 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 134AEC43 for ; Wed, 19 Oct 2016 08:30:59 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: by mailman.ysv.freebsd.org (Postfix) id 125F4C18A65; Wed, 19 Oct 2016 08:30:59 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 118C3C18A64 for ; Wed, 19 Oct 2016 08:30:59 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from pv36p08im-ztdg07111401.me.com (pv36p08im-ztdg07111401.me.com [17.142.193.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E7478C41 for ; Wed, 19 Oct 2016 08:30:58 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from process-dkim-sign-daemon.pv36p08im-ztdg07111401.me.com by pv36p08im-ztdg07111401.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) id <0OFA00L00CBK0700@pv36p08im-ztdg07111401.me.com> for stable@freebsd.org; Wed, 19 Oct 2016 08:30:47 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mac.com; s=4d515a; t=1476865847; bh=dJIkzP4cK9IfMDUDDaIGnOwjuUDlLNnlrdKtt5hoaj8=; h=Content-type:MIME-version:Subject:From:Date:Message-id:To; b=QyaTYbT9uIIyy+vIR/mSfG2mJFkzN5K6O5Y/hkR6GliqbC4CI3S189SEqh76hUL0h QIkr9GQasVTdfEPWLnuEf9dWG4rQA3j9SIJRkrmOFgxFwSS6HKVM/r2PGciIUeU/9H 1lipEdykwKL63furWaa7zgRAVXsU2tLWOx9pJkH2QwWaoiODSFsFXLWYM89G4kxKCr b5Je8LecpJjJFm/m/AEIt7jEcP2dsLwj5kE30xbPAx+iupDCeCCBMhefh7pcbmy1Rl YQYiqfT8XDGOHPlhorCNGb1dRs0ztikQ86Nht5Zbo7VVy/zZ9VYlXiI8TUwYxTG8LC wHImqWtEHE8bg== Received: from [192.168.1.41] (c-73-98-2-193.hsd1.nm.comcast.net [73.98.2.193]) by pv36p08im-ztdg07111401.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) with ESMTPSA id <0OFA00NSPCZ8KT20@pv36p08im-ztdg07111401.me.com>; Wed, 19 Oct 2016 08:30:45 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-10-19_07:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1034 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1603290000 definitions=main-1610190153 MIME-version: 1.0 (1.0) Subject: Re: 11.0 Install Question From: Jonathan Haack X-Mailer: iPhone Mail (14A456) In-reply-to: Date: Wed, 19 Oct 2016 02:30:43 -0600 Cc: Christopher Hall , stable@freebsd.org Message-id: References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> To: Warner Losh Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 08:30:59 -0000 Thanks - worked then reinstalled pkg with force Sincerely, Coach Haack DP Coordinator, Mandela International Magnet School Board of Directors, First Serve NM President, NMCTM www.jonathanhaack.com "It is better to have tried and failed than to have succeeded at doing nothi= ng."=20 Confidentiality Notice: This email, including all attachments is for the so= le use of the intended recipient or recipients and may contain confidential i= nformation. Any unauthorized review, use, disclosure, or distribution, is p= rohibited unless specifically provided under the New Mexico Inspection of Pu= blic Records Act. If you are not the intended recipient, please contact the= sender and destroy all copies of this message. > On Oct 19, 2016, at 2:10 AM, Warner Losh wrote: >=20 > Try pkg-static. >=20 >> On Wed, Oct 19, 2016 at 2:09 AM, Jonathan Haack wrote:= >> Awe geez ... pkg won't work ... says "shared object "libssl.so.7" not fou= nd, required by "pkg" >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >>=20 >> Sincerely, >>=20 >> Coach Haack >>=20 >> DP Coordinator, Mandela International Magnet School >> Board of Directors, First Serve NM >> President, NMCTM >>=20 >> www.jonathanhaack.com >>=20 >> "It is better to have tried and failed than to have succeeded at doing no= thing." >>=20 >> Confidentiality Notice: This email, including all attachments is for the= sole use of the intended recipient or recipients and may contain confidenti= al information. Any unauthorized review, use, disclosure, or distribution, i= s prohibited unless specifically provided under the New Mexico Inspection of= Public Records Act. If you are not the intended recipient, please contact t= he sender and destroy all copies of this message. >>=20 >>> On Oct 19, 2016, at 12:49 AM, Christopher Hall wrote: >>>=20 >>> Hello Jonathan, >>>=20 >>> On Tue, 18 Oct 2016 23:06:10 -0600, Jonathan Haack >>> wrote: >>>=20 >>>> I am running 10.1 fully update with >>>>=20 >>>> freebsd-update fetch >>>> freebsd-update install >>>>=20 >>>> then, every time I do >>>>=20 >>>> freebsd-update -r upgrade 11.0-RELEASE >>>>=20 >>>> It says it failed an integrity check and cowardly refuses =E2=80=A6 I h= ave >>>> followed all forum advise and upgraded pkg, rebooted ran everything >>>> again and again =E2=80=A6 cowardly fails each time >>>>=20 >>>> any way to force / and or re-download and do over? >>>=20 >>> I just noticed a line that appears to zero out bspatch in the procedure >>> on: >>> https://www.freebsd.org/releases/11.0R/announce.html >>> Have you tried that? >>>=20 >>>>=20 >>>>=20 >>>> Sincerely, >>>>=20 >>>> Coach Haack >>>=20 >>> -- >>> Best Regards. >>> Christopher Hall. >> _______________________________________________ >> freebsd-stable@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-stable >> To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"= From owner-freebsd-stable@freebsd.org Wed Oct 19 08:38:37 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34EB4C18C09 for ; Wed, 19 Oct 2016 08:38:37 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 20084180 for ; Wed, 19 Oct 2016 08:38:37 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: by mailman.ysv.freebsd.org (Postfix) id 1CA6CC18C08; Wed, 19 Oct 2016 08:38:37 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1AA25C18C07 for ; Wed, 19 Oct 2016 08:38:37 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CF24117F for ; Wed, 19 Oct 2016 08:38:36 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id F16FF28412; Wed, 19 Oct 2016 10:38:26 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 1CCB228411; Wed, 19 Oct 2016 10:38:26 +0200 (CEST) Subject: Re: 11.0 Install Question To: Jonathan Haack , Christopher Hall Cc: stable@FreeBSD.org References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <58073101.2000203@quip.cz> Date: Wed, 19 Oct 2016 10:38:25 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 08:38:37 -0000 Jonathan Haack wrote on 2016/10/19 10:09: > Awe geez ... pkg won't work ... says "shared object "libssl.so.7" not found, required by "pkg" Do not panic, just run pkg-static to upgrade pkg it-self /usr/local/sbin/pkg-static install -f pkg Miroslav Lachman From owner-freebsd-stable@freebsd.org Wed Oct 19 08:55:09 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE96BC18414 for ; Wed, 19 Oct 2016 08:55:09 +0000 (UTC) (envelope-from matt.xtaz@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B92FA31F for ; Wed, 19 Oct 2016 08:55:09 +0000 (UTC) (envelope-from matt.xtaz@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id B5A1DC18413; Wed, 19 Oct 2016 08:55:09 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B538BC18412 for ; Wed, 19 Oct 2016 08:55:09 +0000 (UTC) (envelope-from matt.xtaz@gmail.com) Received: from mail-wm0-x230.google.com (mail-wm0-x230.google.com [IPv6:2a00:1450:400c:c09::230]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3B7E831E for ; Wed, 19 Oct 2016 08:55:09 +0000 (UTC) (envelope-from matt.xtaz@gmail.com) Received: by mail-wm0-x230.google.com with SMTP id z189so37956469wmb.1 for ; Wed, 19 Oct 2016 01:55:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:in-reply-to:user-agent; bh=YVuw1K2Mp4bt9pruS7avF8XkaebrBsxHjPBToCrvpX0=; b=sJ92n6uhi1oE5Rraz9kbzyMj13b/WRiBuJhb6b72198OSRGKcgfRWKs6JX2eh9sTzo OP6JacZA6FI1vp8Ltx1NlkI+bk4BjbE+PiVBNE4226yNCoHqRLsBc7nSACAK7HL9iQXC ToRffY9mVaMVHnBAwadiKQlv1wVOlHVScvf+LuTZZ3K8qgKuSQIG/yB6kvNKMZlyHWL5 Ao8lM/Eju7L24BAQ7+qoL3vZXHJ0w/FDC59IV66tSoUFS9+Knopedb7KwGdEW3k1bxjo ZzlbDrhU0N1iecWwFTkSg1cuM6LLDW8Noos+q3DlT2MrJ6/Gm1WyseBhGBrm04giVjyz HsXQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to:user-agent; bh=YVuw1K2Mp4bt9pruS7avF8XkaebrBsxHjPBToCrvpX0=; b=mJiEDIVctk6hrFKmABBTYeK4othSMu0AqJg/m7J1qz/Zgvjjb5PvTWv3yj0l7OupDB mleXMRKoRN+B2aFRyjgxB0H9XF8yLW1xDDbkH2mLXd6oApQChEx6MPqhPqqEjUTP0+9a kV9bj0GZbmGa35TCMp/uTsTdktwcqtJ80oVmQoV9siiyTS+dPgzmcpUem1BFpt8e4Oud dHgvqtPaFQsXfZnppFnp3l+508/IpDf1mOV2SVD+p9/5wrlEQsYuaKNRy3ZS5Wz7Roqe zr7aK7K3EU9ESIP91hDhBWmVNGqf9gEXyVKnQclFSxrfmXpnCqLhn1U9IwTAC+/feDZw H79Q== X-Gm-Message-State: AA6/9Rlaqt/DJLm7L/+yC4E0R02UWPmCDv1XEpsS4Dn8+3OzaqooXpzOZIj2fIcyXgVWCQ== X-Received: by 10.28.209.142 with SMTP id i136mr1983062wmg.1.1476867306467; Wed, 19 Oct 2016 01:55:06 -0700 (PDT) Received: from xtaz.uk (tao.xtaz.uk. [2001:8b0:fe33::10]) by smtp.gmail.com with ESMTPSA id wh3sm67473885wjb.49.2016.10.19.01.55.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 19 Oct 2016 01:55:06 -0700 (PDT) Date: Wed, 19 Oct 2016 09:55:04 +0100 From: Matt Smith To: Miroslav Lachman <000.fbsd@quip.cz> Cc: Jonathan Haack , Christopher Hall , stable@FreeBSD.org Subject: Re: 11.0 Install Question Message-ID: <20161019085504.GA37097@xtaz.uk> Mail-Followup-To: Matt Smith , Miroslav Lachman <000.fbsd@quip.cz>, Jonathan Haack , Christopher Hall , stable@FreeBSD.org References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> <58073101.2000203@quip.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Disposition: inline In-Reply-To: <58073101.2000203@quip.cz> User-Agent: Mutt/1.7.1 (2016-10-04) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 08:55:10 -0000 On Oct 19 10:38, Miroslav Lachman wrote: >Jonathan Haack wrote on 2016/10/19 10:09: >>Awe geez ... pkg won't work ... says "shared object "libssl.so.7" not found, required by "pkg" > >Do not panic, just run pkg-static to upgrade pkg it-self > >/usr/local/sbin/pkg-static install -f pkg > I think that these instructions should be in the announcement release notes as this question comes up a lot. The notes just say something like "upgrade all your packages". Most people will then just try and run pkg upgrade and get that error without knowing how to solve it. It's not intuitive unless you have come across this before. -- Matt From owner-freebsd-stable@freebsd.org Wed Oct 19 09:08:23 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABAEAC18A25 for ; Wed, 19 Oct 2016 09:08:23 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [81.2.117.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3E5D8CEE for ; Wed, 19 Oct 2016 09:08:22 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from zero-gravitas.local (unknown [85.199.232.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id DB33DE3C1 for ; Wed, 19 Oct 2016 09:08:12 +0000 (UTC) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org Authentication-Results: smtp.infracaninophile.co.uk/DB33DE3C1; dkim=none; dkim-atps=neutral Subject: Re: 11.0 Install Question To: freebsd-stable@freebsd.org References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> <58073101.2000203@quip.cz> <20161019085504.GA37097@xtaz.uk> From: Matthew Seaman Message-ID: Date: Wed, 19 Oct 2016 10:08:04 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161019085504.GA37097@xtaz.uk> Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Q38B71iJdsSExtvOHvwUeh8bOhrlhwwV4" X-Spam-Status: No, score=-0.4 required=5.0 tests=BAYES_00,RDNS_NONE, SPF_SOFTFAIL autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on smtp.infracaninophile.co.uk X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 09:08:23 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Q38B71iJdsSExtvOHvwUeh8bOhrlhwwV4 Content-Type: multipart/mixed; boundary="pigqo9onFDorkPKrg2KtoIMHiFl0P3bNu"; protected-headers="v1" From: Matthew Seaman To: freebsd-stable@freebsd.org Message-ID: Subject: Re: 11.0 Install Question References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> <58073101.2000203@quip.cz> <20161019085504.GA37097@xtaz.uk> In-Reply-To: <20161019085504.GA37097@xtaz.uk> --pigqo9onFDorkPKrg2KtoIMHiFl0P3bNu Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2016/10/19 09:55, Matt Smith wrote: > On Oct 19 10:38, Miroslav Lachman wrote: >> Jonathan Haack wrote on 2016/10/19 10:09: >>> Awe geez ... pkg won't work ... says "shared object "libssl.so.7" not= >>> found, required by "pkg" >> >> Do not panic, just run pkg-static to upgrade pkg it-self >> >> /usr/local/sbin/pkg-static install -f pkg >> >=20 > I think that these instructions should be in the announcement release > notes as this question comes up a lot. The notes just say something lik= e > "upgrade all your packages". Most people will then just try and run pkg= > upgrade and get that error without knowing how to solve it. It's not > intuitive unless you have come across this before. pkg(8) does have a built-in warning when it detects an OS major version upgrade, which is to do a forced upgrade of pkg(8) so it matches the ABI versions of system shlibs. Did that warning appear in this case? Cheers, Matthew --pigqo9onFDorkPKrg2KtoIMHiFl0P3bNu-- --Q38B71iJdsSExtvOHvwUeh8bOhrlhwwV4 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQJ8BAEBCgBmBQJYBzf8XxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQxOUYxNTRFQ0JGMTEyRTUwNTQ0RTNGMzAw MDUxM0YxMEUwQTlFNEU3AAoJEABRPxDgqeTnUzYP+QHxHNwSw9niwmcMOtL/kUDq uvTA2AijSKYMYlsslPKnRW9WNdqzhjW7t28w0f+KmqPhepNWh7P9XxKo9JlG9Dz4 I7n0MYTG0XkHytdZoVpmw3DCZHCn8D0Xozheege/XT9zz51U/XEMr2itBiFE23uf 3FN9nwr8Ft14hBeKWWEfZQ4+GpVRpQJfbUKV9D9YZ5AXrbadmz0Qtk/rrmjxVzVz fy4GZGOTxxrQfyhtfSIyKXfrLonaP4lLfC2UbXr25aoO0di6Ko4CFQ7d+P8Fohsk SgCIy10/HSnVZrSxVI5fqF6WrPAv4V8lMVdh4WkH8lm8PJR1zJRFEehmLxyTqesi 1Blhsqj1HP7AlaACvR2KCBGhtROktO+IlQZTwA3/Ul2xHV8ggXIfsQeNFlZUpYyU eBPDI+VqSGqNeEBQQhSxXjIgo4hMSx68hKRmKldSHtL5Z2kQcCiZD7ZJ3bMIKOt3 +bEuz6p5TGUdAvwhJcmfYVTUUFEGj5Prxin/YdhzbYi0cKlOg8PIyNsxRaqCYfaw tQhdUnF0TB2o35MySgvIwRsaGu8CqMdxtO3B4fb02PlsPEz9iLRluu+gc78Cu3vL eSSWG6tVqgvxj7ulLIAAffFO1qbxO4IhhVgIVo1q71ccr1NJ8bIOaNJGL8HYgV0C o4HuCTmUdIssxhYcStpa =s0bm -----END PGP SIGNATURE----- --Q38B71iJdsSExtvOHvwUeh8bOhrlhwwV4-- From owner-freebsd-stable@freebsd.org Wed Oct 19 09:37:30 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DB22AC15209 for ; Wed, 19 Oct 2016 09:37:30 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9FE752B2; Wed, 19 Oct 2016 09:37:30 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id BA21F28454; Wed, 19 Oct 2016 11:37:28 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 01EC328433; Wed, 19 Oct 2016 11:37:27 +0200 (CEST) Subject: Re: 11.0 Install Question To: Matthew Seaman , freebsd-stable@freebsd.org References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> <58073101.2000203@quip.cz> <20161019085504.GA37097@xtaz.uk> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <58073ED7.3030504@quip.cz> Date: Wed, 19 Oct 2016 11:37:27 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 09:37:30 -0000 Matthew Seaman wrote on 2016/10/19 11:08: > On 2016/10/19 09:55, Matt Smith wrote: >> On Oct 19 10:38, Miroslav Lachman wrote: >>> Jonathan Haack wrote on 2016/10/19 10:09: >>>> Awe geez ... pkg won't work ... says "shared object "libssl.so.7" not >>>> found, required by "pkg" >>> >>> Do not panic, just run pkg-static to upgrade pkg it-self >>> >>> /usr/local/sbin/pkg-static install -f pkg >>> >> >> I think that these instructions should be in the announcement release >> notes as this question comes up a lot. The notes just say something like >> "upgrade all your packages". Most people will then just try and run pkg >> upgrade and get that error without knowing how to solve it. It's not >> intuitive unless you have come across this before. > > pkg(8) does have a built-in warning when it detects an OS major version > upgrade, which is to do a forced upgrade of pkg(8) so it matches the ABI > versions of system shlibs. Did that warning appear in this case? I think it is not shown if pkg is missing some libraries. It is shown when OS major version changes and you run pkg befor old libraries are deleted or if you run pkg-static: pkg-static: Warning: Major OS version upgrade detected. Running "pkg-static install -f pkg" recommended Miroslav Lachman From owner-freebsd-stable@freebsd.org Wed Oct 19 09:57:44 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A87EC15AE5 for ; Wed, 19 Oct 2016 09:57:44 +0000 (UTC) (envelope-from adg@a-real.ru) Received: from mbox1.a-real.ru (mbox1.a-real.ru [213.187.98.202]) by mx1.freebsd.org (Postfix) with ESMTP id 27665E36 for ; Wed, 19 Oct 2016 09:57:43 +0000 (UTC) (envelope-from adg@a-real.ru) Received: by mbox1.a-real.ru (Sendmail, from userid 1002) id 9867012548B; Wed, 19 Oct 2016 12:55:20 +0300 (MSK) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=a-real.ru; s=mail; t=1476870920; bh=luH4kUnUE7HCsAIC17wckKQFOrXOxNt1db4AAa0wxnA=; h=Date:From:To:Subject; b=o/2OMAle/p4Z4OlZljWIaaTH2jLRxz9yM4RY10pa5j7KUKiaJ/7h+VUlGDXQakOo/ 0kIkvVXkRTpkICeU66vUNMDWmobPEg+X1cCdpXty2c9tE/ePIOAJ3Gn39ZqFkm2GOK KuHDwE2xm9jwGVhQIxW3ync6xuZOfNa3J0h8W1jY= X-Envelope-Sender: adg@a-real.ru X-Envelope-Recipient: freebsd-stable@freebsd.org X-Original-To: freebsd-stable@freebsd.org Received: from mbox1.a-real.ru (localhost [127.0.0.1]) by mbox1.a-real.ru (Sendmail) with ESMTP id CA85C125489 for ; Wed, 19 Oct 2016 12:55:08 +0300 (MSK) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 19 Oct 2016 12:55:08 +0300 From: adg@a-real.ru To: freebsd-stable@freebsd.org Subject: VT not showing cyrillic characters in text mode Message-ID: <382398c982b00cdf34eccb4f56950871@a-real.ru> X-Sender: adg@a-real.ru User-Agent: Roundcube Webmail/1.1.1 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 09:57:44 -0000 Hello everyone. Cannot figure out if vt console driver supports non-ansi characters when started in textmode. The font in sys/dev/vt/vt_font_default.c seems to include cyrillic glyphs, but either it is not being used in textmode or something else is broken since only '?' are displayed instead of proper symbols. The LANG/LC_ALL doesn't affect anything. The problem occurs on 11.0-RELEASE and 10.3-RELEASE. Textmode is required for me since vt performance in graphical mode is very poor on HyperV and textmode looks like the only option. (sorry about the previous messy messages in this mailing list, didnt expect problems with html emails ;) -- Alexey Guskov From owner-freebsd-stable@freebsd.org Wed Oct 19 10:27:06 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAB59C1789D for ; Wed, 19 Oct 2016 10:27:06 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail.madpilot.net (grunt.madpilot.net [78.47.145.38]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6A92D163; Wed, 19 Oct 2016 10:27:05 +0000 (UTC) (envelope-from mad@madpilot.net) Received: from mail (mail [192.168.254.3]) by mail.madpilot.net (Postfix) with ESMTP id 3szSd816C1zbDF; Wed, 19 Oct 2016 12:20:44 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=madpilot.net; h= content-transfer-encoding:content-type:content-type:in-reply-to :mime-version:user-agent:date:date:message-id:from:from :references:subject:subject:received:received; s=mail; t= 1476872442; x=1478686843; bh=XsX5paRn98M02dVatJyKPCgaEP7MfmJhsrK WzWhZKSA=; b=V8+bDSP9JrQ2ZwZCAfKonREVkZJumBEY1w2uJRVYSFbnvocEtUM kk+lPdNN7UnMmQmtXvA+DrnlpFwIMDAqbOu9oRDBga1ySwgcSuYU63ffCnBwpvAE s+2SuLSuQSfWE4qEqLVomOWsB3/T+JMzMPDdj/1hu9KI0eN2sBx9CLPw= Received: from mail.madpilot.net ([192.168.254.3]) by mail (mail.madpilot.net [192.168.254.3]) (amavisd-new, port 10024) with ESMTP id JPnPGtFr7-cJ; Wed, 19 Oct 2016 12:20:42 +0200 (CEST) Received: from marvin.madpilot.net (micro.madpilot.net [88.149.173.206]) by mail.madpilot.net (Postfix) with ESMTPSA; Wed, 19 Oct 2016 12:20:42 +0200 (CEST) Subject: Re: 11.0 Install Question To: Miroslav Lachman <000.fbsd@quip.cz>, Matthew Seaman , freebsd-stable@freebsd.org References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> <58073101.2000203@quip.cz> <20161019085504.GA37097@xtaz.uk> <58073ED7.3030504@quip.cz> From: Guido Falsi Message-ID: <8ad1fc5b-1b57-c031-a0d1-5a92a64ad3ac@madpilot.net> Date: Wed, 19 Oct 2016 12:20:41 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <58073ED7.3030504@quip.cz> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 10:27:06 -0000 On 10/19/16 11:37, Miroslav Lachman wrote: > Matthew Seaman wrote on 2016/10/19 11:08: >> On 2016/10/19 09:55, Matt Smith wrote: >>> On Oct 19 10:38, Miroslav Lachman wrote: >>>> Jonathan Haack wrote on 2016/10/19 10:09: >>>>> Awe geez ... pkg won't work ... says "shared object "libssl.so.7" not >>>>> found, required by "pkg" >>>> >>>> Do not panic, just run pkg-static to upgrade pkg it-self >>>> >>>> /usr/local/sbin/pkg-static install -f pkg >>>> >>> >>> I think that these instructions should be in the announcement release >>> notes as this question comes up a lot. The notes just say something like >>> "upgrade all your packages". Most people will then just try and run pkg >>> upgrade and get that error without knowing how to solve it. It's not >>> intuitive unless you have come across this before. >> >> pkg(8) does have a built-in warning when it detects an OS major version >> upgrade, which is to do a forced upgrade of pkg(8) so it matches the ABI >> versions of system shlibs. Did that warning appear in this case? > > I think it is not shown if pkg is missing some libraries. > It is shown when OS major version changes and you run pkg befor old > libraries are deleted or if you run pkg-static: > > pkg-static: Warning: Major OS version upgrade detected. Running > "pkg-static install -f pkg" recommended > The correct procedure (and I think it's documented somewhere) is to run "pkg upgrade -f" before removing the old libraries. If compiling from sources that is before make delete-old-libraries and is explicitly stated when running make delete-old to upgrade ports before going on. If upgrading using freebsd-update that's before running "freebsd-update install" for the third time(if I remember correctly) and I think I remember freebsd-update stating something about upgrading installed packages, but I'm not 100% sure. -- Guido Falsi From owner-freebsd-stable@freebsd.org Wed Oct 19 10:47:56 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B5F6BC17D13 for ; Wed, 19 Oct 2016 10:47:56 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id A827EC78; Wed, 19 Oct 2016 10:47:56 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id D4C9E117; Wed, 19 Oct 2016 10:47:56 +0000 (UTC) Date: Wed, 19 Oct 2016 10:47:56 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <506245977.10.1476874076104.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <1319508584.7.1476852514221.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1319508584.7.1476852514221.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #432 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 10:47:56 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/432/ From owner-freebsd-stable@freebsd.org Wed Oct 19 14:07:46 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 64184C18D6C for ; Wed, 19 Oct 2016 14:07:46 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 42F02F4F for ; Wed, 19 Oct 2016 14:07:46 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: by mailman.ysv.freebsd.org (Postfix) id 42521C18D69; Wed, 19 Oct 2016 14:07:46 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41FA6C18D67 for ; Wed, 19 Oct 2016 14:07:46 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from pv36p08im-ztdg07111401.me.com (pv36p08im-ztdg07111401.me.com [17.142.193.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 232C0F4E for ; Wed, 19 Oct 2016 14:07:46 +0000 (UTC) (envelope-from oemb1905@mac.com) Received: from process-dkim-sign-daemon.pv36p08im-ztdg07111401.me.com by pv36p08im-ztdg07111401.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) id <0OFA00A00S3A5100@pv36p08im-ztdg07111401.me.com> for stable@FreeBSD.org; Wed, 19 Oct 2016 14:07:37 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mac.com; s=4d515a; t=1476886057; bh=+0UQ9ml7QpEqmOkL5CxtyxWHKmFBkZeeqyPF7X1OPfs=; h=Content-type:MIME-version:Subject:From:Date:Message-id:To; b=bEqbPpzX0/jUQ3dYmaAyJzTjIghIFmd+SdYTh3AtqpYatJy2Hr27V8OhWKqidXQQx 5dOAhUfMizeOB1lqX07vTmlWY3zNzeYL4Mb54LV4upfTO1yBCxPGneSqHPG6Y5Ngun ry85bhhbFWlNHdvA4/grtYukTP52Omo6gZb2230Jw1LwjADBokSq/EAtPPp92QWHKt iMiwI0pQPRsnQgu3+zfLJn7lTbDPVZnFRG9f3mPd5ccSgPRvMM7WKgziD19QuwH5H5 nlgreBC7mP7VTXQfdgH1Q6OZn79kAX6hjFUzeKzTQ3j7oL6LDfJIu19IoZ2YINzJbp IST/yi5Wnc38A== Received: from [192.168.1.41] (c-73-98-2-193.hsd1.nm.comcast.net [73.98.2.193]) by pv36p08im-ztdg07111401.me.com (Oracle Communications Messaging Server 7.0.5.38.0 64bit (built Feb 26 2016)) with ESMTPSA id <0OFA00FGWSKM6H00@pv36p08im-ztdg07111401.me.com>; Wed, 19 Oct 2016 14:07:36 +0000 (GMT) X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-10-19_07:,, signatures=0 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 clxscore=1034 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1603290000 definitions=main-1610190254 MIME-version: 1.0 (1.0) Subject: Re: 11.0 Install Question From: Jonathan Haack X-Mailer: iPhone Mail (14A456) In-reply-to: <20161019085504.GA37097@xtaz.uk> Date: Wed, 19 Oct 2016 08:07:34 -0600 Cc: Miroslav Lachman <000.fbsd@quip.cz>, Christopher Hall , stable@FreeBSD.org Message-id: <75FE1B87-FD4B-4894-9408-08CD164AF349@mac.com> References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> <58073101.2000203@quip.cz> <20161019085504.GA37097@xtaz.uk> To: Matt Smith Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 14:07:46 -0000 Totally agree and since even though I had followed the instructions prior to= that step, it had also fried my mate installation and GUI so ... I could no= t access control center or anything since it was gone ... I almost wiped and= reverted back to 10.1 misinterpreting the outcome first ... once I ran that= command, I reinstalled everything again and it fixed the mate package and m= achine was back to normal ... Thanks again rocking now Sincerely, Coach Haack DP Coordinator, Mandela International Magnet School Board of Directors, First Serve NM President, NMCTM www.jonathanhaack.com "It is better to have tried and failed than to have succeeded at doing nothi= ng."=20 Confidentiality Notice: This email, including all attachments is for the so= le use of the intended recipient or recipients and may contain confidential i= nformation. Any unauthorized review, use, disclosure, or distribution, is p= rohibited unless specifically provided under the New Mexico Inspection of Pu= blic Records Act. If you are not the intended recipient, please contact the= sender and destroy all copies of this message. > On Oct 19, 2016, at 2:55 AM, Matt Smith wrote: >=20 >> On Oct 19 10:38, Miroslav Lachman wrote: >> Jonathan Haack wrote on 2016/10/19 10:09: >>> Awe geez ... pkg won't work ... says "shared object "libssl.so.7" not fo= und, required by "pkg" >>=20 >> Do not panic, just run pkg-static to upgrade pkg it-self >>=20 >> /usr/local/sbin/pkg-static install -f pkg >>=20 >=20 > I think that these instructions should be in the announcement release note= s as this question comes up a lot. The notes just say something like "upgrad= e all your packages". Most people will then just try and run pkg upgrade and= get that error without knowing how to solve it. It's not intuitive unless y= ou have come across this before. >=20 > --=20 > Matt From owner-freebsd-stable@freebsd.org Wed Oct 19 15:00:46 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D94DEC18F01 for ; Wed, 19 Oct 2016 15:00:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x231.google.com (mail-it0-x231.google.com [IPv6:2607:f8b0:4001:c0b::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A078918A for ; Wed, 19 Oct 2016 15:00:46 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x231.google.com with SMTP id 66so30635145itl.1 for ; Wed, 19 Oct 2016 08:00:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=CVhm4RUazQZyNmB5TwGUweQsPXOQ40QR5VwXi7Mi9rg=; b=L5Pfs3pSGsMewY5htE3rjZXW6e5wx8uA3Bn7+KWGQDc/zjWN533EPdFCMVo9nIlZGu nQIze5nHDBa2sJwdxkjh90VGQ6CFjgP4f61U463ffcpmTIwk8jXao84Rp0KWpp3TbKWg dqJY5LqcoHDaztguu2FVguYGSfW8mnfnAbmvgGCYlwB3O6qjicJlBUlvJtYiMNXiaEsH 4UxEBi/wWBSaMOb+DDGX0mEK/ZbCkAGiru/Ipa1abbIPc5BzKGoru1cHV5yQaiN2ivoe Th82Rg2oFI+npKOZG4PVUaQRTyy6oGfubVOc+w2PopGB/RPvmB7Gsqa3gUzlvQrce8ID 3Wgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=CVhm4RUazQZyNmB5TwGUweQsPXOQ40QR5VwXi7Mi9rg=; b=hg0n7oXeVgoV3gIcMKDhvp6ITLryZFFskYTfpZ1DE++gUZRL6M+RfCafqSxZJt4q1w YtLBcq7h/WG19s6XTsSztJn9gP4oI3e0e2WAsNZZRNE3VFQV+QRB3zK5UDxKl1ZoXzEH 3omAfdKtbDT511DH5PSTim4X1c3JtHGvGZrXNMTW4h7B3OIGGRtzZB0rctEpKVcmYU7Z pHOIFWtCzz5MQKkDcDAxzMguOKWz4RGLvuGItQRH/Jm41yjQ2nIjmmUT4wT1ncuMqZTH sjMq9frqMZQX5SiaoVGb+CVP2kwhDDKHsF4yaoEfSAbSkVEhxaflbCLdoJWAVaAqpP7F jq0w== X-Gm-Message-State: AA6/9RlvDymcd72LUc2u8S9k8JiX/cBvlvtHMxP+BFcOYh3w7AxsUD4x6mDbeu53+kEbEFKIArVsiGQtiYuUSw== X-Received: by 10.36.19.147 with SMTP id 141mr3318426itz.85.1476889245981; Wed, 19 Oct 2016 08:00:45 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.77.85 with HTTP; Wed, 19 Oct 2016 08:00:45 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: <84890108-CB52-4B2B-B795-C650879469E9@FreeBSD.org> References: <20161016122041.e20ae504c58d4f9224eec6bf@getmail.no> <426D0291-D785-4CFE-A15D-9BE688499ACB@FreeBSD.org> <84890108-CB52-4B2B-B795-C650879469E9@FreeBSD.org> From: Warner Losh Date: Wed, 19 Oct 2016 09:00:45 -0600 X-Google-Sender-Auth: RqDyaeQz0UrOofyuUHmVBIZedcI Message-ID: Subject: Re: Building FreeBSD 11.0-stable on FreeBSD 10.1-stable fails To: Dimitry Andric Cc: Bryan Drewery , Torfinn Ingolfsen , FreeBSD-STABLE Mailing List Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 15:00:46 -0000 On Sun, Oct 16, 2016 at 12:18 PM, Dimitry Andric wrote: > On 16 Oct 2016, at 17:22, Warner Losh wrote: >> >> On Sun, Oct 16, 2016 at 5:34 AM, Dimitry Andric wrote: >>> On 16 Oct 2016, at 12:20, Torfinn Ingolfsen wrote: >>>> I am trying to build FreeBSD 11.0-stable on a machine which runs: >>>> tingo@kg-v7$ uname -a >>>> FreeBSD kg-v7.kg4.no 10.1-STABLE FreeBSD 10.1-STABLE #0 r278322: Fri Feb 6 21:36:01 CET 2015 >>>> root@kg-v7.kg4.no:/usr/obj/usr/src/sys/GENERIC amd64 >>>> >>>> I have emptied /usr/src and /usr/obj and fetched the latest stable/11 via subversion: >>>> tingo@kg-v7$ egrep "^BRANCH|^REVISION" /usr/src/sys/conf/newvers.sh >>>> REVISION="11.0" >>>> BRANCH="STABLE" >>>> >>>> But building it (per the procedure in the handbook) fails at the buildworld stage. Both 'make -j5 buildworld' and 'make buildworld' fails, like this: >>>> >>>> c++: error: unable to execute command: Segmentation fault (core dumped) > ... >>> Please make sure your stable/10 is at least r286033. >> >> What's the issue this fixes? > > It fixes a possible crash in clang 3.4, which can occur if newer > versions of llvm are compiled. Unfortunately this fix only went in > after 10.3-RELEASE. I think you're confused. In prepping a patch, it looks like it went in after 9.3-RELEASE on stable/9, but it went in between 10.2 and 10.3 on stable/10. So 10.1R and 10.2R are busted, but 10.3R should work. I've updated UPDATING to reflect this. Warner From owner-freebsd-stable@freebsd.org Wed Oct 19 15:17:10 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 00FFCC185DF; Wed, 19 Oct 2016 15:17:10 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Received: from mel1.ec-m.fr (melout.ec-m.fr [147.94.19.33]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B12E2155; Wed, 19 Oct 2016 15:17:08 +0000 (UTC) (envelope-from dgeo@centrale-marseille.fr) Subject: Re: LOR in mpr(4) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=centrale-marseille.fr; s=smtp0; t=1476889837; bh=3Q4QABkDoWOPUt6vBS7UekHdYExpY5WwsFoUVWSs2Ts=; h=Subject:To:References:From:Date:In-Reply-To; b=B7/aMMVrmoPlQxcScXBWVLprMV27bpe3nfbdNDIGpHL3A/aBwSrHMYYBP/kNexfQf 5q+Id1TExD/9nN9Y5NQ1cp8fcAFTXtU+Y5Bc0DzDr+y9z5syH9r/w4wV3EuaPc2X9D FIGgo4M4geiRlC9F3q12KFxKcHf8YPy8XkAPbeyo+mERtop/UoSUUPiEPwK7ca4XZe AakKjG2cqR2NPayautS9rMo+oTX4+dGYk6U7xVZCw1uyurg+bk7H+4kwoS5lyuB0Tx G60SZfci69bRSEDRAh0nXJTbP6FhvU5fSGvHkBVI7h+ahMOitvcx416irWtRczcDrf 00GQTTwIt5L5w== To: Pete Wright , current@freebsd.org, freebsd-stable@FreeBSD.org References: <5644D014.4080601@nomadlogic.org> <564B917E.4000205@nomadlogic.org> From: geoffroy desvernay Openpgp: id=E15095B3F06A1012EE2921023FCFF4094587A0F0 Message-ID: <9fc856a5-c91d-cdfa-5000-5d8fc8ea20f1@centrale-marseille.fr> Date: Wed, 19 Oct 2016 17:10:31 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <564B917E.4000205@nomadlogic.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="UDifcBq6JcAxma9IkvI9nI6cJpcWrqj9N" X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 15:17:10 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --UDifcBq6JcAxma9IkvI9nI6cJpcWrqj9N Content-Type: multipart/mixed; boundary="U4VW04IbcVi4a8sNMqML5ekh2TMcPn22x"; protected-headers="v1" From: geoffroy desvernay To: Pete Wright , current@freebsd.org, freebsd-stable@FreeBSD.org Message-ID: <9fc856a5-c91d-cdfa-5000-5d8fc8ea20f1@centrale-marseille.fr> Subject: Re: LOR in mpr(4) References: <5644D014.4080601@nomadlogic.org> <564B917E.4000205@nomadlogic.org> In-Reply-To: <564B917E.4000205@nomadlogic.org> --U4VW04IbcVi4a8sNMqML5ekh2TMcPn22x Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 11/17/2015 21:43, Pete Wright wrote: >=20 >=20 > On 11/12/15 09:44, Pete Wright wrote: >> Hi All, >> Just wanted a sanity check before filing a PR. I am running r290688 a= nd >> am seeing a LOR being triggered in the mpr(4) device: >> >> $ uname -ar >> FreeBSD srd0013 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r290688: Wed Nov = 11 >> 21:28:26 PST 2015 root@srd0013:/usr/obj/usr/src/sys/GENERIC amd64= >> >> >> lock order reversal: >> 1st 0xfffff8000d26bc60 CAM device lock (CAM device lock) @ >> /usr/src/sys/cam/cam_xpt.c:784 >> 2nd 0xfffffe00012811c0 MPR lock (MPR lock) @ >> /usr/src/sys/cam/cam_xpt.c:2620 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >> 0xfffffe04608ee890 >> witness_checkorder() at witness_checkorder+0xe79/frame 0xfffffe04608ee= 910 >> __mtx_lock_flags() at __mtx_lock_flags+0xa4/frame 0xfffffe04608ee960 >> xpt_action_default() at xpt_action_default+0xb6c/frame 0xfffffe04608ee= 9b0 >> scsi_scan_bus() at scsi_scan_bus+0x1d5/frame 0xfffffe04608eea20 >> xpt_scanner_thread() at xpt_scanner_thread+0x15c/frame 0xfffffe04608ee= a70 >> fork_exit() at fork_exit+0x84/frame 0xfffffe04608eeab0 >> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe04608eeab0 >> --- trap 0, rip =3D 0, rsp =3D 0, rbp =3D 0 --- >> >=20 > FWIW I filed the following PR as I can still reproduce this on boot: >=20 > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D204614 >=20 > cheers, > -pete >=20 Hi all, Sorry for cross-posting, let me know where this should go please, I didn't figured it out :( On 11-RELEASE-p1 here (but replying on current@ where I found something around mpr(4)) Not sure if it's related, but on a fresh new machine with Avago SAS3008 and a 24 disks enclosure (single attached). I see a bunch of: mpr0: Found device <401,End Device> <12.0Gbps> handle<0x001b> enclosureHandle<0x0002> slot 8 (da0:mpr0:0:8:0): UNMAPPED (da0:mpr0:0:8:0): CAM status: SCSI Status Error (da0:mpr0:0:8:0): SCSI status: Check Condition (da0:mpr0:0:8:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code) (da0:mpr0:0:8:0): Error 22, Unretryable error 10:0): UNMAPPED (da0:mpr0:0:8:0): READ(10). CDB: 28 00 e8 e0 88 71 00 00 04 00 (da0:mpr0:0:8:0): CAM status: SCSI Status Error (da0:mpr0:0:8:0): SCSI status: Check Condition (da0:mpr0:0:8:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command operation code) (da0:mpr0:0:8:0): Error 22, Unretryable error ses0: da0: Element descriptor: 'Drive Slot 0' ses0: da0: SAS Device Slot Element: 2 Phys at Slot 0 ses0: phy 0: SAS device type 1 id 0 ses0: phy 0: protocols: Initiator( None ) Target( SSP ) ses0: phy 0: parent 520474729974b57f addr 5000c50097ce8215 ses0: phy 1: SAS device type 1 id 1 ses0: phy 1: protocols: Initiator( None ) Target( SSP ) ses0: phy 1: parent 520474729974b5ff addr 5000c50097ce8216 (more complete dmesg.boot here: http://dgeo.perso.ec-m.fr/dmesg.boot ) Later, no way to use these disks with zfs: # zpool create tank da0 cannot create 'tank': invalid argument for this pool operation I can dd if=3D/dev/zero of=3D/dev/da0 though not tested until disk is ful= l=85 Can this be related ? Must I open a pr ? How can I help debugging this ? I'm not kernel/driver hacker, but I'd like to help this be figured out :)= Yours, --=20 *geoffroy desvernay* C.R.I - Administration syst=E8mes et r=E9seaux Ecole Centrale de Marseille --U4VW04IbcVi4a8sNMqML5ekh2TMcPn22x-- --UDifcBq6JcAxma9IkvI9nI6cJpcWrqj9N Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQIcBAEBCAAGBQJYB4zrAAoJED/P9AlFh6DwErcP/2ZZwg4jr+wsC8NJHAxHbQXk rmWc1VucMbYDBEZDH3tjjcIm3wlCYXbklMUzPamRGWqU1WOC3YQ8YHsra467psls SVeaC2VNOr8/6vWEvpzjv8nwxVXDj84tVOb0bcHt3exkHff6F0OwE9pEHoxNblfq EqJzxO6A8r2nAHGAvnhdHNKxGlmfWeG7Tb9dRi9vhOwXbE5c55rjlKpul0JF63+A 9hMHNd0TyTk3uNTr9IDhAlbADMdYcCpqVyHfBIK37EwevK7zzakIPV9baDu/gl54 iRQYSW1h+Nuf3SlhlyU4I0ls1kHiBLkXTW7nvTSzbxVG0YZH317E+RKMt6bAyoeM G79tkQd+sOiEkQsyj+TohqsPSQd8Ai4uVvBxFJ8zS95nHnVMSyGFd71pOs1nmh/z sTOst78pxn0wR5HheCof22C/ijI0ivymk9+jCIB5fYuY9meq/vTz7DPK4vVvI9Mk bN2lniqot4F36ZCyC/sO8akNCzE8trLE9EMc4+IgKZJXQWBSSA7/8D7DiKdPH5Ex jZgY0U2Rfod49sp5vTaWFSovWsppSOIx7DarDfBzNCHdxVXbTc4/idLDtnND7atZ jARSpcF99SIWiPvAUWSn0nRECf1Gbn/x+6tbMro5+DYu2hGMKDoUdxCssi43TfK5 lGKR/MDejJJvJU6I2+kA =L047 -----END PGP SIGNATURE----- --UDifcBq6JcAxma9IkvI9nI6cJpcWrqj9N-- From owner-freebsd-stable@freebsd.org Wed Oct 19 16:39:28 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2323C18C34; Wed, 19 Oct 2016 16:39:28 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from vps-mail.nomadlogic.org (unknown [IPv6:2607:f2f8:a098::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CA31FF01; Wed, 19 Oct 2016 16:39:28 +0000 (UTC) (envelope-from pete@nomadlogic.org) Received: from la-dgt-31327.usa.tribune.com (nat-192-187-90-113.nat.tribpub.com [192.187.90.113]) by vps-mail.nomadlogic.org (OpenSMTPD) with ESMTPSA id 5fac46bc TLS version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO; Wed, 19 Oct 2016 09:39:27 -0700 (PDT) Subject: Re: LOR in mpr(4) To: geoffroy desvernay , current@freebsd.org, freebsd-stable@FreeBSD.org References: <5644D014.4080601@nomadlogic.org> <564B917E.4000205@nomadlogic.org> <9fc856a5-c91d-cdfa-5000-5d8fc8ea20f1@centrale-marseille.fr> From: Pete Wright Message-ID: Date: Wed, 19 Oct 2016 09:39:27 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <9fc856a5-c91d-cdfa-5000-5d8fc8ea20f1@centrale-marseille.fr> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 16:39:29 -0000 On 10/19/16 8:10 AM, geoffroy desvernay wrote: > On 11/17/2015 21:43, Pete Wright wrote: >> >> >> On 11/12/15 09:44, Pete Wright wrote: >>> Hi All, >>> Just wanted a sanity check before filing a PR. I am running r290688 and >>> am seeing a LOR being triggered in the mpr(4) device: >>> >>> $ uname -ar >>> FreeBSD srd0013 11.0-CURRENT FreeBSD 11.0-CURRENT #1 r290688: Wed Nov 11 >>> 21:28:26 PST 2015 root@srd0013:/usr/obj/usr/src/sys/GENERIC amd64 >>> >>> >>> lock order reversal: >>> 1st 0xfffff8000d26bc60 CAM device lock (CAM device lock) @ >>> /usr/src/sys/cam/cam_xpt.c:784 >>> 2nd 0xfffffe00012811c0 MPR lock (MPR lock) @ >>> /usr/src/sys/cam/cam_xpt.c:2620 >>> KDB: stack backtrace: >>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >>> 0xfffffe04608ee890 >>> witness_checkorder() at witness_checkorder+0xe79/frame 0xfffffe04608ee910 >>> __mtx_lock_flags() at __mtx_lock_flags+0xa4/frame 0xfffffe04608ee960 >>> xpt_action_default() at xpt_action_default+0xb6c/frame 0xfffffe04608ee9b0 >>> scsi_scan_bus() at scsi_scan_bus+0x1d5/frame 0xfffffe04608eea20 >>> xpt_scanner_thread() at xpt_scanner_thread+0x15c/frame 0xfffffe04608eea70 >>> fork_exit() at fork_exit+0x84/frame 0xfffffe04608eeab0 >>> fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe04608eeab0 >>> --- trap 0, rip = 0, rsp = 0, rbp = 0 --- >>> >> >> FWIW I filed the following PR as I can still reproduce this on boot: >> >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204614 >> >> cheers, >> -pete >> > Hi all, > > Sorry for cross-posting, let me know where this should go please, I > didn't figured it out :( > > On 11-RELEASE-p1 here (but replying on current@ where I found something > around mpr(4)) > > Not sure if it's related, but on a fresh new machine with Avago SAS3008 > and a 24 disks enclosure (single attached). > > I see a bunch of: > > mpr0: Found device <401,End Device> <12.0Gbps> handle<0x001b> > enclosureHandle<0x0002> slot 8 > (da0:mpr0:0:8:0): UNMAPPED > (da0:mpr0:0:8:0): CAM status: SCSI Status Error > (da0:mpr0:0:8:0): SCSI status: Check Condition > (da0:mpr0:0:8:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command > operation code) > (da0:mpr0:0:8:0): Error 22, Unretryable error > 10:0): UNMAPPED > (da0:mpr0:0:8:0): READ(10). CDB: 28 00 e8 e0 88 71 00 00 04 00 > (da0:mpr0:0:8:0): CAM status: SCSI Status Error > (da0:mpr0:0:8:0): SCSI status: Check Condition > (da0:mpr0:0:8:0): SCSI sense: ILLEGAL REQUEST asc:20,0 (Invalid command > operation code) > (da0:mpr0:0:8:0): Error 22, Unretryable error > ses0: da0: Element descriptor: 'Drive Slot 0' > ses0: da0: SAS Device Slot Element: 2 Phys at Slot 0 > ses0: phy 0: SAS device type 1 id 0 > ses0: phy 0: protocols: Initiator( None ) Target( SSP ) > ses0: phy 0: parent 520474729974b57f addr 5000c50097ce8215 > ses0: phy 1: SAS device type 1 id 1 > ses0: phy 1: protocols: Initiator( None ) Target( SSP ) > ses0: phy 1: parent 520474729974b5ff addr 5000c50097ce8216 > > (more complete dmesg.boot here: http://dgeo.perso.ec-m.fr/dmesg.boot ) > the issue you are seeing is most likely not related to the LOR from the original email and PR I filed. This looks like a media error with the disk device on your RAID controller. A quick google search turn's up quite a few threads on this - ranging from bad RAID/JBOD controllers to out of date firmware. Cheers, -pete -- Pete Wright pete@nomadlogic.org nomadlogicLA From owner-freebsd-stable@freebsd.org Wed Oct 19 16:48:34 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 74947C1919D for ; Wed, 19 Oct 2016 16:48:34 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 66CDEAA8; Wed, 19 Oct 2016 16:48:34 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id D7D1B11E; Wed, 19 Oct 2016 16:48:34 +0000 (UTC) Date: Wed, 19 Oct 2016 16:48:34 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <1784153898.11.1476895714776.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <506245977.10.1476874076104.JavaMail.jenkins@jenkins-9.freebsd.org> References: <506245977.10.1476874076104.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #433 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 16:48:34 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/433/ From owner-freebsd-stable@freebsd.org Wed Oct 19 17:10:31 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95A08C197BF for ; Wed, 19 Oct 2016 17:10:31 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5801B6DF for ; Wed, 19 Oct 2016 17:10:31 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bwuNo-0005v3-66 for freebsd-stable@FreeBSD.org; Wed, 19 Oct 2016 20:10:28 +0300 Date: Wed, 19 Oct 2016 20:10:28 +0300 From: Slawa Olhovchenkov To: freebsd-stable@FreeBSD.org Subject: tcsh is not handled correctly UTF-8 in arguments Message-ID: <20161019171028.GF57876@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 17:10:31 -0000 tcsh called by sshd for invocation of scp: `tcsh -c scp -f òÁÓÐÉÓÁÎÉÅ.pdf` At this time no any LC_* is set. tcsh read .cshrc and set LC_CTYPE=ru_RU.UTF-8 LC_COLLATE=ru_RU.UTF-8. After this invocation of scp will be incorrect: 00007ab0 20 2d 66 20 c3 90 c2 a0 c3 90 c2 b0 c3 91 c2 81 | -f ............| 00007ac0 c3 90 c2 bf c3 90 c2 b8 c3 91 c2 81 c3 90 c2 b0 |................| 00007ad0 c3 90 c2 bd c3 90 c2 b8 c3 90 c2 b5 5f c3 90 c2 |............_...| 00007ae0 a2 c3 90 c2 97 c3 90 c2 98 2e 70 64 66 0a |..........pdf. | Correct invocation must be: 00000000 20 2d 66 20 | -f | 00000010 d0 a0 d0 b0 d1 81 d0 bf d0 b8 d1 81 d0 b0 d0 bd |................| 00000020 d0 b8 d0 b5 5f d0 a2 d0 97 d0 98 2e 70 64 66 0a |...._.......pdf.| `d0` => `c3 90` `a0` => `c2 a0` I.e. every byte re-encoded to utf-8: `d0` => `c3 90` As result imposible to access files w/ non-ascii names. From owner-freebsd-stable@freebsd.org Wed Oct 19 18:21:05 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4E17C19E3A for ; Wed, 19 Oct 2016 18:21:05 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: from mail-lf0-x22d.google.com (mail-lf0-x22d.google.com [IPv6:2a00:1450:4010:c07::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 63153CEE for ; Wed, 19 Oct 2016 18:21:05 +0000 (UTC) (envelope-from rosti.bsd@gmail.com) Received: by mail-lf0-x22d.google.com with SMTP id l131so34248271lfl.2 for ; Wed, 19 Oct 2016 11:21:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=Zago0aBfZCSRMcyDqTtwpf0rtPtY+TeSE86py4hZ9no=; b=sIRLxEi6IEFNzx7L1vaDirhx7RGa7RYYQ6pP47p6ZBqIR9lva2WlGS3WsbHcKv/RgT Hv+TYIkUeQLjwSoekreazbL9pr7XFwYmoXpHTvfRUVTuHJDeZelp/oZVYIYMQjpkXkT0 MW826hAVqoCo5zo9CErcHOlX9bJ8cN0LHn5g5IZtCGQo/oOjfku22vdmT2/zJkKHa+1H TsubS1fvR05unGt/kKLCCW3diPwcuG9FJ5tKyGGiK9DPMMVKfr43j4+MluPRIDlZnmTn rSX1uJiJt+kUORWAOJs2bRVDpxthY2A3Xsm4hlsjXrSOXioSWV/pHGdFGnIvVcXdoaWq EYig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=Zago0aBfZCSRMcyDqTtwpf0rtPtY+TeSE86py4hZ9no=; b=bipbSaahxOFZGGSvTo4KJ4JpljZYuNDJPbxr6OpCKMncpTpI0Kx4UBoNMV9gsSVAH1 G6FolYx5Os0HRXPyz8PVOblZLP4UY5bmWlhKxFKJYoeemBvoqcuXQDmmESo/5TVEfGu3 qItsIAGpYmLn1Bm//l/gDjLVtEMoOS0z7yf71q09wysV9wWzmF44IsEklGgr2RieAeDY eV0hTTlfOx9GRHjEfQF0K93m+aB1Htl9UiY2JTdSjb8zC2d50/HfY+o6clEl1Uyvr9ra Aq2Re4QCk8ExBOZrQ9NFykuc6+WhGFug313NEikfszp8akD0d7jcaf08jGeOCLks3NPP mCHw== X-Gm-Message-State: AA6/9RlsIzs8D1BwK8CZoywX5uedx91W52FoT26Exah5vvd1ZVg5R2WbB3b1QJPBVEb0zSPkE3sI/u2ybT4Znw== X-Received: by 10.28.228.5 with SMTP id b5mr5824934wmh.101.1476901261594; Wed, 19 Oct 2016 11:21:01 -0700 (PDT) MIME-Version: 1.0 Received: by 10.194.158.228 with HTTP; Wed, 19 Oct 2016 11:21:00 -0700 (PDT) In-Reply-To: References: From: Rostislav Krasny Date: Wed, 19 Oct 2016 21:21:00 +0300 Message-ID: Subject: Re: I'm upset about FreeBSD To: freebsd-stable Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 18:21:05 -0000 On Tue, Oct 18, 2016 at 21:57:29 +1100, Ian Smith wrote: > > If FreeBSD GPT images (and Kindle readers) can trigger this, so could a > theoretically unlimited combination of data on block 2 of USB media; > modifying FreeBSD to fix a Windows bug should be out of the question. Not modifying FreeBSD and not fixing Windows bug but modifying the FreeBSD installation media and working around the Windows bug to let people install FreeBSD without disappointing at very beginning. Why GPT is used in the memstick images at all? Why they can't be MBR based? I think they can. From owner-freebsd-stable@freebsd.org Wed Oct 19 18:27:17 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0B385C1800A for ; Wed, 19 Oct 2016 18:27:17 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from mail.in-addr.com (mail.in-addr.com [IPv6:2a01:4f8:191:61e8::2525:2525]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CB2C5127; Wed, 19 Oct 2016 18:27:16 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by mail.in-addr.com with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bwva5-0007DE-Tr; Wed, 19 Oct 2016 19:27:13 +0100 Date: Wed, 19 Oct 2016 19:27:13 +0100 From: Gary Palmer To: "Kevin P. Neal" Cc: Matthew Seaman , freebsd-stable@freebsd.org Subject: Re: 11.0 Install Question Message-ID: <20161019182713.GA73102@in-addr.com> References: <53CC5785-5ABC-4235-9E3B-ECF6A9E0A74B@mac.com> <20161019144953.34e63e30@gmail.com> <58073101.2000203@quip.cz> <20161019085504.GA37097@xtaz.uk> <20161019175523.GA37473@neutralgood.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161019175523.GA37473@neutralgood.org> X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: gpalmer@freebsd.org X-SA-Exim-Scanned: No (on mail.in-addr.com); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 18:27:17 -0000 On Wed, Oct 19, 2016 at 01:55:23PM -0400, Kevin P. Neal wrote: > On Wed, Oct 19, 2016 at 10:08:04AM +0100, Matthew Seaman wrote: > > On 2016/10/19 09:55, Matt Smith wrote: > > > On Oct 19 10:38, Miroslav Lachman wrote: > > >> Jonathan Haack wrote on 2016/10/19 10:09: > > >>> Awe geez ... pkg won't work ... says "shared object "libssl.so.7" not > > >>> found, required by "pkg" > > >> > > >> Do not panic, just run pkg-static to upgrade pkg it-self > > >> > > >> /usr/local/sbin/pkg-static install -f pkg > > >> > > > > > > I think that these instructions should be in the announcement release > > > notes as this question comes up a lot. The notes just say something like > > > "upgrade all your packages". Most people will then just try and run pkg > > > upgrade and get that error without knowing how to solve it. It's not > > > intuitive unless you have come across this before. > > > > pkg(8) does have a built-in warning when it detects an OS major version > > upgrade, which is to do a forced upgrade of pkg(8) so it matches the ABI > > versions of system shlibs. Did that warning appear in this case? > > Loading of shared libraries generally happens before main() is entered. > It follows that pkg wouldn't have a chance to do anything useful before > dying if a library is missing. Hi, I don't think this is strictly true On one of my systems at least (10.x), the main pkg binary is in /usr/sbin which is a shim which either offers to install a full pkg binary or calls the full pkg binary with whatever arguments you passed it. The main pkg binary lives in /usr/local/sbin/pkg. It seems it should be possible to add some kind of check to the shim to "do the right thing" and offer to use pkg-static to upgrade the main pkg binary if the dynamically linked one is broken. Or maybe it should default to calling pkg-static instead of pkg, which would prevent the problem in the first place. Of course this assumes the user has /usr/sbin before /usr/local/sbin on their path, but I suspect that is true in most cases Regards, Gary From owner-freebsd-stable@freebsd.org Wed Oct 19 19:46:56 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 211B9C17961 for ; Wed, 19 Oct 2016 19:46:56 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 135B5ACA; Wed, 19 Oct 2016 19:46:56 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id AECE2122; Wed, 19 Oct 2016 19:46:55 +0000 (UTC) Date: Wed, 19 Oct 2016 19:46:51 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <443838324.12.1476906411451.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <1784153898.11.1476895714776.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1784153898.11.1476895714776.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #434 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 19:46:56 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/434/ From owner-freebsd-stable@freebsd.org Wed Oct 19 22:39:14 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C33CC192AF for ; Wed, 19 Oct 2016 22:39:14 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22a.google.com (mail-it0-x22a.google.com [IPv6:2607:f8b0:4001:c0b::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1338C9F9 for ; Wed, 19 Oct 2016 22:39:14 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22a.google.com with SMTP id 66so48846614itl.1 for ; Wed, 19 Oct 2016 15:39:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=PeZ/qie/TnDZLEspM55S8CjEJT9OfpN4FP06dHuAfT4=; b=K8Fdrq/oEZxareKTWy4996U18AB7CDmLYCvIlnJmUvc5C4/pMI5DfysKk+qiy6JJd9 vUje/vNasURuXfLcsveKwGTzL81WtwdxzxpDGePRHgW1R3Nv3bRlvTy4dHMc74AfziIl IS9GyUSzxRATLMClFAx5fCPcy7LVTVw4XFaEEWbKLfTCYmybODJD8oCdbDonp96qs/g+ 0RLwFSW9G6e3NdJf/H0V4fTP3qjCGJ5Nzf7S0gxnIPbKxxy01xAbEJRji4k2TKMjbpmx aQb81PhDe5g2hjhkEl5nLKYeQx4jzWaCrXhIyRH57Rl2XwUqQfxmh19SGKNqHX55kcRv Qsag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=PeZ/qie/TnDZLEspM55S8CjEJT9OfpN4FP06dHuAfT4=; b=m48BbgbObq0/7KgBSi0Zfj+IdiFScY0nPYFuw5JHzATN/+AEyUdY9Z9aeEic1v9dFL ynwz2eeJhXS5+tJu+bQ8u3hGgu+eVMpDGHqUI2FXu6FpqZdbiXxxMYwqMWKqeRsWxV51 bhGXrLGD9gAIHbuziWP5CaIq+F5G7qqcxMMOY1vYgLdU/sUfhHPVZoBoMJk95QaKmnVN ADNyAcUZSXSyMCnzYnSeeg4IRecHkYoIjc1wmZgFaPLNi2DTR0DTk9r9qcToxHxqexlf 09YniVT+YO8L3mjO+6LLAv/U5NZugeApzI5aPVdIjAOwQafYQRAW0BkhA6+9+jDdkTSX jVgw== X-Gm-Message-State: AA6/9RkPY4l1vcgmhReh/h9nSPw6tDvnArWYjxyvRoqfq5JvqosZdsh939gjM10i1+hP6cs0FWJll6lSSMCgUg== X-Received: by 10.107.15.222 with SMTP id 91mr9383090iop.19.1476916753472; Wed, 19 Oct 2016 15:39:13 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.77.85 with HTTP; Wed, 19 Oct 2016 15:39:12 -0700 (PDT) X-Originating-IP: [50.253.99.174] In-Reply-To: References: From: Warner Losh Date: Wed, 19 Oct 2016 16:39:12 -0600 X-Google-Sender-Auth: _UAUrMbZsbmV60m5yMKRiNlPJy0 Message-ID: Subject: Re: I'm upset about FreeBSD To: Rostislav Krasny Cc: freebsd-stable Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 22:39:14 -0000 On Wed, Oct 19, 2016 at 12:21 PM, Rostislav Krasny wrote: > On Tue, Oct 18, 2016 at 21:57:29 +1100, Ian Smith wrote: >> >> If FreeBSD GPT images (and Kindle readers) can trigger this, so could a >> theoretically unlimited combination of data on block 2 of USB media; >> modifying FreeBSD to fix a Windows bug should be out of the question. > > Not modifying FreeBSD and not fixing Windows bug but modifying the > FreeBSD installation media and working around the Windows bug to let > people install FreeBSD without disappointing at very beginning. Why > GPT is used in the memstick images at all? Why they can't be MBR > based? I think they can. Can't boot UEFI off of MBR disks on all BIOSes. Warner From owner-freebsd-stable@freebsd.org Wed Oct 19 23:30:10 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EB8F9C191A8 for ; Wed, 19 Oct 2016 23:30:10 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: from mail-vk0-x22e.google.com (mail-vk0-x22e.google.com [IPv6:2607:f8b0:400c:c05::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A28D6B12 for ; Wed, 19 Oct 2016 23:30:10 +0000 (UTC) (envelope-from kob6558@gmail.com) Received: by mail-vk0-x22e.google.com with SMTP id 2so49017954vkb.3 for ; Wed, 19 Oct 2016 16:30:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=qDXWsjjDALDFfU28sq3a1HjzD0r+Bi/IwEsw0v6qaUo=; b=KwB7vUYaIyozYubpDgJBV0SF3g2RjjhEB1wkRsD5hR24DzW/tDJ4DgRQr5aX+j1RT5 eZa/Fki5djYhnx4XQo3ArcOfi6ExhaYe5uCDRAb6x4gb0ay2Yrpm/oGQ7nkXWTx1fiu8 Gby9gY2N7XVYbiNIyt27pI9FV0edC7IwQXgbVOuClUIak5P20820VLyk7BtoZZ1AqV66 /E4LV64Zh2RXNxK6k7GFqo9KEfbD4ikOVM3h3SSwFXRBhZxjwae8FP/uWRRUqRXvttt1 W9nr1M7GvguBqKn77apMwb5SS+nw/FVlOWsaO7YUIySsYKsRo+2wxGiDg1FctZwKU7Lv quxg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=qDXWsjjDALDFfU28sq3a1HjzD0r+Bi/IwEsw0v6qaUo=; b=M3xiOD0N0KJkqbIljxPjnN7SVoJK6GW5PsSyl7+c8vuA47HS6EjkkQSIN5EOrOA0nR Dd7EIeP3CeEFjPS0n+SS48WpRkMS/2wYulupAFsn4Nalfq+CBv+dUS2D1R4IInjr3rwN G3fsFRozu7/BrEKD4KBij75QqXWovHy/gf49VtREeD48abKeL8BfILeSYZUmb5/MjUri PB17u2ulRKtIybTCYCl56dwZpvycErzMYv8wq+4xui8HTom6uVLr3/EnAGyMItlTCGiR /sHuG1B6cGVshy/hxAkGI+SPnRPkrlKVo9cSLDUDszJl8tMcB4vSqph7aw6Z/bb6l4t8 CVqw== X-Gm-Message-State: AA6/9RnCzIjHYpcgUcEixz7R1XBdqKs4JCUIJw5EX3jZrmFUi/QVyjF7Vdzve8jfgiBOtJqCkQcngj7jVNuv6A== X-Received: by 10.31.229.133 with SMTP id c127mr8005289vkh.153.1476919809600; Wed, 19 Oct 2016 16:30:09 -0700 (PDT) MIME-Version: 1.0 Sender: kob6558@gmail.com Received: by 10.103.146.21 with HTTP; Wed, 19 Oct 2016 16:30:09 -0700 (PDT) In-Reply-To: References: From: Kevin Oberman Date: Wed, 19 Oct 2016 16:30:09 -0700 X-Google-Sender-Auth: oMfpl1Xb9Q7BvTKQCgwnL5fVJ4g Message-ID: Subject: Re: I'm upset about FreeBSD To: Warner Losh Cc: Rostislav Krasny , freebsd-stable Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 23:30:11 -0000 On Wed, Oct 19, 2016 at 3:39 PM, Warner Losh wrote: > On Wed, Oct 19, 2016 at 12:21 PM, Rostislav Krasny > wrote: > > On Tue, Oct 18, 2016 at 21:57:29 +1100, Ian Smith > wrote: > >> > >> If FreeBSD GPT images (and Kindle readers) can trigger this, so could a > >> theoretically unlimited combination of data on block 2 of USB media; > >> modifying FreeBSD to fix a Windows bug should be out of the question. > > > > Not modifying FreeBSD and not fixing Windows bug but modifying the > > FreeBSD installation media and working around the Windows bug to let > > people install FreeBSD without disappointing at very beginning. Why > > GPT is used in the memstick images at all? Why they can't be MBR > > based? I think they can. > > Can't boot UEFI off of MBR disks on all BIOSes. > > Warner > I'll go one farther. You can't boot many new PCs with traditional MBR disks. And. please don't confuse GPT with UEFI. I have yet to find an amd64 computer that has a problem with a GPT format with MBR. Due to broken BIOS, my 5-year old ThinkPad won't boot UEFI, but it has no problem with MBR, whether GPT formatted or not. As far as I know, the 11.0 memstick images are still MBR, just on a GPT structured disk, not UEFI. (Let me know if I am mistaken on this.) I do accept that some early amd64 systems and, perhaps, many i386 systems may have problems with GPT, but GPT was designed to be compatible with traditional disk formats and, while they may have problems, they really should work for single partition disks. And I understand that it is frustrating if you hit one of these cases where it fails. -- Kevin Oberman, Part time kid herder and retired Network Engineer E-mail: rkoberman@gmail.com PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683 From owner-freebsd-stable@freebsd.org Thu Oct 20 10:49:43 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4A24BC18E5C for ; Thu, 20 Oct 2016 10:49:43 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 38BCAA24; Thu, 20 Oct 2016 10:49:43 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 493C413B; Thu, 20 Oct 2016 10:49:43 +0000 (UTC) Date: Thu, 20 Oct 2016 10:49:43 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <1725723762.13.1476960583086.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <443838324.12.1476906411451.JavaMail.jenkins@jenkins-9.freebsd.org> References: <443838324.12.1476906411451.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #435 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 10:49:43 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/435/ From owner-freebsd-stable@freebsd.org Thu Oct 20 13:47:28 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2EFF1C19B9D for ; Thu, 20 Oct 2016 13:47:28 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A4E35279 for ; Thu, 20 Oct 2016 13:47:27 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e] ([IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9KDlMw6019309 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 20 Oct 2016 18:47:23 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1476971243; bh=0np+EwXrxDgdAz8gVQbWAM45oc3km6QW3M1pDsD6MT0=; h=To:From:Subject:Date; b=Q7X2uyTQIeJvr/LGESKHVWCJHqnCboM2QKVpm0uwS6S3PmHg8CZKvyCz29H91xE7I E5lHsE/QTdEUcWs2Ptp5PvgeLrEhdZ4LYo5GYcR9A1e/ahjNZlyg3x4lC8LiAmW2RO DcKM/xzwQ5EiNYghUN71ShA731p670K601OFl2EI= To: freebsd-stable@FreeBSD.org From: "Eugene M. Zheganin" Subject: zfs, a directory that used to hold lot of files and listing pause Message-ID: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> Date: Thu, 20 Oct 2016 18:47:28 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 13:47:28 -0000 Hi. I have FreeBSD 10.2-STABLE r289293 (but I have observed this situation on different releases) and a zfs. I also have one directory that used to have a lot of (tens of thousands) files. I surely takes a lot of time to get a listing of it. But now I have 2 files and a couple of dozens directories in it (I sorted files into directories). Surprisingly, there's still a lag between "ls" and an output: ===Cut=== # /usr/bin/time -h ls .recycle 2016-01 2016-04 2016-07 2016-10 sort-files.sh 2014 2016-02 2016-05 2016-08 ktrace.out sort-months.sh 2015 2016-03 2016-06 2016-09 old sounds 5.75s real 0.00s user 0.02s sys ===Cut=== I've seen this situation before, on other servers, so it's not the first time I encounter this. However, it's not 100% reproducible (I mean, if I fill the directory with dozens of thousands of files, I will not certainly get this lag after the deletion). Has anyone seen this and does anyone know how to resolve this ? It's not critical issue, but it makes thing uncomfortable here. One method I'm aware of: you can move the contents of this directory to some other place, then delete it and create again. But it's kind of a nasty workaround. Thanks. Eugene. From owner-freebsd-stable@freebsd.org Thu Oct 20 13:54:42 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 56617C19F1D for ; Thu, 20 Oct 2016 13:54:42 +0000 (UTC) (envelope-from nicolas.gilles@gmail.com) Received: from mail-lf0-x233.google.com (mail-lf0-x233.google.com [IPv6:2a00:1450:4010:c07::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D30BDAFF for ; Thu, 20 Oct 2016 13:54:41 +0000 (UTC) (envelope-from nicolas.gilles@gmail.com) Received: by mail-lf0-x233.google.com with SMTP id b75so83089712lfg.3 for ; Thu, 20 Oct 2016 06:54:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=yE2qyBgvTlzY8BqDKgz2v6pbJxu9X8fot3pd//qm0RE=; b=dod05uDh84+9zNj7Pf0eUpK73l/CioFibVgT2q6BHIKy21xEtzF0/zKAf5ZU8sxin4 9es4eAvhJFD4j/yV70Q9xPAFZZQsPTcXAsFfy35TkugHc8R7pJqJC7wxHl6ruQ2aXO8i kxbKOSbKs/3Spxbkx9GBE8+KHFP9J5WUmQA5aXSc3Cz4zyCHgiLeY4AToQA8c2UrBnAO TDhZcHoWrzCD+wYczDTev2707kWk/6Dv+KYeoDuxP1i4DIo3UkYXo3RMtXOhq5KKeFk5 vaB8zczX93UhyJZPoKbgFOuBF3Vdsh6hocfMcCva15tOgR63qt16RBvrX1dl1ewgM8wu 7DUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=yE2qyBgvTlzY8BqDKgz2v6pbJxu9X8fot3pd//qm0RE=; b=Y4eTkms3I/l8J6Y+x64RcTr1Kqc3HUmKNwiU5wtmj/8r4Z4IepRvexHmXEgyCeLEcQ HV9nTTffblX5m1h+HRoL+NxVK/SDsYSX8pKPdG6iHBfCiqd8SeKR1jmS1Chw3eRqFBYP +b9/3pyLp8jlsqd+hvz5BcFUWJgDMgSc9JmG1FaWc1XjY7J0N99qDkGm7pEJ+UTh7OxS H5IoW6gB1uLJU+5v/L+HGNuG5mo9aQfud0mCBJz0eSlEdfxP3tyrXn5Vk/XTESEfWvbi 52eTxJyt3ui0YPtiweDxpNzCI5HELfOivv8QfFQ8nEkqUQrukw/teEY/XklX4s1TVH3H jCBg== X-Gm-Message-State: AA6/9RkVO3A8ksyd7RzpcinR53ASv8ZKNT4hu2J9OtOk06jKbF7vWl3uC0ivqH83DRSUFzCnfIveYljnd/WJmQ== X-Received: by 10.25.190.71 with SMTP id o68mr1827895lff.23.1476971679577; Thu, 20 Oct 2016 06:54:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.195.147 with HTTP; Thu, 20 Oct 2016 06:54:38 -0700 (PDT) In-Reply-To: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> From: Nicolas Gilles Date: Thu, 20 Oct 2016 15:54:38 +0200 Message-ID: Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: "Eugene M. Zheganin" Cc: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 13:54:42 -0000 On Thu, Oct 20, 2016 at 3:47 PM, Eugene M. Zheganin wrote: > Hi. > > I have FreeBSD 10.2-STABLE r289293 (but I have observed this situation on > different releases) and a zfs. I also have one directory that used to have a > lot of (tens of thousands) files. I surely takes a lot of time to get a > listing of it. But now I have 2 files and a couple of dozens directories in > it (I sorted files into directories). Surprisingly, there's still a lag > between "ls" and an output: > > > ===Cut=== > > # /usr/bin/time -h ls > .recycle 2016-01 2016-04 2016-07 2016-10 > sort-files.sh > 2014 2016-02 2016-05 2016-08 ktrace.out > sort-months.sh > 2015 2016-03 2016-06 2016-09 old > sounds > 5.75s real 0.00s user 0.02s sys Looks like it's not taking up any processing time, so my guess is the lag probably comes from stalled I/O ... bad disk? Does a second "ls" immediately returned (ie. metadata has been cached) ? > > ===Cut=== > > > I've seen this situation before, on other servers, so it's not the first > time I encounter this. However, it's not 100% reproducible (I mean, if I > fill the directory with dozens of thousands of files, I will not certainly > get this lag after the deletion). > > Has anyone seen this and does anyone know how to resolve this ? It's not > critical issue, but it makes thing uncomfortable here. One method I'm aware > of: you can move the contents of this directory to some other place, then > delete it and create again. But it's kind of a nasty workaround. > > > Thanks. > > Eugene. > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu Oct 20 14:03:18 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D0508C1A1CD for ; Thu, 20 Oct 2016 14:03:18 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 968F2F56 for ; Thu, 20 Oct 2016 14:03:18 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 13CD428435; Thu, 20 Oct 2016 16:03:10 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 4FDAD28434; Thu, 20 Oct 2016 16:03:09 +0200 (CEST) Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: "Eugene M. Zheganin" , freebsd-stable@FreeBSD.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <5808CE9D.8080305@quip.cz> Date: Thu, 20 Oct 2016 16:03:09 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 In-Reply-To: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 14:03:18 -0000 Eugene M. Zheganin wrote on 2016/10/20 15:47: > Hi. > > I have FreeBSD 10.2-STABLE r289293 (but I have observed this situation > on different releases) and a zfs. I also have one directory that used to > have a lot of (tens of thousands) files. I surely takes a lot of time to > get a listing of it. But now I have 2 files and a couple of dozens > directories in it (I sorted files into directories). Surprisingly, > there's still a lag between "ls" and an output: [...] > I've seen this situation before, on other servers, so it's not the first > time I encounter this. However, it's not 100% reproducible (I mean, if I > fill the directory with dozens of thousands of files, I will not > certainly get this lag after the deletion). What about snapshots? Are there any snapshots on this filesystem? Or scrub running in the background? Miroslav Lachman From owner-freebsd-stable@freebsd.org Thu Oct 20 14:12:55 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 893E2C1A5A6 for ; Thu, 20 Oct 2016 14:12:55 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (ingresso-1-pt.tunnel.tserv1.lon2.ipv6.he.net [IPv6:2001:470:1f1c:411::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 54CC67DF for ; Thu, 20 Oct 2016 14:12:55 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6] helo=dilbert.ingresso.co.uk) by constantine.ingresso.co.uk with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bxE5V-000Hl9-30; Thu, 20 Oct 2016 14:12:53 +0000 Received: from petefrench by dilbert.ingresso.co.uk with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bxE5V-000LCg-1t; Thu, 20 Oct 2016 15:12:53 +0100 To: emz@norma.perm.ru, nicolas.gilles@gmail.com Subject: Re: zfs, a directory that used to hold lot of files and listing pause Cc: freebsd-stable@freebsd.org In-Reply-To: Message-Id: From: Pete French Date: Thu, 20 Oct 2016 15:12:53 +0100 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 14:12:55 -0000 Have ignored this thread untiul now, but I observed the same behaviour on mysystems over the last week or so. In my case its an exim spool directory, which was hugely full as some point (thousands of files) and now takes an awfully long time to open and list. I delet and remake them and the problem goes away, but I belive it is the same thing. I am running 10.3-STABLE, r303832 -pete. From owner-freebsd-stable@freebsd.org Thu Oct 20 14:31:46 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7738AC191D5 for ; Thu, 20 Oct 2016 14:31:46 +0000 (UTC) (envelope-from dr.klepp@gmx.at) Received: from vie01a-qmta-at50-1.mx.upcmail.net (vie01a-qmta-at50-1.mx.upcmail.net [62.179.121.166]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 30FD9847 for ; Thu, 20 Oct 2016 14:31:45 +0000 (UTC) (envelope-from dr.klepp@gmx.at) Received: from [172.31.218.21] (helo=vie01a-dmta-at52-3.mx.upcmail.net) by vie01a-pqmta-at50.mx.upcmail.net with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.87) (envelope-from ) id 1bxEB3-0003Ay-RA for freebsd-stable@freebsd.org; Thu, 20 Oct 2016 16:18:37 +0200 Received: from [172.31.216.41] (helo=vie01a-pemc-psmtp-at50) by vie01a-dmta-at52.mx.upcmail.net with esmtp (Exim 4.87) (envelope-from ) id 1bxEAv-0006ah-8w for freebsd-stable@freebsd.org; Thu, 20 Oct 2016 16:18:29 +0200 Received: from t60.lan ([85.126.97.210]) by vie01a-pemc-psmtp-at50 with SMTP @ mailcloud.upcmail.net id xqJU1t01r4YLlkt0BqJVCL; Thu, 20 Oct 2016 16:18:29 +0200 X-SourceIP: 85.126.97.210 X-CNFS-Analysis: v=2.2 cv=FpZHxCjq c=1 sm=2 tr=0 a=/Ac8Q0O/YFE5LOLfUiYZVw==:117 a=/Ac8Q0O/YFE5LOLfUiYZVw==:17 a=8nJEP1OIZ-IA:10 a=A4C9TOB8URguu_oj8IAA:9 a=wPNLvfGTeEIA:10 From: "Dr. Nikolaus Klepp" To: freebsd-stable@freebsd.org Subject: Re: zfs, a directory that used to hold lot of files and listing pause Date: Thu, 20 Oct 2016 16:18:31 +0200 User-Agent: KMail/1.9.10 (enterprise35 0.20100827.1168748) References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> In-Reply-To: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> X-KMail-QuotePrefix: > MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <201610201618.31186.dr.klepp@gmx.at> X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 14:31:46 -0000 Am Donnerstag, 20. Oktober 2016 schrieb Eugene M. Zheganin: > Hi. > > I have FreeBSD 10.2-STABLE r289293 (but I have observed this situation > on different releases) and a zfs. I also have one directory that used to > have a lot of (tens of thousands) files. I surely takes a lot of time to > get a listing of it. But now I have 2 files and a couple of dozens > directories in it (I sorted files into directories). Surprisingly, > there's still a lag between "ls" and an output: > > > ===Cut=== > > # /usr/bin/time -h ls > .recycle 2016-01 2016-04 2016-07 2016-10 > sort-files.sh > 2014 2016-02 2016-05 2016-08 ktrace.out > sort-months.sh > 2015 2016-03 2016-06 2016-09 old > sounds > 5.75s real 0.00s user 0.02s sys > > ===Cut=== > > > I've seen this situation before, on other servers, so it's not the first > time I encounter this. However, it's not 100% reproducible (I mean, if I > fill the directory with dozens of thousands of files, I will not > certainly get this lag after the deletion). > > Has anyone seen this and does anyone know how to resolve this ? It's not > critical issue, but it makes thing uncomfortable here. One method I'm > aware of: you can move the contents of this directory to some other > place, then delete it and create again. But it's kind of a nasty workaround. Hi! I've the same issue, but only if the ZFS resides on a LSI MegaRaid and one RAID0 for each disk. Nik -- Please do not email me anything that you are not comfortable also sharing with the NSA. From owner-freebsd-stable@freebsd.org Thu Oct 20 14:54:07 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46D23C19A45 for ; Thu, 20 Oct 2016 14:54:07 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-qk0-x22d.google.com (mail-qk0-x22d.google.com [IPv6:2607:f8b0:400d:c09::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 022D193D for ; Thu, 20 Oct 2016 14:54:07 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-qk0-x22d.google.com with SMTP id o68so98822663qkf.3 for ; Thu, 20 Oct 2016 07:54:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc:content-transfer-encoding; bh=ufnjfgYcU4VxZv/A2jgF2ZNSAAAjaQekhxbQhUTVlpM=; b=oWybttdYOM55nomcN8Ct9D7iIiHtwvXbGYE+oLTdSwXcX49484IueWlcXpNAsDZrfU eTfqszAkLeOKbHapARfRNbwkcIC/484n4GMAnhp/IwCMifcXvfXzEUe+/3RpuwE4tRZr YKbBbamv4HyEJDwKn8WPCR6aD4exG8Eu6Fbu+SEYD5531+bbGAQTLCC/LcNY5BKNG2O2 dAkIWHFMOy400tCAD4xeHsTsr+9XsUQ9DKo33gEVhTaiYA3kRqWOAFVORkI/hObSYYoT Vq8e9xt/ktbN4DFU3FDsUq/ikNKNaJoV22F765D14x9uzR0d7y21SyySsmBAFCor6BCJ 6akQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc:content-transfer-encoding; bh=ufnjfgYcU4VxZv/A2jgF2ZNSAAAjaQekhxbQhUTVlpM=; b=eiSjmZsiEqJLiQbAQWzdhNiGOZJ1/z/kYM/6oQQ8+01DdPcSoN+INOXdi1jZ++ousy cyIGXo6vh9YlJg1XjrdG9XGhh+0gskDNEWDOC5SHAyLdlbARaNi39wYCa0qN32rlZBgZ aAbiKKAIDd7wZUEDD3Wd3PPn2CJtsGdpE7qfeGOdbPcfk9m+qe0nFKC4vCbMNf0e50m4 /psj90CWXKj13BSoK7yL/3uKmjFytASR3s4HSa5XgsthYshGFZdZdqhkR9MonROoMMrW VAvvZr+5Z8PEnshDiieUm1w5jADjRawqGnO12JmWXgi8ytBXw3K+0LkvK2ihY/K/cHrj RZ2g== X-Gm-Message-State: ABUngveLO5+qL8LVLt9jR+SUp3d76U7oFUCkEcaXsqnYW4/fBKrhjrB0uf7QT8x55CnYuu6hbiEkRSe6i+JYhQ== X-Received: by 10.55.92.193 with SMTP id q184mr1001844qkb.94.1476975246157; Thu, 20 Oct 2016 07:54:06 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 10.12.138.251 with HTTP; Thu, 20 Oct 2016 07:54:05 -0700 (PDT) In-Reply-To: <20161019171028.GF57876@zxy.spb.ru> References: <20161019171028.GF57876@zxy.spb.ru> From: Alan Somers Date: Thu, 20 Oct 2016 08:54:05 -0600 X-Google-Sender-Auth: Ay_okkI1KHMfPEZ2zjmjV2tI3L0 Message-ID: Subject: Re: tcsh is not handled correctly UTF-8 in arguments To: Slawa Olhovchenkov Cc: FreeBSD Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 14:54:07 -0000 On Wed, Oct 19, 2016 at 11:10 AM, Slawa Olhovchenkov wrote= : > tcsh called by sshd for invocation of scp: `tcsh -c scp -f =D0=A0=D0=B0= =D1=81=D0=BF=D0=B8=D1=81=D0=B0=D0=BD=D0=B8=D0=B5.pdf` > At this time no any LC_* is set. > tcsh read .cshrc and set LC_CTYPE=3Dru_RU.UTF-8 LC_COLLATE=3Dru_RU.UTF-8. > After this invocation of scp will be incorrect: > > 00007ab0 20 2d 66 20 c3 90 c2 a0 c3 90 c2 b0 c3 91 c2 81 | -f ........= ....| > 00007ac0 c3 90 c2 bf c3 90 c2 b8 c3 91 c2 81 c3 90 c2 b0 |............= ....| > 00007ad0 c3 90 c2 bd c3 90 c2 b8 c3 90 c2 b5 5f c3 90 c2 |............= _...| > 00007ae0 a2 c3 90 c2 97 c3 90 c2 98 2e 70 64 66 0a |..........pd= f. | > > Correct invocation must be: > > 00000000 20 2d 66 20 | = -f | > 00000010 d0 a0 d0 b0 d1 81 d0 bf d0 b8 d1 81 d0 b0 d0 bd |............= ....| > 00000020 d0 b8 d0 b5 5f d0 a2 d0 97 d0 98 2e 70 64 66 0a |...._.......= pdf.| > > `d0` =3D> `c3 90` > `a0` =3D> `c2 a0` > > I.e. every byte re-encoded to utf-8: `d0` =3D> `c3 90` > > As result imposible to access files w/ non-ascii names. This might be related to PR213013. Could you please try on head after r306= 782 ? From owner-freebsd-stable@freebsd.org Thu Oct 20 15:04:51 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DEC60C19FA8 for ; Thu, 20 Oct 2016 15:04:51 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A0D8B1AF; Thu, 20 Oct 2016 15:04:51 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bxEtk-000K9C-GH; Thu, 20 Oct 2016 18:04:48 +0300 Date: Thu, 20 Oct 2016 18:04:48 +0300 From: Slawa Olhovchenkov To: Alan Somers Cc: FreeBSD Subject: Re: tcsh is not handled correctly UTF-8 in arguments Message-ID: <20161020150448.GU57714@zxy.spb.ru> References: <20161019171028.GF57876@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=koi8-r Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 15:04:52 -0000 On Thu, Oct 20, 2016 at 08:54:05AM -0600, Alan Somers wrote: > On Wed, Oct 19, 2016 at 11:10 AM, Slawa Olhovchenkov wrote: > > tcsh called by sshd for invocation of scp: `tcsh -c scp -f òÁÓÐÉÓÁÎÉÅ.pdf` > > At this time no any LC_* is set. > > tcsh read .cshrc and set LC_CTYPE=ru_RU.UTF-8 LC_COLLATE=ru_RU.UTF-8. > > After this invocation of scp will be incorrect: > > > > 00007ab0 20 2d 66 20 c3 90 c2 a0 c3 90 c2 b0 c3 91 c2 81 | -f ............| > > 00007ac0 c3 90 c2 bf c3 90 c2 b8 c3 91 c2 81 c3 90 c2 b0 |................| > > 00007ad0 c3 90 c2 bd c3 90 c2 b8 c3 90 c2 b5 5f c3 90 c2 |............_...| > > 00007ae0 a2 c3 90 c2 97 c3 90 c2 98 2e 70 64 66 0a |..........pdf. | > > > > Correct invocation must be: > > > > 00000000 20 2d 66 20 | -f | > > 00000010 d0 a0 d0 b0 d1 81 d0 bf d0 b8 d1 81 d0 b0 d0 bd |................| > > 00000020 d0 b8 d0 b5 5f d0 a2 d0 97 d0 98 2e 70 64 66 0a |...._.......pdf.| > > > > `d0` => `c3 90` > > `a0` => `c2 a0` > > > > I.e. every byte re-encoded to utf-8: `d0` => `c3 90` > > > > As result imposible to access files w/ non-ascii names. > > This might be related to PR213013. Could you please try on head after r306782 ? I think not related. PR213013 is about character classification, my report is about unnecessary encoding shell arguments. From owner-freebsd-stable@freebsd.org Thu Oct 20 15:34:58 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E2DDAC1ABE1 for ; Thu, 20 Oct 2016 15:34:58 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4581911E for ; Thu, 20 Oct 2016 15:34:57 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e] ([IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9KFYsr1024981 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 20 Oct 2016 20:34:54 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1476977694; bh=lLUzJ2vf4omuiBb6nMiVtcSvVZ/Me9NjEWYQnmWkR4E=; h=Subject:To:References:From:Date:In-Reply-To; b=VSxHadqb6t9ZSvZejLMxtYelQqLxs9en5sXHtw7Mh1Plkh3JmSaqAiwulIjJiuqqU Phf6odPUwjsTA6g22wpLaFIzp0UaOqEdeOYZ5eEcqPo1LlyfF6V//uxjD8BwcWxxD8 A1QagnbjFt+6b5DOScjemhCZSIwz0fbg5N047q1Y= Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> From: "Eugene M. Zheganin" Message-ID: Date: Thu, 20 Oct 2016 20:34:59 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 15:34:59 -0000 Hi. On 20.10.2016 18:54, Nicolas Gilles wrote: > Looks like it's not taking up any processing time, so my guess is > the lag probably comes from stalled I/O ... bad disk? Well, I cannot rule this out completely, but first time I've seen this lag on this particular server about two months ago, and I guess two months is enough time for zfs on a redundant pool to ger errors, but as you can see: ]# zpool status pool: zroot state: ONLINE status: One or more devices are configured to use a non-native block size. Expect reduced performance. action: Replace affected devices with devices that support the configured block size, or migrate data to a properly configured pool. scan: resilvered 5.74G in 0h31m with 0 errors on Wed Jun 8 11:54:14 2016 config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/zroot0 ONLINE 0 0 0 block size: 512B configured, 4096B native gpt/zroot1 ONLINE 0 0 0 errors: No known data errors there's none. Yup, disks have different sector size, but this issue happened with one particular directory, not all of them. So I guess this is irrelevant. > Does a second "ls" immediately returned (ie. metadata has been > cached) ? Nope. Although the lag varies slightly: 4.79s real 0.00s user 0.02s sys 5.51s real 0.00s user 0.02s sys 4.78s real 0.00s user 0.02s sys 6.88s real 0.00s user 0.02s sys Thanks. Eugene. From owner-freebsd-stable@freebsd.org Thu Oct 20 15:37:16 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F41C0C1AD07 for ; Thu, 20 Oct 2016 15:37:15 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 7514829B for ; Thu, 20 Oct 2016 15:37:15 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e] ([IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9KFbCIZ025111 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 20 Oct 2016 20:37:12 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1476977833; bh=p0NEBhjhAEsu2XfFb2kuNwsEh0TDs73Y47ULeZ73OW4=; h=Subject:To:References:From:Date:In-Reply-To; b=JxxNVWDnVaqVc2pt0/fCaDgr9HLMWgLye7TuANljgNtVWZwqs/Juak0XrIhZMQfuQ k98aIQdQXsW57y4NwNjLaxWP+0dwYnxR7RzQI+7huw85lly5eFqS9+u3ItHqul8xra 504yIY5LdYVDxmJZHD0DQomom9gG9TXgPnhK1+KQ= Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@FreeBSD.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <5808CE9D.8080305@quip.cz> From: "Eugene M. Zheganin" Message-ID: <8a5a4eb3-4515-601d-e61c-5178b2f1e08a@norma.perm.ru> Date: Thu, 20 Oct 2016 20:37:18 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <5808CE9D.8080305@quip.cz> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 15:37:16 -0000 Hi. On 20.10.2016 19:03, Miroslav Lachman wrote: > > What about snapshots? Are there any snapshots on this filesystem? Nope. # zfs list -t all NAME USED AVAIL REFER MOUNTPOINT zroot 245G 201G 1.17G legacy zroot/tmp 10.1M 201G 10.1M /tmp zroot/usr 9.78G 201G 7.36G /usr zroot/usr/home 77.9M 201G 77.9M /usr/home zroot/usr/ports 1.41G 201G 857M /usr/ports zroot/usr/ports/distfiles 590M 201G 590M /usr/ports/distfiles zroot/usr/ports/packages 642K 201G 642K /usr/ports/packages zroot/usr/src 949M 201G 949M /usr/src zroot/var 234G 201G 233G /var zroot/var/crash 21.5K 201G 21.5K /var/crash zroot/var/db 127M 201G 121M /var/db zroot/var/db/pkg 6.28M 201G 6.28M /var/db/pkg zroot/var/empty 20K 201G 20K /var/empty zroot/var/log 631M 201G 631M /var/log zroot/var/mail 24.6M 201G 24.6M /var/mail zroot/var/run 54K 201G 54K /var/run zroot/var/tmp 198K 201G 198K /var/tmp > > Or scrub running in the background? > No. Thanks. Eugene. From owner-freebsd-stable@freebsd.org Thu Oct 20 15:38:18 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8F33AC1ADD3 for ; Thu, 20 Oct 2016 15:38:18 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0BD8E666 for ; Thu, 20 Oct 2016 15:38:17 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e] ([IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9KFcEfl025153 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 20 Oct 2016 20:38:15 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1476977895; bh=bxCKtDevfR0O4vfH1pkMrlsghchJoFS2pBLi32Mt97I=; h=Subject:To:References:From:Date:In-Reply-To; b=a3uZJ10h6skHWIrqL4xYC1W3czylz24erJ9GT7+K5BFuLGR0wXYuwu5O2wrA6H+Pd 40/HCaluKxYsEQ94cWQcwAegCSGKwl8F8zR4nHNeCeMPkRWvoUabcojYedVnupZ5wm mYyFJnSnhx31GUYs7Lin1LYfDs/XjyUW2ZeiL+7I= Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: From: "Eugene M. Zheganin" Message-ID: <2125e741-efef-776e-806d-466372fc29e0@norma.perm.ru> Date: Thu, 20 Oct 2016 20:38:20 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 15:38:18 -0000 Hi, On 20.10.2016 19:12, Pete French wrote: > Have ignored this thread untiul now, but I observed the same behaviour > on mysystems over the last week or so. In my case its an exim spool > directory, which was hugely full as some point (thousands of > files) and now takes an awfully long time to open and list. I delet > and remake them and the problem goes away, but I belive it is the same thing. > > I am running 10.3-STABLE, r303832 > Yup, saw this once on a sendmail spool directory. Thanks. Eugene. From owner-freebsd-stable@freebsd.org Thu Oct 20 15:40:11 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2B910C1A00F for ; Thu, 20 Oct 2016 15:40:11 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 9BAF091D for ; Thu, 20 Oct 2016 15:40:10 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e] ([IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9KFe7Bv025243 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Thu, 20 Oct 2016 20:40:07 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1476978008; bh=07RrT+qjjLy++zvF+5DU/5VOZ+Roaw5HAhTegaRKzNI=; h=Subject:To:References:From:Date:In-Reply-To; b=FG8jHdHYBuoRI5ryHzs/Wi1EYxDOJIkmhjAdheTOufbUyy/0QddiImKYU6bux5/CY PfbAgVntklJ3jwn1Jrr7oZLFVkgV2Lsff5toUsog0k/t/dbqdlRwBRZ8RMXllumDk2 fjm3YOczy5gXTkbrw+SvfP2bAFQuulqp4d4923sE= Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <201610201618.31186.dr.klepp@gmx.at> From: "Eugene M. Zheganin" Message-ID: <0991d4a1-4d0b-54f7-485d-47bfaca33db9@norma.perm.ru> Date: Thu, 20 Oct 2016 20:40:13 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <201610201618.31186.dr.klepp@gmx.at> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 15:40:11 -0000 Hi. On 20.10.2016 19:18, Dr. Nikolaus Klepp wrote: > > I've the same issue, but only if the ZFS resides on a LSI MegaRaid and one RAID0 for each disk. > Not in my case, both pool disks are attached to the Intel ICH7 SATA300 controller. Thanks. Eugene. From owner-freebsd-stable@freebsd.org Thu Oct 20 15:57:40 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 10C2EC1A72E for ; Thu, 20 Oct 2016 15:57:40 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (ingresso-1-pt.tunnel.tserv1.lon2.ipv6.he.net [IPv6:2001:470:1f1c:411::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CB00D685 for ; Thu, 20 Oct 2016 15:57:39 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6] helo=dilbert.ingresso.co.uk) by constantine.ingresso.co.uk with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bxFip-000Jri-Br; Thu, 20 Oct 2016 15:57:35 +0000 Received: from petefrench by dilbert.ingresso.co.uk with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bxFip-0000jz-Ag; Thu, 20 Oct 2016 16:57:35 +0100 To: emz@norma.perm.ru, freebsd-stable@freebsd.org Subject: Re: zfs, a directory that used to hold lot of files and listing pause In-Reply-To: <0991d4a1-4d0b-54f7-485d-47bfaca33db9@norma.perm.ru> Message-Id: From: Pete French Date: Thu, 20 Oct 2016 16:57:35 +0100 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 15:57:40 -0000 > > I've the same issue, but only if the ZFS resides on a LSI MegaRaid and one RAID0 for each disk. > > > Not in my case, both pool disks are attached to the Intel ICH7 SATA300 > controller. Nor my case - my discs are on this: ahci0: From owner-freebsd-stable@freebsd.org Thu Oct 20 16:17:20 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DBD3FC1AE29 for ; Thu, 20 Oct 2016 16:17:20 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-qk0-x22a.google.com (mail-qk0-x22a.google.com [IPv6:2607:f8b0:400d:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9C1111A9 for ; Thu, 20 Oct 2016 16:17:20 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-qk0-x22a.google.com with SMTP id n189so103441258qke.0 for ; Thu, 20 Oct 2016 09:17:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=3AbUc1AM5W3NYDjH0T7UO3RYtrwROSd6SFNk8rCztlQ=; b=f/pBz9Zh6caE4rIpYeZthTqtWpnS7BqKNRo7EH1fMo7A9XVUpDVOySe3XETdax37n0 tRMZciQgt9VPFNDEYM8ie/1PivPxVI8pxBwmQ+hSEqDzLb+LOGbqEy2eKkV7+A+Irnn5 /+WdvJ14+ip+xqNQFThqGxuHBUjlE21mk4Fr3l+lfGUcm5wIMGk7SQ0J8asfXTsPytGU iDPg+8x8fMFkjWV+B5phIe6zA34eVhGxne0lrYDwi546zZDYtvVC9YyM3FBc4JlmfrHE DedTL0kHf3mnnlBz6X5j/513psHCPReakbcDxOenI5/6FmHn+JfSUP5iIKwoRlIZm/8i 15aA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=3AbUc1AM5W3NYDjH0T7UO3RYtrwROSd6SFNk8rCztlQ=; b=V9lMBVhjpQTI9wYQhkYe8gQwBtXMAha0lu83s/FjlOwQuBNQC3vknqpr/L4mKel7HK 52Dvm1ag9fCTPq1Q4+tP9r5VLjUab9vlviJK/qKzQTi87gm4G/7LyWiUeKxfaVR0+Auu WZG4t3jaGhyXsm+UI11x3W9gU+6HcoTIrnKPm8rAe1CVWCrSg09EG7deX0zEzu1crVGP YprsGpl7j73PxwKbEx4YfU9KFokQmx+WJrF6Cw8ubwWfSNEF58k3r7cKLFzZEqMga6Al HIPfSVPb9ODKPS2JxenkL8cHkcU9rhh4GvZDDMYRLj3MeWtcLN+wYRH7rRo6A+Anaoys b5/g== X-Gm-Message-State: AA6/9RlvXYU+kuIb5Mi/LCdLKQoXgSl1zxJwJ6zH3SexYubk0/pnSbcDn3JAWiVMre3sMWBX X-Received: by 10.194.84.40 with SMTP id v8mr875886wjy.51.1476980238764; Thu, 20 Oct 2016 09:17:18 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id us3sm79972856wjb.32.2016.10.20.09.17.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Oct 2016 09:17:17 -0700 (PDT) Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> From: Steven Hartland Message-ID: <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> Date: Thu, 20 Oct 2016 17:17:47 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 16:17:20 -0000 Do you have atime enabled for the relevant volume? If so disable it and see if that helps: zfs set atime=off Regards Steve On 20/10/2016 14:47, Eugene M. Zheganin wrote: > Hi. > > I have FreeBSD 10.2-STABLE r289293 (but I have observed this situation > on different releases) and a zfs. I also have one directory that used > to have a lot of (tens of thousands) files. I surely takes a lot of > time to get a listing of it. But now I have 2 files and a couple of > dozens directories in it (I sorted files into directories). > Surprisingly, there's still a lag between "ls" and an output: > > > ===Cut=== > > # /usr/bin/time -h ls > .recycle 2016-01 2016-04 2016-07 > 2016-10 sort-files.sh > 2014 2016-02 2016-05 2016-08 > ktrace.out sort-months.sh > 2015 2016-03 2016-06 2016-09 > old sounds > 5.75s real 0.00s user 0.02s sys > > ===Cut=== > > > I've seen this situation before, on other servers, so it's not the > first time I encounter this. However, it's not 100% reproducible (I > mean, if I fill the directory with dozens of thousands of files, I > will not certainly get this lag after the deletion). > > Has anyone seen this and does anyone know how to resolve this ? It's > not critical issue, but it makes thing uncomfortable here. One method > I'm aware of: you can move the contents of this directory to some > other place, then delete it and create again. But it's kind of a nasty > workaround. > > > Thanks. > > Eugene. > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu Oct 20 16:43:19 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B2CDC1A5B8 for ; Thu, 20 Oct 2016 16:43:19 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CE697179 for ; Thu, 20 Oct 2016 16:43:18 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id u9KGh1HR003329; Fri, 21 Oct 2016 03:43:01 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Fri, 21 Oct 2016 03:43:01 +1100 (EST) From: Ian Smith To: Kevin Oberman cc: Warner Losh , Rostislav Krasny , freebsd-stable Subject: Re: I'm upset about FreeBSD In-Reply-To: Message-ID: <20161021024744.B6806@sola.nimnet.asn.au> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 16:43:19 -0000 On Wed, 19 Oct 2016 16:38:09 -0700, Kevin Oberman wrote: > On Wed, Oct 19, 2016 at 3:39 PM, Warner Losh wrote: > > > On Wed, Oct 19, 2016 at 12:21 PM, Rostislav Krasny > > wrote: > > > On Tue, Oct 18, 2016 at 21:57:29 +1100, Ian Smith > > wrote: > > >> > > >> If FreeBSD GPT images (and Kindle readers) can trigger this, so could a > > >> theoretically unlimited combination of data on block 2 of USB media; > > >> modifying FreeBSD to fix a Windows bug should be out of the question. > > > > > > Not modifying FreeBSD and not fixing Windows bug but modifying the > > > FreeBSD installation media and working around the Windows bug to let > > > people install FreeBSD without disappointing at very beginning. Why > > > GPT is used in the memstick images at all? Why they can't be MBR > > > based? I think they can. > > > > Can't boot UEFI off of MBR disks on all BIOSes. > > > > Warner > > I'll go one farther. You can't boot many new PCs with traditional MBR > disks. And. please don't confuse GPT with UEFI. I have yet to find an > amd64 computer that has a problem with a GPT format with MBR. Due to broken > BIOS, my 5-year old ThinkPad won't boot UEFI, but it has no problem with > MBR, whether GPT formatted or not. As far as I know, the 11.0 memstick > images are still MBR, just on a GPT structured disk, not UEFI. (Let me know > if I am mistaken on this.) Well, GPT with protective MBR. Wikipedia calls FreeBSD's GPT 'hybrid', I gathered because it still works also with older BIOS booting. root@x200:/extra/images # mdconfig -lv md0 vnode 700M /home/smithi/FreeBSD-11.0-RELEASE-amd64-memstick.img root@x200:/extra/images # gpart show -p md0 => 3 1433741 md0 GPT (700M) 3 1600 md0p1 efi (800k) 1603 125 md0p2 freebsd-boot (62k) 1728 1429968 md0p3 freebsd-ufs (698M) 1431696 2048 md0p4 freebsd-swap (1.0M) [wondering vaguely what use 1.0M of swap might be?] > I do accept that some early amd64 systems and, perhaps, many i386 systems > may have problems with GPT, but GPT was designed to be compatible with > traditional disk formats and, while they may have problems, they really > should work for single partition disks. And I understand that it is > frustrating if you hit one of these cases where it fails. My 8yo Thinkpad X200 knows nothing of UEFI and boots 11.0 amd64 memstick fine as is. It also happily boots from a sliced MBR memstick w/ boot0, after dd'ing md0p3 to (in this case) da0s2, adding bootcode after a bit of fiddling recreating da0s2 & da0s2a after the above dd clobbers it .. So it's not hard making an MBR sliced memstick with up to 4 bootables; I'm hoping to convert dvd1 to one of these, maybe it'll work this time, and - at least theoretically - couldn't "kill Windows" by mere presence. cheers, Ian From owner-freebsd-stable@freebsd.org Thu Oct 20 16:48:30 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44D45C1A8A8 for ; Thu, 20 Oct 2016 16:48:30 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id 3640B64D; Thu, 20 Oct 2016 16:48:30 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 26D75142; Thu, 20 Oct 2016 16:48:30 +0000 (UTC) Date: Thu, 20 Oct 2016 16:48:29 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <1695454902.14.1476982109716.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <1725723762.13.1476960583086.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1725723762.13.1476960583086.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #436 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 16:48:30 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/436/ From owner-freebsd-stable@freebsd.org Thu Oct 20 21:13:37 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 44CCFC1A21B for ; Thu, 20 Oct 2016 21:13:37 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "uucp.dinoex.sub.de", Issuer "StartCom Class 1 DV Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A8F9CB0B for ; Thu, 20 Oct 2016 21:13:36 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.sub.de (8.15.2/8.14.9) with ESMTPS id u9KLD76i045795 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 20 Oct 2016 23:13:08 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.15.2/8.14.9/Submit) with UUCP id u9KLD71m045794 for freebsd-stable@FreeBSD.ORG; Thu, 20 Oct 2016 23:13:07 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.14.9/8.14.9) with ESMTP id u9KKPJM0067829 for ; Thu, 20 Oct 2016 22:25:19 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.14.9/8.14.9) with ESMTP id u9KKO6xD067551 for ; Thu, 20 Oct 2016 22:24:07 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: (from news@localhost) by gate.oper.dinoex.org (8.14.9/8.14.9/Submit) id u9KKO6oF067548 for freebsd-stable@FreeBSD.ORG; Thu, 20 Oct 2016 22:24:06 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-Authentication-Warning: gate.oper.dinoex.org: news set sender to li-fbsd@citylink.dinoex.sub.org using -f From: Peter Subject: Re: zfs, a directory that used to hold lot of files and listing pause Date: Thu, 20 Oct 2016 22:09:55 +0200 Organization: even some more stinky socks Message-ID: References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Date: Thu, 20 Oct 2016 20:10:02 +0000 (UTC) Injection-Info: oper.dinoex.de; logging-data="65645"; mail-complaints-to="usenet@citylink.dinoex.sub.org" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 In-Reply-To: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> Sender: li-fbsd@citylink.dinoex.sub.org To: freebsd-stable@FreeBSD.ORG X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (uucp.dinoex.sub.de [194.45.71.2]); Thu, 20 Oct 2016 23:13:09 +0200 (CEST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 21:13:37 -0000 Eugene M. Zheganin wrote: > Hi. > > I have FreeBSD 10.2-STABLE r289293 (but I have observed this situation > on different releases) and a zfs. I also have one directory that used to > have a lot of (tens of thousands) files. I surely takes a lot of time to > get a listing of it. But now I have 2 files and a couple of dozens > directories in it (I sorted files into directories). Surprisingly, > there's still a lag between "ls" and an output: I see this on my pgsql_tmp dirs (where Postgres stores intermediate query data that gets too big for mem - usually lots of files) - in normal operation these dirs are completely empty, but make heavy disk activity (even writing!) when doing ls. Seems normal, I dont care as long as the thing is stable. One would need to check how ZFS stores directories and what kind of fragmentation can happen there. Or wait for some future feature that would do housekeeping. ;) From owner-freebsd-stable@freebsd.org Thu Oct 20 21:13:38 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 445C1C1A21D for ; Thu, 20 Oct 2016 21:13:38 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [IPv6:2001:1440:5001:1::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "uucp.dinoex.sub.de", Issuer "StartCom Class 1 DV Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C4FA0B0C for ; Thu, 20 Oct 2016 21:13:37 +0000 (UTC) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from uucp.dinoex.sub.de (uucp.dinoex.sub.de [194.45.71.2]) by uucp.dinoex.sub.de (8.15.2/8.14.9) with ESMTPS id u9KLD9eK045810 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Thu, 20 Oct 2016 23:13:09 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-MDaemon-Deliver-To: Received: from citylink.dinoex.sub.org (uucp@localhost) by uucp.dinoex.sub.de (8.15.2/8.14.9/Submit) with UUCP id u9KLD9ES045809 for freebsd-stable@FreeBSD.ORG; Thu, 20 Oct 2016 23:13:09 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by citylink.dinoex.sub.de (8.14.9/8.14.9) with ESMTP id u9KKPJM2067829 for ; Thu, 20 Oct 2016 22:25:19 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: from gate.oper.dinoex.org (gate-e [192.168.98.2]) by gate.oper.dinoex.org (8.14.9/8.14.9) with ESMTP id u9KKO8F0067566 for ; Thu, 20 Oct 2016 22:24:08 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) Received: (from news@localhost) by gate.oper.dinoex.org (8.14.9/8.14.9/Submit) id u9KKO80M067564 for freebsd-stable@FreeBSD.ORG; Thu, 20 Oct 2016 22:24:08 +0200 (CEST) (envelope-from li-fbsd@citylink.dinoex.sub.org) X-Authentication-Warning: gate.oper.dinoex.org: news set sender to li-fbsd@citylink.dinoex.sub.org using -f From: Peter Subject: Re: Nightly disk-related panic since upgrade to 10.3 Date: Thu, 20 Oct 2016 22:12:26 +0200 Organization: even some more stinky socks Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Injection-Date: Thu, 20 Oct 2016 20:12:27 +0000 (UTC) Injection-Info: oper.dinoex.de; logging-data="66049"; mail-complaints-to="usenet@citylink.dinoex.sub.org" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 In-Reply-To: Sender: li-fbsd@citylink.dinoex.sub.org To: freebsd-stable@FreeBSD.ORG X-Milter: Spamilter (Reciever: uucp.dinoex.sub.de; Sender-ip: 194.45.71.2; Sender-helo: uucp.dinoex.sub.de; ) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (uucp.dinoex.sub.de [194.45.71.2]); Thu, 20 Oct 2016 23:13:10 +0200 (CEST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 21:13:38 -0000 Andrea Venturoli wrote: > Hello. > > Last week I upgraded a 9.3/amd64 box to 10.3: since then, it crashed and > rebooted at least once every night. Hi, I have quite similar issue, crash dumps every night, but then my stacktrace is different (crashing mostly in cam/scsi/scsi.c), and my env is also quite different (old i386, individual disks, extensive use of ZFS), so here is very likely a different reason. Also here the upgrade is not the only change, I also replaced a burnt powersupply recently and added an SSD cache. Basically You have two options: A) fire up kgdb, go into the code and try and understand what exactly is happening. This depends if You have clue enough to go that way; I found "man 4 gdb" and especially the "Debugging Kernel Problems" pdf by Greg Lehey quite helpful. B) systematically change parameters. Start by figuring from the logs the exact time of crash and what was happening then, try to reproduce that. Then change things and isolate the cause. Having a RAID controller is a bit ugly in this regard, as it is more or less a blackbox, and difficult to change parameters or swap components. > The only exception was on Friday, when it locked without rebooting: it > still answered ping request and logins through HTTP would half work; I'm > under the impression that the disk subsystem was hung, so ICMP would > work since it does no I/O and HTTP too worked as far as no disk access > was required. Yep. That tends to happen. It doesnt give much clue, except that there is a disk related problem. From owner-freebsd-stable@freebsd.org Thu Oct 20 21:18:53 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D30A5C1A522 for ; Thu, 20 Oct 2016 21:18:53 +0000 (UTC) (envelope-from jonc@chen.org.nz) Received: from mail-it0-x22c.google.com (mail-it0-x22c.google.com [IPv6:2607:f8b0:4001:c0b::22c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A52FCFA9 for ; Thu, 20 Oct 2016 21:18:53 +0000 (UTC) (envelope-from jonc@chen.org.nz) Received: by mail-it0-x22c.google.com with SMTP id m138so118235393itm.0 for ; Thu, 20 Oct 2016 14:18:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chen-org-nz.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=89FTUVWG4zxiJ6P+HMPoNFZUxmn9F4QUZjeJFWeyO60=; b=MEukHmiV+smD6ZTqV8AAmmQaBeRZG8YEvOnU+D6RKyoxFSRVMuhOKrz4qlasR6ZCd1 MmgEaLzCT87iSJ+PsT1BMXH7i8k349XOtE+ajhdxRJu2iv2LDA+YI+Gv7HUgrIzbucNi uHtqT17/qu+vsAYj+UPapvPv/Umrh+uyJlDzpTKeRs72v+AWAjqLm6Bf5U4d+TSZOfEr qeB4/CLlppmVzMgtMPdf8tNLNLAjMo6+VRt5Qm6g0gBA4rHxBYwEraLBKYAw44NjwnT0 ApEWk2EqWKa5w7QTzCAra1/nQxFtkXOobAQTgqhsuiHMa0MKkw1bMRl7FNzV+ixgTtq6 TE3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=89FTUVWG4zxiJ6P+HMPoNFZUxmn9F4QUZjeJFWeyO60=; b=OOQaXHwft4/Mtt9FAP99F9TZahOZIg5KpXDyvWhnS5SkgB5JbaWKEak+tSW+wgahrz iwdiB86NToQfHqcGJnheEN9X10VxBHFx9z+Y5dbObyvyDrpj1hrPVZGqg1DI/WMyboqq dks0kW/KJhQWdcpgl4B6rCC+FmWynWTcrWTVuVx1/xfVnJiNLZgS4Ztc3zyzOIzT8K0i 7LfNXaqmc9Af3JMoZmq/Yinz9yKPs+7GEoZjiXirYtzM3+Pv5+8b1wI69P1GG57cBvmt GrFAXHZna2O6zX5DnasWeCTOxJdxs3EujDhgaUy7BMcQ34R6fJDLTRy1Xxi1cWJE7pFG jjIQ== X-Gm-Message-State: AA6/9Rmk2bmzwKwxXnYeD7iY4bpmvUdJSTMW7GA3b8akRJ72IFAJIXdjdS/85Slm8WbkYILMvoafhmchjEKWZw== X-Received: by 10.36.250.195 with SMTP id v186mr9910937ith.119.1476998333006; Thu, 20 Oct 2016 14:18:53 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.38.87 with HTTP; Thu, 20 Oct 2016 14:18:52 -0700 (PDT) X-Originating-IP: [203.99.129.1] In-Reply-To: References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> From: Jonathan Chen Date: Fri, 21 Oct 2016 10:18:52 +1300 Message-ID: Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: Peter Cc: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 21:18:53 -0000 On 21 October 2016 at 09:09, Peter wrote: [...] > > I see this on my pgsql_tmp dirs (where Postgres stores intermediate > query data that gets too big for mem - usually lots of files) - in > normal operation these dirs are completely empty, but make heavy disk > activity (even writing!) when doing ls. > Seems normal, I dont care as long as the thing is stable. One would need > to check how ZFS stores directories and what kind of fragmentation can > happen there. Or wait for some future feature that would do > housekeeping. ;) I'm seeing this as well with an Odoo ERP running on Postgresql. This lag does matter to me as this is huge performance hit when running Postgresql on ZFS, and it would be good to see this resolved. pg_restores can make the system crawl as well. Cheers. -- Jonathan Chen From owner-freebsd-stable@freebsd.org Thu Oct 20 22:07:28 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2203C1B240 for ; Thu, 20 Oct 2016 22:07:28 +0000 (UTC) (envelope-from bsdunix44@gmail.com) Received: from mail-qt0-x235.google.com (mail-qt0-x235.google.com [IPv6:2607:f8b0:400d:c0d::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BB5B2DA3 for ; Thu, 20 Oct 2016 22:07:28 +0000 (UTC) (envelope-from bsdunix44@gmail.com) Received: by mail-qt0-x235.google.com with SMTP id s49so70417774qta.0 for ; Thu, 20 Oct 2016 15:07:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=references:in-reply-to:mime-version:content-transfer-encoding :message-id:cc:from:subject:date:to; bh=L/TJ/B5UGoyeOf8TgSW9PjyPSgVBDU5Z3YdKQQ1NIU4=; b=fQyoUDXHmmhM9E0jBDyGWxhHDTIS/7wBYkd55wQ7SmGTuAZh0n9yIP5RqaBsfazZKY B4XL4YGp4fydQts1M1FRz/6fT8A4lPELWfvQDr0wsia7llbWSnlLaEJlhPzKoJPcoAls WpVbN4ESQPKApmpUWG9JJ+bUNHl8Wqb04M10vmAWAulPx3D4RzwLmBSM1psE62toxVM0 gdJ+IBfG9CCWUKsXFHz4BCLKZUnfdILihvH0/VXokD2HFre/26nnPwRWBtXg4dpL+qQZ +ND3vnXSvase4SSrWii85cKTyD9nlQxB6Ij3coy9UhNPfpMbGPsSm2w3tHRrXG5KCCWL +nZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:references:in-reply-to:mime-version :content-transfer-encoding:message-id:cc:from:subject:date:to; bh=L/TJ/B5UGoyeOf8TgSW9PjyPSgVBDU5Z3YdKQQ1NIU4=; b=Pdv7ISO4uj4dZndkipOux0qeOVRivcdY20r4W/t6nSiCMdvp0tHFXLvQSQnmR4HPec OIeUfoKqj65deevSRnYuWg2HVyZvofmgtF/soXl3b9TSaOmU9ZXLmsvhxPS0oqkSO1Uv iPAjNeW0C7avqryn/P9NrAG2j+FR786Uco1UuymGlAJs4epJjL3PIn9+z090u0Nka8fA i+aEm4cf1AqeR/ApCiENYymbvDChYmNsNfGaKrTMKscLZzYJQ9fE6Uy/rATY7pVdj5mY q3CAJZOqqDks7PXn+9Lgv900H19Ms/Ubw1C9WXhHdj/4QtExV4fz5+HbPmug6Cg09Job DsSg== X-Gm-Message-State: ABUngvcCAPbacTntCZcZ2kFE/958lv6pHSVVoIXkwlGpO2M6jLKDe5RP7FZschnEEHr6qQ== X-Received: by 10.200.57.58 with SMTP id s55mr2844362qtb.68.1477001247793; Thu, 20 Oct 2016 15:07:27 -0700 (PDT) Received: from [30.83.188.127] ([172.56.2.145]) by smtp.gmail.com with ESMTPSA id q190sm24711959qke.31.2016.10.20.15.07.26 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 20 Oct 2016 15:07:26 -0700 (PDT) References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> In-Reply-To: Mime-Version: 1.0 (1.0) Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Message-Id: Cc: Peter , freebsd-stable@freebsd.org X-Mailer: iPhone Mail (14A456) From: Chris Watson Subject: Re: zfs, a directory that used to hold lot of files and listing pause Date: Thu, 20 Oct 2016 17:07:23 -0500 To: Jonathan Chen X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 22:07:29 -0000 While I have yet to encounter this with PG on ZFS, knock on wood, this obvio= usly is not an isolated issue and if possible those experiencing it should d= o as much investigation as possible and open a PR. This seems like something= I'm going to read about FreeBSD and PG/ZFS over at Hacker News from a Linux= switcher as they explain why they went back to Linux in the future. It's no= t a show stopper but it's obviously an issue.=20 Chris Sent from my iPhone 5 > On Oct 20, 2016, at 4:18 PM, Jonathan Chen wrote: >=20 >> On 21 October 2016 at 09:09, Peter wrote: >> [...] >>=20 >> I see this on my pgsql_tmp dirs (where Postgres stores intermediate >> query data that gets too big for mem - usually lots of files) - in >> normal operation these dirs are completely empty, but make heavy disk >> activity (even writing!) when doing ls. >> Seems normal, I dont care as long as the thing is stable. One would need >> to check how ZFS stores directories and what kind of fragmentation can >> happen there. Or wait for some future feature that would do >> housekeeping. ;) >=20 > I'm seeing this as well with an Odoo ERP running on Postgresql. This > lag does matter to me as this is huge performance hit when running > Postgresql on ZFS, and it would be good to see this resolved. > pg_restores can make the system crawl as well. >=20 > Cheers. > --=20 > Jonathan Chen > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Thu Oct 20 22:26:39 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 78A72C1B5E7 for ; Thu, 20 Oct 2016 22:26:39 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-lf0-x231.google.com (mail-lf0-x231.google.com [IPv6:2a00:1450:4010:c07::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F40CD7E4 for ; Thu, 20 Oct 2016 22:26:38 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-lf0-x231.google.com with SMTP id b81so103736984lfe.1 for ; Thu, 20 Oct 2016 15:26:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=Brgjd+8aMljf9vJ+h1R62UYCNdX/UtTBUAt1fQosF9I=; b=s9kPHvoCtMQBRuGtTsr0cfhlypB1dJXq+ET60DiBr5IHvEQTGjW7hIaNydd+6VRMT+ yYM46S/Jur3ilZLZVFBpN9puDgMBWFqFryN2anTaJMzrJONSmj4hxgi4H9D3zYR6Slad CsNGGaKVe5qpELz9aAVu84Tp+zMpxoQz2RZLFOwvtEryBqdk3lkRpXNhbH6z2CoSqQrB 9CrAAXTQgnGl4CWl5SI/Ltmt4uWa9Q3wJSeg70/6wsttzg3bJpMMKHX1qnZriFcQdTUK jBKYSaw2Ha1IrSXm7811TALyksKPkU2/HjbA86PyBJ7yiNK6ew0RUOJsGQsQxHGJQWQS j4qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=Brgjd+8aMljf9vJ+h1R62UYCNdX/UtTBUAt1fQosF9I=; b=hPoZJpwAX/gm3+i7AQVncrKuSY1O03DqAnOilmREjCGIcoBsjmrIymtWEBnAHmjnFS +TOdPl/4rn+ReADO2V8n6KVzwmuCnvE8a7KXj8GZGD0QlPsn7yvch06xP07MaOtvoK/p QxnCmeFkmfvDmsShZmCGWUG7qTjxcO/2xacWfHRX/86BYKvRioTOgpIWc3rC/Zc4B9P+ kFPn8ssYc7u8nhbk0L5HhDKg6K2TO1nJROm4atkmMOTCNrMIu3QEa18tQMovQ4f3eqYa JRbDpeAO0ZHc6mg2lJ8hiu+8B7CzdpGUVOfENxurlJmOwUWRLucadiByhvnHcoA3QBwW 0vAw== X-Gm-Message-State: AA6/9RkeZeEJMkvD/wrJOLHDohNt6OqXUeF+rj58Ozi18TXaxqt5HvYVA1JXpmni039PE4Iy X-Received: by 10.28.158.203 with SMTP id h194mr291283wme.59.1477002393802; Thu, 20 Oct 2016 15:26:33 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id us3sm81856679wjb.32.2016.10.20.15.26.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Oct 2016 15:26:32 -0700 (PDT) Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> From: Steven Hartland Message-ID: <92607b3b-a2d1-2391-5bd6-9781f426d7a6@multiplay.co.uk> Date: Thu, 20 Oct 2016 23:27:02 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 22:26:39 -0000 On 20/10/2016 22:18, Jonathan Chen wrote: > On 21 October 2016 at 09:09, Peter wrote: > [...] >> I see this on my pgsql_tmp dirs (where Postgres stores intermediate >> query data that gets too big for mem - usually lots of files) - in >> normal operation these dirs are completely empty, but make heavy disk >> activity (even writing!) when doing ls. >> Seems normal, I dont care as long as the thing is stable. One would need >> to check how ZFS stores directories and what kind of fragmentation can >> happen there. Or wait for some future feature that would do >> housekeeping. ;) > I'm seeing this as well with an Odoo ERP running on Postgresql. This > lag does matter to me as this is huge performance hit when running > Postgresql on ZFS, and it would be good to see this resolved. > pg_restores can make the system crawl as well. As mentioned before could you confirm you have disable atime? From owner-freebsd-stable@freebsd.org Thu Oct 20 22:48:10 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A79F3C1BA18 for ; Thu, 20 Oct 2016 22:48:10 +0000 (UTC) (envelope-from jonc@chen.org.nz) Received: from mail-it0-x233.google.com (mail-it0-x233.google.com [IPv6:2607:f8b0:4001:c0b::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6D515FEA for ; Thu, 20 Oct 2016 22:48:09 +0000 (UTC) (envelope-from jonc@chen.org.nz) Received: by mail-it0-x233.google.com with SMTP id 66so117915801itl.1 for ; Thu, 20 Oct 2016 15:48:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chen-org-nz.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=SFSE3J2E1u/qEsMWKnuQVHv6xmexM6zYMUqcB645cz8=; b=lbKQll4zMh/0B6VYSLEqLHFzC2qA4JP1qYfQGEzd6bgXdF8GYA1C8XUZM3B7wPFl2G IU+N/kD6sZHcll01dKo5eH/DWj3Og9LZHmDB7WPsCl6rgLXgMRvaX4kecm4BErpwCa+g Tm64eBRrefzXcpO5Z5b+JbHUIOTtzFKEI/OD5w+AHz61FYM7Rb0NtwCILpUyhnoj/DsU V1x+VlY2onN3iFyYOnNGbqffY/dl+i4PcKwRm2fQzs/nu2cO0eAPryWUfJwOMTCDMGX9 0wbYd29vRRcRuM6f1FWQbKVGBS03LAUZni9TviFErkZGXwr6le1cimQkXmbX/N+RSOs1 wyHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=SFSE3J2E1u/qEsMWKnuQVHv6xmexM6zYMUqcB645cz8=; b=Rq+ojVK4jY2+PGquzxtN0vVC4ZUhIwirF7UNhVG8hlLxlafHWiX4gVQtPon8lFQ0jA 1ZXa62ebACGdYX59QHO3fSeHwobE0xCFrqwzrZfnAfhaC950i741vBcS9ai0rCWG16Qr bH52h+MlMsPIIx5mZ1Fg+PFWbnYGy+2tGaQqLm5DXj8vDQQdqmYdUGd1mjdnC+rb03Fb 15R78LKVzrfRE93GQcshEv4M8W4m5IrA3OGazr9lstQa6M4AgGncqhpeEVVPqiOoEeMD E9IhCUQq8CyO+MA9yNEjguu4yrJYT+JAZyyGg2om4dQd8blHHRqS6XNCet2F2xqI+gZQ +0PA== X-Gm-Message-State: ABUngvdHQNDJmgA404L9xLKGuiZmym7KW1ZvDFOMcgyWQa6wYf8AsK5KlD/De6PdvkQiabDue64nwuuDP13WEw== X-Received: by 10.107.34.199 with SMTP id i190mr3386513ioi.90.1477003688545; Thu, 20 Oct 2016 15:48:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.38.87 with HTTP; Thu, 20 Oct 2016 15:48:07 -0700 (PDT) X-Originating-IP: [203.99.129.1] In-Reply-To: <92607b3b-a2d1-2391-5bd6-9781f426d7a6@multiplay.co.uk> References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <92607b3b-a2d1-2391-5bd6-9781f426d7a6@multiplay.co.uk> From: Jonathan Chen Date: Fri, 21 Oct 2016 11:48:07 +1300 Message-ID: Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: Steven Hartland Cc: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 22:48:10 -0000 On 21 October 2016 at 11:27, Steven Hartland wrote: > On 20/10/2016 22:18, Jonathan Chen wrote: >> >> On 21 October 2016 at 09:09, Peter wrote: >> [...] >>> >>> I see this on my pgsql_tmp dirs (where Postgres stores intermediate >>> query data that gets too big for mem - usually lots of files) - in >>> normal operation these dirs are completely empty, but make heavy disk >>> activity (even writing!) when doing ls. >>> Seems normal, I dont care as long as the thing is stable. One would need >>> to check how ZFS stores directories and what kind of fragmentation can >>> happen there. Or wait for some future feature that would do >>> housekeeping. ;) >> >> I'm seeing this as well with an Odoo ERP running on Postgresql. This >> lag does matter to me as this is huge performance hit when running >> Postgresql on ZFS, and it would be good to see this resolved. >> pg_restores can make the system crawl as well. > > As mentioned before could you confirm you have disable atime? Yup, also set the blocksize to 4K. 11:46am# zfs get all irontree/postgresql NAME PROPERTY VALUE SOURCE irontree/postgresql type filesystem - irontree/postgresql creation Wed Sep 23 15:07 2015 - irontree/postgresql used 43.8G - irontree/postgresql available 592G - irontree/postgresql referenced 43.8G - irontree/postgresql compressratio 1.00x - irontree/postgresql mounted yes - irontree/postgresql quota none default irontree/postgresql reservation none default irontree/postgresql recordsize 8K local irontree/postgresql mountpoint /postgresql inherited from irontree irontree/postgresql sharenfs off default irontree/postgresql checksum on default irontree/postgresql compression off default irontree/postgresql atime off local irontree/postgresql devices on default irontree/postgresql exec on default irontree/postgresql setuid on default irontree/postgresql readonly off default irontree/postgresql jailed off default irontree/postgresql snapdir hidden default irontree/postgresql aclmode discard default irontree/postgresql aclinherit restricted default irontree/postgresql canmount on default irontree/postgresql xattr off temporary irontree/postgresql copies 1 default irontree/postgresql version 5 - irontree/postgresql utf8only off - irontree/postgresql normalization none - irontree/postgresql casesensitivity sensitive - irontree/postgresql vscan off default irontree/postgresql nbmand off default irontree/postgresql sharesmb off default irontree/postgresql refquota none default irontree/postgresql refreservation none default irontree/postgresql primarycache all default irontree/postgresql secondarycache all default irontree/postgresql usedbysnapshots 0 - irontree/postgresql usedbydataset 43.8G - irontree/postgresql usedbychildren 0 - irontree/postgresql usedbyrefreservation 0 - irontree/postgresql logbias latency default irontree/postgresql dedup off default irontree/postgresql mlslabel - irontree/postgresql sync standard default irontree/postgresql refcompressratio 1.00x - irontree/postgresql written 43.8G - irontree/postgresql logicalused 43.4G - irontree/postgresql logicalreferenced 43.4G - irontree/postgresql volmode default default irontree/postgresql filesystem_limit none default irontree/postgresql snapshot_limit none default irontree/postgresql filesystem_count none default irontree/postgresql snapshot_count none default irontree/postgresql redundant_metadata all default -- Jonathan Chen From owner-freebsd-stable@freebsd.org Thu Oct 20 23:56:08 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9B74BC16A6C for ; Thu, 20 Oct 2016 23:56:08 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-lf0-x22e.google.com (mail-lf0-x22e.google.com [IPv6:2a00:1450:4010:c07::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0DF4E92D for ; Thu, 20 Oct 2016 23:56:07 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-lf0-x22e.google.com with SMTP id b81so107419570lfe.1 for ; Thu, 20 Oct 2016 16:56:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=cLZAEXiCh4PJ93dQ1ew2ZptO+0Ygx2Kl2///fFTCkbk=; b=zQhUq943NeeRt8mchcP4bwQGmrdnFtiRuMjbE31UJmaS1mGa3H+Pgpf5ovgylW3UCy 679SB3h6UWQcSirwD9UmomGjbdKh+hJu4kDWzo53r6SOjOGSwIoKPFKacXwGgLsS5Ut5 /Hrli3sjQRNLrAKfTGqHeLlKK4WfC+29brJcqPOvR+c/bhUUvsVMAmkBWGmMLCI2u68o Yn7Qo4ymT1l5qJ0B6pb/U6BTxmhlbAb0tjcYAwXh1cG0idP/x8v+ecTHTIJcV8H+uVLy zRqYmhRQs/mMwo22dn9oAI/K/PGICkfPp5hjiDK7n55nrL9I9jSi7u7R8V8ImL0A7RPX lpKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=cLZAEXiCh4PJ93dQ1ew2ZptO+0Ygx2Kl2///fFTCkbk=; b=Z+X1ZENNtUtrMT02CTIbRY8D+FY9Lu4vY7jrFuBhqY5N5RIgzO5wm2b7tpPgACoaqw ATRwfxo2p1rcZ1AGTM9qBkLGv6JjcR1R0B5S2UJ6VziUWzmvrDbR7MQ5/NaSNq24wyr8 nj1NloJ2F/7ZwPm73AISWzz4ouflxcnqzVs2I0VKxZNwdxEZujG1nKo95UPZxE8ZdPsN qSGlcUvQ0e/Tf68CJld/et8tfsudVQET8K9tXByr7W/dWkXD/CmVFj+ovCjatnGUO3MY uOUBsc+6dOfRlkO4AeVPuFmDAYXgrmpwA3C6ubNQiLoBzlluQrW6YmHZOoqI26E3ZkZE SBiw== X-Gm-Message-State: AA6/9RkwgttcGUINNL7TCUJUrmTFEnae8TsQOdYzNMV6pygcAu7stfEWSKR9BTBQRrLGurnK X-Received: by 10.28.60.7 with SMTP id j7mr4858189wma.7.1477007764601; Thu, 20 Oct 2016 16:56:04 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id au8sm81900802wjc.12.2016.10.20.16.56.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Oct 2016 16:56:03 -0700 (PDT) Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: Jonathan Chen References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <92607b3b-a2d1-2391-5bd6-9781f426d7a6@multiplay.co.uk> Cc: freebsd-stable@freebsd.org From: Steven Hartland Message-ID: <1b187ab5-cf67-bf66-ab5f-8ec5ae29c140@multiplay.co.uk> Date: Fri, 21 Oct 2016 00:56:33 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Oct 2016 23:56:08 -0000 On 20/10/2016 23:48, Jonathan Chen wrote: > On 21 October 2016 at 11:27, Steven Hartland wrote: >> On 20/10/2016 22:18, Jonathan Chen wrote: >>> On 21 October 2016 at 09:09, Peter wrote: >>> [...] >>>> I see this on my pgsql_tmp dirs (where Postgres stores intermediate >>>> query data that gets too big for mem - usually lots of files) - in >>>> normal operation these dirs are completely empty, but make heavy disk >>>> activity (even writing!) when doing ls. >>>> Seems normal, I dont care as long as the thing is stable. One would need >>>> to check how ZFS stores directories and what kind of fragmentation can >>>> happen there. Or wait for some future feature that would do >>>> housekeeping. ;) >>> I'm seeing this as well with an Odoo ERP running on Postgresql. This >>> lag does matter to me as this is huge performance hit when running >>> Postgresql on ZFS, and it would be good to see this resolved. >>> pg_restores can make the system crawl as well. >> As mentioned before could you confirm you have disable atime? > Yup, also set the blocksize to 4K. > > 11:46am# zfs get all irontree/postgresql > NAME PROPERTY VALUE SOURCE > irontree/postgresql type filesystem - > irontree/postgresql creation Wed Sep 23 15:07 2015 - > irontree/postgresql used 43.8G - > irontree/postgresql available 592G - > irontree/postgresql referenced 43.8G - > irontree/postgresql compressratio 1.00x - > irontree/postgresql mounted yes - > irontree/postgresql quota none default > irontree/postgresql reservation none default > irontree/postgresql recordsize 8K local > irontree/postgresql mountpoint /postgresql > inherited from irontree > irontree/postgresql sharenfs off default > irontree/postgresql checksum on default > irontree/postgresql compression off default > irontree/postgresql atime off local > irontree/postgresql devices on default > irontree/postgresql exec on default > irontree/postgresql setuid on default > irontree/postgresql readonly off default > irontree/postgresql jailed off default > irontree/postgresql snapdir hidden default > irontree/postgresql aclmode discard default > irontree/postgresql aclinherit restricted default > irontree/postgresql canmount on default > irontree/postgresql xattr off temporary > irontree/postgresql copies 1 default > irontree/postgresql version 5 - > irontree/postgresql utf8only off - > irontree/postgresql normalization none - > irontree/postgresql casesensitivity sensitive - > irontree/postgresql vscan off default > irontree/postgresql nbmand off default > irontree/postgresql sharesmb off default > irontree/postgresql refquota none default > irontree/postgresql refreservation none default > irontree/postgresql primarycache all default > irontree/postgresql secondarycache all default > irontree/postgresql usedbysnapshots 0 - > irontree/postgresql usedbydataset 43.8G - > irontree/postgresql usedbychildren 0 - > irontree/postgresql usedbyrefreservation 0 - > irontree/postgresql logbias latency default > irontree/postgresql dedup off default > irontree/postgresql mlslabel - > irontree/postgresql sync standard default > irontree/postgresql refcompressratio 1.00x - > irontree/postgresql written 43.8G - > irontree/postgresql logicalused 43.4G - > irontree/postgresql logicalreferenced 43.4G - > irontree/postgresql volmode default default > irontree/postgresql filesystem_limit none default > irontree/postgresql snapshot_limit none default > irontree/postgresql filesystem_count none default > irontree/postgresql snapshot_count none default > irontree/postgresql redundant_metadata all default > When you see the stalling what does gstat -pd and top -SHz show? Regards Steve From owner-freebsd-stable@freebsd.org Fri Oct 21 00:55:01 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A40CC19B78 for ; Fri, 21 Oct 2016 00:55:01 +0000 (UTC) (envelope-from jonc@chen.org.nz) Received: from mail-it0-x229.google.com (mail-it0-x229.google.com [IPv6:2607:f8b0:4001:c0b::229]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 04DF8363 for ; Fri, 21 Oct 2016 00:55:01 +0000 (UTC) (envelope-from jonc@chen.org.nz) Received: by mail-it0-x229.google.com with SMTP id 4so206684715itv.0 for ; Thu, 20 Oct 2016 17:55:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chen-org-nz.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=MUKTpbGALqtjtL6af361fPTs/BIUAOIiS9FCaqWOOdk=; b=PdmO//UuYiVoKuuff1pLvjbL1kB89IyG/muhwFUrCMgJiyB3BxsF0497ztIytZvegn 1mSrKF/F5fmD4XDhrV5W5P2EZbhXM2pG9imGXbyayGIVWE5P4VWoJDgTLqe/A0Ks6tdg I+ijpwjhRZMtXV+EBdYRJDbUBZQ3qdqYM46QNmtldqGL2tYOskQeqPiMcbd+n1mmmzL2 4tyG1CtMtoSV4ogCjk23TKWAsLBmb8bBJCuUq4XXs9vm/cxfZhMfH2VWlDAtvfiCXW8n osmY/cIJL9AJ93ZcdOgpWeJcCby01RfBtlSUPzfksJFSqlyfzryU5HxV78Te6GQN/np0 JxsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=MUKTpbGALqtjtL6af361fPTs/BIUAOIiS9FCaqWOOdk=; b=KhKek/gWilP2JSKhZFwNYDS1Qekg/A4/Z4r8MC3oqLEKl4gJDyf60u5f2ugSZlbAOF bM5pNO1cWtsxznOQvm92RoHwPbe5nCwpOkOkmb/Te7bw5kDn8mHXaY6/M61TfsYnPc+I x9IMvPz/mE9JN9Vj4/eGInhhhgDKMu6wWbcwFPDObQP1bFq9p4adEl2TkVy1brfomIim gSl3v6zyqZxBSfd8grIwqes2xMQvQ9CgRDQiqCeiF8zq2Y+hlgfG+v2/rWxMs0B+KM0W vMhcfOQz84lLl9V2+awvM9OAqPcIx9iicwLEoNfZQdFqwHTLC7hMpR8dvB1dP/vQnUF0 h0zw== X-Gm-Message-State: AA6/9RnmmgByOjTiEuV1SbTHTyms43iVHRsM9ltdAd5BVbUqjGJkB3RR8J2DXPQY9qQgcEgOte6cR9OHcgxH8Q== X-Received: by 10.36.44.80 with SMTP id i77mr3677718iti.119.1477011298481; Thu, 20 Oct 2016 17:54:58 -0700 (PDT) MIME-Version: 1.0 Received: by 10.79.38.87 with HTTP; Thu, 20 Oct 2016 17:54:57 -0700 (PDT) X-Originating-IP: [203.99.129.1] In-Reply-To: <1b187ab5-cf67-bf66-ab5f-8ec5ae29c140@multiplay.co.uk> References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <92607b3b-a2d1-2391-5bd6-9781f426d7a6@multiplay.co.uk> <1b187ab5-cf67-bf66-ab5f-8ec5ae29c140@multiplay.co.uk> From: Jonathan Chen Date: Fri, 21 Oct 2016 13:54:57 +1300 Message-ID: Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: Steven Hartland Cc: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 00:55:01 -0000 On 21 October 2016 at 12:56, Steven Hartland wrote: [...] > When you see the stalling what does gstat -pd and top -SHz show? On my dev box: 1:38pm# uname -a FreeBSD irontree 10.3-STABLE FreeBSD 10.3-STABLE #0 r307401: Mon Oct 17 10:17:22 NZDT 2016 root@irontree:/usr/obj/usr/src/sys/GENERIC amd64 1:49pm# gstat -pd dT: 1.004s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| cd0 18 618 1 128 41.4 606 52854 17.2 0 0 0.0 100.5| ada0 ^C 1:49pm# top -SHz last pid: 83284; load averages: 0.89, 0.68, 0.46 up 4+03:11:32 13:49:05 565 processes: 9 running, 517 sleeping, 17 zombie, 22 waiting CPU: 3.7% user, 0.0% nice, 1.9% system, 0.0% interrupt, 94.3% idle Mem: 543M Active, 2153M Inact, 11G Wired, 10M Cache, 2132M Free ARC: 7249M Total, 1325M MFU, 4534M MRU, 906M Anon, 223M Header, 261M Other Swap: 32G Total, 201M Used, 32G Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 83149 postgres 38 0 2197M 528M zio->i 5 1:13 23.19% postgres 83148 jonc 22 0 36028K 13476K select 2 0:11 3.86% pg_restore 852 postgres 20 0 2181M 2051M select 5 0:27 0.68% postgres 0 root -15 - 0K 4240K - 6 0:50 0.49% kernel{zio_write_issue_} 0 root -15 - 0K 4240K - 6 0:50 0.39% kernel{zio_write_issue_} 0 root -15 - 0K 4240K - 6 0:50 0.39% kernel{zio_write_issue_} 0 root -15 - 0K 4240K - 7 0:50 0.39% kernel{zio_write_issue_} 0 root -15 - 0K 4240K - 7 0:50 0.39% kernel{zio_write_issue_} 0 root -15 - 0K 4240K - 7 0:50 0.29% kernel{zio_write_issue_} 3 root -8 - 0K 112K zio->i 6 1:50 0.20% zfskern{txg_thread_enter} 12 root -88 - 0K 352K WAIT 0 1:07 0.20% intr{irq268: ahci0} 0 root -16 - 0K 4240K - 4 0:29 0.20% kernel{zio_write_intr_4} 0 root -16 - 0K 4240K - 7 0:29 0.10% kernel{zio_write_intr_6} 0 root -16 - 0K 4240K - 0 0:29 0.10% kernel{zio_write_intr_1} 0 root -16 - 0K 4240K - 5 0:29 0.10% kernel{zio_write_intr_2} 0 root -16 - 0K 4240K - 1 0:29 0.10% kernel{zio_write_intr_5} ... Taking another look at the internal dir structure for postgres, I'm not too sure whether this is related to the original poster's problem though. Cheers. -- Jonathan Chen From owner-freebsd-stable@freebsd.org Fri Oct 21 03:52:51 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AFDE0C1A123 for ; Fri, 21 Oct 2016 03:52:51 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2C558B8C for ; Fri, 21 Oct 2016 03:52:50 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e] ([IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9L3qkIA066150 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 21 Oct 2016 08:52:47 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1477021967; bh=usLO2QxLQ/QWfEq23XAQATp0xNkugQgMcaVxdgwV6aw=; h=Subject:To:References:From:Date:In-Reply-To; b=Mv7+86Qwa54UEhKTUmU0bobLIsx2cOBR08RlQB6Sc2gNFfnFhHXixIhc1nYXoxJdb RMY28Ji61BEYw2ECxGrtRt/SRvgoORSmaVQKEoDfaeuPY7s5NKvHfLrX26UMq7QYXt bs9G9jMwYXwE3hHWAOF1ZzcUd4zmvwH7G3PCTgGE= Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> From: "Eugene M. Zheganin" Message-ID: <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> Date: Fri, 21 Oct 2016 08:52:52 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 03:52:51 -0000 Hi. On 20.10.2016 21:17, Steven Hartland wrote: > Do you have atime enabled for the relevant volume? I do. > > If so disable it and see if that helps: > zfs set atime=off > Nah, it doesn't help at all. Thanks. Eugene. From owner-freebsd-stable@freebsd.org Fri Oct 21 04:21:21 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2227FC1A7BA for ; Fri, 21 Oct 2016 04:21:21 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-lf0-x22b.google.com (mail-lf0-x22b.google.com [IPv6:2a00:1450:4010:c07::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B82063B for ; Fri, 21 Oct 2016 04:21:20 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-lf0-x22b.google.com with SMTP id b81so118009725lfe.1 for ; Thu, 20 Oct 2016 21:21:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to; bh=nOZlYr2hvEi0k4W3P+b27OAlmal8HSOKHvz6DK+I79I=; b=Uty3ZArEwRE9CTMLUaiRTjy1daVkfuk1oe6v/AkiWp425RJaGZzhOE2Kcc3nRiodOj p1psbRmX6/NU04T3IsHHybs1/mLBdTSJd94GWfnOvS0Q/qVfA4L8a9nWEPwSOTj1pm+9 zV5BxWXZK7VMdZ+A7I2cgH64yNACzFOGUzWywdgpEMPS/Gu8ho7S6AnG2S4Xm3EPhBLb sT3Q+fwN/x9f+B0+Z9D99Cm2hYGZyhmK+dl1nupgc3673da4qPLCHyXjMDxyXKfcDp1/ 74A1/zboGT+A7Ux/qIPmwh13fhqADUdqIL2mMcMQQAVm+BHn20tk99RZzXsFMU00CmSl uMjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to; bh=nOZlYr2hvEi0k4W3P+b27OAlmal8HSOKHvz6DK+I79I=; b=l3v1QGHBglL32hN+Q3Xtc/XlAk6nFeV3GjBlLW5fVzGjJFe3OZueVctPKYPvD81sYA SyACZjjuO7wK9gEl4B1/3KC2BvF2Ywz+dxjl7WU9pIDPJoa1WlaML8q0VwCmJgs4f51+ t3dtIKyN6BSo71R8f2wvYqfyc2SXIyhHS99+Fpo85rmggfAkx3gyLshx7al5pKMKEQiJ Bp6qyFypzTxk8ldDrH1npyRppSdWCGelkR1NFGuMSiIRbwmkuc1svK2E9rw11bXkg5Wy QrBSXeeRJGYoaUeVR5VfXo9chZQrOlh0uczR5ZKpHWYjFfGiapbaP7+XK8D1/yxaFRxm 3jUA== X-Gm-Message-State: AA6/9RmebIec8PR190Txd7MCLQ/YMduUDraa5mdpavEmRF8+ZGFkmVpcDk/8VMHBGfCqfjbj X-Received: by 10.28.142.82 with SMTP id q79mr1016713wmd.20.1477023676395; Thu, 20 Oct 2016 21:21:16 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id q135sm1997724wmd.16.2016.10.20.21.21.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Oct 2016 21:21:15 -0700 (PDT) Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: Jonathan Chen References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <92607b3b-a2d1-2391-5bd6-9781f426d7a6@multiplay.co.uk> <1b187ab5-cf67-bf66-ab5f-8ec5ae29c140@multiplay.co.uk> Cc: freebsd-stable@freebsd.org From: Steven Hartland Message-ID: Date: Fri, 21 Oct 2016 05:21:45 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 04:21:21 -0000 In your case there your vdev (ada0) is saturated with writes from postgres. You should consider more / faster disks. You might also want to consider enabling lz4 compression on the PG volume as its works well in IO bound situations. On 21/10/2016 01:54, Jonathan Chen wrote: > On 21 October 2016 at 12:56, Steven Hartland wrote: > [...] >> When you see the stalling what does gstat -pd and top -SHz show? > On my dev box: > > 1:38pm# uname -a > FreeBSD irontree 10.3-STABLE FreeBSD 10.3-STABLE #0 r307401: Mon Oct > 17 10:17:22 NZDT 2016 root@irontree:/usr/obj/usr/src/sys/GENERIC > amd64 > 1:49pm# gstat -pd > dT: 1.004s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps > ms/d %busy Name > 0 0 0 0 0.0 0 0 0.0 0 0 > 0.0 0.0| cd0 > 18 618 1 128 41.4 606 52854 17.2 0 0 > 0.0 100.5| ada0 > ^C > 1:49pm# top -SHz > last pid: 83284; load averages: 0.89, 0.68, 0.46 > up > 4+03:11:32 13:49:05 > 565 processes: 9 running, 517 sleeping, 17 zombie, 22 waiting > CPU: 3.7% user, 0.0% nice, 1.9% system, 0.0% interrupt, 94.3% idle > Mem: 543M Active, 2153M Inact, 11G Wired, 10M Cache, 2132M Free > ARC: 7249M Total, 1325M MFU, 4534M MRU, 906M Anon, 223M Header, 261M Other > Swap: 32G Total, 201M Used, 32G Free > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 83149 postgres 38 0 2197M 528M zio->i 5 1:13 23.19% postgres > 83148 jonc 22 0 36028K 13476K select 2 0:11 3.86% pg_restore > 852 postgres 20 0 2181M 2051M select 5 0:27 0.68% postgres > 0 root -15 - 0K 4240K - 6 0:50 0.49% > kernel{zio_write_issue_} > 0 root -15 - 0K 4240K - 6 0:50 0.39% > kernel{zio_write_issue_} > 0 root -15 - 0K 4240K - 6 0:50 0.39% > kernel{zio_write_issue_} > 0 root -15 - 0K 4240K - 7 0:50 0.39% > kernel{zio_write_issue_} > 0 root -15 - 0K 4240K - 7 0:50 0.39% > kernel{zio_write_issue_} > 0 root -15 - 0K 4240K - 7 0:50 0.29% > kernel{zio_write_issue_} > 3 root -8 - 0K 112K zio->i 6 1:50 0.20% > zfskern{txg_thread_enter} > 12 root -88 - 0K 352K WAIT 0 1:07 0.20% > intr{irq268: ahci0} > 0 root -16 - 0K 4240K - 4 0:29 0.20% > kernel{zio_write_intr_4} > 0 root -16 - 0K 4240K - 7 0:29 0.10% > kernel{zio_write_intr_6} > 0 root -16 - 0K 4240K - 0 0:29 0.10% > kernel{zio_write_intr_1} > 0 root -16 - 0K 4240K - 5 0:29 0.10% > kernel{zio_write_intr_2} > 0 root -16 - 0K 4240K - 1 0:29 0.10% > kernel{zio_write_intr_5} > ... > > Taking another look at the internal dir structure for postgres, I'm > not too sure whether this is related to the original poster's problem > though. > > Cheers. From owner-freebsd-stable@freebsd.org Fri Oct 21 04:22:30 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E35B2C1A968 for ; Fri, 21 Oct 2016 04:22:30 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-lf0-x22d.google.com (mail-lf0-x22d.google.com [IPv6:2a00:1450:4010:c07::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5F2A5958 for ; Fri, 21 Oct 2016 04:22:30 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-lf0-x22d.google.com with SMTP id b75so120773507lfg.3 for ; Thu, 20 Oct 2016 21:22:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=xpxid8DTsm7zzlfMRamN0hrtiXN+ZNjO+wJxe2HP8pM=; b=hHOeXAAajjhXIf4X82yiA8PEKQbdS/VvVcT2oPbedsRhhUE0/2Irj/ogPj/W3RNaOd bQ5J7V8ibXjLvWIgmdgc0AUQxQRO4bVzUrbUjRvv9SCw2m2PIiu0cAfOWkLVIGfNGXhm pxclfnizZgr8l0HPJK31OSUXq6K0hQReQYt5BosQFIGE/2UrKrLtIfO0Z1EFDi/6d8Ai QaEAfFynSM99FYF3HDvUu01Ze3EYODkjzFbi47vliIR4NQlPMl3gUj05B45vzl9BHjvc tgf5i4SdEF7QmCS/QqoLLsCf5x0mQURgzaCKjK44bWvHTHGvwOXtWPK5l+ZIXc4+BDg2 OL7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=xpxid8DTsm7zzlfMRamN0hrtiXN+ZNjO+wJxe2HP8pM=; b=Rbxkj4o4C4fPhV5ft1K2qyKOh/NV3eYQWl+RNw/XTgyYJUkavVZ8Km20WpvmIrqfo7 CuWdwoZUrOkZLMIIwxyzDr2/n36RtN0eQ0DlQO35+gs6sU5h6zNewucnopg2NHwlA5OV 0fkoPd8wqjn5wTeX8s8iha7x2EFhY9J2/axY0uw3lyn9ww9YtMtn9GkvG2HwX6vSJlSC ahd8hRsyuq7WVdDkTXPVgRaQFCS6NlFQOHPH8I26FSoYr9gpo+zI1E6+ozMwFs0nqMBk /3Zb8Z8cGq8LnlwhF+ElXtfm6BqtRDct52/l7HLYu9Lmj9C2bnFW3P0ynpiRS2rgMZLE yvJw== X-Gm-Message-State: AA6/9Rl2ZsHfvFYzyikI0EZ7xNk+w6eTU+c9agZBfSrC3aC9kwjP9yg+3ScbXKxDoXlABAeU X-Received: by 10.28.103.7 with SMTP id b7mr1059881wmc.48.1477023746835; Thu, 20 Oct 2016 21:22:26 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id o62sm1952631wmg.9.2016.10.20.21.22.25 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Oct 2016 21:22:26 -0700 (PDT) Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> From: Steven Hartland Message-ID: <38a84fce-fd97-a6a1-5820-89e578d76f0b@multiplay.co.uk> Date: Fri, 21 Oct 2016 05:22:56 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 04:22:31 -0000 On 21/10/2016 04:52, Eugene M. Zheganin wrote: > Hi. > > On 20.10.2016 21:17, Steven Hartland wrote: >> Do you have atime enabled for the relevant volume? > I do. >> >> If so disable it and see if that helps: >> zfs set atime=off >> > Nah, it doesn't help at all. As per with Jonathon what does gstat -pd and top -SHz show? From owner-freebsd-stable@freebsd.org Fri Oct 21 05:58:22 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F15B9C1BACD for ; Fri, 21 Oct 2016 05:58:22 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vps.rulingia.com (vps.rulingia.com [103.243.244.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.rulingia.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 901F39F6 for ; Fri, 21 Oct 2016 05:58:21 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp59-167-167-3.static.internode.on.net [59.167.167.3]) by vps.rulingia.com (8.15.2/8.15.2) with ESMTPS id u9L5To0Y062556 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 21 Oct 2016 16:29:57 +1100 (AEDT) (envelope-from peter@rulingia.com) X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.15.2/8.15.2) with ESMTPS id u9L5Ti59063528 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 21 Oct 2016 16:29:44 +1100 (AEDT) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.15.2/8.15.2/Submit) id u9L5ThWv063527; Fri, 21 Oct 2016 16:29:43 +1100 (AEDT) (envelope-from peter) Date: Fri, 21 Oct 2016 16:29:43 +1100 From: Peter Jeremy To: "Eugene M. Zheganin" Cc: freebsd-stable@freebsd.org Subject: Re: zfs, a directory that used to hold lot of files and listing pause Message-ID: <20161021052943.GJ62187@server.rulingia.com> References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="neYutvxvOLaeuPCA" Content-Disposition: inline In-Reply-To: <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp User-Agent: Mutt/1.6.1 (2016-04-27) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 05:58:23 -0000 --neYutvxvOLaeuPCA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Have you done any ZFS tuning? Could you try installing ports/sysutils/zfs-stats and posting the output =66rom "zfs-stats -a". That might point to a bottleneck or poor cache tuning. --=20 Peter Jeremy --neYutvxvOLaeuPCA Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJYCafHXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRFRUIyOTg2QzMwNjcxRTc0RTY1QzIyN0Ux NkE1OTdBMEU0QTIwQjM0AAoJEBall6Dkogs0HJcP/jLEqwC+EnzfdsZXplwsSL3T 3qyvSRVu40SaKtbWCCzPRJkDX8uXH8pitUj2wUrDX/PSFPJS98q866/83mESZSBo 3ebLl1+mMaMFZl54XBFLomTNuzDhZRHk1S8wHDlAgRix1HOAAPOzMfavQcmcukKG Fl1t5wvd2ONQ2FZqvZRD26BgNyQBY0a08mDAlQE9Q6zONLg+tFqlc357eo0ysXUT W1ts9q+lAyb4v+wlfy2v9QxMdqNnKdDUHdV5LFLWdovEtzHvFQ5Pc+MRmbibbu9k 4kvxP++Q5OoRbPDNujXYzCQnUpJM9kttbnxP02OHcMcKywrFtYB5HWPn0GPiSdFG ne5A6etweSl4mPm+2InxC8CiWRLtfi+lf30ghhmiGu3rtzHhX5W9/PvgsENCJMU5 EmUPLGyoi5Jk8Y1+tEjnM0TiEL23HdnjWPzdXjl3gm1wLGmcqkx51mz6vx9btLz4 zGJ5MwZlqEBz08WDIG0S3OdUeW7MV/4VLQ1Va9+GbYM4NEULGh1s3oHiWSltVWJN lbHr+88Th2pzeIQFrtix7huHOYBwRb3KUJbiA2KZ7+2egsg744ki6weM1AgbW1xz gig1mPvpksYWFA3QZ19ynwAzAJE5Fyu7xsrM6UnSjfTSOcM7ooK8SUlfyv77CGEs N5IyXJiKzk8885ciQPbX =GM+m -----END PGP SIGNATURE----- --neYutvxvOLaeuPCA-- From owner-freebsd-stable@freebsd.org Fri Oct 21 06:30:02 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 503C9C1B229; Fri, 21 Oct 2016 06:30:02 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: from mail-it0-x22b.google.com (mail-it0-x22b.google.com [IPv6:2607:f8b0:4001:c0b::22b]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B7C8CA1; Fri, 21 Oct 2016 06:30:02 +0000 (UTC) (envelope-from aryeh.friedman@gmail.com) Received: by mail-it0-x22b.google.com with SMTP id 4so221457973itv.0; Thu, 20 Oct 2016 23:30:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to; bh=EASGg0K1S6KiKHAa8/U55ygXpBzu4ukxFWKiJop0iMg=; b=ZTsjYIsOhXjjhIVT+FxhM2a3ivC5Ysg7itrBYdIEcGf8lAZ81j7qjxDjbWjVKgbfye g4TOcmh1ReVedMV3Wh4KSIdVsOf63CP6KYXuzJ4eUu5yk+g88Y7GN6Nh/GyGd7lPOXd9 Thxe1+zq6cd5uYAhys2FEcGGn7c6UKgVQA2chcsYi60zoNSHdm1Z+6PnBqLmmKOkHB4c N3LLiFo69TNiJFCkHBZ87fA/D8ukJMP0PLiXmCRm+3JpocCEkE3gR+V0jFq2omGpIn1H q3O1KyDeLGPODt6J30FwDLvVqXNkA8eyu1Ovw1RqoMo/gCUmvTMX52QtQjce9mUE3HxJ iqWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=EASGg0K1S6KiKHAa8/U55ygXpBzu4ukxFWKiJop0iMg=; b=jjj9TKiv9+IpTqSld7nWBe3dJHupVfuDH7xJtolD9TCoNDCPppw8CImU+A8pdSrOVD vOg0fxAuwp1Vrvu1zxPARS2O3p/zzeV4LBq4p/fkzQnIcw7IED1QSQ/Zvlx5kGJtu25Q jPur8aydIFPT9ytlj7gw2Gdi47DsWKIr6o8ws9ULMKQkuW7D1n4SvXTpQ3tQAyf3b7rj AU0ym8CP0KGlqEMNLC4HpkphYKFcWj8AoRFR7ruwV1om+/BGUPe3sk8y48DCe3eAMrHi MGtH02yq61K2c33Zjjez9G/667yTAjF5x3C+9O1Nglcvp04M//RtV3VkQ0EkecTj+snM ZxmQ== X-Gm-Message-State: AA6/9RkK0lUheGrV8+YQaL9qCkNN56kLOTcm+/o28ozoqxw5X13DhpIl0LFJUN+0Xgi+NTIkRbfYo2duH3aHYw== X-Received: by 10.107.172.133 with SMTP id v127mr5250992ioe.49.1477031399675; Thu, 20 Oct 2016 23:29:59 -0700 (PDT) MIME-Version: 1.0 Received: by 10.36.127.88 with HTTP; Thu, 20 Oct 2016 23:29:59 -0700 (PDT) From: Aryeh Friedman Date: Fri, 21 Oct 2016 02:29:59 -0400 Message-ID: Subject: port collection severally broken on 11-STABLE (it also breaks make world) To: FreeBSD Ports ML , FreeBSD Stable List Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 06:30:02 -0000 On FreeBSD lilith 10.3-STABLE FreeBSD 10.3-STABLE #0 r307694: Fri Oct 21 00:10:20 EDT 2016 root@lilith:/usr/obj/usr/src/sys/GENERIC amd64 In order to install java/openjdk8 and devel/subversion I had to install the following as packages instead of compiling it from source also make world also is broken (not before) the attempt to install openjdk8 (see below package listing for makeworld output): gettext db5 boehm-gc pcre gmp binutils jsoncpp cmake ninja llvm37 libclc gnutls cups openjdk openjdk8 root@lilith:/usr/src # make DESTDIR=/ world kernel -------------------------------------------------------------- >>> make world started on Fri Oct 21 02:27:38 EDT 2016 -------------------------------------------------------------- -------------------------------------------------------------- >>> World build started on Fri Oct 21 02:27:39 EDT 2016 -------------------------------------------------------------- -------------------------------------------------------------- >>> Rebuilding the temporary build tree -------------------------------------------------------------- rm -rf /usr/obj/usr/src/tmp/legacy/usr/include rm -f /usr/obj/usr/src/usr.bin/kdump/ioctl.c rm -f /usr/obj/usr/src/usr.bin/kdump/kdump_subr.c rm -f /usr/obj/usr/src/usr.bin/truss/ioctl.c mkdir -p /usr/obj/usr/src/tmp/lib mkdir -p /usr/obj/usr/src/tmp/usr mkdir -p /usr/obj/usr/src/tmp/legacy/bin mkdir -p /usr/obj/usr/src/tmp/legacy/usr mtree -deU -f /usr/src/etc/mtree/BSD.usr.dist -p /usr/obj/usr/src/tmp/legacy/usr >/dev/null mtree -deU -f /usr/src/etc/mtree/BSD.groff.dist -p /usr/obj/usr/src/tmp/legacy/usr >/dev/null mtree -deU -f /usr/src/etc/mtree/BSD.usr.dist -p /usr/obj/usr/src/tmp/usr >/dev/null mtree -deU -f /usr/src/etc/mtree/BSD.include.dist -p /usr/obj/usr/src/tmp/usr/include >/dev/null ln -sf /usr/src/sys /usr/obj/usr/src/tmp -------------------------------------------------------------- >>> stage 1.1: legacy release compatibility shims -------------------------------------------------------------- cd /usr/src; MAKEOBJDIRPREFIX=/usr/obj/usr/src/tmp INSTALL="sh /usr/src/tools/install.sh" PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/usr/games:/usr/obj/usr/src/tmp/legacy/bin:/sbin:/bin:/usr/sbin:/usr/bin WORLDTMP=/usr/obj/usr/src/tmp VERSION="FreeBSD 10.3-STABLE amd64 1003509" MAKEFLAGS="-m /usr/src/tools/build/mk -m /usr/src/share/mk" COMPILER_TYPE=clang make -f Makefile.inc1 DESTDIR= BOOTSTRAPPING=1003509 SSP_CFLAGS= -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT -DWITHOUT_MAN -DNO_PIC -DNO_PROFILE -DNO_SHARED _BOOTSTRAP_MAKEINFO=yes -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD -DNO_TESTS legacy ===> tools/build (obj,includes,depend,all,install) set -e; cd /usr/src/tools/build; make buildincludes; make installincludes cc -O2 -pipe -I/usr/src/tools/build/../../contrib/libc-pwcache -I/usr/src/tools/build/../../lib/libc/include -std=gnu99 -I/usr/obj/usr/src/tmp/legacy/usr/include -c /usr/src/tools/build/../../contrib/libc-pwcache/pwcache.c -o pwcache.o building static egacy library ranlib -D libegacy.a sh /usr/src/tools/install.sh -C -o root -g wheel -m 444 libegacy.a /usr/obj/usr/src/tmp/legacy/usr/lib -------------------------------------------------------------- >>> stage 1.2: bootstrap tools -------------------------------------------------------------- cd /usr/src; MAKEOBJDIRPREFIX=/usr/obj/usr/src/tmp INSTALL="sh /usr/src/tools/install.sh" PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/usr/games:/usr/obj/usr/src/tmp/legacy/bin:/sbin:/bin:/usr/sbin:/usr/bin WORLDTMP=/usr/obj/usr/src/tmp VERSION="FreeBSD 10.3-STABLE amd64 1003509" MAKEFLAGS="-m /usr/src/tools/build/mk -m /usr/src/share/mk" COMPILER_TYPE=clang make -f Makefile.inc1 DESTDIR= BOOTSTRAPPING=1003509 SSP_CFLAGS= -DWITHOUT_HTML -DWITHOUT_INFO -DNO_LINT -DWITHOUT_MAN -DNO_PIC -DNO_PROFILE -DNO_SHARED _BOOTSTRAP_MAKEINFO=yes -DNO_CPU_CFLAGS -DNO_WARNS -DNO_CTF -DEARLY_BUILD -DNO_TESTS bootstrap-tools ===> lib/clang/libllvmsupport (obj,depend,all,install) c++ -O2 -pipe -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/tools/clang/include -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support -I. -I/usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/../../lib/clang/include -DLLVM_ON_UNIX -DLLVM_ON_FREEBSD -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -fno-strict-aliasing -DLLVM_DEFAULT_TARGET_TRIPLE=\"x86_64-unknown-freebsd10.3\" -DLLVM_HOST_TRIPLE=\"x86_64-unknown-freebsd10.3\" -DDEFAULT_SYSROOT=\"\" -I/usr/obj/usr/src/tmp/legacy/usr/include -fno-exceptions -fno-rtti -c /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/APFloat.cpp -o APFloat.o In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/APFloat.cpp:15: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/APFloat.h:20: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/APInt.h:19: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/ArrayRef.h:14: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/SmallVector.h:17: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/Support/AlignOf.h:19: /usr/include/c++/v1/cstddef:58:30: error: redefinition of 'nullptr_t' struct _LIBCPP_TYPE_VIS_ONLY nullptr_t ^ /usr/include/c++/v1/__nullptr:24:30: note: previous definition is here struct _LIBCPP_TYPE_VIS_ONLY nullptr_t ^ In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/APFloat.cpp:15: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/APFloat.h:20: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/APInt.h:19: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/ArrayRef.h:14: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/SmallVector.h:17: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/Support/AlignOf.h:19: /usr/include/c++/v1/cstddef:85:58: error: redefinition of '__get_nullptr_t' inline _LIBCPP_ALWAYS_INLINE _LIBCPP_CONSTEXPR nullptr_t __get_nullptr_t() {return nullptr_t(0);} ^ /usr/include/c++/v1/__nullptr:51:58: note: previous definition is here inline _LIBCPP_ALWAYS_INLINE _LIBCPP_CONSTEXPR nullptr_t __get_nullptr_t() {return nullptr_t(0);} ^ In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/APFloat.cpp:15: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/APFloat.h:20: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/APInt.h:19: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/ArrayRef.h:14: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/SmallVector.h:19: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/Support/MathExtras.h:18: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/Support/SwapByteOrder.h:18: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/../../lib/clang/include/llvm/Support/DataTypes.h:36: In file included from /usr/include/c++/v1/cmath:301: In file included from /usr/include/c++/v1/math.h:309: /usr/include/c++/v1/type_traits:472:114: error: expected class name : public integral_constant(nullptr)) == 1> {}; ^ In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/lib/Support/APFloat.cpp:15: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/APFloat.h:20: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/APInt.h:19: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/ArrayRef.h:14: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/ADT/SmallVector.h:19: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/Support/MathExtras.h:18: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/include/llvm/Support/SwapByteOrder.h:18: In file included from /usr/src/lib/clang/libllvmsupport/../../../contrib/llvm/../../lib/clang/include/llvm/Support/DataTypes.h:36: /usr/include/c++/v1/cmath:679:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY float acos(float __lcpp_x) _NOEXCEPT {return acosf(__lcpp_x);} ^ /usr/include/c++/v1/math.h:660:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY float acos(float __lcpp_x) _NOEXCEPT {return acosf(__lcpp_x);} ^ /usr/include/c++/v1/cmath:675:9: note: using declaration using ::acos; ^ /usr/include/c++/v1/cmath:680:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY long double acos(long double __lcpp_x) _NOEXCEPT {return acosl(__lcpp_x);} ^ /usr/include/c++/v1/math.h:661:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY long double acos(long double __lcpp_x) _NOEXCEPT {return acosl(__lcpp_x);} ^ /usr/include/c++/v1/cmath:675:9: note: using declaration using ::acos; ^ /usr/include/c++/v1/cmath:686:1: error: declaration conflicts with target of using declaration already in scope acos(_A1 __lcpp_x) _NOEXCEPT {return acos((double)__lcpp_x);} ^ /usr/include/c++/v1/math.h:667:1: note: target of using declaration acos(_A1 __lcpp_x) _NOEXCEPT {return acos((double)__lcpp_x);} ^ /usr/include/c++/v1/cmath:675:9: note: using declaration using ::acos; ^ /usr/include/c++/v1/cmath:694:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY float asin(float __lcpp_x) _NOEXCEPT {return asinf(__lcpp_x);} ^ /usr/include/c++/v1/math.h:672:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY float asin(float __lcpp_x) _NOEXCEPT {return asinf(__lcpp_x);} ^ /usr/include/c++/v1/cmath:690:9: note: using declaration using ::asin; ^ /usr/include/c++/v1/cmath:695:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY long double asin(long double __lcpp_x) _NOEXCEPT {return asinl(__lcpp_x);} ^ /usr/include/c++/v1/math.h:673:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY long double asin(long double __lcpp_x) _NOEXCEPT {return asinl(__lcpp_x);} ^ /usr/include/c++/v1/cmath:690:9: note: using declaration using ::asin; ^ /usr/include/c++/v1/cmath:701:1: error: declaration conflicts with target of using declaration already in scope asin(_A1 __lcpp_x) _NOEXCEPT {return asin((double)__lcpp_x);} ^ /usr/include/c++/v1/math.h:679:1: note: target of using declaration asin(_A1 __lcpp_x) _NOEXCEPT {return asin((double)__lcpp_x);} ^ /usr/include/c++/v1/cmath:690:9: note: using declaration using ::asin; ^ /usr/include/c++/v1/cmath:709:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY float atan(float __lcpp_x) _NOEXCEPT {return atanf(__lcpp_x);} ^ /usr/include/c++/v1/math.h:684:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY float atan(float __lcpp_x) _NOEXCEPT {return atanf(__lcpp_x);} ^ /usr/include/c++/v1/cmath:705:9: note: using declaration using ::atan; ^ /usr/include/c++/v1/cmath:710:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY long double atan(long double __lcpp_x) _NOEXCEPT {return atanl(__lcpp_x);} ^ /usr/include/c++/v1/math.h:685:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY long double atan(long double __lcpp_x) _NOEXCEPT {return atanl(__lcpp_x);} ^ /usr/include/c++/v1/cmath:705:9: note: using declaration using ::atan; ^ /usr/include/c++/v1/cmath:716:1: error: declaration conflicts with target of using declaration already in scope atan(_A1 __lcpp_x) _NOEXCEPT {return atan((double)__lcpp_x);} ^ /usr/include/c++/v1/math.h:691:1: note: target of using declaration atan(_A1 __lcpp_x) _NOEXCEPT {return atan((double)__lcpp_x);} ^ /usr/include/c++/v1/cmath:705:9: note: using declaration using ::atan; ^ /usr/include/c++/v1/cmath:724:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY float atan2(float __lcpp_y, float __lcpp_x) _NOEXCEPT {retur... ^ /usr/include/c++/v1/math.h:696:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY float atan2(float __lcpp_y, float __lcpp_x) _NOEXCEPT {retur... ^ /usr/include/c++/v1/cmath:720:9: note: using declaration using ::atan2; ^ /usr/include/c++/v1/cmath:725:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY long double atan2(long double __lcpp_y, long double __lcpp_x) _NOEXCEPT {retur... ^ /usr/include/c++/v1/math.h:697:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY long double atan2(long double __lcpp_y, long double __lcpp_x) _NOEXCEPT {retur... ^ /usr/include/c++/v1/cmath:720:9: note: using declaration using ::atan2; ^ /usr/include/c++/v1/cmath:736:1: error: declaration conflicts with target of using declaration already in scope atan2(_A1 __lcpp_y, _A2 __lcpp_x) _NOEXCEPT ^ /usr/include/c++/v1/math.h:708:1: note: target of using declaration atan2(_A1 __lcpp_y, _A2 __lcpp_x) _NOEXCEPT ^ /usr/include/c++/v1/cmath:720:9: note: using declaration using ::atan2; ^ /usr/include/c++/v1/cmath:750:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY float ceil(float __lcpp_x) _NOEXCEPT {return ceilf(__lcpp_x);} ^ /usr/include/c++/v1/math.h:719:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY float ceil(float __lcpp_x) _NOEXCEPT {return ceilf(__lcpp_x);} ^ /usr/include/c++/v1/cmath:746:9: note: using declaration using ::ceil; ^ /usr/include/c++/v1/cmath:751:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY long double ceil(long double __lcpp_x) _NOEXCEPT {return ceill(__lcpp_x);} ^ /usr/include/c++/v1/math.h:720:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY long double ceil(long double __lcpp_x) _NOEXCEPT {return ceill(__lcpp_x);} ^ /usr/include/c++/v1/cmath:746:9: note: using declaration using ::ceil; ^ /usr/include/c++/v1/cmath:757:1: error: declaration conflicts with target of using declaration already in scope ceil(_A1 __lcpp_x) _NOEXCEPT {return ceil((double)__lcpp_x);} ^ /usr/include/c++/v1/math.h:726:1: note: target of using declaration ceil(_A1 __lcpp_x) _NOEXCEPT {return ceil((double)__lcpp_x);} ^ /usr/include/c++/v1/cmath:746:9: note: using declaration using ::ceil; ^ /usr/include/c++/v1/cmath:765:46: error: declaration conflicts with target of using declaration already in scope inline _LIBCPP_INLINE_VISIBILITY float cos(float __lcpp_x) _NOEXCEPT {return cosf(__lcpp_x);} ^ /usr/include/c++/v1/math.h:731:46: note: target of using declaration inline _LIBCPP_INLINE_VISIBILITY float cos(float __lcpp_x) _NOEXCEPT {return cosf(__lcpp_x);} ^ /usr/include/c++/v1/cmath:761:9: note: using declaration using ::cos; ^ fatal error: too many errors emitted, stopping now [-ferror-limit=] 20 errors generated. *** Error code 1 Stop. make[3]: stopped in /usr/src/lib/clang/libllvmsupport *** Error code 1 Stop. make[2]: stopped in /usr/src *** Error code 1 Stop. make[1]: stopped in /usr/src *** Error code 1 Stop. make: stopped in /usr/src -- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org From owner-freebsd-stable@freebsd.org Fri Oct 21 08:14:47 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B7BCEC183FD for ; Fri, 21 Oct 2016 08:14:47 +0000 (UTC) (envelope-from ml@netfence.it) Received: from smtp207.alice.it (smtp207.alice.it [82.57.200.103]) by mx1.freebsd.org (Postfix) with ESMTP id 43F44E6A for ; Fri, 21 Oct 2016 08:14:46 +0000 (UTC) (envelope-from ml@netfence.it) Received: from soth.ventu (79.46.7.147) by smtp207.alice.it (8.6.060.28) (authenticated as acanedi@alice.it) id 57C7E4C0093942D1 for freebsd-stable@freebsd.org; Fri, 21 Oct 2016 10:14:27 +0200 Received: from alamar.ventu (alamar.local.netfence.it [10.1.2.18]) by soth.ventu (8.15.2/8.15.2) with ESMTP id u9L8EQcT009469 for ; Fri, 21 Oct 2016 10:14:26 +0200 (CEST) (envelope-from ml@netfence.it) X-Authentication-Warning: soth.ventu: Host alamar.local.netfence.it [10.1.2.18] claimed to be alamar.ventu Subject: Re: Nightly disk-related panic since upgrade to 10.3 To: "freebsd-stable@freebsd.org" References: From: Andrea Venturoli Message-ID: <76d65036-0f4b-28fc-d1ef-f6527a9299a1@netfence.it> Date: Fri, 21 Oct 2016 10:14:26 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=iso-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 08:14:47 -0000 On 10/20/16 22:12, Peter wrote: Hello. > Basically You have two options: A) fire up kgdb, go into the code and > try and understand what exactly is happening. This depends > if You have clue enough to go that way; I found "man 4 gdb" and > especially the "Debugging Kernel Problems" pdf by Greg Lehey quite > helpful. I've tried this way, but altough I'm quite proficient with [k]gdb I tend to get lost in FreeBSD's kernel's source code, which, unfortunately, I'm not familiar with. BTW, I had read that book years ago; I searched for it now, but a 2005 edition still comes up. Has it ever been updated? > B) systematically change parameters. Start by figuring from the logs > the exact time of crash and what was happening then, try to reproduce > that. Then change things and isolate the cause. Again, I already tried, but without luck. Since I had one hang one night during the creation of a snapshot, yesterday I tried creating/deleting around 40 of them: I hoped to get the system to hang again, but it all worked perfectly. Since backups are run at night (possibly at the time of the hangs/panics and doing snapshots), I launched several backup jobs, but they all worked perfectly. I checked that at the times of the panics there is usually no cron job, periodic job or whatever. At least not something I could identify. There was in fact once a periodic running, but that's not the rule. "ps -axl -M /var/crash/vmcore.x" showed nothing unusual. bye & Thanks av. From owner-freebsd-stable@freebsd.org Fri Oct 21 09:04:26 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CBDB6C1B7A1 for ; Fri, 21 Oct 2016 09:04:26 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 28CDFAE4 for ; Fri, 21 Oct 2016 09:04:25 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e] ([IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9L94Lkq087314 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 21 Oct 2016 14:04:22 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1477040662; bh=qj0Xq6yt4qCryY18cnwrQuhqhoN7Rd3wvB2RHURkvdg=; h=Subject:To:References:From:Date:In-Reply-To; b=vg6XtfL5pj7dklY42w73krvkpikSjMneQG2m1SjBWyCuvmOfpfOZ0rJiuEYx+2Tdv GLEAztQdzVnyBPBaSGWu8aZ286b0JVCAjniWLqCmabm6AKvX/HTf9TBd1HOiz0Cjgo XoTNKfFnzvWjee56Fqc8eBvAHWeFahMSO+2Rgy2Q= Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> <38a84fce-fd97-a6a1-5820-89e578d76f0b@multiplay.co.uk> From: "Eugene M. Zheganin" Message-ID: <91f34889-7f4f-661d-c88f-8034402c39cd@norma.perm.ru> Date: Fri, 21 Oct 2016 14:04:27 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <38a84fce-fd97-a6a1-5820-89e578d76f0b@multiplay.co.uk> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 09:04:26 -0000 Hi. On 21.10.2016 9:22, Steven Hartland wrote: > On 21/10/2016 04:52, Eugene M. Zheganin wrote: >> Hi. >> >> On 20.10.2016 21:17, Steven Hartland wrote: >>> Do you have atime enabled for the relevant volume? >> I do. >>> >>> If so disable it and see if that helps: >>> zfs set atime=off >>> >> Nah, it doesn't help at all. > As per with Jonathon what does gstat -pd and top -SHz show? gstat (while ls'ing): dT: 1.005s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name 1 49 49 2948 13.5 0 0 0.0 0 0 0.0 65.0| ada0 0 32 32 1798 11.1 0 0 0.0 0 0 0.0 35.3| ada1 gstat (while idling): dT: 1.003s w: 1.000s L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps ms/d %busy Name 0 0 0 0 0.0 0 0 0.0 0 0 0.0 0.0| ada0 0 2 2 255 0.8 0 0 0.0 0 0 0.0 0.1| ada1 top -SHz output doesn't really differ while ls'ing or idling: last pid: 12351; load averages: 0.46, 0.49, 0.46 up 39+14:41:02 14:03:05 376 processes: 3 running, 354 sleeping, 19 waiting CPU: 5.8% user, 0.0% nice, 16.3% system, 0.0% interrupt, 77.9% idle Mem: 21M Active, 646M Inact, 931M Wired, 2311M Free ARC: 73M Total, 3396K MFU, 21M MRU, 545K Anon, 1292K Header, 47M Other Swap: 4096M Total, 4096M Free PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND 600 root 39 0 27564K 5072K nanslp 1 295.0H 24.56% monit 0 root -17 0 0K 2608K - 1 75:24 0.00% kernel{zio_write_issue} 767 freeswitch 20 0 139M 31668K uwait 0 48:29 0.00% freeswitch{freeswitch} 683 asterisk 20 0 806M 483M uwait 0 41:09 0.00% asterisk{asterisk} 0 root -8 0 0K 2608K - 0 37:43 0.00% kernel{metaslab_group_t} [... others lines are just 0% ...] Thanks. Eugene. From owner-freebsd-stable@freebsd.org Fri Oct 21 10:02:30 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CC10CC1A64F for ; Fri, 21 Oct 2016 10:02:30 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: from mail-qk0-x22a.google.com (mail-qk0-x22a.google.com [IPv6:2607:f8b0:400d:c09::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A43AE2D for ; Fri, 21 Oct 2016 10:02:30 +0000 (UTC) (envelope-from killing@multiplay.co.uk) Received: by mail-qk0-x22a.google.com with SMTP id f128so136668526qkb.1 for ; Fri, 21 Oct 2016 03:02:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to; bh=09G4124X6kZpe2hIpYKMaBi3cxzY033SjX2MKU2LqF0=; b=X8w9XuNX3Eqmo860o/JY0CvTA+jIR4cyUCtClh982DKte02yWE8j110Q6/mKigM6Ds 1v9EWFN8mTZRQIPQ2xh0/+BSz5QxNhYMjf7JJPSJAGgGyvmr3207xED639vqg668XUoR C0uhQJQYihcJsWxILNdEzMU6i5OmExycatbcdkOZS5gTU9fnVBAMrhKglZmpAl4hiwgK 8KB+FWrs5vc9iNW5T3Y1gomoLSU/2srC9lXjDElSAqsYA+zafOcoB04gHgqQaPiHL69L FSxmQ+SexaXJjIlQ9vkZ3eaJa0hWbEIfN99wxaVlI2JCmBVo8Q/LKdGLu8akLT72XZCv UHZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to; bh=09G4124X6kZpe2hIpYKMaBi3cxzY033SjX2MKU2LqF0=; b=FkW+pJddUjg3rvJA7fr7UobdHCjyE5ima58EEz2D6ufTmTiZmazU6jssfMgXd76Bwu cxUOw4bnxFLV2CfLj3LhQ9PPIVdBXK2PYgz7GBoRJxm0dvBqu1vxWsFZIgOpTXVoMaep vHeOQ1Vr3lQ9Ae7blxsv2GhoEChXRtVtlbwlj91ltMRsCF09wONbG1D8wtmvorBHAA97 ztgHqiWRPrUxfN8zDKOMQywBm0N7BGI0oF4JwsjfUDJHaM0ZkYQmjwPWeUjAVKBQSlIw wWrwi8i6WFd9Yey3lJjk6X5yJia41dtLItWt0IP8/o3XoMerrG/Eq34hCbF06/aR01pP bfmQ== X-Gm-Message-State: ABUngvc8rVoIvf+9smg+VSs+lylwFXVxHTXl4VhQ0nzN+I7xpENL16lmi/BAhtRRnOGRGnrg X-Received: by 10.194.238.162 with SMTP id vl2mr19326wjc.39.1477044148897; Fri, 21 Oct 2016 03:02:28 -0700 (PDT) Received: from [10.10.1.58] ([185.97.61.26]) by smtp.gmail.com with ESMTPSA id pe5sm2041771wjb.15.2016.10.21.03.02.27 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 21 Oct 2016 03:02:27 -0700 (PDT) Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> <38a84fce-fd97-a6a1-5820-89e578d76f0b@multiplay.co.uk> <91f34889-7f4f-661d-c88f-8034402c39cd@norma.perm.ru> From: Steven Hartland Message-ID: <19a45e94-65e6-53a9-202d-c048055460d1@multiplay.co.uk> Date: Fri, 21 Oct 2016 11:02:57 +0100 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <91f34889-7f4f-661d-c88f-8034402c39cd@norma.perm.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 10:02:30 -0000 On 21/10/2016 10:04, Eugene M. Zheganin wrote: > Hi. > > On 21.10.2016 9:22, Steven Hartland wrote: >> On 21/10/2016 04:52, Eugene M. Zheganin wrote: >>> Hi. >>> >>> On 20.10.2016 21:17, Steven Hartland wrote: >>>> Do you have atime enabled for the relevant volume? >>> I do. >>>> >>>> If so disable it and see if that helps: >>>> zfs set atime=off >>>> >>> Nah, it doesn't help at all. >> As per with Jonathon what does gstat -pd and top -SHz show? > > gstat (while ls'ing): > > dT: 1.005s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps > ms/d %busy Name > 1 49 49 2948 13.5 0 0 0.0 0 0 0.0 > 65.0| ada0 > 0 32 32 1798 11.1 0 0 0.0 0 0 0.0 > 35.3| ada1 > Averagely busy then on rust. > gstat (while idling): > > dT: 1.003s w: 1.000s > L(q) ops/s r/s kBps ms/r w/s kBps ms/w d/s kBps > ms/d %busy Name > 0 0 0 0 0.0 0 0 0.0 0 0 0.0 > 0.0| ada0 > 0 2 2 255 0.8 0 0 0.0 0 0 0.0 > 0.1| ada1 > > top -SHz output doesn't really differ while ls'ing or idling: > > last pid: 12351; load averages: 0.46, 0.49, > 0.46 up 39+14:41:02 14:03:05 > 376 processes: 3 running, 354 sleeping, 19 waiting > CPU: 5.8% user, 0.0% nice, 16.3% system, 0.0% interrupt, 77.9% idle > Mem: 21M Active, 646M Inact, 931M Wired, 2311M Free > ARC: 73M Total, 3396K MFU, 21M MRU, 545K Anon, 1292K Header, 47M Other > Swap: 4096M Total, 4096M Free > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 600 root 39 0 27564K 5072K nanslp 1 295.0H 24.56% monit > 0 root -17 0 0K 2608K - 1 75:24 0.00% > kernel{zio_write_issue} > 767 freeswitch 20 0 139M 31668K uwait 0 48:29 0.00% > freeswitch{freeswitch} > 683 asterisk 20 0 806M 483M uwait 0 41:09 0.00% > asterisk{asterisk} > 0 root -8 0 0K 2608K - 0 37:43 0.00% > kernel{metaslab_group_t} > [... others lines are just 0% ...] This looks like you only have ~4Gb ram which is pretty low for ZFS I suspect vfs.zfs.prefetch_disable will be 1, which will crash the performance. Regards Steve From owner-freebsd-stable@freebsd.org Fri Oct 21 10:20:59 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 361D4C1AD31 for ; Fri, 21 Oct 2016 10:20:59 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E979480D for ; Fri, 21 Oct 2016 10:20:58 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bxWwV-000967-3f; Fri, 21 Oct 2016 13:20:51 +0300 Date: Fri, 21 Oct 2016 13:20:51 +0300 From: Slawa Olhovchenkov To: Steven Hartland Cc: freebsd-stable@freebsd.org Subject: Re: zfs, a directory that used to hold lot of files and listing pause Message-ID: <20161021102051.GH57876@zxy.spb.ru> References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> <38a84fce-fd97-a6a1-5820-89e578d76f0b@multiplay.co.uk> <91f34889-7f4f-661d-c88f-8034402c39cd@norma.perm.ru> <19a45e94-65e6-53a9-202d-c048055460d1@multiplay.co.uk> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <19a45e94-65e6-53a9-202d-c048055460d1@multiplay.co.uk> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 10:20:59 -0000 On Fri, Oct 21, 2016 at 11:02:57AM +0100, Steven Hartland wrote: > > Mem: 21M Active, 646M Inact, 931M Wired, 2311M Free > > ARC: 73M Total, 3396K MFU, 21M MRU, 545K Anon, 1292K Header, 47M Other > > Swap: 4096M Total, 4096M Free > > > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND > > 600 root 39 0 27564K 5072K nanslp 1 295.0H 24.56% monit > > 0 root -17 0 0K 2608K - 1 75:24 0.00% > > kernel{zio_write_issue} > > 767 freeswitch 20 0 139M 31668K uwait 0 48:29 0.00% > > freeswitch{freeswitch} > > 683 asterisk 20 0 806M 483M uwait 0 41:09 0.00% > > asterisk{asterisk} > > 0 root -8 0 0K 2608K - 0 37:43 0.00% > > kernel{metaslab_group_t} > > [... others lines are just 0% ...] > This looks like you only have ~4Gb ram which is pretty low for ZFS I > suspect vfs.zfs.prefetch_disable will be 1, which will crash the > performance. ZFS prefetch affect performance dpeneds of workload (independed of RAM size): for some workloads wins, for some workloads lose (for my workload prefetch is lose and manualy disabled with 128GB RAM). Anyway, this system have only 24MB in ARC by 2.3GB free, this is may be too low for this workload. From owner-freebsd-stable@freebsd.org Fri Oct 21 10:47:20 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 09732C1B948 for ; Fri, 21 Oct 2016 10:47:20 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (jenkins-9.freebsd.org [8.8.178.209]) by mx1.freebsd.org (Postfix) with ESMTP id EF8B0B10; Fri, 21 Oct 2016 10:47:19 +0000 (UTC) (envelope-from jenkins-admin@FreeBSD.org) Received: from jenkins-9.freebsd.org (localhost [127.0.0.1]) by jenkins-9.freebsd.org (Postfix) with ESMTP id 0C6CD1D8; Fri, 21 Oct 2016 10:47:20 +0000 (UTC) Date: Fri, 21 Oct 2016 10:47:19 +0000 (GMT) From: jenkins-admin@FreeBSD.org To: jenkins-admin@FreeBSD.org, freebsd-stable@FreeBSD.org Message-ID: <396235140.37.1477046839885.JavaMail.jenkins@jenkins-9.freebsd.org> In-Reply-To: <1695454902.14.1476982109716.JavaMail.jenkins@jenkins-9.freebsd.org> References: <1695454902.14.1476982109716.JavaMail.jenkins@jenkins-9.freebsd.org> Subject: Jenkins build is still unstable: FreeBSD_stable_10 #437 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Instance-Identity: MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkKKb2VAfYQKfu1t7qk4nR5qzUBEI+UqT4BPec4qHVhqUy0FFdq50sMH+3y9bCDNOufctov6VqTNffZ3YXArnZK95YF0OX97fh+E9txYOUX1adc+TikcKjuYpHmL5dE62eaZTI+4A5jnRonskQ1PaoIFz0Kbu4mWzkFsmdiXTraGzomXq4cHUCATA2+K4eDYgjXEQI30z3GOMmmZ4t/+6QGk1cMb/BqMWHbn80AsRCb4tU7Hpd72XLDpsuO7YRP1Q0CjmNAuBOTj+sFiiOe6U9HpqOlQN+iFUvBdZo/ybuy5Kh71cAaYQNL68cYdZJ6binH/DkG3KY/fS7DFYAeuwjwIDAQAB X-Jenkins-Job: FreeBSD_stable_10 X-Jenkins-Result: UNSTABLE X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 10:47:20 -0000 https://jenkins.FreeBSD.org/job/FreeBSD_stable_10/437/ From owner-freebsd-stable@freebsd.org Fri Oct 21 11:35:07 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D1D1C1B9F8 for ; Fri, 21 Oct 2016 11:35:07 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id A6364A72 for ; Fri, 21 Oct 2016 11:35:06 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id OAA08899; Fri, 21 Oct 2016 14:34:58 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1bxY6E-0004p7-OW; Fri, 21 Oct 2016 14:34:58 +0300 Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: "Eugene M. Zheganin" , freebsd-stable@FreeBSD.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> From: Andriy Gapon Message-ID: <87197469-f69e-04ae-5cd5-30d0499bb5c7@FreeBSD.org> Date: Fri, 21 Oct 2016 14:34:03 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 11:35:07 -0000 Instead of the guesswork and black magic, you could try to use tools to analyze the problem. E.g., determine if the delay is because a CPU does a lot of work or it is because of waiting. Find the bottleneck, etc. pmcstat, dtrace are your friends :-) -- Andriy Gapon From owner-freebsd-stable@freebsd.org Fri Oct 21 11:51:34 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 47602C1B122 for ; Fri, 21 Oct 2016 11:51:34 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id B5A0C3A7 for ; Fri, 21 Oct 2016 11:51:33 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e] ([IPv6:2a02:2698:27:b4d9:28e5:c182:e05b:6a1e]) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id u9LBpTPY098847 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO) for ; Fri, 21 Oct 2016 16:51:30 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1477050690; bh=C27w+cY1oxEbOzuIxxQxcVFq9rqXAL4f9YE7mjYRbYQ=; h=Subject:To:References:From:Date:In-Reply-To; b=txOU6bkhXjurdBpfYYRtuWnZ8Hpox7H7SR4yYF29rue2TylgiOAjeMEu+r4A5gPW4 JwIQ8dbbt36L1QThuo8CfY8SHSFguiBFgqK8fWqbjrevEdZefp243ydUyZkYrHuGQI D8SY6Vuc9lHhOz48Xb+fNu+RrQGRI48WhlakOCcQ= Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> <38a84fce-fd97-a6a1-5820-89e578d76f0b@multiplay.co.uk> <91f34889-7f4f-661d-c88f-8034402c39cd@norma.perm.ru> <19a45e94-65e6-53a9-202d-c048055460d1@multiplay.co.uk> <20161021102051.GH57876@zxy.spb.ru> From: "Eugene M. Zheganin" Message-ID: <499a3693-d8ea-8b47-138c-77876077967c@norma.perm.ru> Date: Fri, 21 Oct 2016 16:51:36 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: <20161021102051.GH57876@zxy.spb.ru> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 11:51:34 -0000 Hi. On 21.10.2016 15:20, Slawa Olhovchenkov wrote: > > ZFS prefetch affect performance dpeneds of workload (independed of RAM > size): for some workloads wins, for some workloads lose (for my > workload prefetch is lose and manualy disabled with 128GB RAM). > > Anyway, this system have only 24MB in ARC by 2.3GB free, this is may > be too low for this workload. You mean - "for getting a list of a directory with 20 subdirectories" ? Why then does only this directory have this issue with pause, not /usr/ports/..., which has more directories in it ? (and yes, /usr/ports/www isn't empty and holds 2410 entities) /usr/bin/time -h ls -1 /usr/ports/www [...] 0.14s real 0.00s user 0.00s sys Thanks. Eugene. From owner-freebsd-stable@freebsd.org Fri Oct 21 12:02:06 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE51FC1BBEB for ; Fri, 21 Oct 2016 12:02:06 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7EAE3EB9 for ; Fri, 21 Oct 2016 12:02:06 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate2.intern.punkt.de with ESMTP id u9LBdZlO036282 for ; Fri, 21 Oct 2016 13:39:35 +0200 (CEST) Received: from [217.29.44.206] ([217.29.44.206]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id u9LBdYUn044121 for ; Fri, 21 Oct 2016 13:39:34 +0200 (CEST) (envelope-from hausen@punkt.de) From: "Patrick M. Hausen" Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: boot0cfg on does not set default selection on gmirror device Message-Id: <14FD5FE6-6277-4EBE-8EE9-630A735F8BEA@punkt.de> Date: Fri, 21 Oct 2016 13:39:57 +0200 To: freebsd-stable Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 12:02:07 -0000 Hi, all, we are repeatedly bitten by the following misbehaviour of boot0cfg: root@hd45:/usr/local # boot0cfg -s 1 mirror/m0 root@hd45:/usr/local # boot0cfg -v mirror/m0 # flag start chs type end chs offset size 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 version=3D1.0 drive=3D0x80 mask=3D0xf ticks=3D182 bell=3D (0x7) options=3Dpacket,update,nosetdrv default_selection=3DF2 (Slice 2) So, while it should have set the default to slice 1, it simply didn't. gpart on the other hand works as expected: root@hd45:/usr/local # gpart set -a active -i 1 mirror/m0 active set on mirror/m0s1 root@hd45:/usr/local # gpart show mirror/m0 =3D> 63 1953525104 mirror/m0 MBR (932G) 63 16002 - free - (7.8M) 16065 16418430 1 freebsd [active] (7.8G) 16434495 16418430 2 freebsd (7.8G) 32852925 1920667140 3 freebsd (916G) 1953520065 5102 - free - (2.5M) But the "active" flag alone is not enough to convince boot0 to actually = boot that partition. Additional info: root@hd45:/usr/local # uname -a FreeBSD hd45.hosting.punkt.de 10.3-RELEASE-p10 FreeBSD 10.3-RELEASE-p10 = #0 r306942: Mon Oct 10 10:29:14 UTC 2016 = root@:/usr/obj/nanobsd.hosting/usr/src/sys/GENERIC amd64 root@hd45:/usr/local # gmirror status Name Status Components mirror/m0 COMPLETE ada0 (ACTIVE) ada1 (ACTIVE) The only way to actually switch the boot0 default selection is: root@hd45:/usr/local # sysctl kern.geom.debugflags=3D16 kern.geom.debugflags: 0 -> 16 root@hd45:/usr/local # boot0cfg -s 1 ada0 root@hd45:/usr/local # boot0cfg -s 1 ada1 root@hd45:/usr/local # boot0cfg -v mirror/m0 # flag start chs type end chs offset size 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 version=3D1.0 drive=3D0x80 mask=3D0xf ticks=3D182 bell=3D (0x7) options=3Dpacket,update,nosetdrv default_selection=3DF1 (Slice 1) Any hints what's going on, here? Obviously it is possible to manipulate the MBR of a gmirror device - as gpart proves. The boot0cfg pops up since FreeBSD 8 when we started using a mirrored NanoBSD setup. Thanks and kind regards, Patrick --=20 punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe Tel. 0721 9109 0 * Fax 0721 9109 100 info@punkt.de http://www.punkt.de Gf: J=C3=BCrgen Egeling AG Mannheim 108285 From owner-freebsd-stable@freebsd.org Fri Oct 21 12:05:39 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76BECC1BD43 for ; Fri, 21 Oct 2016 12:05:39 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 36603D1 for ; Fri, 21 Oct 2016 12:05:39 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bxYZt-000CIf-04; Fri, 21 Oct 2016 15:05:37 +0300 Date: Fri, 21 Oct 2016 15:05:36 +0300 From: Slawa Olhovchenkov To: "Eugene M. Zheganin" Cc: freebsd-stable@freebsd.org Subject: Re: zfs, a directory that used to hold lot of files and listing pause Message-ID: <20161021120536.GI57876@zxy.spb.ru> References: <4d9269af-ed64-bb73-eb7f-98a3f5ffd5a2@norma.perm.ru> <40fa9fd6-15aa-d8f7-b958-8783e763e6bc@multiplay.co.uk> <577ab7b2-46c1-5cf0-b6ad-50895978d957@norma.perm.ru> <38a84fce-fd97-a6a1-5820-89e578d76f0b@multiplay.co.uk> <91f34889-7f4f-661d-c88f-8034402c39cd@norma.perm.ru> <19a45e94-65e6-53a9-202d-c048055460d1@multiplay.co.uk> <20161021102051.GH57876@zxy.spb.ru> <499a3693-d8ea-8b47-138c-77876077967c@norma.perm.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <499a3693-d8ea-8b47-138c-77876077967c@norma.perm.ru> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 12:05:39 -0000 On Fri, Oct 21, 2016 at 04:51:36PM +0500, Eugene M. Zheganin wrote: > Hi. > > On 21.10.2016 15:20, Slawa Olhovchenkov wrote: > > > > ZFS prefetch affect performance dpeneds of workload (independed of RAM > > size): for some workloads wins, for some workloads lose (for my > > workload prefetch is lose and manualy disabled with 128GB RAM). > > > > Anyway, this system have only 24MB in ARC by 2.3GB free, this is may > > be too low for this workload. > You mean - "for getting a list of a directory with 20 subdirectories" ? > Why then does only this directory have this issue with pause, not > /usr/ports/..., which has more directories in it ? > > (and yes, /usr/ports/www isn't empty and holds 2410 entities) > > /usr/bin/time -h ls -1 /usr/ports/www > [...] > 0.14s real 0.00s user 0.00s sys You wrote: "(tens of thousands) files". In bad case metadata of every file will be placed in random place of disk. ls need access to metadata of every file before start of output listing. I.e. in bad case you will be need tens of thousands seeks over disk capable only 72 seeks per seconds. Perhaps /usr/ports/www created at once and metadata of all entries placed near each other, need less seeks. If zfs property primarycache/secondarycache not off. From owner-freebsd-stable@freebsd.org Fri Oct 21 12:25:42 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 89818C1A3FB for ; Fri, 21 Oct 2016 12:25:42 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from mail.soaustin.net (mail.soaustin.net [192.108.105.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.soaustin.net", Issuer "StartCom Class 2 IV Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6F298D09 for ; Fri, 21 Oct 2016 12:25:42 +0000 (UTC) (envelope-from linimon@lonesome.com) Received: from lonesome.com (bones.soaustin.net [192.108.105.22]) by mail.soaustin.net (Postfix) with ESMTPSA id 96DF9672; Fri, 21 Oct 2016 07:25:34 -0500 (CDT) Date: Fri, 21 Oct 2016 07:25:33 -0500 From: Mark Linimon To: Andrea Venturoli Cc: "freebsd-stable@freebsd.org" Subject: Re: Nightly disk-related panic since upgrade to 10.3 Message-ID: <20161021122533.GB2446@lonesome.com> References: <76d65036-0f4b-28fc-d1ef-f6527a9299a1@netfence.it> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <76d65036-0f4b-28fc-d1ef-f6527a9299a1@netfence.it> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 12:25:42 -0000 On Fri, Oct 21, 2016 at 10:14:26AM +0200, Andrea Venturoli wrote: > I've tried this way, but altough I'm quite proficient with [k]gdb I tend to > get lost in FreeBSD's kernel's source code, which, unfortunately, I'm not > familiar with. > > BTW, I had read that book years ago; I searched for it now, but a 2005 > edition still comes up. Has it ever been updated? My usual go-to documentation John Baldwin's paper: http://www.bsdcan.org/2008/schedule/attachments/45_article.pdf mcl From owner-freebsd-stable@freebsd.org Fri Oct 21 12:47:12 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EFD94C1AAA1 for ; Fri, 21 Oct 2016 12:47:12 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (ingresso-1-pt.tunnel.tserv1.lon2.ipv6.he.net [IPv6:2001:470:1f1c:411::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C087AABF for ; Fri, 21 Oct 2016 12:47:12 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6] helo=dilbert.ingresso.co.uk) by constantine.ingresso.co.uk with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bxZE4-000A3c-91; Fri, 21 Oct 2016 12:47:08 +0000 Received: from petefrench by dilbert.ingresso.co.uk with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bxZE4-0003ja-7Y; Fri, 21 Oct 2016 13:47:08 +0100 To: emz@norma.perm.ru, slw@zxy.spb.ru Subject: Re: zfs, a directory that used to hold lot of files and listing pause Cc: freebsd-stable@freebsd.org In-Reply-To: <20161021120536.GI57876@zxy.spb.ru> Message-Id: From: Pete French Date: Fri, 21 Oct 2016 13:47:08 +0100 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 12:47:13 -0000 > In bad case metadata of every file will be placed in random place of disk. > ls need access to metadata of every file before start of output listing. Umm, are we not talkong abut an issue where the directoyr no longer contains any files. It used to have lots, now it has none. > I.e. in bad case you will be need tens of thousands seeks over disk > capable only 72 seeks per seconds. Why does it need to seek all over the disc if there are no files (and hence no metadata surely) ? I am not bothered if a hufge directoyr takes a while to list, thats something I am happy to deal with. What I dont like is when it is back down to zero that it still takes a long time to list. That doesnt make much sense. -pete. From owner-freebsd-stable@freebsd.org Fri Oct 21 13:08:04 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DFA83C1AF8E for ; Fri, 21 Oct 2016 13:08:04 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A30B98DA for ; Fri, 21 Oct 2016 13:08:04 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bxZYE-000EG8-KA; Fri, 21 Oct 2016 16:07:58 +0300 Date: Fri, 21 Oct 2016 16:07:58 +0300 From: Slawa Olhovchenkov To: Pete French Cc: emz@norma.perm.ru, freebsd-stable@freebsd.org Subject: Re: zfs, a directory that used to hold lot of files and listing pause Message-ID: <20161021130758.GJ57876@zxy.spb.ru> References: <20161021120536.GI57876@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 13:08:05 -0000 On Fri, Oct 21, 2016 at 01:47:08PM +0100, Pete French wrote: > > In bad case metadata of every file will be placed in random place of disk. > > ls need access to metadata of every file before start of output listing. > > Umm, are we not talkong abut an issue where the directoyr no longer contains > any files. It used to have lots, now it has none. > > > I.e. in bad case you will be need tens of thousands seeks over disk > > capable only 72 seeks per seconds. > > Why does it need to seek all over the disc if there are no files (and hence > no metadata surely) ? > > I am not bothered if a hufge directoyr takes a while to list, > thats something I am happy to deal with. What I dont like is > when it is back down to zero that it still takes a long time > to list. That doesnt make much sense. OK, this case may be differ. May be zdb can help. ls -li /parent/dir Take inode number zdb -vvvv zfs_set inode_number also do ktrace ls and anaylyse `kdump -E` From owner-freebsd-stable@freebsd.org Fri Oct 21 13:41:16 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34C29C1BBE3 for ; Fri, 21 Oct 2016 13:41:16 +0000 (UTC) (envelope-from bennett@sdf.org) Received: from sdf.lonestar.org (mx.sdf.org [192.94.73.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "ol.sdf.org", Issuer "ol.sdf.org" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id E08CB6C6 for ; Fri, 21 Oct 2016 13:41:14 +0000 (UTC) (envelope-from bennett@sdf.org) Received: from sdf.org (norge.freeshell.org [192.94.73.17]) by sdf.lonestar.org (8.15.2/8.14.5) with ESMTPS id u9LDesrG010930 (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256 bits) verified NO); Fri, 21 Oct 2016 13:40:54 GMT Received: (from bennett@localhost) by sdf.org (8.15.2/8.12.8/Submit) id u9LDer6D018453; Fri, 21 Oct 2016 08:40:53 -0500 (CDT) From: Scott Bennett Message-Id: <201610211340.u9LDer6D018453@sdf.org> Date: Fri, 21 Oct 2016 08:40:52 -0500 To: freebsd-stable@freebsd.org Subject: Re: zfs, a directory that used to hold lot of files and listing pause Cc: "Eugene M. Zheganin" User-Agent: Heirloom mailx 12.5 6/20/10 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 13:41:16 -0000 On Fri, 21 Oct 2016 16:51:36 +0500 "Eugene M. Zheganin" wrote: >On 21.10.2016 15:20, Slawa Olhovchenkov wrote: >> >> ZFS prefetch affect performance dpeneds of workload (independed of RAM >> size): for some workloads wins, for some workloads lose (for my >> workload prefetch is lose and manualy disabled with 128GB RAM). >> >> Anyway, this system have only 24MB in ARC by 2.3GB free, this is may >> be too low for this workload. >You mean - "for getting a list of a directory with 20 subdirectories" ? >Why then does only this directory have this issue with pause, not >/usr/ports/..., which has more directories in it ? > >(and yes, /usr/ports/www isn't empty and holds 2410 entities) > >/usr/bin/time -h ls -1 /usr/ports/www >[...] >0.14s real 0.00s user 0.00s sys > Oh, my goodness, how far afield nonsense has gotten! Have all the good folks posting in this thread forgotten how directory blocks are allocated in UNIX? This isn't even a BSD-specific thing; it's really ancient. What Eugene has complained of is exactly what is to be expected-- on really old hardware. The only eyebrow-raiser is that he has created a use case so extreme that a live human can actually notice the delays on modern hardware. I quote from his original posting: "I also have one directory that used to have a lot of (tens of thousands) files." and "But now I have 2 files and a couple of dozens directories in it". A directory with tens of thousands of files in it at one point in time most likely has somewhere well over one thousand blocks allocated. Directories don't shrink. Directory entries do not get moved around within directories when files are added or deleted. Directories can remain the same length or they can grow in length. If a directory once had many tens of thousands of filenames and links to their primary inodes, then the directory is still that big, even if it now only contains two [+ 20 to 30 directory], probably widely separated, entries. To read a file's entry, all blocks must be searched until the desired filename is found. Likewise, to list the contents of a directory, all blocks must be read until the number of files found matches the link count for the directory. IOW, if you want the performance to go back to what it was when the directory was fresh (and still small), you have to create a new directory and then move the remaining entries from the old directory into the new (small) directory. The only real difference here between UFS (or even the early AT&T filesystem) and ZFS is that the two remaining entries in a formerly huge directory are likely to be in different directory blocks that could be at effectively random locations scattered around the space of a partition for one filesystem in UFS or over an entire pool of potentially many filesystems and much more space in ZFS. Scott Bennett, Comm. ASMELG, CFIAG ********************************************************************** * Internet: bennett at sdf.org *xor* bennett at freeshell.org * *--------------------------------------------------------------------* * "A well regulated and disciplined militia, is at all times a good * * objection to the introduction of that bane of all free governments * * -- a standing army." * * -- Gov. John Hancock, New York Journal, 28 January 1790 * ********************************************************************** From owner-freebsd-stable@freebsd.org Fri Oct 21 14:04:49 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB2A7C1B2FA for ; Fri, 21 Oct 2016 14:04:49 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from constantine.ingresso.co.uk (ingresso-1-pt.tunnel.tserv1.lon2.ipv6.he.net [IPv6:2001:470:1f1c:411::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8C7AB637 for ; Fri, 21 Oct 2016 14:04:49 +0000 (UTC) (envelope-from petefrench@ingresso.co.uk) Received: from dilbert.london-internal.ingresso.co.uk ([10.64.50.6] helo=dilbert.ingresso.co.uk) by constantine.ingresso.co.uk with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bxaRA-000BCX-BM; Fri, 21 Oct 2016 14:04:44 +0000 Received: from petefrench by dilbert.ingresso.co.uk with local (Exim 4.87 (FreeBSD)) (envelope-from ) id 1bxaRA-0003uq-9q; Fri, 21 Oct 2016 15:04:44 +0100 To: bennett@sdf.org, freebsd-stable@freebsd.org Subject: Re: zfs, a directory that used to hold lot of files and listing pause Cc: emz@norma.perm.ru In-Reply-To: <201610211340.u9LDer6D018453@sdf.org> Message-Id: From: Pete French Date: Fri, 21 Oct 2016 15:04:44 +0100 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 14:04:49 -0000 > Oh, my goodness, how far afield nonsense has gotten! Have all the > good folks posting in this thread forgotten how directory blocks are > allocated in UNIX? Not forgotten, just under the impression that ZFS shrinks directories unlike good old UFS. Apparenrly not, and yes, if thats true then the behaviour is not surprising in the slightest. Live and learn... ;-) -pete. [old enough to have used 32V on a Vax, a loooong time ago...] From owner-freebsd-stable@freebsd.org Fri Oct 21 14:23:12 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3288C1B8C9 for ; Fri, 21 Oct 2016 14:23:12 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp.infracaninophile.co.uk [IPv6:2001:8b0:151:1:c4ea:bd49:619b:6cb3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.infracaninophile.co.uk", Issuer "infracaninophile.co.uk" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4689EF16 for ; Fri, 21 Oct 2016 14:23:12 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from zero-gravitas.local (unknown [85.199.232.226]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: m.seaman@infracaninophile.co.uk) by smtp.infracaninophile.co.uk (Postfix) with ESMTPSA id 5AA0BEF9B for ; Fri, 21 Oct 2016 14:23:07 +0000 (UTC) Authentication-Results: smtp.infracaninophile.co.uk; dmarc=none header.from=FreeBSD.org Authentication-Results: smtp.infracaninophile.co.uk/5AA0BEF9B; dkim=none; dkim-atps=neutral Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: freebsd-stable@freebsd.org References: From: Matthew Seaman Message-ID: <944c8d53-7ec9-c825-03ab-daf947ef8d8e@FreeBSD.org> Date: Fri, 21 Oct 2016 15:23:00 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="1k7eXcaHHQgGSMuASwShd5ht61m3ojDjj" X-Spam-Status: No, score=-0.4 required=5.0 tests=BAYES_00,RDNS_NONE, SPF_SOFTFAIL autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on smtp.infracaninophile.co.uk X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 14:23:12 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --1k7eXcaHHQgGSMuASwShd5ht61m3ojDjj Content-Type: multipart/mixed; boundary="vNT5fHR2R6t5k0S1i1iPOSXAMhw7ebq3T"; protected-headers="v1" From: Matthew Seaman To: freebsd-stable@freebsd.org Message-ID: <944c8d53-7ec9-c825-03ab-daf947ef8d8e@FreeBSD.org> Subject: Re: zfs, a directory that used to hold lot of files and listing pause References: In-Reply-To: --vNT5fHR2R6t5k0S1i1iPOSXAMhw7ebq3T Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 2016/10/21 13:47, Pete French wrote: >> In bad case metadata of every file will be placed in random place of d= isk. >> ls need access to metadata of every file before start of output listin= g. >=20 > Umm, are we not talkong abut an issue where the directoyr no longer con= tains > any files. It used to have lots, now it has none. >=20 >> I.e. in bad case you will be need tens of thousands seeks over disk >> capable only 72 seeks per seconds. >=20 > Why does it need to seek all over the disc if there are no files (and h= ence > no metadata surely) ? >=20 > I am not bothered if a hufge directoyr takes a while to list, > thats something I am happy to deal with. What I dont like is > when it is back down to zero that it still takes a long time > to list. That doesnt make much sense. Interesting. Is this somehow related to the old Unixy thing with directories, where the directory node would grow in size as you created more and more files or sub-directories (as you might expect), but it wouldn't shrink immediately if you simply deleted many files -- it would only shrink later when you next created a new file in that directory. This was a performance feature IIRC -- it avoided shrinking and re-growing directory nodes in quick succession for what was apparently a fairly common usage pattern of clearing out a directory and then refilling it. Can't see how that would apply to ZFS though, as the CoW nature means there should be no benefit to not immediately adjusting the size of the directory node to fit the amount of contents. Cheers, Matthew --vNT5fHR2R6t5k0S1i1iPOSXAMhw7ebq3T-- --1k7eXcaHHQgGSMuASwShd5ht61m3ojDjj Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQJ8BAEBCgBmBQJYCiTLXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQxOUYxNTRFQ0JGMTEyRTUwNTQ0RTNGMzAw MDUxM0YxMEUwQTlFNEU3AAoJEABRPxDgqeTni2IP/3Zdgpe492bmoD1Kv3SneXEH PG5SBXSWae610nTRWMglM299xkvmqLFZZYaEEu+0aEsPuqsvFxYnuXKZYojlwqQ0 coKH2PrHIUVEMkJtTvfpS2/kZc6mWClZerby/VTrPNh5Axdav82zL0ZizaYnNOyL ID6ZYb3wewu77q31R6dUTAkdZTD/vu4rV/LCcT5yxli4874X5qLadcjXry9jiBBa ecCC8ahrzymkPJywR9J9khKMLZpWtKrqXwVTIfRYAw+oCfHE78Uw/QN8M4uMuflo u6fEQHhTgGrLPohznmtWWLNM1yfjum8jebnPpzrtLqkItE8hGkqpq/xUgUQB6Mj5 lqMi8iBx1X0FPxMy19N3VUxrDTI3BXjhflrFU7EEPvducprhCVgCgriit9lG/fXT irhMWPexwO1mFMl2rDy82NBJTFQPRefusPidnTc4tTdR/e2oF/42RuzrNYocs9cf yHJQJn1Y4omFg35sWCuZ7cTc6j66rRolVa6ezVjyUWelQ9+lRURGx6pRaqc/BdsM FgJr075Na3E/tyAWFrdFoym3TZYEbWB89QjZVj4pE7EVVip1etDCwXNxvW7Em+El Sr3gJ1Ya1Z2VMui9YtgOjt7VaEkdji53UNCMu1ja8OiCIeAfukz8eTMA9VOS+Ty7 JBiHjsXM1SikdTpyorjA =9mah -----END PGP SIGNATURE----- --1k7eXcaHHQgGSMuASwShd5ht61m3ojDjj-- From owner-freebsd-stable@freebsd.org Fri Oct 21 14:41:37 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7B098C1A01E for ; Fri, 21 Oct 2016 14:41:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x22f.google.com (mail-it0-x22f.google.com [IPv6:2607:f8b0:4001:c0b::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4975B171 for ; Fri, 21 Oct 2016 14:41:37 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x22f.google.com with SMTP id m138so168671324itm.0 for ; Fri, 21 Oct 2016 07:41:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=VNH46PtAv0LmXxquMTXcBCFhB2McOuFlvYm4TtS042g=; b=tZIiRl96T3dbk2WcO2JRkRZnbTYAbvSIyIVguLH9mZwQFckjL7lBQcIw1BrxI0eHfq BsYiF3mRrRn5Yr5a3T0nPiX3q9zCCLsinSVvSV3Ec5+TYaOPGLsUqo4U+HTEOLCgkNIT avmxfUtP1R60YS1sFBa9R3pfD+WIHn5Df72/avQ3xolqCeWw4d9bL23TQXiLibSaeGgR NJY2XpPxkpGl5KMYdArVmrMuUAC0FLIY8XvS8gXg94sRpkPmJsXfzUNZB1Ltcs6BxIzO jf2QI+WW7DQ16kNk2cDpstIQaam12QMOmfCM0deKcsr/t/ibmLlFnJKX8Jn5Uu6yED1i h8qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=VNH46PtAv0LmXxquMTXcBCFhB2McOuFlvYm4TtS042g=; b=Vm5etB4mnXAgkmhPkT405FhwC6xRECCfk2HNISzalcnY0Zi4qt40enM9HccMER9Oxq 1Xq/cUIWCQPsiVoW7r6JrAPqrBAq7O0uMp4ZJC2spBa/lMZgOPXq27/naUlIBPm4pUVM 7z5BaXWYwN+F+pE8JE+DEVYWPQnWwf9fla8JEH/OfggZqRaXxOubYySFwRra3eTgRfRu F3x+4L/YIUClHt5Xjp/AP/hBosQpwViLAAI3N+gjh++wEpdA6gQ9a02ipdRA6XM2cbng 2oWm7e8Spk7XHk5rJCOFK8AtuZL/68SOda23rc6L/0t28hxwPRDemycGRBAGEbu1zcZq /sUg== X-Gm-Message-State: ABUngvcvmnsBBLdPDKhR9/YsYX5k4NSVR2MFv4d6nV92ecmSAz6ulKpWY7sW4IPfbcvGY02BVkyg3ZCxM/3TmA== X-Received: by 10.36.53.214 with SMTP id k205mr1240342ita.85.1477060894231; Fri, 21 Oct 2016 07:41:34 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.77.85 with HTTP; Fri, 21 Oct 2016 07:41:33 -0700 (PDT) X-Originating-IP: [69.53.245.200] In-Reply-To: <14FD5FE6-6277-4EBE-8EE9-630A735F8BEA@punkt.de> References: <14FD5FE6-6277-4EBE-8EE9-630A735F8BEA@punkt.de> From: Warner Losh Date: Fri, 21 Oct 2016 08:41:33 -0600 X-Google-Sender-Auth: OTWgT5Z9-OZNMFF2SwnH8KYBAKA Message-ID: Subject: Re: boot0cfg on does not set default selection on gmirror device To: "Patrick M. Hausen" Cc: freebsd-stable Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 14:41:37 -0000 On Fri, Oct 21, 2016 at 5:39 AM, Patrick M. Hausen wrote: > Hi, all, > > we are repeatedly bitten by the following misbehaviour of boot0cfg: > > root@hd45:/usr/local # boot0cfg -s 1 mirror/m0 > root@hd45:/usr/local # boot0cfg -v mirror/m0 > # flag start chs type end chs offset size > 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 > 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 > 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 > > version=1.0 drive=0x80 mask=0xf ticks=182 bell= (0x7) > options=packet,update,nosetdrv > default_selection=F2 (Slice 2) > > So, while it should have set the default to slice 1, it simply didn't. > > gpart on the other hand works as expected: > > root@hd45:/usr/local # gpart set -a active -i 1 mirror/m0 > active set on mirror/m0s1 > root@hd45:/usr/local # gpart show mirror/m0 > => 63 1953525104 mirror/m0 MBR (932G) > 63 16002 - free - (7.8M) > 16065 16418430 1 freebsd [active] (7.8G) > 16434495 16418430 2 freebsd (7.8G) > 32852925 1920667140 3 freebsd (916G) > 1953520065 5102 - free - (2.5M) > > But the "active" flag alone is not enough to convince boot0 to actually boot that partition. > > Additional info: > > root@hd45:/usr/local # uname -a > FreeBSD hd45.hosting.punkt.de 10.3-RELEASE-p10 FreeBSD 10.3-RELEASE-p10 #0 r306942: Mon Oct 10 10:29:14 UTC 2016 root@:/usr/obj/nanobsd.hosting/usr/src/sys/GENERIC amd64 > root@hd45:/usr/local # gmirror status > Name Status Components > mirror/m0 COMPLETE ada0 (ACTIVE) > ada1 (ACTIVE) > > > The only way to actually switch the boot0 default selection is: > > root@hd45:/usr/local # sysctl kern.geom.debugflags=16 > kern.geom.debugflags: 0 -> 16 > root@hd45:/usr/local # boot0cfg -s 1 ada0 > root@hd45:/usr/local # boot0cfg -s 1 ada1 > root@hd45:/usr/local # boot0cfg -v mirror/m0 > # flag start chs type end chs offset size > 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 > 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 > 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 > > version=1.0 drive=0x80 mask=0xf ticks=182 bell= (0x7) > options=packet,update,nosetdrv > default_selection=F1 (Slice 1) > > > Any hints what's going on, here? Obviously it is possible to manipulate > the MBR of a gmirror device - as gpart proves. The boot0cfg pops up > since FreeBSD 8 when we started using a mirrored NanoBSD setup. Any chance you can migrate to using gpart? Is boot0cfg still referenced in NanoBSD somewhere? Warner From owner-freebsd-stable@freebsd.org Fri Oct 21 14:43:51 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9AB5EC1A2E2 for ; Fri, 21 Oct 2016 14:43:51 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0B1717E3 for ; Fri, 21 Oct 2016 14:43:50 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id u9LEhklw048073; Sat, 22 Oct 2016 01:43:46 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sat, 22 Oct 2016 01:43:46 +1100 (EST) From: Ian Smith To: "Patrick M. Hausen" cc: freebsd-stable Subject: Re: boot0cfg on does not set default selection on gmirror device In-Reply-To: <20161022005837.G6806@sola.nimnet.asn.au> Message-ID: <20161022010007.U6806@sola.nimnet.asn.au> References: <14FD5FE6-6277-4EBE-8EE9-630A735F8BEA@punkt.de> <20161022005837.G6806@sola.nimnet.asn.au> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 14:43:51 -0000 On Fri, 21 Oct 2016 13:39:57 +0200, Patrick M. Hausen wrote: > Hi, all, > > we are repeatedly bitten by the following misbehaviour of boot0cfg: > > root@hd45:/usr/local # boot0cfg -s 1 mirror/m0 > root@hd45:/usr/local # boot0cfg -v mirror/m0 > # flag start chs type end chs offset size > 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 > 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 > 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 > > version=1.0 drive=0x80 mask=0xf ticks=182 bell= (0x7) > options=packet,update,nosetdrv > default_selection=F2 (Slice 2) > > So, while it should have set the default to slice 1, it simply didn't. boot0cfg isn't mirror-aware as such and thinks it's using BIOS services to write to a specific drive, and likely did write to one of the disks, but it seems gmirror isn't updating both disks' MBRs - which might not be too surprising. Does it work 'sometimes'? > gpart on the other hand works as expected: > > root@hd45:/usr/local # gpart set -a active -i 1 mirror/m0 > active set on mirror/m0s1 > root@hd45:/usr/local # gpart show mirror/m0 > => 63 1953525104 mirror/m0 MBR (932G) > 63 16002 - free - (7.8M) > 16065 16418430 1 freebsd [active] (7.8G) > 16434495 16418430 2 freebsd (7.8G) > 32852925 1920667140 3 freebsd (916G) > 1953520065 5102 - free - (2.5M) > > But the "active" flag alone is not enough to convince boot0 to > actually boot that partition. boot0cfg stashes -s selected nextboot slice (-1) at 0x1b5, using that to choose the default boot slice - if nothing else was selected. It doesn't set the active flag (0x80) until just before booting via BIOS, when it also updates the selection if another slice, or disk, were chosen. That is, boot0 doesn't care which active flag was set before setting it, and gpart doesn't know about non-MBR bytes in the boot sector, so can't influence boot0's behaviour. > Additional info: > > root@hd45:/usr/local # uname -a > FreeBSD hd45.hosting.punkt.de 10.3-RELEASE-p10 FreeBSD 10.3-RELEASE-p10 #0 r306942: Mon Oct 10 10:29:14 UTC 2016 root@:/usr/obj/nanobsd.hosting/usr/src/sys/GENERIC amd64 > root@hd45:/usr/local # gmirror status > Name Status Components > mirror/m0 COMPLETE ada0 (ACTIVE) > ada1 (ACTIVE) > > > The only way to actually switch the boot0 default selection is: > > root@hd45:/usr/local # sysctl kern.geom.debugflags=16 > kern.geom.debugflags: 0 -> 16 > root@hd45:/usr/local # boot0cfg -s 1 ada0 > root@hd45:/usr/local # boot0cfg -s 1 ada1 > root@hd45:/usr/local # boot0cfg -v mirror/m0 > # flag start chs type end chs offset size > 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 > 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 > 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 > > version=1.0 drive=0x80 mask=0xf ticks=182 bell= (0x7) > options=packet,update,nosetdrv > default_selection=F1 (Slice 1) > > > Any hints what's going on, here? Obviously it is possible to manipulate > the MBR of a gmirror device - as gpart proves. The boot0cfg pops up > since FreeBSD 8 when we started using a mirrored NanoBSD setup. You might need to script the above, ie setting -s on both disks, unless someone who actually knows something about gmirror has a better clue. cheers, Ian PS sorry for busted threading, my pine had trouble quoting your message. From owner-freebsd-stable@freebsd.org Fri Oct 21 14:57:45 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5585CC1A8E8 for ; Fri, 21 Oct 2016 14:57:45 +0000 (UTC) (envelope-from ardovm@yahoo.it) Received: from nm21-vm5.bullet.mail.ir2.yahoo.com (nm21-vm5.bullet.mail.ir2.yahoo.com [212.82.97.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E18BFDE for ; Fri, 21 Oct 2016 14:57:44 +0000 (UTC) (envelope-from ardovm@yahoo.it) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.it; s=s2048; t=1477061715; bh=st91+1VnHe7bzv6w4sBnPzybjgH1nspQFiuJhe9sbGs=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From:Subject; b=eFPsTyuQOadak1Zlf9QXIizGU9JW2LbHkrniFvmv57tgKIEUWOlndCeFscTexNapT9COFO3uMXgzmtRzVsAvYk0L6PQDS5/iNthV9FEGFHEEyd94LFsx0GcRREwBOnRkT6HxCRzzfsB0b7pqFkCF+02lfj/9ZiwhwpDJpI7D0QhqKh31ZTD5i7fnMhljf9mLLmLI7Plqsszrmbyv8Z/iKylUHy4LRjeOO+vZ/TvhLBcuU57xrl+6k1OnOFfPXwO2ekUsGnS7EYwgZR9U0CBlHlAL05fxM70ze5JEGbZFRnI+DZTDd0trSJjF+1tKSMRbhvigRJyzlvWftZDuXSJAIw== Received: from [212.82.98.52] by nm21.bullet.mail.ir2.yahoo.com with NNFMP; 21 Oct 2016 14:55:15 -0000 Received: from [46.228.39.110] by tm5.bullet.mail.ir2.yahoo.com with NNFMP; 21 Oct 2016 14:55:15 -0000 Received: from [127.0.0.1] by smtp147.mail.ir2.yahoo.com with NNFMP; 21 Oct 2016 14:55:15 -0000 X-Yahoo-Newman-Id: 595006.76256.bm@smtp147.mail.ir2.yahoo.com X-Yahoo-Newman-Property: ymail-3 X-YMail-OSG: F8M7osoVM1nZdkBSWxZxeP.R9GpfQIXoTLAq3JFiQ8O3sSW IzBCEbTNdtg5frQhTXhajdsYqRIa7Q9Dw_awHse5fcnmOUDEq5WYfT.mDQFd cq.FnmfqsvH9wzOcefZSe.mQwX2i82SCTA9VxzSMX2bRv0c5THg7VAQ3H.B9 Rx7ErYLQDYYQOxHZ5UjqMo0IFj3CMBIiM6XuTjs8VpHDKXM2Fm_ZEGLesyTl joJei_gIyAWMa_QKHWuEhbATj1E_ckRbrO1tUlaNBxhq4OR2lPEckMzKkrqa G.wSBVn03AvaUxMP9geD4WFfxkPSlkx3beO8LCHoYw6yb2uhOFmmR7mkMSZS LRgdssMa5KhGjNTud6ozVkoK8yncGKSro9iu.2A9rt3rUG0zqWmZxtxIiIuo FdA9p2Ky3HI5IvE0r.vtevF239SNao84s4Ius0Xo2oV7iCKtmPITDOmqdSDw PjXgXC5f0U3Y3wPYg7W1j51es0Q8LB8Sr.3W4SBXN74cxz5p2oEvkxJnbB7H aIAr0U2gFVJGKRWo4MhA47NafjmeofmYJN1JCPUwFDHWd9FeB X-Yahoo-SMTP: WU.IBxeswBAAnLcBZV3tEZIK0A-- Received: by nuvolo.localdomain (Postfix, from userid 1001) id 55C141AF109; Fri, 21 Oct 2016 16:55:14 +0200 (CEST) Date: Fri, 21 Oct 2016 16:55:14 +0200 From: Arrigo Marchiori To: Warner Losh Cc: "Patrick M. Hausen" , freebsd-stable Subject: Re: boot0cfg on does not set default selection on gmirror device Message-ID: <20161021145514.GC1450@nuvolo> References: <14FD5FE6-6277-4EBE-8EE9-630A735F8BEA@punkt.de> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.7.0 (2016-08-17) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 14:57:45 -0000 Hello all, On Fri, Oct 21, 2016 at 08:41:33AM -0600, Warner Losh wrote: > On Fri, Oct 21, 2016 at 5:39 AM, Patrick M. Hausen wrote: > > Hi, all, > > > > we are repeatedly bitten by the following misbehaviour of boot0cfg: > > > > root@hd45:/usr/local # boot0cfg -s 1 mirror/m0 > > root@hd45:/usr/local # boot0cfg -v mirror/m0 > > # flag start chs type end chs offset size > > 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 > > 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 > > 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 > > > > version=1.0 drive=0x80 mask=0xf ticks=182 bell= (0x7) > > options=packet,update,nosetdrv > > default_selection=F2 (Slice 2) > > > > So, while it should have set the default to slice 1, it simply didn't. > > > > gpart on the other hand works as expected: > > > > root@hd45:/usr/local # gpart set -a active -i 1 mirror/m0 > > active set on mirror/m0s1 > > root@hd45:/usr/local # gpart show mirror/m0 > > => 63 1953525104 mirror/m0 MBR (932G) > > 63 16002 - free - (7.8M) > > 16065 16418430 1 freebsd [active] (7.8G) > > 16434495 16418430 2 freebsd (7.8G) > > 32852925 1920667140 3 freebsd (916G) > > 1953520065 5102 - free - (2.5M) > > > > But the "active" flag alone is not enough to convince boot0 to actually boot that partition. > > > > Additional info: > > > > root@hd45:/usr/local # uname -a > > FreeBSD hd45.hosting.punkt.de 10.3-RELEASE-p10 FreeBSD 10.3-RELEASE-p10 #0 r306942: Mon Oct 10 10:29:14 UTC 2016 root@:/usr/obj/nanobsd.hosting/usr/src/sys/GENERIC amd64 > > root@hd45:/usr/local # gmirror status > > Name Status Components > > mirror/m0 COMPLETE ada0 (ACTIVE) > > ada1 (ACTIVE) > > > > > > The only way to actually switch the boot0 default selection is: > > > > root@hd45:/usr/local # sysctl kern.geom.debugflags=16 > > kern.geom.debugflags: 0 -> 16 > > root@hd45:/usr/local # boot0cfg -s 1 ada0 > > root@hd45:/usr/local # boot0cfg -s 1 ada1 > > root@hd45:/usr/local # boot0cfg -v mirror/m0 > > # flag start chs type end chs offset size > > 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 > > 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 > > 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 > > > > version=1.0 drive=0x80 mask=0xf ticks=182 bell= (0x7) > > options=packet,update,nosetdrv > > default_selection=F1 (Slice 1) > > > > > > Any hints what's going on, here? Obviously it is possible to manipulate > > the MBR of a gmirror device - as gpart proves. The boot0cfg pops up > > since FreeBSD 8 when we started using a mirrored NanoBSD setup. > > Any chance you can migrate to using gpart? Is boot0cfg still > referenced in NanoBSD somewhere? Ahem... For what it's worth... I cannot help not pointing this old PR out: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=186030 Best regards, -- rigo http://rigo.altervista.org From owner-freebsd-stable@freebsd.org Fri Oct 21 15:22:00 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 02656C1B299 for ; Fri, 21 Oct 2016 15:22:00 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B8CE31723 for ; Fri, 21 Oct 2016 15:21:59 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (localhost [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id 95E6528472; Fri, 21 Oct 2016 17:21:51 +0200 (CEST) Received: from illbsd.quip.test (ip-86-49-16-209.net.upcbroadband.cz [86.49.16.209]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id ACCD22846B; Fri, 21 Oct 2016 17:21:50 +0200 (CEST) Subject: Re: boot0cfg on does not set default selection on gmirror device To: Ian Smith , "Patrick M. Hausen" Cc: freebsd-stable References: <14FD5FE6-6277-4EBE-8EE9-630A735F8BEA@punkt.de> <20161022005837.G6806@sola.nimnet.asn.au> <20161022010007.U6806@sola.nimnet.asn.au> From: Miroslav Lachman <000.fbsd@quip.cz> Message-ID: <580A328E.4040504@quip.cz> Date: Fri, 21 Oct 2016 17:21:50 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:42.0) Gecko/20100101 Firefox/42.0 SeaMonkey/2.39 MIME-Version: 1.0 In-Reply-To: <20161022010007.U6806@sola.nimnet.asn.au> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 15:22:00 -0000 Ian Smith wrote on 2016/10/21 16:43: > On Fri, 21 Oct 2016 13:39:57 +0200, Patrick M. Hausen wrote: > > Hi, all, > > > > we are repeatedly bitten by the following misbehaviour of boot0cfg: > > > > root@hd45:/usr/local # boot0cfg -s 1 mirror/m0 > > root@hd45:/usr/local # boot0cfg -v mirror/m0 > > # flag start chs type end chs offset size > > 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 > > 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 > > 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 > > > > version=1.0 drive=0x80 mask=0xf ticks=182 bell= (0x7) > > options=packet,update,nosetdrv > > default_selection=F2 (Slice 2) > > > > So, while it should have set the default to slice 1, it simply didn't. > > boot0cfg isn't mirror-aware as such and thinks it's using BIOS services > to write to a specific drive, and likely did write to one of the disks, > but it seems gmirror isn't updating both disks' MBRs - which might not > be too surprising. Does it work 'sometimes'? We are using gmirror for whole drives mirroring from the time when it was introduced. It was always working with MRB/BSD. gmirror label gm0 ada0 ada1 And then you can use fdisk + bsdlabel or gpart to create slices and partitions and set it bootable on /dev/mirror/gm0. I didn't tried it with FreeBSD 10.3, but it works with 8.x (we skipped 9.x and all 8.x boxes were upgraded to 10.2 then 10.3) Miroslav Lachman From owner-freebsd-stable@freebsd.org Fri Oct 21 17:44:29 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5FC51C1BD6D for ; Fri, 21 Oct 2016 17:44:29 +0000 (UTC) (envelope-from allbery.b@gmail.com) Received: from mail-qt0-x22f.google.com (mail-qt0-x22f.google.com [IPv6:2607:f8b0:400d:c0d::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1A1BCDAB for ; Fri, 21 Oct 2016 17:44:29 +0000 (UTC) (envelope-from allbery.b@gmail.com) Received: by mail-qt0-x22f.google.com with SMTP id q7so93795058qtq.1 for ; Fri, 21 Oct 2016 10:44:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=dXj6pRtw6VQcE4Rlj6lEPHPMurp1VerGJgGtySjDkOA=; b=ynQLcBje+IEKKz/YDU9BqhBf9Sb6WDQbs+IGw5HA/Du0AABhcyKe++K5mesl+TKR3h 1lsU2xfq4xLJHDUF+Uvz8qtgZF8VVcMKswAjYoc9dJi3dWdWPVQtjZ37TWc3jNhi0vOk +dcKirBhJCl3bRE1HXY9zr8OaytDhfrkhT0mHWzeK4VDbM24YRRsDwuMO4T0S3c16vOv b8B95tDstfGG/3qINvx0fFMOzjH6dYxLgAI3O760uah+enD0h3tGviWjhL6i30SNWncP W1pxtNqxJkMXul2e8h8a3aeyxEyq+JEmPOFjya74QVjgyyMl1TLlpjUui8AaCZ5fibwq mTqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=dXj6pRtw6VQcE4Rlj6lEPHPMurp1VerGJgGtySjDkOA=; b=drlsCf7/nN/vC3WWLkO0H8yEc7j+vM5RHs72B5epezFxWODsYdE3U4/47PXF19/z3V vovrW9pkOy6hJ7GTYlbxEiiP6fBWrRxDD1cRc+8ExU1zIC8pCuc0KmA1U0PmAHdxKixC CEYlc3C+yuYPXlvB3hpNsepMqnecTGsWzEplr0T/RlyiSaC3NhclDTx3pgcGeKcXcADj 0jgn6+xgO0ng7nFrzyGtcX17prHe87dm8ZwGhG0P753xPPi3b7R9RI1+YV//UtCz8wys itVQsVuCZTIoV3Y1WtKTyYXJE3qtwbzixSP87BumoNzn2u6/1YqnLCU39ToOYMEu+jyJ dokQ== X-Gm-Message-State: ABUngvdu4gLju/FracAWZCFv6q0Ltyyh/B10ac8rfOHJg61cxhCXwQcWN2XtLtuf5Gu56HiQuYjM0pl50d+YEA== X-Received: by 10.237.39.71 with SMTP id n65mr2501830qtd.9.1477071868312; Fri, 21 Oct 2016 10:44:28 -0700 (PDT) MIME-Version: 1.0 Received: by 10.233.216.132 with HTTP; Fri, 21 Oct 2016 10:44:27 -0700 (PDT) In-Reply-To: References: <201610211340.u9LDer6D018453@sdf.org> From: Brandon Allbery Date: Fri, 21 Oct 2016 13:44:27 -0400 Message-ID: Subject: Re: zfs, a directory that used to hold lot of files and listing pause To: Pete French Cc: bennett@sdf.org, freebsd-stable , emz@norma.perm.ru Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 17:44:29 -0000 On Fri, Oct 21, 2016 at 10:04 AM, Pete French wrote: > Not forgotten, just under the impression that ZFS shrinks directories > unlike good old UFS. Apparenrly not, > Someone offhandedly mentioned this earlier (it's apparently intended for the future sometime). I at least hope they do something smarter than double indirect blocks these days.... -- brandon s allbery kf8nh sine nomine associates allbery.b@gmail.com ballbery@sinenomine.net unix, openafs, kerberos, infrastructure, xmonad http://sinenomine.net From owner-freebsd-stable@freebsd.org Fri Oct 21 17:56:59 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1DF6C1C150 for ; Fri, 21 Oct 2016 17:56:59 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8FDFA7F6 for ; Fri, 21 Oct 2016 17:56:59 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate2.intern.punkt.de with ESMTP id u9LHuunM039687 for ; Fri, 21 Oct 2016 19:56:56 +0200 (CEST) Received: from [217.29.46.116] ([217.29.46.116]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id u9LHuuDd093763 for ; Fri, 21 Oct 2016 19:56:56 +0200 (CEST) (envelope-from hausen@punkt.de) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: boot0cfg on does not set default selection on gmirror device From: "Patrick M. Hausen" In-Reply-To: Date: Fri, 21 Oct 2016 19:57:20 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <54B556F6-B834-4B0C-A9A4-90B9AD80A668@punkt.de> References: <14FD5FE6-6277-4EBE-8EE9-630A735F8BEA@punkt.de> To: freebsd-stable X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 17:57:00 -0000 Hi, all, > Am 21.10.2016 um 16:41 schrieb Warner Losh : > Any chance you can migrate to using gpart? Is boot0cfg still > referenced in NanoBSD somewhere? Not in NanoBSD but how would you configure boot0's default slice with gpart? It doesn't pay attention to the "active" flag. See Miroslav's mails for all the details. gpart would only be an option if we did not use the FreeBSD boot manager. But we need the "F1 ..., F2 ..." prompt, because being able to roll back to the last known-good system via the console is the entire point of using this NanoBSD setup. There's a presentation on the EuroBSDCon 2010 page about motivation and setup. Wonder who did that talk ... :-))) BTW: thanks, Miroslav. As for your question: it does work on the only two systems that use hardware RAID, yet have a gmirror built of only a single component to get consistent device names accross all servers. I'm not quite sure if it works from time to time, I've come to accept the "kern.geom.debugflags" dance. I had opened a similar discussion years ago for 7.x/8.x and I was told that geom was to provide an API for fdisk, boot0cfg and friends to manipulate the MBR. Because back in the days boot0cfg and fdisk both threw an error message when trying to work on a whole-disk mirror. I thought that was long solved - at least no error, anymore. But it's still not working in 10.x. Thanks to all and take care, Patrick --=20 punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe Tel. 0721 9109 0 * Fax 0721 9109 100 info@punkt.de http://www.punkt.de Gf: J=C3=BCrgen Egeling AG Mannheim 108285 From owner-freebsd-stable@freebsd.org Fri Oct 21 18:25:31 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C6C2C1C8AA for ; Fri, 21 Oct 2016 18:25:31 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-it0-x233.google.com (mail-it0-x233.google.com [IPv6:2607:f8b0:4001:c0b::233]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D992F97A for ; Fri, 21 Oct 2016 18:25:30 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: by mail-it0-x233.google.com with SMTP id 4so253574269itv.0 for ; Fri, 21 Oct 2016 11:25:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20150623.gappssmtp.com; s=20150623; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=iX1hmYschdhoLWvHZJvMU44FtNKEWYbY9RYLqWeaF5U=; b=DD5LTMs1sAtlIrjcbk/DKOCjFn6xC6Qkt0aODNQGBn7xM2OIQiDi2WVlWJH7uEIyy+ n2zvaFVl9wKuNGIl/alY2n9Lbw30ebkJIESJ4mgtO+aQZZ2cVabCDj/XmR9hzODEhYKz LIjV6uWzI6nurUoO6OaOX7+OV690rhtDPJYjle908a/lbnXH3D93AcTUhySN3+6F9piH H0nmnO6pOClEEuRhnwv1w+xg98zzcx+tF/WynitIf8lK9mCQ7LODevm8+k2pd/ti73Nv IiIT2IWXgRweVyGvav19WU0GEewLYmYEStR4JBabdU3FiqcbgasSaZ0zsaF2MFg8gPIr c7HA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=iX1hmYschdhoLWvHZJvMU44FtNKEWYbY9RYLqWeaF5U=; b=kbEBisCOrLDzVb2hp1DrVAVDpZJXsf+bI7NTe5jzw2bdT/UMOT5V/Mv/NCDBqbYPsm W0PC2655IQvuA5pRiAWfI+YKsaDIafR+iIWq4yXRcLb3EWuMe7ASRnBMcM8psF7eOSvt 8BC+490/VDK1cwaD8N315FpF/iegEqMQ9yTou5JsK0+TjYSdttpXFGc9uUJC7ZaDpC73 z+f1Sf5YJIyuzWqkEHeTbcgVOoPWngakZlXxS/4aUKUroLzv7g9eEemiETr7DPvtb9eT t14rIyt8ojiH9t3mvPuVzJqRv2/b06MD1vwcyrW9EtYdHamZvCzFIiETdNjaiiD7ecMP epBA== X-Gm-Message-State: ABUngvd8wDvj1LaenobneIkNWyM8miVi1HoGaRI2wpne+EWgWXEwLCEf5seO6kW2g7pddIAM5exqUnQe93iA4A== X-Received: by 10.36.53.214 with SMTP id k205mr2301797ita.85.1477074329910; Fri, 21 Oct 2016 11:25:29 -0700 (PDT) MIME-Version: 1.0 Sender: wlosh@bsdimp.com Received: by 10.79.77.85 with HTTP; Fri, 21 Oct 2016 11:25:29 -0700 (PDT) X-Originating-IP: [69.53.245.200] In-Reply-To: <54B556F6-B834-4B0C-A9A4-90B9AD80A668@punkt.de> References: <14FD5FE6-6277-4EBE-8EE9-630A735F8BEA@punkt.de> <54B556F6-B834-4B0C-A9A4-90B9AD80A668@punkt.de> From: Warner Losh Date: Fri, 21 Oct 2016 12:25:29 -0600 X-Google-Sender-Auth: _TL265P6lN81FmQIgGldK_liNG8 Message-ID: Subject: Re: boot0cfg on does not set default selection on gmirror device To: "Patrick M. Hausen" Cc: freebsd-stable Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 18:25:31 -0000 On Fri, Oct 21, 2016 at 11:57 AM, Patrick M. Hausen wrote: > Hi, all, > >> Am 21.10.2016 um 16:41 schrieb Warner Losh : >> Any chance you can migrate to using gpart? Is boot0cfg still >> referenced in NanoBSD somewhere? > > Not in NanoBSD but how would you configure boot0's default > slice with gpart? It doesn't pay attention to the "active" flag. > See Miroslav's mails for all the details. > > gpart would only be an option if we did not use the FreeBSD > boot manager. Ah! OK, I thought this was the active flag issue, not the default in boot0 issue. > But we need the "F1 ..., F2 ..." prompt, because > being able to roll back to the last known-good system via the > console is the entire point of using this NanoBSD setup. > There's a presentation on the EuroBSDCon 2010 page about > motivation and setup. Wonder who did that talk ... :-))) I think I sat in the talk :) > BTW: thanks, Miroslav. As for your question: it does work on > the only two systems that use hardware RAID, yet have a > gmirror built of only a single component to get consistent > device names accross all servers. > > I'm not quite sure if it works from time to time, I've come to > accept the "kern.geom.debugflags" dance. > > I had opened a similar discussion years ago for 7.x/8.x and > I was told that geom was to provide an API for fdisk, boot0cfg > and friends to manipulate the MBR. Because back in the days > boot0cfg and fdisk both threw an error message when trying > to work on a whole-disk mirror. It certainly looks like this code has that conversion in it. Looks like it's been there quite a while. I'd have expected it to "JUST WORK" [tm]. > I thought that was long solved - at least no error, anymore. > But it's still not working in 10.x. Can you give us the strace output? It looks like it is reading the current blocks, setting the options, and then writing it back to the device. If the write back fails, it opens the device with geom and sends either the bootcode verb to geom (for the PART (aka gpart)) case or the data for the MBR case. strace should show that clearly. There's nothing in dmesg, right? Try this again but set geom.debug_flags to 128 instead of 16. This will give a verbose error in dmesg if there's any errors from the control message. Warner From owner-freebsd-stable@freebsd.org Fri Oct 21 18:47:00 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 45EAEC1CE2E for ; Fri, 21 Oct 2016 18:47:00 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from kagate.punkt.de (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E57D1B35 for ; Fri, 21 Oct 2016 18:46:59 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from hugo10.ka.punkt.de (hugo10.ka.punkt.de [217.29.44.10]) by gate2.intern.punkt.de with ESMTP id u9LIkuYh039990; Fri, 21 Oct 2016 20:46:56 +0200 (CEST) Received: from [217.29.46.116] ([217.29.46.116]) by hugo10.ka.punkt.de (8.14.2/8.14.2) with ESMTP id u9LIkuI7099791; Fri, 21 Oct 2016 20:46:56 +0200 (CEST) (envelope-from hausen@punkt.de) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: boot0cfg on does not set default selection on gmirror device From: "Patrick M. Hausen" In-Reply-To: Date: Fri, 21 Oct 2016 20:47:20 +0200 Cc: freebsd-stable Content-Transfer-Encoding: quoted-printable Message-Id: References: <14FD5FE6-6277-4EBE-8EE9-630A735F8BEA@punkt.de> <54B556F6-B834-4B0C-A9A4-90B9AD80A668@punkt.de> To: Warner Losh X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 18:47:00 -0000 Hi, Warner, > Am 21.10.2016 um 20:25 schrieb Warner Losh : > Can you give us the strace output? amd64 - no strace. I need a hand here, what precisely do I need to = enter? > It looks like it is reading the current blocks, setting the options, > and then writing it back to the device. If the write back fails, it > opens the device with geom and sends either the bootcode verb to geom > (for the PART (aka gpart)) case or the data for the MBR case. strace > should show that clearly. There's nothing in dmesg, right? Try this > again but set geom.debug_flags to 128 instead of 16. This will give a > verbose error in dmesg if there's any errors from the control message. I set the flag, then tried to change the slice from 1 to 2. Result: Dump of gctl request at 0xfffffe02392bd9e0: param: "class" [R5] =3D "PART" param: "arg0" [R10] =3D "mirror/m0" param: "verb" [R9] =3D "bootcode" param: "bootcode" [R512] =3D fc 31 c0 8e c0 8e d8 8e d0 bc 00 = 7c 89 e6 bf 00 06 b9 00 01 f3 a5 89 fd b1 08 f3 ab fe 45 f2 e9 00 8a f6 = 46 bb 20 75 08 84 d2 78 07 80 4e bb 40 8a 56 ba 88 56 00 e8 fc 00 52 bb = c2 07 31 d2 88 6f fc 0f a3 56 bb 73 19 8a 07 bf 87 07 b1 03 f2 ae 74 0e = b1 0b f2 ae 83 c7 09 8a 0d 01 cf e8 c5 00 42 80 c3 10 73 d8 58 2c 7f 3a = 06 75 04 72 05 48 74 0d 30 c0 04 b0 88 46 b8 bf b2 07 e8 a6 00 be 7b 07 = e8 b2 00 8a 56 b9 4e e8 8e 00 eb 05 b0 07 e8 b0 00 30 e4 cd 1a 89 d7 03 = 7e bc b4 01 cd 16 75 0d 30 e4 cd 1a 39 fa 72 f2 8a 46 b9 eb 16 30 e4 cd = 16 88 e0 3c 1c 74 f1 2c 3b 3c 04 76 06 2c c7 3c 04 77 c9 98 0f a3 46 0c = 73 c2 88 46 b9 be 00 08 8a 14 89 f3 3c 04 9c 74 0a c0 e0 04 05 be 07 93 = c6 07 80 53 f6 46 bb 40 75 08 bb 00 06 b4 03 e8 59 00 5e 9d 75 06 8a 56 = b8 80 ea 30 bb 00 7c b4 02 e8 47 00 72 86 81 bf fe 01 55 aa 0f 85 7c ff = be 85 07 e8 19 00 ff e3 b0 46 e8 24 00 b0 31 00 d0 eb 17 0f ab 56 0c be = 78 07 e8 eb ff 89 fe e8 03 00 be 85 07 ac a8 80 75 05 e8 04 00 eb f6 24 = 7f 53 bb 07 00 b4 0e cd 10 5b c3 8a 74 01 8b 4c 02 b0 01 56 89 e7 f6 46 = bb 80 74 13 66 6a 00 66 ff 74 08 06 53 6a 01 6a 10 89 e6 48 80 cc 40 cd = 13 89 fc 5e c3 20 20 a0 0a 44 65 66 61 75 6c 74 3a a0 0d 8a 00 05 0f 01 = 06 07 0b 0c 0e 83 a5 a6 a9 0d 0c 0b 0a 09 08 0a 0e 11 10 01 3f bf 44 4f = d3 4c 69 6e 75 f8 46 72 65 65 42 53 c4 66 bb 44 72 69 76 65 20 b1 01 80 = 8f b6 00 80 00 01 01 a5 fe ff fe c1 3e 00 00 7e 86 fa 00 00 00 c1 ff a5 = fe ff fc 3f c5 fa 00 7e 86 fa 00 00 00 c1 fd a5 fe ff 00 bd 4b f5 01 04 = 0e 7b 72 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa param: "flags" [R2] =3D "C" root@hd45:~ # boot0cfg -v mirror/m0 # flag start chs type end chs offset size 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 version=3D1.0 drive=3D0x80 mask=3D0xf ticks=3D182 bell=3D (0x7) options=3Dpacket,update,nosetdrv default_selection=3DF1 (Slice 1) So again, no change. Thanks, Patrick --=20 punkt.de GmbH * Kaiserallee 13a * 76133 Karlsruhe Tel. 0721 9109 0 * Fax 0721 9109 100 info@punkt.de http://www.punkt.de Gf: J=C3=BCrgen Egeling AG Mannheim 108285 From owner-freebsd-stable@freebsd.org Fri Oct 21 19:21:32 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2BE90C1BAE4; Fri, 21 Oct 2016 19:21:32 +0000 (UTC) (envelope-from fidaj@ukr.net) Received: from frv25.fwdcdn.com (frv25.fwdcdn.com [212.42.77.25]) by mx1.freebsd.org (Postfix) with ESMTP id C8F82647; Fri, 21 Oct 2016 19:21:31 +0000 (UTC) (envelope-from fidaj@ukr.net) Received: from [10.10.14.26] (helo=frv157.fwdcdn.com) by frv25.fwdcdn.com QID:1bxf79-000OvQ-H3/RC:2; Fri, 21 Oct 2016 22:04:23 +0300 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=fsm; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID: Subject:To:From:Date:Sender:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: In-Reply-To:References:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=MeO5X/1NhJ4mRvMXQQnFxA8XZjxDGr6/pjO2MenPEUk=; b=YG7S5clkzojMaKYrUwhqGFu/Z/ +rlJqrN5VpoiC1mtklNgMRHfCE7ICVnVpkqEoCOag2zu6QfuxDU/1WFrJ8OnCdvyyAhz8kbsIrQCP mDFJ/Bvh+JD1qq0jQVOuKxpcBkmFl/IoYQ6Y/Wn8qjXKJPKIPNrYcond30wgmyDxEXOw=; Received: from [37.229.193.176] (helo=nonamehost) by frv157.fwdcdn.com with esmtpsa ID 1bxf70-0000M3-81 ; Fri, 21 Oct 2016 22:04:14 +0300 Date: Fri, 21 Oct 2016 22:04:13 +0300 From: Ivan Klymenko To: freebsd-bugs@freebsd.org, freebsd-stable@freebsd.org Subject: panics collections on FreeBSD 11.0-RC1 RC2 PRERELEASE RELEASE STABLE Message-ID: <20161021220413.1d130f5c@nonamehost> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEWpqak/Pz/i4uIfHx8GBwZwcHAQEBA6o92AAAACHElEQVQ4jWWUTY7bMAyF6QzUPSEoa8PFHEBgqwuM4bVVg7MvZOj+R+ijpMTpjIwgkT7z75EKrdfattpXERG6zqvUOtAr2LCRYfEKcB4l/Q+2cc6XjQH7hv+2YZYreIk5nevZEPvuzUzptizHLzgDMnC5Wpbl7ewJlOEqlQF+DlCjgVLki0WV6FMDMsBxjlJiQulIznwZ+DxHiQyDyIg0wN3Oo6o6ZQ5s5AIfar+W2Wlmz+kCcb8tg6j3voMEwNrBQk69dDBDqw/urpqJH+m+Q6u/4QnoAeYpnUXC/s1iup9rhCd6xMgAqdDyAyFegbKkVAHeLCcOulPLawaoUIDos4M88iLNrVkU7uu5ccTDO6naJzWLum51C6Yb7y4HKKbdArLWir0PBiS8glJRBZHeyHl7J9lENpAC6qT9NlNG4u5hsVYDyJP6mlJJtY3oVju4WSUzHal1sDU17NASoBWSk40J2eBLBJhYrVmzC5gVALGpNIAiQgN6eGstOp9Oa6zFbbLTISYi28BGZDRUJKWeroECkCEkzXjUtbmmaKMfAx2RfbT69/cO+tgHcmx6AfyZOmj3NDIah0F0GB66d4CrdIoplNFFGHSpSheRxbo0W4S8azNItEoMWbw3uXAeJgCrmX5joz7CGXqSg6PcryEhnFr/C1C2ntPxBOYbdwY+8dO3+wZJyFlbMX9s8zNnvp/tLwAv03NB4j3HVpn8Awwm+GrlP6MVAAAAAElFTkSuQmCC MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Authentication-Result: IP=37.229.193.176; mail.from=fidaj@ukr.net; dkim=pass; header.d=ukr.net X-Ukrnet-Yellow: 0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 19:21:32 -0000 Hi, friendly community. I present to you my collection. I have : //=======================================================// System Information Manufacturer: Supermicro Product Name: X10SLH-F/X10SLM+-F //=======================================================// pciconf -lv hostb0@pci0:0:0:0: class=0x060000 card=0x080315d9 chip=0x0c088086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = 'Xeon E3-1200 v3 Processor DRAM Controller' class = bridge subclass = HOST-PCI xhci0@pci0:0:20:0: class=0x0c0330 card=0x080315d9 chip=0x8c318086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = '8 Series/C220 Series Chipset Family USB xHCI' class = serial bus subclass = USB none0@pci0:0:22:0: class=0x078000 card=0x080315d9 chip=0x8c3a8086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = '8 Series/C220 Series Chipset Family MEI Controller' class = simple comms none1@pci0:0:22:1: class=0x078000 card=0x080315d9 chip=0x8c3b8086 rev=0x04 hdr=0x00 vendor = 'Intel Corporation' device = '8 Series/C220 Series Chipset Family MEI Controller' class = simple comms ehci0@pci0:0:26:0: class=0x0c0320 card=0x080315d9 chip=0x8c2d8086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = '8 Series/C220 Series Chipset Family USB EHCI' class = serial bus subclass = USB pcib1@pci0:0:28:0: class=0x060400 card=0x080315d9 chip=0x8c108086 rev=0xd5 hdr=0x01 vendor = 'Intel Corporation' device = '8 Series/C220 Series Chipset Family PCI Express Root Port' class = bridge subclass = PCI-PCI pcib3@pci0:0:28:2: class=0x060400 card=0x080315d9 chip=0x8c148086 rev=0xd5 hdr=0x01 vendor = 'Intel Corporation' device = '8 Series/C220 Series Chipset Family PCI Express Root Port' class = bridge subclass = PCI-PCI pcib4@pci0:0:28:3: class=0x060400 card=0x080315d9 chip=0x8c168086 rev=0xd5 hdr=0x01 vendor = 'Intel Corporation' device = '8 Series/C220 Series Chipset Family PCI Express Root Port' class = bridge subclass = PCI-PCI ehci1@pci0:0:29:0: class=0x0c0320 card=0x080315d9 chip=0x8c268086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = '8 Series/C220 Series Chipset Family USB EHCI' class = serial bus subclass = USB isab0@pci0:0:31:0: class=0x060100 card=0x080315d9 chip=0x8c548086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = 'C224 Series Chipset Family Server Standard SKU LPC Controller' class = bridge subclass = PCI-ISA ahci0@pci0:0:31:2: class=0x010601 card=0x080315d9 chip=0x8c028086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = '8 Series/C220 Series Chipset Family 6-port SATA Controller 1 [AHCI mode]' class = mass storage subclass = SATA ichsmb0@pci0:0:31:3: class=0x0c0500 card=0x080315d9 chip=0x8c228086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = '8 Series/C220 Series Chipset Family SMBus Controller' class = serial bus subclass = SMBus none2@pci0:0:31:6: class=0x118000 card=0x080315d9 chip=0x8c248086 rev=0x05 hdr=0x00 vendor = 'Intel Corporation' device = '8 Series Chipset Family Thermal Management Controller' class = dasp pcib2@pci0:1:0:0: class=0x060400 card=0x080315d9 chip=0x11501a03 rev=0x03 hdr=0x01 vendor = 'ASPEED Technology, Inc.' device = 'AST1150 PCI-to-PCI Bridge' class = bridge subclass = PCI-PCI vgapci0@pci0:2:0:0: class=0x030000 card=0x080315d9 chip=0x20001a03 rev=0x30 hdr=0x00 vendor = 'ASPEED Technology, Inc.' device = 'ASPEED Graphics Family' class = display subclass = VGA igb0@pci0:3:0:0: class=0x020000 card=0x153315d9 chip=0x15338086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = 'I210 Gigabit Network Connection' class = network subclass = ethernet igb1@pci0:4:0:0: class=0x020000 card=0x153315d9 chip=0x15338086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = 'I210 Gigabit Network Connection' class = network subclass = ethernet //=======================================================// In fact, a collection of panics: //=======================================================// Sep 15 11:57:22 ns kernel: Fatal trap 12: page fault while in kernel mode Sep 15 11:57:22 ns kernel: cpuid = 6; apic id = 06 Sep 15 11:57:22 ns kernel: fault virtual address = 0x8 Sep 15 11:57:22 ns kernel: fault code = supervisor read data, page not present Sep 15 11:57:22 ns kernel: instruction pointer = 0x20:0xffffffff80baca70 Sep 15 11:57:22 ns kernel: stack pointer = 0x28:0xfffffe07c9eba540 Sep 15 11:57:22 ns kernel: frame pointer = 0x28:0xfffffe07c9eba580 Sep 15 11:57:22 ns kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Sep 15 11:57:22 ns kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Sep 15 11:57:22 ns kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Sep 15 11:57:22 ns kernel: current process = 12 (swi5: fast taskq) Sep 15 11:57:22 ns kernel: trap number = 12 Sep 15 11:57:22 ns kernel: panic: page fault Sep 15 11:57:22 ns kernel: cpuid = 6 Sep 15 11:57:22 ns kernel: KDB: stack backtrace: Sep 15 11:57:22 ns kernel: #0 0xffffffff80b5c6f7 at kdb_backtrace+0x67 Sep 15 11:57:22 ns kernel: #1 0xffffffff80b103f2 at vpanic+0x182 Sep 15 11:57:22 ns kernel: #2 0xffffffff80b10263 at panic+0x43 Sep 15 11:57:22 ns kernel: #3 0xffffffff81055f10 at trap_fatal+0x350 Sep 15 11:57:22 ns kernel: #4 0xffffffff81056103 at trap_pfault+0x1e3 Sep 15 11:57:22 ns kernel: #5 0xffffffff8105568d at trap+0x26d Sep 15 11:57:22 ns kernel: #6 0xffffffff81038011 at calltrap+0x8 Sep 15 11:57:22 ns kernel: #7 0xffffffff80d4fa10 at tcp_do_segment+0x19a0 Sep 15 11:57:22 ns kernel: #8 0xffffffff80d4d8c7 at tcp_input+0x1487 Sep 15 11:57:22 ns kernel: #9 0xffffffff80cb254e at ip_input+0x18e Sep 15 11:57:22 ns kernel: #10 0xffffffff80c46b6d at netisr_dispatch_src+0xad Sep 15 11:57:22 ns kernel: #11 0xffffffff80c2e25a at ether_demux+0x13a Sep 15 11:57:22 ns kernel: #12 0xffffffff82b6471c at vboxNetFltFreeBSDinput+0x27c Sep 15 11:57:22 ns kernel: #13 0xffffffff80b6fd6a at taskqueue_run_locked+0x14a Sep 15 11:57:22 ns kernel: #14 0xffffffff80b6fb5f at taskqueue_run+0xbf Sep 15 11:57:22 ns kernel: #15 0xffffffff80ac8faf at intr_event_execute_handlers+0x20f Sep 15 11:57:22 ns kernel: #16 0xffffffff80ac9216 at ithread_loop+0xc6 Sep 15 11:57:22 ns kernel: #17 0xffffffff80ac59f5 at fork_exit+0x85 Sep 15 11:57:22 ns kernel: Uptime: 14d13h3m30s Sep 17 12:48:33 ns syslogd: kernel boot file is /boot/kernel/kernel Sep 17 12:48:33 ns kernel: Sep 17 12:48:33 ns kernel: Sep 17 12:48:33 ns kernel: Fatal trap 12: page fault while in kernel mode Sep 17 12:48:33 ns kernel: cpuid = 1; apic id = 01 Sep 17 12:48:33 ns kernel: fault virtual address = 0x78 Sep 17 12:48:33 ns kernel: fault code = supervisor read data, page not present Sep 17 12:48:33 ns kernel: instruction pointer = 0x20:0xffffffff80cafab7 Sep 17 12:48:33 ns kernel: stack pointer = 0x28:0xfffffe07c9eba770 Sep 17 12:48:33 ns kernel: frame pointer = 0x28:0xfffffe07c9eba800 Sep 17 12:48:33 ns kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Sep 17 12:48:33 ns kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Sep 17 12:48:33 ns kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Sep 17 12:48:33 ns kernel: current process = 12 (swi5: fast taskq) Sep 17 12:48:33 ns kernel: trap number = 12 Sep 17 12:48:33 ns kernel: panic: page fault Sep 17 12:48:33 ns kernel: cpuid = 1 Sep 17 12:48:33 ns kernel: KDB: stack backtrace: Sep 17 12:48:33 ns kernel: #0 0xffffffff80b5c6f7 at kdb_backtrace+0x67 Sep 17 12:48:33 ns kernel: #1 0xffffffff80b103f2 at vpanic+0x182 Sep 17 12:48:33 ns kernel: #2 0xffffffff80b10263 at panic+0x43 Sep 17 12:48:33 ns kernel: #3 0xffffffff81055f10 at trap_fatal+0x350 Sep 17 12:48:33 ns kernel: #4 0xffffffff81056103 at trap_pfault+0x1e3 Sep 17 12:48:33 ns kernel: #5 0xffffffff8105568d at trap+0x26d Sep 17 12:48:33 ns kernel: #6 0xffffffff81038011 at calltrap+0x8 Sep 17 12:48:33 ns kernel: #7 0xffffffff80cb27bc at ip_input+0x3fc Sep 17 12:48:33 ns kernel: #8 0xffffffff80c46b6d at netisr_dispatch_src+0xad Sep 17 12:48:33 ns kernel: #9 0xffffffff80c2e25a at ether_demux+0x13a Sep 17 12:48:33 ns kernel: #10 0xffffffff82b6471c at vboxNetFltFreeBSDinput+0x27c Sep 17 12:48:33 ns kernel: #11 0xffffffff80b6fd6a at taskqueue_run_locked+0x14a Sep 17 12:48:33 ns kernel: #12 0xffffffff80b6fb5f at taskqueue_run+0xbf Sep 17 12:48:33 ns kernel: #13 0xffffffff80ac8faf at intr_event_execute_handlers+0x20f Sep 17 12:48:33 ns kernel: #14 0xffffffff80ac9216 at ithread_loop+0xc6 Sep 17 12:48:33 ns kernel: #15 0xffffffff80ac59f5 at fork_exit+0x85 Sep 17 12:48:33 ns kernel: #16 0xffffffff8103854e at fork_trampoline+0xe Sep 17 12:48:33 ns kernel: Uptime: 2d0h48m48s Sep 17 12:48:33 ns kernel: Dumping 7325 out of 32688 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Sep 19 18:23:27 ns kernel: panic: sbsndptr: sockbuf 0xfffff80086aed1b8 and mbuf 0xfffff8024e034600 clashing Sep 19 18:23:27 ns kernel: cpuid = 1 Sep 19 18:23:27 ns kernel: KDB: stack backtrace: Sep 19 18:23:27 ns kernel: #0 0xffffffff80b5cf57 at kdb_backtrace+0x67 Sep 19 18:23:27 ns kernel: #1 0xffffffff80b10902 at vpanic+0x182 Sep 19 18:23:27 ns kernel: #2 0xffffffff80b10773 at panic+0x43 Sep 19 18:23:27 ns kernel: #3 0xffffffff80bad5da at sbsndptr+0xda Sep 19 18:23:27 ns kernel: #4 0xffffffff80d5530a at tcp_output+0x105a Sep 19 18:23:27 ns kernel: #5 0xffffffff80d6669d at tcp_usr_send+0x3ed Sep 19 18:23:27 ns kernel: #6 0xffffffff80bb0357 at sosend_generic+0x5e7 Sep 19 18:23:27 ns kernel: #7 0xffffffff80bb07fc at sosend+0x3c Sep 19 18:23:27 ns kernel: #8 0xffffffff80bb83db at kern_sendit+0x22b Sep 19 18:23:27 ns kernel: #9 0xffffffff80bb87df at sendit+0x19f Sep 19 18:23:27 ns kernel: #10 0xffffffff80bb862d at sys_sendto+0x4d Sep 19 18:23:27 ns kernel: #11 0xffffffff81057878 at amd64_syscall+0x4d8 Sep 19 18:23:27 ns kernel: #12 0xffffffff81038e2b at Xfast_syscall+0xfb Sep 19 18:23:27 ns kernel: Uptime: 5m6s Oct 6 09:34:11 ns kernel: Fatal trap 12: page fault while in kernel mode Oct 6 09:34:11 ns kernel: cpuid = 3; apic id = 03 Oct 6 09:34:11 ns kernel: fault virtual address = 0x360 Oct 6 09:34:11 ns kernel: fault code = supervisor read data, page not present Oct 6 09:34:11 ns kernel: instruction pointer = 0x20:0xffffffff80c481f7 Oct 6 09:34:11 ns kernel: stack pointer = 0x28:0xfffffe085dd1c960 Oct 6 09:34:11 ns kernel: frame pointer = 0x28:0xfffffe085dd1c9e0 Oct 6 09:34:11 ns kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Oct 6 09:34:11 ns kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Oct 6 09:34:11 ns kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Oct 6 09:34:11 ns kernel: current process = 12 (swi1: netisr 3) Oct 6 09:34:11 ns kernel: trap number = 12 Oct 6 09:34:11 ns kernel: panic: page fault Oct 6 09:34:11 ns kernel: cpuid = 3 Oct 6 09:34:11 ns kernel: KDB: stack backtrace: Oct 6 09:34:11 ns kernel: #0 0xffffffff80b5d1e7 at kdb_backtrace+0x67 Oct 6 09:34:11 ns kernel: #1 0xffffffff80b10b92 at vpanic+0x182 Oct 6 09:34:11 ns kernel: #2 0xffffffff80b10a03 at panic+0x43 Oct 6 09:34:11 ns kernel: #3 0xffffffff81058110 at trap_fatal+0x350 Oct 6 09:34:11 ns kernel: #4 0xffffffff81058303 at trap_pfault+0x1e3 Oct 6 09:34:11 ns kernel: #5 0xffffffff8105788d at trap+0x26d Oct 6 09:34:11 ns kernel: #6 0xffffffff810392f1 at calltrap+0x8 Oct 6 09:34:11 ns kernel: #7 0xffffffff80ac96ef at intr_event_execute_handlers+0x20f Oct 6 09:34:11 ns kernel: #8 0xffffffff80ac9956 at ithread_loop+0xc6 Oct 6 09:34:11 ns kernel: #9 0xffffffff80ac6135 at fork_exit+0x85 Oct 6 09:34:11 ns kernel: #10 0xffffffff8103982e at fork_trampoline+0xe Oct 6 09:34:11 ns kernel: Uptime: 33s Oct 6 09:34:11 ns kernel: Dumping 1344 out of 32688 MB:..2%..11%..21%..31%..41%..52%..61%..71%..81%..91% Oct 6 22:37:42 ns kernel: Fatal trap 12: page fault while in kernel mode Oct 6 22:37:42 ns kernel: cpuid = 7; apic id = 07 Oct 6 22:37:42 ns kernel: fault virtual address = 0x8 Oct 6 22:37:42 ns kernel: fault code = supervisor read data, page not present Oct 6 22:37:42 ns kernel: instruction pointer = 0x20:0xffffffff80baea30 Oct 6 22:37:42 ns kernel: stack pointer = 0x28:0xfffffe07c9eba540 Oct 6 22:37:42 ns kernel: frame pointer = 0x28:0xfffffe07c9eba580 Oct 6 22:37:42 ns kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Oct 6 22:37:42 ns kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Oct 6 22:37:42 ns kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Oct 6 22:37:42 ns kernel: current process = 12 (swi5: fast taskq) Oct 6 22:37:42 ns kernel: trap number = 12 Oct 6 22:37:42 ns kernel: panic: page fault Oct 6 22:37:42 ns kernel: cpuid = 7 Oct 6 22:37:42 ns kernel: KDB: stack backtrace: Oct 6 22:37:42 ns kernel: #0 0xffffffff80b5e5b7 at kdb_backtrace+0x67 Oct 6 22:37:42 ns kernel: #1 0xffffffff80b11e92 at vpanic+0x182 Oct 6 22:37:42 ns kernel: #2 0xffffffff80b11d03 at panic+0x43 Oct 6 22:37:42 ns kernel: #3 0xffffffff81059280 at trap_fatal+0x350 Oct 6 22:37:42 ns kernel: #4 0xffffffff81059473 at trap_pfault+0x1e3 Oct 6 22:37:42 ns kernel: #5 0xffffffff810589fd at trap+0x26d Oct 6 22:37:42 ns kernel: #6 0xffffffff8103a3d1 at calltrap+0x8 Oct 6 22:37:42 ns kernel: #7 0xffffffff80d51ade at tcp_do_segment+0x1a1e Oct 6 22:37:42 ns kernel: #8 0xffffffff80d4f917 at tcp_input+0x1487 Oct 6 22:37:42 ns kernel: #9 0xffffffff80cb459e at ip_input+0x18e Oct 6 22:37:42 ns kernel: #10 0xffffffff80c48bbd at netisr_dispatch_src+0xad Oct 6 22:37:42 ns kernel: #11 0xffffffff80c302ca at ether_demux+0x13a Oct 6 22:37:42 ns kernel: #12 0xffffffff82b6671c at vboxNetFltFreeBSDinput+0x27c Oct 6 22:37:42 ns kernel: #13 0xffffffff80b71d1a at taskqueue_run_locked+0x14a Oct 6 22:37:42 ns kernel: #14 0xffffffff80b71b0f at taskqueue_run+0xbf Oct 6 22:37:42 ns kernel: #15 0xffffffff80aca9ff at intr_event_execute_handlers+0x20f Oct 6 22:37:42 ns kernel: #16 0xffffffff80acac66 at ithread_loop+0xc6 Oct 6 22:37:42 ns kernel: #17 0xffffffff80ac7445 at fork_exit+0x85 Oct 6 22:37:42 ns kernel: Uptime: 3h57m42s Oct 6 22:37:42 ns kernel: Dumping 5958 out of 32688 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Oct 13 13:20:54 ns kernel: panic: sbsndptr: sockbuf 0xfffff80756718518 and mbuf 0xfffff80042986200 clashing Oct 13 13:20:54 ns kernel: cpuid = 1 Oct 13 13:20:54 ns kernel: KDB: stack backtrace: Oct 13 13:20:54 ns kernel: #0 0xffffffff80b5ec97 at kdb_backtrace+0x67 Oct 13 13:20:54 ns kernel: #1 0xffffffff80b12572 at vpanic+0x182 Oct 13 13:20:54 ns kernel: #2 0xffffffff80b123e3 at panic+0x43 Oct 13 13:20:54 ns kernel: #3 0xffffffff80baf7fa at sbsndptr+0xda Oct 13 13:20:54 ns kernel: #4 0xffffffff80d576f8 at tcp_output+0x1168 Oct 13 13:20:54 ns kernel: #5 0xffffffff80d53c66 at tcp_do_segment+0x30d6 Oct 13 13:20:54 ns kernel: #6 0xffffffff80d503e7 at tcp_input+0x1487 Oct 13 13:20:54 ns kernel: #7 0xffffffff80cb506e at ip_input+0x18e Oct 13 13:20:54 ns kernel: #8 0xffffffff80c4968d at netisr_dispatch_src+0xad Oct 13 13:20:54 ns kernel: #9 0xffffffff80c30d9a at ether_demux+0x13a Oct 13 13:20:54 ns kernel: #10 0xffffffff82b6671c at vboxNetFltFreeBSDinput+0x27c Oct 13 13:20:54 ns kernel: #11 0xffffffff80b724fa at taskqueue_run_locked+0x14a Oct 13 13:20:54 ns kernel: #12 0xffffffff80b722ef at taskqueue_run+0xbf Oct 13 13:20:54 ns kernel: #13 0xffffffff80acb0df at intr_event_execute_handlers+0x20f Oct 13 13:20:54 ns kernel: #14 0xffffffff80acb346 at ithread_loop+0xc6 Oct 13 13:20:54 ns kernel: #15 0xffffffff80ac7b25 at fork_exit+0x85 Oct 13 13:20:54 ns kernel: #16 0xffffffff8103b3ee at fork_trampoline+0xe Oct 13 13:20:54 ns kernel: Uptime: 16h23m3s Oct 20 17:19:54 ns kernel: Fatal trap 12: page fault while in kernel mode Oct 20 17:19:54 ns kernel: cpuid = 7; apic id = 07 Oct 20 17:19:54 ns kernel: fault virtual address = 0x8 Oct 20 17:19:54 ns kernel: fault code = supervisor read data, page not present Oct 20 17:19:54 ns kernel: instruction pointer = 0x20:0xffffffff80baf500 Oct 20 17:19:54 ns kernel: stack pointer = 0x28:0xfffffe07c65c1430 Oct 20 17:19:54 ns kernel: frame pointer = 0x28:0xfffffe07c65c1470 Oct 20 17:19:54 ns kernel: code segment = base 0x0, limit 0xfffff, type 0x1b Oct 20 17:19:54 ns kernel: = DPL 0, pres 1, long 1, def32 0, gran 1 Oct 20 17:19:54 ns kernel: processor eflags = interrupt enabled, resume, IOPL = 0 Oct 20 17:19:54 ns kernel: current process = 12 (irq273: igb1:que 3) Oct 20 17:19:54 ns kernel: trap number = 12 Oct 20 17:19:54 ns kernel: panic: page fault Oct 20 17:19:54 ns kernel: cpuid = 7 Oct 20 17:19:54 ns kernel: KDB: stack backtrace: Oct 20 17:19:54 ns kernel: #0 0xffffffff80b5ec97 at kdb_backtrace+0x67 Oct 20 17:19:54 ns kernel: #1 0xffffffff80b12572 at vpanic+0x182 Oct 20 17:19:54 ns kernel: #2 0xffffffff80b123e3 at panic+0x43 Oct 20 17:19:54 ns kernel: #3 0xffffffff81059280 at trap_fatal+0x350 Oct 20 17:19:54 ns kernel: #4 0xffffffff81059473 at trap_pfault+0x1e3 Oct 20 17:19:54 ns kernel: #5 0xffffffff810589fd at trap+0x26d Oct 20 17:19:54 ns kernel: #6 0xffffffff8103aeb1 at calltrap+0x8 Oct 20 17:19:54 ns kernel: #7 0xffffffff80d525ae at tcp_do_segment+0x1a1e Oct 20 17:19:54 ns kernel: #8 0xffffffff80d503e7 at tcp_input+0x1487 Oct 20 17:19:54 ns kernel: #9 0xffffffff80cb506e at ip_input+0x18e Oct 20 17:19:54 ns kernel: #10 0xffffffff80c4968d at netisr_dispatch_src+0xad Oct 20 17:19:54 ns kernel: #11 0xffffffff80c30d9a at ether_demux+0x13a Oct 20 17:19:54 ns kernel: #12 0xffffffff80c31b21 at ether_nh_input+0x341 Oct 20 17:19:54 ns kernel: #13 0xffffffff80c4968d at netisr_dispatch_src+0xad Oct 20 17:19:54 ns kernel: #14 0xffffffff80c31054 at ether_input+0x54 Oct 20 17:19:54 ns kernel: #15 0xffffffff8057307c at igb_rxeof+0x82c Oct 20 17:19:54 ns kernel: #16 0xffffffff805723b8 at igb_msix_que+0x158 Oct 20 17:19:54 ns kernel: #17 0xffffffff80acb0df at intr_event_execute_handlers+0x20f Oct 20 17:19:54 ns kernel: Uptime: 1m54s Oct 20 17:19:54 ns kernel: Dumping 1462 out of 32688 MB:..2%..11%..21%..31%..41%..51%..61%..71%..81%..91% Oct 21 21:28:14 ns syslogd: kernel boot file is /boot/kernel/kernel Oct 21 21:28:14 ns kernel: panic: sbsndptr: sockbuf 0xfffff8048c1001b8 and mbuf 0xfffff80097b78400 clashing Oct 21 21:28:14 ns kernel: cpuid = 5 Oct 21 21:28:14 ns kernel: KDB: stack backtrace: Oct 21 21:28:14 ns kernel: #0 0xffffffff80b5ec97 at kdb_backtrace+0x67 Oct 21 21:28:14 ns kernel: #1 0xffffffff80b12572 at vpanic+0x182 Oct 21 21:28:14 ns kernel: #2 0xffffffff80b123e3 at panic+0x43 Oct 21 21:28:14 ns kernel: #3 0xffffffff80baf7fa at sbsndptr+0xda Oct 21 21:28:14 ns kernel: #4 0xffffffff80d576f8 at tcp_output+0x1168 Oct 21 21:28:14 ns kernel: #5 0xffffffff80d53c66 at tcp_do_segment+0x30d6 Oct 21 21:28:14 ns kernel: #6 0xffffffff80d503e7 at tcp_input+0x1487 Oct 21 21:28:14 ns kernel: #7 0xffffffff80cb506e at ip_input+0x18e Oct 21 21:28:14 ns kernel: #8 0xffffffff80c4968d at netisr_dispatch_src+0xad Oct 21 21:28:14 ns kernel: #9 0xffffffff80c30d9a at ether_demux+0x13a Oct 21 21:28:14 ns kernel: #10 0xffffffff80c31b21 at ether_nh_input+0x341 Oct 21 21:28:14 ns kernel: #11 0xffffffff80c4968d at netisr_dispatch_src+0xad Oct 21 21:28:14 ns kernel: #12 0xffffffff80c31054 at ether_input+0x54 Oct 21 21:28:14 ns kernel: #13 0xffffffff8057307c at igb_rxeof+0x82c Oct 21 21:28:14 ns kernel: #14 0xffffffff805723b8 at igb_msix_que+0x158 Oct 21 21:28:14 ns kernel: #15 0xffffffff80acb0df at intr_event_execute_handlers+0x20f Oct 21 21:28:14 ns kernel: #16 0xffffffff80acb346 at ithread_loop+0xc6 Oct 21 21:28:14 ns kernel: #17 0xffffffff80ac7b25 at fork_exit+0x85 Oct 21 21:28:14 ns kernel: Uptime: 12h10m59s Oct 21 21:28:14 ns kernel: Dumping 6363 out of 32688 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% Thanks. From owner-freebsd-stable@freebsd.org Fri Oct 21 19:29:17 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 06467C1BF8E; Fri, 21 Oct 2016 19:29:17 +0000 (UTC) (envelope-from fidaj@ukr.net) Received: from frv157.fwdcdn.com (frv157.fwdcdn.com [212.42.77.157]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B844BD22; Fri, 21 Oct 2016 19:29:16 +0000 (UTC) (envelope-from fidaj@ukr.net) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=fsm; h=Content-Transfer-Encoding:Content-Type:MIME-Version:References: In-Reply-To:Message-ID:Subject:To:From:Date:Sender:Reply-To:Cc:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=35Oq3ZbWVdYEOd268HvraVp61y/iA5nFudZXdJYMfmk=; b=nx9fZvLUDnt/rRdg1hdsf43hEH fOxLf1LaTkIL2Ki5h0kufEzYTdtxuMRUzjI2aR64X3dBg2TUXMpwnJBQ/ibpJ37lSXZxYqL5FVTJS rWmYSj+vyOZXF1CNnK5qrLaiQw8rNhjTSZnApP9uPdFCdKOemOieJ19uEVZjerORDyME=; Received: from [37.229.193.176] (helo=nonamehost) by frv157.fwdcdn.com with esmtpsa ID 1bxfVA-000LdG-NE ; Fri, 21 Oct 2016 22:29:12 +0300 Date: Fri, 21 Oct 2016 22:29:11 +0300 From: Ivan Klymenko To: freebsd-bugs@freebsd.org, freebsd-stable@freebsd.org Subject: Re: panics collections on FreeBSD 11.0-RC1 RC2 PRERELEASE RELEASE STABLE Message-ID: <20161021222911.10c407e1@nonamehost> In-Reply-To: <20161021220413.1d130f5c@nonamehost> References: <20161021220413.1d130f5c@nonamehost> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.30; x86_64-pc-linux-gnu) Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAFVBMVEWpqak/Pz/i4uIfHx8GBwZwcHAQEBA6o92AAAACHElEQVQ4jWWUTY7bMAyF6QzUPSEoa8PFHEBgqwuM4bVVg7MvZOj+R+ijpMTpjIwgkT7z75EKrdfattpXERG6zqvUOtAr2LCRYfEKcB4l/Q+2cc6XjQH7hv+2YZYreIk5nevZEPvuzUzptizHLzgDMnC5Wpbl7ewJlOEqlQF+DlCjgVLki0WV6FMDMsBxjlJiQulIznwZ+DxHiQyDyIg0wN3Oo6o6ZQ5s5AIfar+W2Wlmz+kCcb8tg6j3voMEwNrBQk69dDBDqw/urpqJH+m+Q6u/4QnoAeYpnUXC/s1iup9rhCd6xMgAqdDyAyFegbKkVAHeLCcOulPLawaoUIDos4M88iLNrVkU7uu5ccTDO6naJzWLum51C6Yb7y4HKKbdArLWir0PBiS8glJRBZHeyHl7J9lENpAC6qT9NlNG4u5hsVYDyJP6mlJJtY3oVju4WSUzHal1sDU17NASoBWSk40J2eBLBJhYrVmzC5gVALGpNIAiQgN6eGstOp9Oa6zFbbLTISYi28BGZDRUJKWeroECkCEkzXjUtbmmaKMfAx2RfbT69/cO+tgHcmx6AfyZOmj3NDIah0F0GB66d4CrdIoplNFFGHSpSheRxbo0W4S8azNItEoMWbw3uXAeJgCrmX5joz7CGXqSg6PcryEhnFr/C1C2ntPxBOYbdwY+8dO3+wZJyFlbMX9s8zNnvp/tLwAv03NB4j3HVpn8Awwm+GrlP6MVAAAAAElFTkSuQmCC MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Authentication-Result: IP=37.229.193.176; mail.from=fidaj@ukr.net; dkim=pass; header.d=ukr.net X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 Oct 2016 19:29:17 -0000 Next Oct 21 22:06:26 ns syslogd: kernel boot file is /boot/kernel/kernel Oct 21 22:06:26 ns kernel: panic: sbsndptr: sockbuf 0xfffff80201038518 and mbuf 0xfffff802820a1200 clashing Oct 21 22:06:26 ns kernel: cpuid = 3 Oct 21 22:06:26 ns kernel: KDB: stack backtrace: Oct 21 22:06:26 ns kernel: #0 0xffffffff80b5f0e7 at kdb_backtrace+0x67 Oct 21 22:06:26 ns kernel: #1 0xffffffff80b129c2 at vpanic+0x182 Oct 21 22:06:26 ns kernel: #2 0xffffffff80b12833 at panic+0x43 Oct 21 22:06:26 ns kernel: #3 0xffffffff80bafc4a at sbsndptr+0xda Oct 21 22:06:26 ns kernel: #4 0xffffffff80d57b88 at tcp_output+0x1168 Oct 21 22:06:26 ns kernel: #5 0xffffffff80d540f6 at tcp_do_segment+0x30d6 Oct 21 22:06:26 ns kernel: #6 0xffffffff80d508b6 at tcp_input+0x14a6 Oct 21 22:06:26 ns kernel: #7 0xffffffff80cb54be at ip_input+0x18e Oct 21 22:06:26 ns kernel: #8 0xffffffff80c49add at netisr_dispatch_src+0xad Oct 21 22:06:26 ns kernel: #9 0xffffffff80c3815e at tunwrite+0x2ee Oct 21 22:06:26 ns kernel: #10 0xffffffff809b54e7 at devfs_write_f+0xe7 Oct 21 22:06:26 ns kernel: #11 0xffffffff80b7e227 at dofilewrite+0x87 Oct 21 22:06:26 ns kernel: #12 0xffffffff80b7ddf9 at sys_write+0xd9 Oct 21 22:06:26 ns kernel: #13 0xffffffff8105ac2e at amd64_syscall+0x51e Oct 21 22:06:26 ns kernel: #14 0xffffffff8103bbcb at Xfast_syscall+0xfb Oct 21 22:06:26 ns kernel: Uptime: 37m25s Oct 21 22:06:26 ns kernel: Dumping 1500 out of 32688 MB:..2%..11%..21%..31%..41%..51%..61%..71%..82%..91% From owner-freebsd-stable@freebsd.org Sat Oct 22 03:37:06 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 96E45C1C8D5 for ; Sat, 22 Oct 2016 03:37:06 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D7172A86 for ; Sat, 22 Oct 2016 03:37:03 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id u9M3alkH081381; Sat, 22 Oct 2016 14:36:48 +1100 (EST) (envelope-from smithi@nimnet.asn.au) Date: Sat, 22 Oct 2016 14:36:47 +1100 (EST) From: Ian Smith To: "Patrick M. Hausen" cc: Warner Losh , freebsd-stable Subject: Re: boot0cfg on does not set default selection on gmirror device In-Reply-To: Message-ID: <20161022140755.S6806@sola.nimnet.asn.au> References: <14FD5FE6-6277-4EBE-8EE9-630A735F8BEA@punkt.de> <54B556F6-B834-4B0C-A9A4-90B9AD80A668@punkt.de> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Oct 2016 03:37:06 -0000 On Fri, 21 Oct 2016 20:47:20 +0200, Patrick M. Hausen wrote: Again, trouble quoting your message properly, so quotes by hand .. > I set the flag, then tried to change the slice from 1 to 2. > Result: [..] > root@hd45:~ # boot0cfg -v mirror/m0 > # flag start chs type end chs offset size > 1 0x80 1: 0: 1 0xa5 1022:254:63 16065 16418430 > 2 0x00 1023: 0: 1 0xa5 1020:254:63 16434495 16418430 > 3 0x00 1021: 0: 1 0xa5 768:254:63 32852925 1920667140 > > version=1.0 drive=0x80 mask=0xf ticks=182 bell= (0x7) > options=packet,update,nosetdrv > default_selection=F1 (Slice 1) > > So again, no change. In my previous message I said that the boot selection would be stored in 0x1b5. That was ASSuming we'd be looking at a version=2.0 boot0, which from the above is not the case. For version=1.0 that byte is at 0x1b9. Discombobulating the dump: 0x1b0: 66 bb 44 72 69 76 65 20 b1 01 80 8f b6 00 80 00 v2 v1 ^active 0x1c0: 01 01 a5 fe ff fe c1 3e 00 00 7e 86 fa 00 00 00 0x1d0: c1 ff a5 fe ff fc 3f c5 fa 00 7e 86 fa 00 00 00 0x1e0: c1 fd a5 fe ff 00 bd 4b f5 01 04 0e 7b 72 00 00 0x1f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 aa So 0x1b9 = 1, +1 = 2 (for F2). It appears correct in the dump but not in what boot0cfg then reports. I wonder two things: Do 'boot0cfg -v ada0' and 'boot0cfg -v ada1' both report the same? Might it work properly if you upgraded the boot sectors to version 2, which is what you should get if you reinstall from current boot0cfg, presumably without touching the MBR data, but you'll have backups .. Only noticing because I made a memstick with boot0 the other day from FreeBSD 9.3 and it showed version=2.0 with a (dummy) volume serial ID, the same time as finding that setting active with gpart had no effect. cheers, ian From owner-freebsd-stable@freebsd.org Sat Oct 22 05:51:11 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 63978C1C154 for ; Sat, 22 Oct 2016 05:51:11 +0000 (UTC) (envelope-from jakub_lach@mailplus.pl) Received: from mbob.nabble.com (mbob.nabble.com [162.253.133.15]) by mx1.freebsd.org (Postfix) with ESMTP id 52E2ADE3 for ; Sat, 22 Oct 2016 05:51:10 +0000 (UTC) (envelope-from jakub_lach@mailplus.pl) Received: from static.162.255.23.37.macminivault.com (unknown [162.255.23.37]) by mbob.nabble.com (Postfix) with ESMTP id 1F576343E161 for ; Fri, 21 Oct 2016 22:44:13 -0700 (PDT) Date: Fri, 21 Oct 2016 22:51:09 -0700 (MST) From: Jakub Lach To: freebsd-stable@freebsd.org Message-ID: <1477115469564-6138663.post@n6.nabble.com> Subject: moused(?) touchpad issue after updating to FreeBSD 11.0-STABLE #0 r307755 amd64 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Oct 2016 05:51:11 -0000 Namely, the touchpad ceased to simulate mouse buttons (doesn't respond to tapping). Any pointers? It must be recent, source from few days ago worked ok. -- View this message in context: http://freebsd.1045724.x6.nabble.com/moused-touchpad-issue-after-updating-to-FreeBSD-11-0-STABLE-0-r307755-amd64-tp6138663.html Sent from the freebsd-stable mailing list archive at Nabble.com. From owner-freebsd-stable@freebsd.org Sat Oct 22 06:04:02 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BE599C1C498 for ; Sat, 22 Oct 2016 06:04:02 +0000 (UTC) (envelope-from jakub_lach@mailplus.pl) Received: from mbob.nabble.com (mbob.nabble.com [162.253.133.15]) by mx1.freebsd.org (Postfix) with ESMTP id A510C31B for ; Sat, 22 Oct 2016 06:04:02 +0000 (UTC) (envelope-from jakub_lach@mailplus.pl) Received: from static.162.255.23.37.macminivault.com (unknown [162.255.23.37]) by mbob.nabble.com (Postfix) with ESMTP id 72459343E29A for ; Fri, 21 Oct 2016 22:57:05 -0700 (PDT) Date: Fri, 21 Oct 2016 23:04:01 -0700 (MST) From: Jakub Lach To: freebsd-stable@freebsd.org Message-ID: <1477116241463-6138664.post@n6.nabble.com> In-Reply-To: <1477115469564-6138663.post@n6.nabble.com> References: <1477115469564-6138663.post@n6.nabble.com> Subject: Re: moused(?) touchpad issue after updating to FreeBSD 11.0-STABLE #0 r307755 amd64 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 22 Oct 2016 06:04:02 -0000 I'm going afk for a couple days, all I've got so far from moused is- /dev/psm0 ps/2 sysmouse generic moused: proto params: f8 80 00 00 8 00 ff The laptop in question is Thinkpad T400. -- View this message in context: http://freebsd.1045724.x6.nabble.com/moused-touchpad-issue-after-updating-to-FreeBSD-11-0-STABLE-0-r307755-amd64-tp6138663p6138664.html Sent from the freebsd-stable mailing list archive at Nabble.com.