From owner-freebsd-fs@freebsd.org Mon Aug 10 19:46:05 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A527C99E70F for ; Mon, 10 Aug 2015 19:46:05 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 89C821F8 for ; Mon, 10 Aug 2015 19:46:05 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [IPv6:::1]) by freefall.freebsd.org (Postfix) with ESMTP id 7E7AF1FAA for ; Mon, 10 Aug 2015 19:46:05 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [172.31.3.2]) by mail.xzibition.com (Postfix) with ESMTP id 3A20D10060 for ; Mon, 10 Aug 2015 19:46:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at mail.xzibition.com Received: from mail.xzibition.com ([172.31.3.2]) by mail.xzibition.com (mail.xzibition.com [172.31.3.2]) (amavisd-new, port 10026) with LMTP id kyALYtf4hHyu for ; Mon, 10 Aug 2015 19:46:02 +0000 (UTC) Subject: Re: solaris assert: t == ZIO_TYPE_READ || t == ZIO_TYPE_WRITE, file: ...zfs/vdev_queue.c, line: 302 DKIM-Filter: OpenDKIM Filter v2.9.2 mail.xzibition.com F2E1E1005A To: freebsd-fs@freebsd.org References: <20150810194122.GE48727@vega.codepro.be> From: Bryan Drewery Organization: FreeBSD Message-ID: <55C8FF76.7060202@FreeBSD.org> Date: Mon, 10 Aug 2015 12:45:58 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <20150810194122.GE48727@vega.codepro.be> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Aug 2015 19:46:05 -0000 On 8/10/15 12:41 PM, Kristof Provost wrote: > Hi, > > Booting CURRENT (r286582) earlier I saw the following panic: > panic: solaris assert: t == ZIO_TYPE_READ || t == ZIO_TYPE_WRITE, file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c, line: 302 > cpuid = 1 > Uptime: 41s > > ... > (kgdb) #0 doadump (textdump=1) at pcpu.h:221 > #1 0xffffffff80a19e65 in kern_reboot (howto=260) > at /usr/src/sys/kern/kern_shutdown.c:329 > #2 0xffffffff80a1a458 in vpanic (fmt=, > ap=) at /usr/src/sys/kern/kern_shutdown.c:626 > #3 0xffffffff80a1a4a3 in panic (fmt=0x0) > at /usr/src/sys/kern/kern_shutdown.c:557 > #4 0xffffffff8242622a in assfail (a=, > f=, l=) > at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81 > #5 0xffffffff8213f8be in vdev_queue_io (zio=0xfffff8000ede1000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:302 > #6 0xffffffff82163406 in zio_vdev_io_start (zio=0xfffff8000ede1000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2774 > #7 0xffffffff8215eaa9 in zio_execute (zio=0xfffff8000ede1000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1496 > #8 0xffffffff8215de5b in zio_nowait (zio=0xfffff8000ede1000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1550 > #9 0xffffffff8219f23c in trim_map_commit (spa=0xfffffe00032b7000, > zio=0xfffff8000eda6760, vd=0xfffff8000e7dd800) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c:476 > #10 0xffffffff8219f046 in trim_map_commit (spa=0xfffffe00032b7000, > zio=0xfffff8000eda6760, vd=0xfffff8000e9b4800) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c:532 > #11 0xffffffff8219f046 in trim_map_commit (spa=0xfffffe00032b7000, > zio=0xfffff8000eda6760, vd=0xfffff8000e9b4000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c:532 > #12 0xffffffff8219ed9f in trim_thread (arg=0xfffffe00032b7000) > at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c:579 > #13 0xffffffff809e0374 in fork_exit ( > callout=0xffffffff8219ecb0 , arg=0xfffffe00032b7000, > frame=0xfffffe0239c09c00) at /usr/src/sys/kern/kern_fork.c:1006 > #14 0xffffffff80e5e7ae in fork_trampoline () > at /usr/src/sys/amd64/amd64/exception.S:610 > #15 0x0000000000000000 in ?? () > > This is a regular amd64 machine with 4 disks in raidz. I'd pulled one drive and > replaced it with an empty one. It's possible that's related to the panic. > Same here... panic: solaris assert: t == ZIO_TYPE_READ || t == ZIO_TYPE_WRITE, file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c, line: 302 cpuid = 1 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe35545c6780 vpanic() at vpanic+0x189/frame 0xfffffe35545c6800 panic() at panic+0x43/frame 0xfffffe35545c6860 assfail() at assfail+0x1a/frame 0xfffffe35545c6870 vdev_queue_io() at vdev_queue_io+0x17e/frame 0xfffffe35545c68c0 zio_vdev_io_start() at zio_vdev_io_start+0x446/frame 0xfffffe35545c6920 zio_execute() at zio_execute+0x2e9/frame 0xfffffe35545c6970 zio_nowait() at zio_nowait+0x6b/frame 0xfffffe35545c6990 trim_map_commit() at trim_map_commit+0x2dc/frame 0xfffffe35545c6a30 trim_map_commit() at trim_map_commit+0xe6/frame 0xfffffe35545c6ad0 trim_map_commit() at trim_map_commit+0xe6/frame 0xfffffe35545c6b70 trim_thread() at trim_thread+0xef/frame 0xfffffe35545c6bb0 fork_exit() at fork_exit+0x84/frame 0xfffffe35545c6bf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe35545c6bf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic [ thread pid 6 tid 100684 ] Stopped at kdb_enter+0x3e: movq $0,kdb_why 2 disk mirror + an SSD log/cache. I have no way to debug it currently, rolling back is hard enough on this system. -- Regards, Bryan Drewery