From owner-freebsd-fs@freebsd.org Mon Aug 10 20:01:36 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3A40799EC4E for ; Mon, 10 Aug 2015 20:01:36 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 2658B225 for ; Mon, 10 Aug 2015 20:01:36 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [IPv6:::1]) by freefall.freebsd.org (Postfix) with ESMTP id 1AEBB1838 for ; Mon, 10 Aug 2015 20:01:36 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [172.31.3.2]) by mail.xzibition.com (Postfix) with ESMTP id B65231010C for ; Mon, 10 Aug 2015 20:01:35 +0000 (UTC) X-Virus-Scanned: amavisd-new at mail.xzibition.com Received: from mail.xzibition.com ([172.31.3.2]) by mail.xzibition.com (mail.xzibition.com [172.31.3.2]) (amavisd-new, port 10026) with LMTP id Y3Z5O6wWydBf for ; Mon, 10 Aug 2015 20:01:33 +0000 (UTC) Subject: Re: solaris assert: t == ZIO_TYPE_READ || t == ZIO_TYPE_WRITE, file: ...zfs/vdev_queue.c, line: 302 DKIM-Filter: OpenDKIM Filter v2.9.2 mail.xzibition.com CAF3710103 To: freebsd-fs@freebsd.org References: <20150810194122.GE48727@vega.codepro.be> <55C8FF76.7060202@FreeBSD.org> From: Bryan Drewery Organization: FreeBSD Message-ID: <55C9031C.60901@FreeBSD.org> Date: Mon, 10 Aug 2015 13:01:32 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <55C8FF76.7060202@FreeBSD.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Aug 2015 20:01:36 -0000 On 8/10/15 12:45 PM, Bryan Drewery wrote: > On 8/10/15 12:41 PM, Kristof Provost wrote: >> Hi, >> >> Booting CURRENT (r286582) earlier I saw the following panic: >> panic: solaris assert: t == ZIO_TYPE_READ || t == ZIO_TYPE_WRITE, file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c, line: 302 >> cpuid = 1 >> Uptime: 41s >> >> ... >> (kgdb) #0 doadump (textdump=1) at pcpu.h:221 >> #1 0xffffffff80a19e65 in kern_reboot (howto=260) >> at /usr/src/sys/kern/kern_shutdown.c:329 >> #2 0xffffffff80a1a458 in vpanic (fmt=, >> ap=) at /usr/src/sys/kern/kern_shutdown.c:626 >> #3 0xffffffff80a1a4a3 in panic (fmt=0x0) >> at /usr/src/sys/kern/kern_shutdown.c:557 >> #4 0xffffffff8242622a in assfail (a=, >> f=, l=) >> at /usr/src/sys/cddl/compat/opensolaris/kern/opensolaris_cmn_err.c:81 >> #5 0xffffffff8213f8be in vdev_queue_io (zio=0xfffff8000ede1000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c:302 >> #6 0xffffffff82163406 in zio_vdev_io_start (zio=0xfffff8000ede1000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:2774 >> #7 0xffffffff8215eaa9 in zio_execute (zio=0xfffff8000ede1000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1496 >> #8 0xffffffff8215de5b in zio_nowait (zio=0xfffff8000ede1000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c:1550 >> #9 0xffffffff8219f23c in trim_map_commit (spa=0xfffffe00032b7000, >> zio=0xfffff8000eda6760, vd=0xfffff8000e7dd800) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c:476 >> #10 0xffffffff8219f046 in trim_map_commit (spa=0xfffffe00032b7000, >> zio=0xfffff8000eda6760, vd=0xfffff8000e9b4800) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c:532 >> #11 0xffffffff8219f046 in trim_map_commit (spa=0xfffffe00032b7000, >> zio=0xfffff8000eda6760, vd=0xfffff8000e9b4000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c:532 >> #12 0xffffffff8219ed9f in trim_thread (arg=0xfffffe00032b7000) >> at /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c:579 >> #13 0xffffffff809e0374 in fork_exit ( >> callout=0xffffffff8219ecb0 , arg=0xfffffe00032b7000, >> frame=0xfffffe0239c09c00) at /usr/src/sys/kern/kern_fork.c:1006 >> #14 0xffffffff80e5e7ae in fork_trampoline () >> at /usr/src/sys/amd64/amd64/exception.S:610 >> #15 0x0000000000000000 in ?? () >> >> This is a regular amd64 machine with 4 disks in raidz. I'd pulled one drive and >> replaced it with an empty one. It's possible that's related to the panic. >> > > > Same here... > > panic: solaris assert: t == ZIO_TYPE_READ || t == ZIO_TYPE_WRITE, file: > /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_queue.c, > line: 302 > cpuid = 1 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe35545c6780 > vpanic() at vpanic+0x189/frame 0xfffffe35545c6800 > panic() at panic+0x43/frame 0xfffffe35545c6860 > assfail() at assfail+0x1a/frame 0xfffffe35545c6870 > vdev_queue_io() at vdev_queue_io+0x17e/frame 0xfffffe35545c68c0 > zio_vdev_io_start() at zio_vdev_io_start+0x446/frame 0xfffffe35545c6920 > zio_execute() at zio_execute+0x2e9/frame 0xfffffe35545c6970 > zio_nowait() at zio_nowait+0x6b/frame 0xfffffe35545c6990 > trim_map_commit() at trim_map_commit+0x2dc/frame 0xfffffe35545c6a30 > trim_map_commit() at trim_map_commit+0xe6/frame 0xfffffe35545c6ad0 > trim_map_commit() at trim_map_commit+0xe6/frame 0xfffffe35545c6b70 > trim_thread() at trim_thread+0xef/frame 0xfffffe35545c6bb0 > fork_exit() at fork_exit+0x84/frame 0xfffffe35545c6bf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe35545c6bf0 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > KDB: enter: panic > [ thread pid 6 tid 100684 ] > Stopped at kdb_enter+0x3e: movq $0,kdb_why > > > 2 disk mirror + an SSD log/cache. > > I have no way to debug it currently, rolling back is hard enough on this > system. > > Booting with vfs.zfs.trim.enabled=0 seems to work fine as a workaround until mav@ has a fix. -- Regards, Bryan Drewery