From owner-freebsd-fs@FreeBSD.ORG Sun May 19 20:41:04 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DCF75863 for ; Sun, 19 May 2013 20:41:04 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: from mail-we0-x232.google.com (mail-we0-x232.google.com [IPv6:2a00:1450:400c:c03::232]) by mx1.freebsd.org (Postfix) with ESMTP id 7C73F949 for ; Sun, 19 May 2013 20:41:04 +0000 (UTC) Received: by mail-we0-f178.google.com with SMTP id q57so4978772wes.9 for ; Sun, 19 May 2013 13:41:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=T3YbiISdMsBj2Bbx7HYzPm8AB6maWIA9FDXQX3zRjgw=; b=e2hGLqEsLwmRvKj4OXOKobLl8FanLkRYfSLWHYw8jNMPxfb60rflXdh9BM9Ls8ZUVl gK033E2QKnMcjG+Bfeyqs6nbIeNSEkxMWI1Z/Mu0yJ+by9qty33Ng7yVMAva3R6XnrtU aLaIxj73vLdpV9sT+ZC97lu34U9TkxZTrhFQFESfIIi16feCpUSeaRcfXMle2QL6M5T/ F/cIpLQ/SbuvsKiSfTkQ8WEb5qLWRcLwx/Zjtuk2nuUNtRsA0xrkJhgWhQ4EEokE0LdS LQ7JbHBg4KLb2xLrUcefhq3p7Lz0usRJewhEFu4Clc0LxNuqpcuF1pJy+faV2DomfDn8 kwsw== MIME-Version: 1.0 X-Received: by 10.180.37.229 with SMTP id b5mr7871871wik.29.1368996063519; Sun, 19 May 2013 13:41:03 -0700 (PDT) Sender: antoine.brodin.freebsd@gmail.com Received: by 10.194.91.143 with HTTP; Sun, 19 May 2013 13:41:03 -0700 (PDT) Date: Sun, 19 May 2013 20:41:03 +0000 X-Google-Sender-Auth: gctiPMoQgDDm0QhxgtfWo2p3F-w Message-ID: Subject: Nullfs - Panic when umounting and lower fs has been removed From: Antoine Brodin To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 May 2013 20:41:04 -0000 Hi there, I encountered a panic on head today with nullfs using poudriere. It is quite easy to reproduce, the script is probably doing something wrong but panicking is not nice %%% mkdir /tmp/.new_packages /mnt/new_packages mount -t nullfs /tmp/.new_packages /mnt/new_packages dd if=/dev/zero of=/mnt/new_packages/bar count=20000 mv /tmp/.new_packages/bar /tmp/ rm -rf /tmp/.new_packages umount -f /mnt/new_packages %%% The panic: userret: returning with the following locks held: exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0010a77548) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:615 panic: witness_warn cpuid = 0 KDB: enter: panic db:1:lockinfo> show locks exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0010a77548) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:615 db:1:locks> show alllocks Process 1032 (umount) thread 0xfffffe0010623920 (100116) exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0010a77548) locked @ /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:615 db:1:alllocks> show lockedvnods Locked vnodes 0xfffffe0010a774e0: tag ufs, type VDIR usecount 0, writecount 0, refcount 0 mountedhere 0 flags (VI_FREE) v_object 0xfffffe0010af0960 ref 0 pages 0 #0 0xffffffff8088cd59 at __lockmgr_args+0x979 #1 0xffffffff80af4df2 at ffs_lock+0x92 #2 0xffffffff80ce96f9 at VOP_LOCK1_APV+0xf9 #3 0xffffffff81a5f8df at null_lock+0xdf #4 0xffffffff80ce96f9 at VOP_LOCK1_APV+0xf9 #5 0xffffffff8095a836 at _vn_lock+0xc6 #6 0xffffffff8094b681 at vflush+0x3a1 #7 0xffffffff81a5eace at nullfs_unmount+0x2e #8 0xffffffff809436cb at dounmount+0x39b #9 0xffffffff8094330b at sys_unmount+0x37b #10 0xffffffff80c41322 at amd64_syscall+0x282 #11 0xffffffff80c2981b at Xfast_syscall+0xfb db:0:kdb.enter.panic> bt Tracing pid 1032 tid 100116 td 0xfffffe0010623920 kdb_enter() at kdb_enter+0x3e/frame 0xffffff8122567940 vpanic() at vpanic+0x146/frame 0xffffff8122567980 kassert_panic() at kassert_panic+0x136/frame 0xffffff81225679f0 witness_warn() at witness_warn+0x4e4/frame 0xffffff8122567ab0 userret() at userret+0x89/frame 0xffffff8122567ae0 amd64_syscall() at amd64_syscall+0x397/frame 0xffffff8122567bf0 Xfast_syscall() at Xfast_syscall+0xfb/frame 0xffffff8122567bf0 --- syscall (0, FreeBSD ELF64, nosys), rip = 0x80087abcc, rsp = 0x7fffffffd398, rbp = 0x800c0a9d0 --- Cheers, Antoine From owner-freebsd-fs@FreeBSD.ORG Sun May 19 23:47:27 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4D4AB555; Sun, 19 May 2013 23:47:27 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: from mail-wi0-x22e.google.com (mail-wi0-x22e.google.com [IPv6:2a00:1450:400c:c05::22e]) by mx1.freebsd.org (Postfix) with ESMTP id B5582E90; Sun, 19 May 2013 23:47:26 +0000 (UTC) Received: by mail-wi0-f174.google.com with SMTP id c10so1630619wiw.1 for ; Sun, 19 May 2013 16:47:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=oOf9moy1ztdnPXpuMr8YEyl6Vu7otMhV6zXGpMSamUU=; b=ntO+m+TGba/fthjiWrkWbo7qA60Vt3a+rfYhyj2yFed+ad3IE8vE1tKUiDj/luWX22 GPj0cx5bYhq89cbfiDpNHz98SgvtRJ7nc5TJh9/prTsGaQVCHaZGWOvCPKkcOkaYIvYN eG0wecVYZpXqUjkdY/3q0ekMVfgvwzolOZ7uLE+tB+QMKA191kW1T7ld7sl42NdQEssI E6NQLuRegW31SGPTjcJuzGIHOqKNxx0/wMQAckxWqWsN9NgY9ElpjxZl6u100XZVkPqV rfZSz/74AhwtafyAGVNR7Zeio4Sy0hTe3Rnk+qBARIPPxZL/rpFXyfanC+kHRrqOa/EV dcqA== MIME-Version: 1.0 X-Received: by 10.180.21.167 with SMTP id w7mr8589194wie.2.1369007245760; Sun, 19 May 2013 16:47:25 -0700 (PDT) Sender: antoine.brodin.freebsd@gmail.com Received: by 10.194.91.143 with HTTP; Sun, 19 May 2013 16:47:25 -0700 (PDT) In-Reply-To: References: Date: Sun, 19 May 2013 23:47:25 +0000 X-Google-Sender-Auth: Dy6zgy6vvT7D1k4LCTR0neY_pL0 Message-ID: Subject: Re: Nullfs - Panic when umounting and lower fs has been removed From: Antoine Brodin To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Cc: Konstantin Belousov X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 19 May 2013 23:47:27 -0000 On Sun, May 19, 2013 at 8:41 PM, Antoine Brodin wrote: > Hi there, > > I encountered a panic on head today with nullfs using poudriere. > > It is quite easy to reproduce, the script is probably doing something > wrong but panicking is not nice > > %%% > mkdir /tmp/.new_packages /mnt/new_packages > mount -t nullfs /tmp/.new_packages /mnt/new_packages > dd if=/dev/zero of=/mnt/new_packages/bar count=20000 > mv /tmp/.new_packages/bar /tmp/ > rm -rf /tmp/.new_packages > umount -f /mnt/new_packages > %%% > > The panic: > > userret: returning with the following locks held: > exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0010a77548) locked @ > /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:615 > panic: witness_warn > cpuid = 0 > KDB: enter: panic > > db:1:lockinfo> show locks > exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0010a77548) locked @ > /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:615 > > db:1:locks> show alllocks > Process 1032 (umount) thread 0xfffffe0010623920 (100116) > exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0010a77548) locked @ > /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:615 > > db:1:alllocks> show lockedvnods > Locked vnodes > 0xfffffe0010a774e0: tag ufs, type VDIR > usecount 0, writecount 0, refcount 0 mountedhere 0 > flags (VI_FREE) > v_object 0xfffffe0010af0960 ref 0 pages 0 > #0 0xffffffff8088cd59 at __lockmgr_args+0x979 > #1 0xffffffff80af4df2 at ffs_lock+0x92 > #2 0xffffffff80ce96f9 at VOP_LOCK1_APV+0xf9 > #3 0xffffffff81a5f8df at null_lock+0xdf > #4 0xffffffff80ce96f9 at VOP_LOCK1_APV+0xf9 > #5 0xffffffff8095a836 at _vn_lock+0xc6 > #6 0xffffffff8094b681 at vflush+0x3a1 > #7 0xffffffff81a5eace at nullfs_unmount+0x2e > #8 0xffffffff809436cb at dounmount+0x39b > #9 0xffffffff8094330b at sys_unmount+0x37b > #10 0xffffffff80c41322 at amd64_syscall+0x282 > #11 0xffffffff80c2981b at Xfast_syscall+0xfb > > db:0:kdb.enter.panic> bt > Tracing pid 1032 tid 100116 td 0xfffffe0010623920 > kdb_enter() at kdb_enter+0x3e/frame 0xffffff8122567940 > vpanic() at vpanic+0x146/frame 0xffffff8122567980 > kassert_panic() at kassert_panic+0x136/frame 0xffffff81225679f0 > witness_warn() at witness_warn+0x4e4/frame 0xffffff8122567ab0 > userret() at userret+0x89/frame 0xffffff8122567ae0 > amd64_syscall() at amd64_syscall+0x397/frame 0xffffff8122567bf0 > Xfast_syscall() at Xfast_syscall+0xfb/frame 0xffffff8122567bf0 > --- syscall (0, FreeBSD ELF64, nosys), rip = 0x80087abcc, rsp = > 0x7fffffffd398, rbp = 0x800c0a9d0 --- Reverting r250505 fixes this issue. Cheers, Antoine From owner-freebsd-fs@FreeBSD.ORG Mon May 20 03:29:13 2013 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6C577390; Mon, 20 May 2013 03:29:13 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 474DD947; Mon, 20 May 2013 03:29:13 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4K3TDXI081510; Mon, 20 May 2013 03:29:13 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4K3TDHh081509; Mon, 20 May 2013 03:29:13 GMT (envelope-from linimon) Date: Mon, 20 May 2013 03:29:13 GMT Message-Id: <201305200329.r4K3TDHh081509@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/178238: [nullfs] nullfs don't release i-nodes on unlink. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 03:29:13 -0000 Old Synopsis: nullfs don't release i-nodes on unlink. New Synopsis: [nullfs] nullfs don't release i-nodes on unlink. Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Mon May 20 03:28:53 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=178238 From owner-freebsd-fs@FreeBSD.ORG Mon May 20 03:29:35 2013 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 26294459; Mon, 20 May 2013 03:29:35 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id F40B0959; Mon, 20 May 2013 03:29:34 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4K3TYCd081560; Mon, 20 May 2013 03:29:34 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4K3TY3r081559; Mon, 20 May 2013 03:29:34 GMT (envelope-from linimon) Date: Mon, 20 May 2013 03:29:34 GMT Message-Id: <201305200329.r4K3TY3r081559@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/178231: [nfs] 8.3 nfsv4 client reports "nfsv4 client/server protocol prob err=10026" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 03:29:35 -0000 Old Synopsis: 8.3 nfsv4 client reports "nfsv4 client/server protocol prob err=10026" New Synopsis: [nfs] 8.3 nfsv4 client reports "nfsv4 client/server protocol prob err=10026" Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Mon May 20 03:29:21 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=178231 From owner-freebsd-fs@FreeBSD.ORG Mon May 20 03:30:50 2013 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id DDCA153E; Mon, 20 May 2013 03:30:50 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id B7DA8978; Mon, 20 May 2013 03:30:50 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4K3UorL083153; Mon, 20 May 2013 03:30:50 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4K3UoWP083152; Mon, 20 May 2013 03:30:50 GMT (envelope-from linimon) Date: Mon, 20 May 2013 03:30:50 GMT Message-Id: <201305200330.r4K3UoWP083152@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/178103: [kernel] [nfs] [patch] Correct support of index files for WebNFS exports X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 03:30:50 -0000 Old Synopsis: Correct support of index files for WebNFS exports New Synopsis: [kernel] [nfs] [patch] Correct support of index files for WebNFS exports Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Mon May 20 03:29:52 UTC 2013 Responsible-Changed-Why: Involves NFS, but also a kernel and binary change. Perhaps the NFS folks can review. http://www.freebsd.org/cgi/query-pr.cgi?pr=178103 From owner-freebsd-fs@FreeBSD.ORG Mon May 20 03:32:55 2013 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3E54F706; Mon, 20 May 2013 03:32:55 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 177D69BE; Mon, 20 May 2013 03:32:55 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4K3WsSH083291; Mon, 20 May 2013 03:32:54 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4K3WsBS083290; Mon, 20 May 2013 03:32:54 GMT (envelope-from linimon) Date: Mon, 20 May 2013 03:32:54 GMT Message-Id: <201305200332.r4K3WsBS083290@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/177335: [nfs] [panic] Sleeping on "vmopar" with the following non-sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode lock) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 03:32:55 -0000 Old Synopsis: Sleeping on "vmopar" with the following non-sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode lock) New Synopsis: [nfs] [panic] Sleeping on "vmopar" with the following non-sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode lock) Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Mon May 20 03:32:02 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=177335 From owner-freebsd-fs@FreeBSD.ORG Mon May 20 04:55:10 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C85E6E7C for ; Mon, 20 May 2013 04:55:10 +0000 (UTC) (envelope-from maxwellahmed_maxwell01@jumpy.it) Received: from ns2283677.ovh.net (ns2283677.ovh.net [5.39.73.124]) by mx1.freebsd.org (Postfix) with ESMTP id 33C8DC10 for ; Mon, 20 May 2013 04:55:09 +0000 (UTC) Received: from [5.39.73.124] by ns2283677.ovh.net id qJwUOPZHWxjL with SMTP; Mon, 20 May 2013 06:55:09 +0200 Date: Mon, 20 May 2013 06:55:09 +0200 From: "Angelina" X-Mailer: The Bat! (v3.09.8) Personal X-Priority: 3 (Normal) Message-ID: <50089358.86344649085335@ns2283677.ovh.net> To: Subject: Re: Rispondere MIME-Version: 1.0 Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 04:55:10 -0000 Gentile utente, Monte Biz In risposta alla Sua richiesta, siamo lieti di inviarvi informazioni dettagliate sui servizi http://www.foto-kirch.de/BestFoto.zip Id 531639 . From owner-freebsd-fs@FreeBSD.ORG Mon May 20 07:34:35 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 26786E62; Mon, 20 May 2013 07:34:35 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) by mx1.freebsd.org (Postfix) with ESMTP id BE6B6155; Mon, 20 May 2013 07:34:34 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.7/8.14.7) with ESMTP id r4K7YRR4053009; Mon, 20 May 2013 10:34:27 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.8.3 kib.kiev.ua r4K7YRR4053009 Received: (from kostik@localhost) by tom.home (8.14.7/8.14.7/Submit) id r4K7YQ6t053008; Mon, 20 May 2013 10:34:26 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 20 May 2013 10:34:26 +0300 From: Konstantin Belousov To: Antoine Brodin Subject: Re: Nullfs - Panic when umounting and lower fs has been removed Message-ID: <20130520073426.GY3047@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="S41ebxBz2Zis128g" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on tom.home Cc: freebsd-fs@freebsd.org, pho@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 07:34:35 -0000 --S41ebxBz2Zis128g Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, May 19, 2013 at 08:41:03PM +0000, Antoine Brodin wrote: > Hi there, >=20 > I encountered a panic on head today with nullfs using poudriere. >=20 > It is quite easy to reproduce, the script is probably doing something > wrong but panicking is not nice >=20 > %%% > mkdir /tmp/.new_packages /mnt/new_packages > mount -t nullfs /tmp/.new_packages /mnt/new_packages > dd if=3D/dev/zero of=3D/mnt/new_packages/bar count=3D20000 > mv /tmp/.new_packages/bar /tmp/ > rm -rf /tmp/.new_packages > umount -f /mnt/new_packages > %%% >=20 > The panic: >=20 > userret: returning with the following locks held: > exclusive lockmgr ufs (ufs) r =3D 0 (0xfffffe0010a77548) locked @ > /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:615 > panic: witness_warn > cpuid =3D 0 > KDB: enter: panic This is strange, do you have witness enabled, but invariants not ? Please try the following change. diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c index ad02236..bece8c8 100644 --- a/sys/fs/nullfs/null_vfsops.c +++ b/sys/fs/nullfs/null_vfsops.c @@ -417,8 +417,13 @@ nullfs_unlink_lowervp(struct mount *mp, struct vnode *= lowervp) */ if (vp->v_usecount =3D=3D 0) { KASSERT((vp->v_iflag & VI_DOOMED) !=3D 0, - ("not reclaimed %p", vp)); + ("not reclaimed nullfs vnode %p", vp)); VOP_UNLOCK(vp, 0); + } else { + ASSERT_VOP_ELOCKED(vp, "unlink_lowervp"); + KASSERT((vp->v_iflag & VI_DOOMED) =3D=3D 0, + ("reclaimed nullfs vnode %p", vp)); + xp->null_flags &=3D ~NULLV_NOUNLOCK; } vdrop(vp); } --S41ebxBz2Zis128g Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.20 (FreeBSD) iQIcBAEBAgAGBQJRmdICAAoJEJDCuSvBvK1BwHIP/3Z4EqI3XBMr1woGPwv4rBw0 z4qg6XLfLbHL8E4q2CQiRvBsdKCyZ3plI7ujwbqXILGsA7lAa+kguKa22vtSq0rn DPzvoLvzXWmj7iFnzxahPIqnORdat2+0Wx9GanGoQmbDL4s2YvlyBoqES/6wUHS5 ccZpSvSYH1xrdLLr1KB4i6RemG6L3TZ5xRJLG8cf7UVkMq8rbobyylxUpAG4QO9s EbTS7tjCsobZXdWWN/xAS59bfFdRcAqkc1QYAODSBiCeTLgLX62l2VVu4JgoxMzw 4jbabey3dJhUAHpfbzRevQ3TFB1uIpXMDefWHFwgUDO40g8Wo16ASY+LxKKT/tKS W0/5fZGAetCt+u+lTKuArwSZ82voWHnmU7oMKCHfdG1gWCtLCmt9IEzKwND4VeAt jtA7/kZob6dKFND4SNxXGnYrQG29l7r5MFytjkQVB2q7e5Mn+H6tOqP0z37e5KXp cdThGzRh4AQobwZ/8iBczY3/LKrI7Srj94QbGAaS26KRZF2tmSvXROpg0QQ0hsC5 xHrB7VKJiFacUdgkynnPvhYqPYtrP7YpODu1D6wuqGARPZWJEC62NHwRdNTcvBui +rZ4dKILgcwsktfRTBmVk9KM0hwLK7yLIg1U/ZfXJOW2glUrgGaGGqpkt0u1JSBZ jHo0iLRgdSZr8nz71pmt =Jaw2 -----END PGP SIGNATURE----- --S41ebxBz2Zis128g-- From owner-freebsd-fs@FreeBSD.ORG Mon May 20 08:54:38 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id EE248B94 for ; Mon, 20 May 2013 08:54:38 +0000 (UTC) (envelope-from pho@holm.cc) Received: from relay01.pair.com (relay01.pair.com [209.68.5.15]) by mx1.freebsd.org (Postfix) with SMTP id A883A3FC for ; Mon, 20 May 2013 08:54:38 +0000 (UTC) Received: (qmail 98765 invoked from network); 20 May 2013 08:54:27 -0000 Received: from 87.58.146.155 (HELO x2.osted.lan) (87.58.146.155) by relay01.pair.com with SMTP; 20 May 2013 08:54:27 -0000 X-pair-Authenticated: 87.58.146.155 Received: from x2.osted.lan (localhost [127.0.0.1]) by x2.osted.lan (8.14.5/8.14.5) with ESMTP id r4K8sRv7024379; Mon, 20 May 2013 10:54:27 +0200 (CEST) (envelope-from pho@x2.osted.lan) Received: (from pho@localhost) by x2.osted.lan (8.14.5/8.14.5/Submit) id r4K8sR7G024378; Mon, 20 May 2013 10:54:27 +0200 (CEST) (envelope-from pho) Date: Mon, 20 May 2013 10:54:27 +0200 From: Peter Holm To: Konstantin Belousov Subject: Re: Nullfs - Panic when umounting and lower fs has been removed Message-ID: <20130520085427.GA24287@x2.osted.lan> References: <20130520073426.GY3047@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20130520073426.GY3047@kib.kiev.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org, Antoine Brodin X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 08:54:39 -0000 On Mon, May 20, 2013 at 10:34:26AM +0300, Konstantin Belousov wrote: > On Sun, May 19, 2013 at 08:41:03PM +0000, Antoine Brodin wrote: > > Hi there, > > > > I encountered a panic on head today with nullfs using poudriere. > > > > It is quite easy to reproduce, the script is probably doing something > > wrong but panicking is not nice > > > > %%% > > mkdir /tmp/.new_packages /mnt/new_packages > > mount -t nullfs /tmp/.new_packages /mnt/new_packages > > dd if=/dev/zero of=/mnt/new_packages/bar count=20000 > > mv /tmp/.new_packages/bar /tmp/ > > rm -rf /tmp/.new_packages > > umount -f /mnt/new_packages > > %%% > > > > The panic: > > > > userret: returning with the following locks held: > > exclusive lockmgr ufs (ufs) r = 0 (0xfffffe0010a77548) locked @ > > /usr/src/sys/modules/nullfs/../../fs/nullfs/null_vnops.c:615 > > panic: witness_warn > > cpuid = 0 > > KDB: enter: panic > > This is strange, do you have witness enabled, but invariants not ? > > Please try the following change. > > diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c > index ad02236..bece8c8 100644 > --- a/sys/fs/nullfs/null_vfsops.c I can confirm the panic http://people.freebsd.org/~pho/stress/log/kostik567.txt and that the patch fixes that problem. - Peter From owner-freebsd-fs@FreeBSD.ORG Mon May 20 09:05:08 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 2A590B6; Mon, 20 May 2013 09:05:08 +0000 (UTC) (envelope-from antoine.brodin.freebsd@gmail.com) Received: from mail-wi0-x233.google.com (mail-wi0-x233.google.com [IPv6:2a00:1450:400c:c05::233]) by mx1.freebsd.org (Postfix) with ESMTP id 92AFE644; Mon, 20 May 2013 09:05:07 +0000 (UTC) Received: by mail-wi0-f179.google.com with SMTP id hq7so1866478wib.0 for ; Mon, 20 May 2013 02:05:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; bh=WdRSkbd1tuTQdeo1HFHmNWxJI1+yzeJE7re0i1zKpQ4=; b=gZb2rifyLfskSvgtiXMStkWM0OT2Fjx7b/iuJ/bWfaTPULYWvFl3UT/GEsrhVD69N3 Odgvx1XMGaVOEMtSaw92gDzRM7UY3kFbhorLlYiny3mEq9KxU5+USU/oEdZpek6cy+AD LU7lxKr51gMHBwditzfWswUC921O1o6XHF+wPmE/1SGWQDVIKQzhBAO2J0Wj/wgylK4j R7TsBDrJmNLQyt+C+Brp6cdX6hJDS2FALPut5QZwuUamZ5wtthjhjbSGTOSS47oZoQcZ Fx4ElHQ1gNYMsuRaeRvWw4mgOzNys7DwOmgAaGFbfQ8nfifwconJV8WRjmwAgX+PSZMM pYcA== MIME-Version: 1.0 X-Received: by 10.180.21.167 with SMTP id w7mr11833635wie.2.1369040706804; Mon, 20 May 2013 02:05:06 -0700 (PDT) Sender: antoine.brodin.freebsd@gmail.com Received: by 10.194.91.143 with HTTP; Mon, 20 May 2013 02:05:06 -0700 (PDT) In-Reply-To: <20130520073426.GY3047@kib.kiev.ua> References: <20130520073426.GY3047@kib.kiev.ua> Date: Mon, 20 May 2013 09:05:06 +0000 X-Google-Sender-Auth: kpWw6tq5ffhqVKbEEydQxgrQnLM Message-ID: Subject: Re: Nullfs - Panic when umounting and lower fs has been removed From: Antoine Brodin To: Konstantin Belousov Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-fs@freebsd.org, pho@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 09:05:08 -0000 On Mon, May 20, 2013 at 7:34 AM, Konstantin Belousov wrote: > This is strange, do you have witness enabled, but invariants not ? > > Please try the following change. > > diff --git a/sys/fs/nullfs/null_vfsops.c b/sys/fs/nullfs/null_vfsops.c > index ad02236..bece8c8 100644 > --- a/sys/fs/nullfs/null_vfsops.c > +++ b/sys/fs/nullfs/null_vfsops.c > @@ -417,8 +417,13 @@ nullfs_unlink_lowervp(struct mount *mp, struct vnode *lowervp) > */ > if (vp->v_usecount == 0) { > KASSERT((vp->v_iflag & VI_DOOMED) != 0, > - ("not reclaimed %p", vp)); > + ("not reclaimed nullfs vnode %p", vp)); > VOP_UNLOCK(vp, 0); > + } else { > + ASSERT_VOP_ELOCKED(vp, "unlink_lowervp"); > + KASSERT((vp->v_iflag & VI_DOOMED) == 0, > + ("reclaimed nullfs vnode %p", vp)); > + xp->null_flags &= ~NULLV_NOUNLOCK; > } > vdrop(vp); > } I have both witness and invariants. And this patch fixes the panic. Thanks! Antoine From owner-freebsd-fs@FreeBSD.ORG Mon May 20 11:06:45 2013 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A7308587 for ; Mon, 20 May 2013 11:06:45 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id 8AF64DA9 for ; Mon, 20 May 2013 11:06:45 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4KB6j1A082875 for ; Mon, 20 May 2013 11:06:45 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4KB6ji7082873 for freebsd-fs@FreeBSD.org; Mon, 20 May 2013 11:06:45 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 20 May 2013 11:06:45 GMT Message-Id: <201305201106.r4KB6ji7082873@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 11:06:45 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/178713 fs [nfs] [patch] Correct WebNFS support in NFS server and o kern/178412 fs [smbfs] Coredump when smbfs mounted o kern/178388 fs [zfs] [patch] allow up to 8MB recordsize o kern/178349 fs [zfs] zfs scrub on deduped data could be much less see o kern/178329 fs [zfs] extended attributes leak o kern/178238 fs [nullfs] nullfs don't release i-nodes on unlink. o kern/178231 fs [nfs] 8.3 nfsv4 client reports "nfsv4 client/server pr o kern/178103 fs [kernel] [nfs] [patch] Correct support of index files o kern/177985 fs [zfs] disk usage problem when copying from one zfs dat o kern/177971 fs [nfs] FreeBSD 9.1 nfs client dirlist problem w/ nfsv3, o kern/177966 fs [zfs] resilver completes but subsequent scrub reports o kern/177658 fs [ufs] FreeBSD panics after get full filesystem with uf o kern/177536 fs [zfs] zfs livelock (deadlock) with high write-to-disk o kern/177445 fs [hast] HAST panic o kern/177335 fs [nfs] [panic] Sleeping on "vmopar" with the following o kern/177240 fs [zfs] zpool import failed with state UNAVAIL but all d o kern/176978 fs [zfs] [panic] zfs send -D causes "panic: System call i o kern/176857 fs [softupdates] [panic] 9.1-RELEASE/amd64/GENERIC panic o bin/176253 fs zpool(8): zfs pool indentation is misleading/wrong o kern/176141 fs [zfs] sharesmb=on makes errors for sharenfs, and still o kern/175950 fs [zfs] Possible deadlock in zfs after long uptime o kern/175897 fs [zfs] operations on readonly zpool hang o kern/175179 fs [zfs] ZFS may attach wrong device on move o kern/175071 fs [ufs] [panic] softdep_deallocate_dependencies: unrecov o kern/174372 fs [zfs] Pagefault appears to be related to ZFS o kern/174315 fs [zfs] chflags uchg not supported o kern/174310 fs [zfs] root point mounting broken on CURRENT with multi o kern/174279 fs [ufs] UFS2-SU+J journal and filesystem corruption o kern/174060 fs [ext2fs] Ext2FS system crashes (buffer overflow?) o kern/173830 fs [zfs] Brain-dead simple change to ZFS error descriptio o kern/173718 fs [zfs] phantom directory in zraid2 pool f kern/173657 fs [nfs] strange UID map with nfsuserd o kern/173363 fs [zfs] [panic] Panic on 'zpool replace' on readonly poo o kern/173136 fs [unionfs] mounting above the NFS read-only share panic o kern/172942 fs [smbfs] Unmounting a smb mount when the server became o kern/172348 fs [unionfs] umount -f of filesystem in use with readonly o kern/172334 fs [unionfs] unionfs permits recursive union mounts; caus o kern/171626 fs [tmpfs] tmpfs should be noisier when the requested siz o kern/171415 fs [zfs] zfs recv fails with "cannot receive incremental o kern/170945 fs [gpt] disk layout not portable between direct connect o bin/170778 fs [zfs] [panic] FreeBSD panics randomly o kern/170680 fs [nfs] Multiple NFS Client bug in the FreeBSD 7.4-RELEA o kern/170497 fs [xfs][panic] kernel will panic whenever I ls a mounted o kern/169945 fs [zfs] [panic] Kernel panic while importing zpool (afte o kern/169480 fs [zfs] ZFS stalls on heavy I/O o kern/169398 fs [zfs] Can't remove file with permanent error o kern/169339 fs panic while " : > /etc/123" o kern/169319 fs [zfs] zfs resilver can't complete o kern/168947 fs [nfs] [zfs] .zfs/snapshot directory is messed up when o kern/168942 fs [nfs] [hang] nfsd hangs after being restarted (not -HU o kern/168158 fs [zfs] incorrect parsing of sharenfs options in zfs (fs o kern/167979 fs [ufs] DIOCGDINFO ioctl does not work on 8.2 file syste o kern/167977 fs [smbfs] mount_smbfs results are differ when utf-8 or U o kern/167688 fs [fusefs] Incorrect signal handling with direct_io o kern/167685 fs [zfs] ZFS on USB drive prevents shutdown / reboot o kern/167612 fs [portalfs] The portal file system gets stuck inside po o kern/167272 fs [zfs] ZFS Disks reordering causes ZFS to pick the wron o kern/167260 fs [msdosfs] msdosfs disk was mounted the second time whe o kern/167109 fs [zfs] [panic] zfs diff kernel panic Fatal trap 9: gene o kern/167105 fs [nfs] mount_nfs can not handle source exports wiht mor o kern/167067 fs [zfs] [panic] ZFS panics the server o kern/167065 fs [zfs] boot fails when a spare is the boot disk o kern/167048 fs [nfs] [patch] RELEASE-9 crash when using ZFS+NULLFS+NF o kern/166912 fs [ufs] [panic] Panic after converting Softupdates to jo o kern/166851 fs [zfs] [hang] Copying directory from the mounted UFS di o kern/166477 fs [nfs] NFS data corruption. o kern/165950 fs [ffs] SU+J and fsck problem o kern/165521 fs [zfs] [hang] livelock on 1 Gig of RAM with zfs when 31 o kern/165392 fs Multiple mkdir/rmdir fails with errno 31 o kern/165087 fs [unionfs] lock violation in unionfs o kern/164472 fs [ufs] fsck -B panics on particular data inconsistency o kern/164370 fs [zfs] zfs destroy for snapshot fails on i386 and sparc o kern/164261 fs [nullfs] [patch] fix panic with NFS served from NULLFS o kern/164256 fs [zfs] device entry for volume is not created after zfs o kern/164184 fs [ufs] [panic] Kernel panic with ufs_makeinode o kern/163801 fs [md] [request] allow mfsBSD legacy installed in 'swap' o kern/163770 fs [zfs] [hang] LOR between zfs&syncer + vnlru leading to o kern/163501 fs [nfs] NFS exporting a dir and a subdir in that dir to o kern/162944 fs [coda] Coda file system module looks broken in 9.0 o kern/162860 fs [zfs] Cannot share ZFS filesystem to hosts with a hyph o kern/162751 fs [zfs] [panic] kernel panics during file operations o kern/162591 fs [nullfs] cross-filesystem nullfs does not work as expe o kern/162519 fs [zfs] "zpool import" relies on buggy realpath() behavi o kern/161968 fs [zfs] [hang] renaming snapshot with -r including a zvo o kern/161864 fs [ufs] removing journaling from UFS partition fails on o bin/161807 fs [patch] add option for explicitly specifying metadata o kern/161579 fs [smbfs] FreeBSD sometimes panics when an smb share is o kern/161533 fs [zfs] [panic] zfs receive panic: system ioctl returnin o kern/161438 fs [zfs] [panic] recursed on non-recursive spa_namespace_ o kern/161424 fs [nullfs] __getcwd() calls fail when used on nullfs mou o kern/161280 fs [zfs] Stack overflow in gptzfsboot o kern/161205 fs [nfs] [pfsync] [regression] [build] Bug report freebsd o kern/161169 fs [zfs] [panic] ZFS causes kernel panic in dbuf_dirty o kern/161112 fs [ufs] [lor] filesystem LOR in FreeBSD 9.0-BETA3 o kern/160893 fs [zfs] [panic] 9.0-BETA2 kernel panic f kern/160860 fs [ufs] Random UFS root filesystem corruption with SU+J o kern/160801 fs [zfs] zfsboot on 8.2-RELEASE fails to boot from root-o o kern/160790 fs [fusefs] [panic] VPUTX: negative ref count with FUSE o kern/160777 fs [zfs] [hang] RAID-Z3 causes fatal hang upon scrub/impo o kern/160706 fs [zfs] zfs bootloader fails when a non-root vdev exists o kern/160591 fs [zfs] Fail to boot on zfs root with degraded raidz2 [r o kern/160410 fs [smbfs] [hang] smbfs hangs when transferring large fil o kern/160283 fs [zfs] [patch] 'zfs list' does abort in make_dataset_ha o kern/159930 fs [ufs] [panic] kernel core o kern/159402 fs [zfs][loader] symlinks cause I/O errors o kern/159357 fs [zfs] ZFS MAXNAMELEN macro has confusing name (off-by- o kern/159356 fs [zfs] [patch] ZFS NAME_ERR_DISKLIKE check is Solaris-s o kern/159351 fs [nfs] [patch] - divide by zero in mountnfs() o kern/159251 fs [zfs] [request]: add FLETCHER4 as DEDUP hash option o kern/159077 fs [zfs] Can't cd .. with latest zfs version o kern/159048 fs [smbfs] smb mount corrupts large files o kern/159045 fs [zfs] [hang] ZFS scrub freezes system o kern/158839 fs [zfs] ZFS Bootloader Fails if there is a Dead Disk o kern/158802 fs amd(8) ICMP storm and unkillable process. o kern/158231 fs [nullfs] panic on unmounting nullfs mounted over ufs o f kern/157929 fs [nfs] NFS slow read o kern/157399 fs [zfs] trouble with: mdconfig force delete && zfs strip o kern/157179 fs [zfs] zfs/dbuf.c: panic: solaris assert: arc_buf_remov o kern/156797 fs [zfs] [panic] Double panic with FreeBSD 9-CURRENT and o kern/156781 fs [zfs] zfs is losing the snapshot directory, p kern/156545 fs [ufs] mv could break UFS on SMP systems o kern/156193 fs [ufs] [hang] UFS snapshot hangs && deadlocks processes o kern/156039 fs [nullfs] [unionfs] nullfs + unionfs do not compose, re o kern/155615 fs [zfs] zfs v28 broken on sparc64 -current o kern/155587 fs [zfs] [panic] kernel panic with zfs p kern/155411 fs [regression] [8.2-release] [tmpfs]: mount: tmpfs : No o kern/155199 fs [ext2fs] ext3fs mounted as ext2fs gives I/O errors o bin/155104 fs [zfs][patch] use /dev prefix by default when importing o kern/154930 fs [zfs] cannot delete/unlink file from full volume -> EN o kern/154828 fs [msdosfs] Unable to create directories on external USB o kern/154491 fs [smbfs] smb_co_lock: recursive lock for object 1 p kern/154228 fs [md] md getting stuck in wdrain state o kern/153996 fs [zfs] zfs root mount error while kernel is not located o kern/153753 fs [zfs] ZFS v15 - grammatical error when attempting to u o kern/153716 fs [zfs] zpool scrub time remaining is incorrect o kern/153695 fs [patch] [zfs] Booting from zpool created on 4k-sector o kern/153680 fs [xfs] 8.1 failing to mount XFS partitions o kern/153418 fs [zfs] [panic] Kernel Panic occurred writing to zfs vol o kern/153351 fs [zfs] locking directories/files in ZFS o bin/153258 fs [patch][zfs] creating ZVOLs requires `refreservation' s kern/153173 fs [zfs] booting from a gzip-compressed dataset doesn't w o bin/153142 fs [zfs] ls -l outputs `ls: ./.zfs: Operation not support o kern/153126 fs [zfs] vdev failure, zpool=peegel type=vdev.too_small o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an f bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs p bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic p kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis p kern/133174 fs [msdosfs] [patch] msdosfs must support multibyte inter o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o kern/118318 fs [nfs] NFS server hangs under special circumstances o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118126 fs [nfs] [patch] Poor NFS server write performance o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with o kern/116583 fs [ffs] [hang] System freezes for short time when using o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 317 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon May 20 15:01:49 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 57843786 for ; Mon, 20 May 2013 15:01:49 +0000 (UTC) (envelope-from josh@signalboxes.net) Received: from mail-ob0-x22c.google.com (mail-ob0-x22c.google.com [IPv6:2607:f8b0:4003:c01::22c]) by mx1.freebsd.org (Postfix) with ESMTP id 2304BF3A for ; Mon, 20 May 2013 15:01:48 +0000 (UTC) Received: by mail-ob0-f172.google.com with SMTP id tb18so7169072obb.17 for ; Mon, 20 May 2013 08:01:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:mime-version:x-received:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=SvsaEs/X0rUnTm/422Xqnkt9jH+8U8MWuYNkT3aSFts=; b=hGclZoL0DiBt1Z3QqBSqHGEVMW0dmTL1tnQ5LHiNq+0rJZ4+e3ZbItjqHJsf3NdJFT MlXCVlPFOXAg9LLupjTB9ccRPwiVEpGZkBAffpZkTCSXbP9QcYuORdSKJ8KsHTb+XLln ByBBt/eF8hmohTeq7dVLU96y+bWj3Ui1rSgYX/bD9BI607GfB2LzN2ACgbDDnxjYOuwf jBmuOmcs6tKbPnVtAiCmpHq2wnJJYlwao38dKwxi2cc/3txkW11MzSzGtfaEXxvhhklR FJ1YBH5IW6KeBOuKC1eFl4hJWMx6aTu8vyFbGLGJh5ARFMLywS61ETHVGZ4+1P2LGYI6 d6YA== X-Received: by 10.60.83.49 with SMTP id n17mr941116oey.80.1369061617022; Mon, 20 May 2013 07:53:37 -0700 (PDT) Received: from mail-oa0-f43.google.com (mail-oa0-f43.google.com [209.85.219.43]) by mx.google.com with ESMTPSA id ns4sm20498252obc.2.2013.05.20.07.53.36 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 20 May 2013 07:53:36 -0700 (PDT) Received: by mail-oa0-f43.google.com with SMTP id o6so7754484oag.2 for ; Mon, 20 May 2013 07:53:36 -0700 (PDT) MIME-Version: 1.0 X-Received: by 10.60.125.232 with SMTP id mt8mr23627038oeb.33.1369061616294; Mon, 20 May 2013 07:53:36 -0700 (PDT) Received: by 10.60.79.194 with HTTP; Mon, 20 May 2013 07:53:36 -0700 (PDT) Date: Mon, 20 May 2013 08:53:36 -0600 Message-ID: Subject: L2ARC and ZIL on same SSD? From: Josh Beard To: freebsd-fs@freebsd.org X-Gm-Message-State: ALoCoQmlKhL3SiJXKbtyO4OLTYCfAntbWt4crxuiQje1piLqUMQjG84suaJh9NxHx+51aLrTakvv Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 15:01:49 -0000 Hello, I have a file server running 9.1-RELEASE with 24 GB of RAM and a 6 TB zpool spread across 12 500GB disks in two raidz1 and 2 spares. This server houses hundreds of home directories for end-users and stays pretty busy with both read and writes. I'm planning to add two SSDs soon (128 GB each) to help performance and was planning to mirror them. I was curious if having the L2ARC and ZIL on the same disks (mirrored) is okay. With some searching, I haven't seen any "official" word on this. I've seen some folks say not to and others recommend it. Adding a second set of SSDs isn't desirable, as I'll start cutting into my available storage capacity (16 bay system). Any advice or experience with this? Thanks, Josh From owner-freebsd-fs@FreeBSD.ORG Mon May 20 15:18:57 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CA5F4D74 for ; Mon, 20 May 2013 15:18:57 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) by mx1.freebsd.org (Postfix) with ESMTP id 6875781 for ; Mon, 20 May 2013 15:18:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.5/8.14.5) with ESMTP id r4KFIn5A011301; Mon, 20 May 2013 19:18:49 +0400 (MSK) (envelope-from marck@rinet.ru) Date: Mon, 20 May 2013 19:18:49 +0400 (MSK) From: Dmitry Morozovsky To: Josh Beard Subject: Re: L2ARC and ZIL on same SSD? In-Reply-To: Message-ID: References: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (woozle.rinet.ru [0.0.0.0]); Mon, 20 May 2013 19:18:49 +0400 (MSK) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 15:18:57 -0000 On Mon, 20 May 2013, Josh Beard wrote: > I have a file server running 9.1-RELEASE with 24 GB of RAM and a 6 TB zpool > spread across 12 500GB disks in two raidz1 and 2 spares. This server > houses hundreds of home directories for end-users and stays pretty busy > with both read and writes. > > I'm planning to add two SSDs soon (128 GB each) to help performance and was > planning to mirror them. I was curious if having the L2ARC and ZIL on the > same disks (mirrored) is okay. With some searching, I haven't seen any > "official" word on this. I've seen some folks say not to and others > recommend it. Adding a second set of SSDs isn't desirable, as I'll start > cutting into my available storage capacity (16 bay system). > > Any advice or experience with this? Well, no official words, but -- I use the following setup for our new database server, very similar to yours (but RAID10-alike as it's database, not generic storage): root@briareus:/usr/local/etc# zpool status pool: br state: ONLINE scan: scrub repaired 0 in 0h0m with 0 errors on Fri Apr 26 15:29:32 2013 config: NAME STATE READ WRITE CKSUM br ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/br0 ONLINE 0 0 0 gpt/br4 ONLINE 0 0 0 mirror-1 ONLINE 0 0 0 gpt/br1 ONLINE 0 0 0 gpt/br5 ONLINE 0 0 0 mirror-2 ONLINE 0 0 0 gpt/br6 ONLINE 0 0 0 gpt/br2 ONLINE 0 0 0 mirror-3 ONLINE 0 0 0 gpt/br3 ONLINE 0 0 0 gpt/br7 ONLINE 0 0 0 logs mirror-4 ONLINE 0 0 0 gpt/br-zil0 ONLINE 0 0 0 gpt/br-zil1 ONLINE 0 0 0 cache gpt/br-cache0 ONLINE 0 0 0 gpt/br-cache1 ONLINE 0 0 0 errors: No known data errors root@briareus:/usr/local/etc# glabel status | egrep 'zil|cache' gpt/br-zil0 N/A da8p1 gpt/br-cache0 N/A da8p2 gpt/br-zil1 N/A da9p1 gpt/br-cache1 N/A da9p2 root@briareus:/usr/local/etc# gpart show da8 da9 => 34 234371453 da8 GPT (111G) 34 2014 - free - (1M) 2048 16777216 1 freebsd-zfs (8.0G) 16779264 217591808 2 freebsd-zfs (103G) 234371072 415 - free - (207k) => 34 234371453 da9 GPT (111G) 34 2014 - free - (1M) 2048 16777216 1 freebsd-zfs (8.0G) 16779264 217591808 2 freebsd-zfs (103G) 234371072 415 - free - (207k) ... and it's working pretty well, givins us some tens of kIOPs on database writes. -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Mon May 20 15:29:01 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C389F1CA for ; Mon, 20 May 2013 15:29:01 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.21.123]) by mx1.freebsd.org (Postfix) with ESMTP id 42CB5F0 for ; Mon, 20 May 2013 15:29:00 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [193.68.6.1]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.6/8.14.6) with ESMTP id r4KFSphj094842 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Mon, 20 May 2013 18:28:51 +0300 (EEST) (envelope-from daniel@digsys.bg) Message-ID: <519A4133.3090502@digsys.bg> Date: Mon, 20 May 2013 18:28:51 +0300 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130520 Thunderbird/17.0.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: L2ARC and ZIL on same SSD? References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 15:29:01 -0000 On 20.05.13 17:53, Josh Beard wrote: > Hello, > > I have a file server running 9.1-RELEASE with 24 GB of RAM and a 6 TB zpool > spread across 12 500GB disks in two raidz1 and 2 spares. This server > houses hundreds of home directories for end-users and stays pretty busy > with both read and writes. > > I'm planning to add two SSDs soon (128 GB each) to help performance and was > planning to mirror them. I was curious if having the L2ARC and ZIL on the > same disks (mirrored) is okay. With some searching, I haven't seen any > "official" word on this. I've seen some folks say not to and others > recommend it. Adding a second set of SSDs isn't desirable, as I'll start > cutting into my available storage capacity (16 bay system). > > Any advice or experience with this? > Indeed, there is no official position on how SSDs should be used with ZFS, because SSDs are so much different from each other. Here is what you should consider, however: There is no need to mirror the L2ARC. On failure, it is simply ignored. Therefore, for L2ARC in general you put as large SSD as you find, that has good read rate. You might want to have this on a separate controller, which provides the highest bandwidth. There is suggestion to mirror the ZIL, in order to avoid pool damage, but in post-v28 pools, the ZIL can be removed and replaced on the fly. Since the ZIL is only ever read after crash, it all boils down what your recovery scenario is. If you can tolerate the loss of most recent data, you really do not need to mirror ZIL. In any case, it is a bad idea to mirror the SSDs before giving them to ZFS. Let ZFS manage it's redundancy, by giving it either the raw devices or raw slices/partittions. You also need to start thinking of SSDs in ZFS as "expense". You are wearing it during use and at some point you will have to replace the SSD. That is, you trade SSD wear for performance. L2ARC and ZIL have quite different wearing characteristics. In theory, ZIL would work best if it is either an high endurance SLC flash drive, or if you use only small portion of it for writes. For example, use only 8GB out of an 30GB (or larger) SSD for ZIL. L2ARC, if sufficiently large would wear much less, because it will be mostly used for reads. But, if your L2ARC is smaller than required -- it will be overwriten many many times and thus wear faster. In both cases you can monitor the "Media Wearout Indicator" of SMART to see how much life you have left. So considering especially the "expense" concept, you may want to have one higher performance but small SSD for ZIL and another lower performance but cheap SSD for L2ARC. If your SSD has sufficient read/write performance, you might use slices of the same SSD for ZIL and L2ARC. But consider consumer grade "performance" SSD wear faster. One good things is that as SSDs wear, larger and cheaper SSDs will be available :) I don't know how much this helps -- you need to put it into perspective for your workload and other constraints. Daniel From owner-freebsd-fs@FreeBSD.ORG Mon May 20 19:30:02 2013 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id CCDEA3BA for ; Mon, 20 May 2013 19:30:02 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id BF5EA1829 for ; Mon, 20 May 2013 19:30:02 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4KJU2Ob094420 for ; Mon, 20 May 2013 19:30:02 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4KJU20w094419; Mon, 20 May 2013 19:30:02 GMT (envelope-from gnats) Date: Mon, 20 May 2013 19:30:02 GMT Message-Id: <201305201930.r4KJU20w094419@freefall.freebsd.org> To: freebsd-fs@FreeBSD.org Cc: From: Jaakko Heinonen Subject: Re: kern/165392: Multiple mkdir/rmdir fails with errno 31 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list Reply-To: Jaakko Heinonen List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 19:30:02 -0000 The following reply was made to PR kern/165392; it has been noted by GNATS. From: Jaakko Heinonen To: Jilles Tjoelker Cc: bug-followup@FreeBSD.org, vvv@colocall.net, mckusick@FreeBSD.org Subject: Re: kern/165392: Multiple mkdir/rmdir fails with errno 31 Date: Mon, 20 May 2013 22:21:34 +0300 Hi! On 2012-02-25, Jilles Tjoelker wrote: > > [mkdir fails with [EMLINK], but link count < LINK_MAX] > > I can reproduce this problem with UFS with soft updates (with or without > journaling). > > A reproduction without C programs is: > > cd empty_dir > mkdir `jot 32766 1` # the last one will fail (correctly) > rmdir 1 > mkdir a # will erroneously fail > > The problem appears to be because the previous rmdir has not yet been > fully completed. It is still holding onto the link count until the > directory is written, which may take up to two minutes. > > The same problem can occur with other calls that increase the link count > such as link() and rename(). > > A workaround is to call fsync() on the directory that contained the > deleted entries. It will then release its hold on the link count and > allow mkdir or other calls. If fsync() is only called when [EMLINK] is > returned, the performance impact should not be very bad, although it > still causes more I/O than necessary. I tried to implement this with the following patch: http://people.freebsd.org/~jh/patches/ufs-check_linkcnt.diff However, VOP_FSYNC(9) with the MNT_WAIT flag seems not to update the i_nlink count for a reason unknown to me. I can verify that also by taking your reproduction recipe above and adding "fsync ." between "rmdir 1" and "mkdir a". Does this mean that fsync(2) is broken for directories on softdep enabled UFS? I have cc'd Kirk in hope he could shed some light on this. -- Jaakko From owner-freebsd-fs@FreeBSD.ORG Mon May 20 20:42:20 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E51CD2EB; Mon, 20 May 2013 20:42:20 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) by mx1.freebsd.org (Postfix) with ESMTP id B7E741877; Mon, 20 May 2013 20:42:20 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 005DBB965; Mon, 20 May 2013 16:42:19 -0400 (EDT) From: John Baldwin To: freebsd-fs@freebsd.org, Rick Macklem Subject: Re: kern/177335: [nfs] [panic] Sleeping on "vmopar" with the following non-sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode lock) Date: Mon, 20 May 2013 15:31:45 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p25; KDE/4.5.5; amd64; ; ) References: <201305200332.r4K3WsBS083290@freefall.freebsd.org> In-Reply-To: <201305200332.r4K3WsBS083290@freefall.freebsd.org> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201305201531.45263.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Mon, 20 May 2013 16:42:20 -0400 (EDT) Cc: kib@freebsd.org, linimon@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 20:42:21 -0000 On Sunday, May 19, 2013 11:32:54 pm linimon@freebsd.org wrote: > Old Synopsis: Sleeping on "vmopar" with the following non-sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode lock) > New Synopsis: [nfs] [panic] Sleeping on "vmopar" with the following non- sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode lock) > > Responsible-Changed-From-To: freebsd-bugs->freebsd-fs > Responsible-Changed-By: linimon > Responsible-Changed-When: Mon May 20 03:32:02 UTC 2013 > Responsible-Changed-Why: > Over to maintainer(s). I think this issue might have been recently fixed in HEAD, specifically in r248567 and r248581. -- John Baldwin From owner-freebsd-fs@FreeBSD.ORG Mon May 20 20:58:47 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 6FBCA106; Mon, 20 May 2013 20:58:47 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id E1EFCEF2; Mon, 20 May 2013 20:58:46 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqAEAAuOmlGDaFvO/2dsb2JhbABahnO+DIEadIIfAQEFIwRSGw4KAgINEgcCWQYTiA2rGpFBgSaNRzQHgkGBEwOXOJFAgViBUyCBag X-IronPort-AV: E=Sophos;i="4.87,709,1363147200"; d="scan'208";a="28948607" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu.net.uoguelph.ca with ESMTP; 20 May 2013 16:58:45 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 2E567B4066; Mon, 20 May 2013 16:58:45 -0400 (EDT) Date: Mon, 20 May 2013 16:58:45 -0400 (EDT) From: Rick Macklem To: John Baldwin Message-ID: <45797708.465808.1369083525132.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <201305201531.45263.jhb@freebsd.org> Subject: Re: kern/177335: [nfs] [panic] Sleeping on "vmopar" with the following non-sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode lock) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: freebsd-fs@freebsd.org, Rick Macklem , kib@freebsd.org, linimon@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 20:58:47 -0000 John Baldwin wrote: > On Sunday, May 19, 2013 11:32:54 pm linimon@freebsd.org wrote: > > Old Synopsis: Sleeping on "vmopar" with the following non-sleepable > > locks > held: exclusive sleep mutex NFSnode lock (NFSnode lock) > > New Synopsis: [nfs] [panic] Sleeping on "vmopar" with the following > > non- > sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode > lock) > > > > Responsible-Changed-From-To: freebsd-bugs->freebsd-fs > > Responsible-Changed-By: linimon > > Responsible-Changed-When: Mon May 20 03:32:02 UTC 2013 > > Responsible-Changed-Why: > > Over to maintainer(s). > > I think this issue might have been recently fixed in HEAD, > specifically in > r248567 and r248581. > I just emailed the reporter with a variant of these patches for 8.3/old NFS client. I suppose I should change the PR status to feedback. And, yes, the panic appears to be the same as what the above commits fix, except in 8.3/old NFS client. rick > -- > John Baldwin From owner-freebsd-fs@FreeBSD.ORG Mon May 20 21:06:19 2013 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id EA424321; Mon, 20 May 2013 21:06:19 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id C3725135D; Mon, 20 May 2013 21:06:19 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4KL6J0f013096; Mon, 20 May 2013 21:06:19 GMT (envelope-from rmacklem@freefall.freebsd.org) Received: (from rmacklem@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4KL6JBc013095; Mon, 20 May 2013 21:06:19 GMT (envelope-from rmacklem) Date: Mon, 20 May 2013 21:06:19 GMT Message-Id: <201305202106.r4KL6JBc013095@freefall.freebsd.org> To: izrodix@gmail.com, rmacklem@FreeBSD.org, freebsd-fs@FreeBSD.org From: rmacklem@FreeBSD.org Subject: Re: kern/177335: [nfs] [panic] Sleeping on "vmopar" with the following non-sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode lock) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 21:06:20 -0000 Synopsis: [nfs] [panic] Sleeping on "vmopar" with the following non-sleepable locks held: exclusive sleep mutex NFSnode lock (NFSnode lock) State-Changed-From-To: open->feedback State-Changed-By: rmacklem State-Changed-When: Mon May 20 21:04:48 UTC 2013 State-Changed-Why: I have emailed the reporter a variant of r248567, r248581 for the 8.3/old NFS client and am waiting to hear from him w.r.t. this. http://www.freebsd.org/cgi/query-pr.cgi?pr=177335 From owner-freebsd-fs@FreeBSD.ORG Mon May 20 21:08:00 2013 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 02DD14AB; Mon, 20 May 2013 21:08:00 +0000 (UTC) (envelope-from rmacklem@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id D08C41451; Mon, 20 May 2013 21:07:59 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4KL7xKH013170; Mon, 20 May 2013 21:07:59 GMT (envelope-from rmacklem@freefall.freebsd.org) Received: (from rmacklem@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4KL7xSM013169; Mon, 20 May 2013 21:07:59 GMT (envelope-from rmacklem) Date: Mon, 20 May 2013 21:07:59 GMT Message-Id: <201305202107.r4KL7xSM013169@freefall.freebsd.org> To: astrange@gmail.com, rmacklem@FreeBSD.org, freebsd-fs@FreeBSD.org From: rmacklem@FreeBSD.org Subject: Re: kern/178231: [nfs] 8.3 nfsv4 client reports "nfsv4 client/server protocol prob err=10026" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 May 2013 21:08:00 -0000 Synopsis: [nfs] 8.3 nfsv4 client reports "nfsv4 client/server protocol prob err=10026" State-Changed-From-To: open->feedback State-Changed-By: rmacklem State-Changed-When: Mon May 20 21:06:30 UTC 2013 State-Changed-Why: I have emailed the reporter and asked him to test without the "intr" mount option, which could cause this. The error indicates that the client got "out of sync" with the server while doing open/lock operations, which are serialized via a sequence number. http://www.freebsd.org/cgi/query-pr.cgi?pr=178231 From owner-freebsd-fs@FreeBSD.ORG Wed May 22 10:54:47 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id AE9A0D74 for ; Wed, 22 May 2013 10:54:47 +0000 (UTC) (envelope-from leslie@eskk.nu) Received: from mx1.bjare.net (mx1.bjare.net [212.31.160.3]) by mx1.freebsd.org (Postfix) with ESMTP id 47522CD5 for ; Wed, 22 May 2013 10:54:46 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mx1.bjare.net (Postfix) with ESMTP id 99EC55E594 for ; Wed, 22 May 2013 12:54:39 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mx1.bjare.net X-Spam-Flag: NO X-Spam-Score: -0.967 X-Spam-Level: X-Spam-Status: No, score=-0.967 tagged_above=-999 required=5 tests=[AWL=0.031, BAYES_00=-2.599, RCVD_IN_PBL=0.905, RDNS_DYNAMIC=0.1, SPF_SOFTFAIL=0.596] Received: from mx1.bjare.net ([127.0.0.1]) by localhost (mx1.bjare.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id QErfxIrVFhwB for ; Wed, 22 May 2013 12:54:34 +0200 (CEST) X-BN-MX1: ja X-BN-MailInfo: BjareNet Received: from bljbsd01.no-ip.org (c224-156-70-80.bjare.net [80.70.156.224]) by mx1.bjare.net (Postfix) with ESMTP id DAB3E5E5BE for ; Wed, 22 May 2013 12:53:25 +0200 (CEST) Message-ID: <519CA3A5.3020109@eskk.nu> Date: Wed, 22 May 2013 12:53:25 +0200 From: Leslie Jensen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130516 Thunderbird/17.0.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Upgrading from 8.3 to 9.1 zfs pool can't attach Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 10:54:47 -0000 I've upgraded a machine with freebsd-update from 8.3 to 9.1. After the first restart I edited /etc/fstab in single user mode because the names on the disks had changed. But the zfs pool I have seem to have a problem and I'm sure on how to recover it. May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0. May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1. May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4. May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5. camcontrol devlist at scbus0 target 0 lun 0 (ada0,pass0) at scbus1 target 0 lun 0 (ada1,pass1) at scbus2 target 0 lun 0 (ada2,pass2) at scbus2 target 1 lun 0 (ada3,pass3) at scbus3 target 0 lun 0 (ada4,pass4) at scbus3 target 1 lun 0 (ada5,pass5) ada2 and ada3 are configured as raid/r0 holding the system. ada0, ada1, ada4 and ada5 should be a zfs pool with /home cat /etc/fstab # Device Mountpoint FStype Options Dump Pass# /dev/raid/r0s1b none swap sw 0 0 /dev/raid/r0s1a / ufs rw 1 1 /dev/raid/r0s1e /tmp ufs rw 2 2 /dev/raid/r0s1f /usr ufs rw 2 2 /dev/raid/r0s1d /var ufs rw 2 2 I'm not sure but shouldn't the mount point for /home also appear in fstab? Maybe I messed up during the manual editing of files during the update process? Can I recover or shall I destroy the pool and rebuild it? Thanks /Leslie From owner-freebsd-fs@FreeBSD.ORG Wed May 22 15:02:22 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1E59DDBD for ; Wed, 22 May 2013 15:02:22 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from smarthost1.greenhost.nl (smarthost1.greenhost.nl [195.190.28.78]) by mx1.freebsd.org (Postfix) with ESMTP id ADEE9A8D for ; Wed, 22 May 2013 15:02:21 +0000 (UTC) Received: from smtp.greenhost.nl ([213.108.104.138]) by smarthost1.greenhost.nl with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.69) (envelope-from ) id 1UfAYO-0002EQ-N0; Wed, 22 May 2013 17:02:13 +0200 Received: from [81.21.138.17] (helo=ronaldradial.versatec.local) by smtp.greenhost.nl with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.72) (envelope-from ) id 1UfAYM-0004o4-C5; Wed, 22 May 2013 17:02:10 +0200 Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-fs@freebsd.org, "Leslie Jensen" Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach References: <519CA3A5.3020109@eskk.nu> Date: Wed, 22 May 2013 17:02:09 +0200 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Ronald Klop" Message-ID: In-Reply-To: <519CA3A5.3020109@eskk.nu> User-Agent: Opera Mail/12.15 (Win32) X-Virus-Scanned: by clamav at smarthost1.samage.net X-Spam-Level: / X-Spam-Score: 0.8 X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.1 X-Scan-Signature: 448baf4759cd3283a5930955cc61e1db X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 15:02:22 -0000 On Wed, 22 May 2013 12:53:25 +0200, Leslie Jensen wrote: > > I've upgraded a machine with freebsd-update from 8.3 to 9.1. > > After the first restart I edited /etc/fstab in single user mode because > the names on the disks had changed. But the zfs pool I have seem to have > a problem and I'm sure on how to recover it. > > > May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0. > May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1. > May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4. > May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5. > > > camcontrol devlist > at scbus0 target 0 lun 0 (ada0,pass0) > at scbus1 target 0 lun 0 (ada1,pass1) > at scbus2 target 0 lun 0 (ada2,pass2) > at scbus2 target 1 lun 0 (ada3,pass3) > at scbus3 target 0 lun 0 (ada4,pass4) > at scbus3 target 1 lun 0 (ada5,pass5) > > ada2 and ada3 are configured as raid/r0 holding the system. > > ada0, ada1, ada4 and ada5 should be a zfs pool with /home > > > cat /etc/fstab > > # Device Mountpoint FStype Options Dump Pass# > /dev/raid/r0s1b none swap sw 0 0 > /dev/raid/r0s1a / ufs rw 1 1 > /dev/raid/r0s1e /tmp ufs rw 2 2 > /dev/raid/r0s1f /usr ufs rw 2 2 > /dev/raid/r0s1d /var ufs rw 2 2 > > > I'm not sure but shouldn't the mount point for /home also appear in > fstab? > > Maybe I messed up during the manual editing of files during the update > process? > > > Can I recover or shall I destroy the pool and rebuild it? > > Thanks > > /Leslie Mounts from zfs do not have to be in fstab. You can set a mountpoint on a zfs volume which kind of mounts itself. Can you send more information about the 'Unable to attach to ada[0-4]' messages? Maybe a complete dmesg (/var/run/dmesg.boot). And how are these disks configured/partitioned? Output from 'gpart list ada0' or 'gpart show ada0' and for the other disks. Are you using stuff like glabel? Ronald. From owner-freebsd-fs@FreeBSD.ORG Wed May 22 15:33:52 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D9AE5197 for ; Wed, 22 May 2013 15:33:52 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qa0-x231.google.com (mail-qa0-x231.google.com [IPv6:2607:f8b0:400d:c00::231]) by mx1.freebsd.org (Postfix) with ESMTP id A2F2AE58 for ; Wed, 22 May 2013 15:33:52 +0000 (UTC) Received: by mail-qa0-f49.google.com with SMTP id j11so1096033qag.8 for ; Wed, 22 May 2013 08:33:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=EFwk/8ESqR1nMtzW/GVreNwnyjKjhK83ORE13WLWDdI=; b=j0EGd2JANobSJj7rrduhG9occdir5lb9PITJNgtUQMcSiQwAPrbzUxOzITI92n00Yz MYdOGggjU8k58m0jPf6Kba2oYvWRl3LyfxjmP6RLon8OVkm4XjIg6rfKNhICZxfqJLlz s+hg1nm0SMeU9gmD4s9IHR5mP8ART4eiQsCDRfu104mwsmVI3BMurZkHIcgozXuXGx9b 5dl7JL3KUk/38wyqbeRAxJBa6EDVzlffIwKtgYpNVCIMf9CRqtdeMhl1taI+xCw7Nfvy cxTzKfilgEwidzlmlamrHTYvvoyBP8KUnGjY2W8kYGEvFmyAs2mimVDgWwWl4rv7bczp S67A== MIME-Version: 1.0 X-Received: by 10.49.35.132 with SMTP id h4mr8541387qej.29.1369236831920; Wed, 22 May 2013 08:33:51 -0700 (PDT) Received: by 10.49.0.66 with HTTP; Wed, 22 May 2013 08:33:51 -0700 (PDT) In-Reply-To: <519CA3A5.3020109@eskk.nu> References: <519CA3A5.3020109@eskk.nu> Date: Wed, 22 May 2013 08:33:51 -0700 Message-ID: Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach From: Freddie Cash To: Leslie Jensen Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 15:33:52 -0000 What's the output of "zpool import" in single-user mode? What are the names of your disk devices? "grep ad /var/log/messages" for example. Are you using whole disks, disk partitions, labels, etc to access your disks? Are you sure your disks haven't been renumbered at all? On Wed, May 22, 2013 at 3:53 AM, Leslie Jensen wrote: > > I've upgraded a machine with freebsd-update from 8.3 to 9.1. > > After the first restart I edited /etc/fstab in single user mode because > the names on the disks had changed. But the zfs pool I have seem to have a > problem and I'm sure on how to recover it. > > > May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0. > May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1. > May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4. > May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5. > > > camcontrol devlist > at scbus0 target 0 lun 0 (ada0,pass0) > at scbus1 target 0 lun 0 (ada1,pass1) > at scbus2 target 0 lun 0 (ada2,pass2) > at scbus2 target 1 lun 0 (ada3,pass3) > at scbus3 target 0 lun 0 (ada4,pass4) > at scbus3 target 1 lun 0 (ada5,pass5) > > ada2 and ada3 are configured as raid/r0 holding the system. > > ada0, ada1, ada4 and ada5 should be a zfs pool with /home > > > cat /etc/fstab > > # Device Mountpoint FStype Options Dump Pass# > /dev/raid/r0s1b none swap sw 0 0 > /dev/raid/r0s1a / ufs rw 1 1 > /dev/raid/r0s1e /tmp ufs rw 2 2 > /dev/raid/r0s1f /usr ufs rw 2 2 > /dev/raid/r0s1d /var ufs rw 2 2 > > > I'm not sure but shouldn't the mount point for /home also appear in fstab? > > Maybe I messed up during the manual editing of files during the update > process? > > > Can I recover or shall I destroy the pool and rebuild it? > > Thanks > > /Leslie > > > > ______________________________**_________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@**freebsd.org > " > -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Wed May 22 17:45:24 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E2C99B75 for ; Wed, 22 May 2013 17:45:24 +0000 (UTC) (envelope-from leslie@eskk.nu) Received: from mx1.bjare.net (mx1.bjare.net [212.31.160.3]) by mx1.freebsd.org (Postfix) with ESMTP id 5C1D3960 for ; Wed, 22 May 2013 17:45:24 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mx1.bjare.net (Postfix) with ESMTP id 23A855E545; Wed, 22 May 2013 19:45:23 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mx1.bjare.net X-Spam-Flag: NO X-Spam-Score: -0.967 X-Spam-Level: X-Spam-Status: No, score=-0.967 tagged_above=-999 required=5 tests=[AWL=0.031, BAYES_00=-2.599, RCVD_IN_PBL=0.905, RDNS_DYNAMIC=0.1, SPF_SOFTFAIL=0.596] Received: from mx1.bjare.net ([127.0.0.1]) by localhost (mx1.bjare.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id b2ten0JTzQ0F; Wed, 22 May 2013 19:45:17 +0200 (CEST) X-BN-MX1: ja X-BN-MailInfo: BjareNet Received: from bljbsd01.no-ip.org (c224-156-70-80.bjare.net [80.70.156.224]) by mx1.bjare.net (Postfix) with ESMTP id D7A245E541; Wed, 22 May 2013 19:45:12 +0200 (CEST) Message-ID: <519D0428.8060800@eskk.nu> Date: Wed, 22 May 2013 19:45:12 +0200 From: Leslie Jensen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130516 Thunderbird/17.0.6 MIME-Version: 1.0 To: Freddie Cash , freebsd-fs@freebsd.org Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach References: <519CA3A5.3020109@eskk.nu> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 17:45:24 -0000 2013-05-22 17:33, Freddie Cash skrev: > What's the output of "zpool import" in single-user mode? > > What are the names of your disk devices? "grep ad /var/log/messages" > for example. > > Are you using whole disks, disk partitions, labels, etc to access your > disks? Are you sure your disks haven't been renumbered at all? > > > On Wed, May 22, 2013 at 3:53 AM, Leslie Jensen > wrote: > > > I've upgraded a machine with freebsd-update from 8.3 to 9.1. > > After the first restart I edited /etc/fstab in single user mode > because the names on the disks had changed. But the zfs pool I have > seem to have a problem and I'm sure on how to recover it. > > > May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0. > May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1. > May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4. > May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5. > > > camcontrol devlist > at scbus0 target 0 lun 0 (ada0,pass0) > at scbus1 target 0 lun 0 (ada1,pass1) > at scbus2 target 0 lun 0 (ada2,pass2) > at scbus2 target 1 lun 0 (ada3,pass3) > at scbus3 target 0 lun 0 (ada4,pass4) > at scbus3 target 1 lun 0 (ada5,pass5) > > ada2 and ada3 are configured as raid/r0 holding the system. > > ada0, ada1, ada4 and ada5 should be a zfs pool with /home > > > cat /etc/fstab > > # Device Mountpoint FStype Options Dump Pass# > /dev/raid/r0s1b none swap sw 0 0 > /dev/raid/r0s1a / ufs rw 1 1 > /dev/raid/r0s1e /tmp ufs rw 2 2 > /dev/raid/r0s1f /usr ufs rw 2 2 > /dev/raid/r0s1d /var ufs rw 2 2 > > > I'm not sure but shouldn't the mount point for /home also appear in > fstab? > > Maybe I messed up during the manual editing of files during the > update process? > > > Can I recover or shall I destroy the pool and rebuild it? > > Thanks > > /Leslie > > > > _________________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/__mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to > "freebsd-fs-unsubscribe@__freebsd.org > " > > > > > -- > Freddie Cash > fjwcash@gmail.com zpool import in single user mode gives: zfs filesystem version 5 zfs storage pool version 28 ZFS WARNING: Unable to attach to ada0. ZFS WARNING: Unable to attach to ada1. ZFS WARNING: Unable to attach to ada4. ZFS WARNING: Unable to attach to ada5. grep ada /var/log/messages -------------------------------------------- May 22 19:34:03 bagnerbsd kernel: ada0 at siisch0 bus 0 scbus0 target 0 lun 0 May 22 19:34:03 bagnerbsd kernel: ada0: ATA-7 SATA 1.x device May 22 19:34:03 bagnerbsd kernel: ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes) May 22 19:34:03 bagnerbsd kernel: ada0: Command Queueing enabled May 22 19:34:03 bagnerbsd kernel: ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) May 22 19:34:03 bagnerbsd kernel: ada0: Previously was known as ad4 May 22 19:34:03 bagnerbsd kernel: ada1 at siisch1 bus 0 scbus1 target 0 lun 0 May 22 19:34:03 bagnerbsd kernel: ada1: ATA-7 SATA 1.x device May 22 19:34:03 bagnerbsd kernel: ada1: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes) May 22 19:34:03 bagnerbsd kernel: ada1: Command Queueing enabled May 22 19:34:03 bagnerbsd kernel: ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) May 22 19:34:03 bagnerbsd kernel: ada1: Previously was known as ad6 May 22 19:34:03 bagnerbsd kernel: ada2 at ata0 bus 0 scbus2 target 0 lun 0 May 22 19:34:03 bagnerbsd kernel: ada2: ATA-6 SATA 1.x device May 22 19:34:03 bagnerbsd kernel: ada2: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) May 22 19:34:03 bagnerbsd kernel: ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) May 22 19:34:03 bagnerbsd kernel: ada2: Previously was known as ad0 May 22 19:34:03 bagnerbsd kernel: ada3 at ata0 bus 0 scbus2 target 1 lun 0 May 22 19:34:03 bagnerbsd kernel: ada3: ATA-6 SATA 1.x device May 22 19:34:03 bagnerbsd kernel: ada3: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) May 22 19:34:03 bagnerbsd kernel: ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) May 22 19:34:03 bagnerbsd kernel: ada3: Previously was known as ad1 May 22 19:34:03 bagnerbsd kernel: ada4 at ata1 bus 0 scbus3 target 0 lun 0 May 22 19:34:03 bagnerbsd kernel: ada4: ATA-7 SATA 1.x device May 22 19:34:03 bagnerbsd kernel: ada4: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) May 22 19:34:03 bagnerbsd kernel: ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) May 22 19:34:03 bagnerbsd kernel: ada4: Previously was known as ad2 May 22 19:34:03 bagnerbsd kernel: ada5 at ata1 bus 0 scbus3 target 1 lun 0 May 22 19:34:03 bagnerbsd kernel: ada5: ATA-7 SATA 1.x device May 22 19:34:03 bagnerbsd kernel: ada5: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) May 22 19:34:03 bagnerbsd kernel: ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) May 22 19:34:03 bagnerbsd kernel: ada5: Previously was known as ad3 May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from NONE to ACTIVE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from NONE to ACTIVE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from NONE to ACTIVE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from NONE to ACTIVE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state changed from NONE to STALE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE. May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state changed from NONE to STALE. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4. May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5 ---------------------------------------------- I'm using whole disks. The pool was created back in 2010 and I have forgot the details on how I did but I think I just created the out of the four disks. The disks might have been renumbered I'm not sure. It would be logical because the disks ada2 and ada3 had to bee changed in fstab as I wrote before. Thanks /Leslie From owner-freebsd-fs@FreeBSD.ORG Wed May 22 17:54:42 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 45A0F3BE for ; Wed, 22 May 2013 17:54:42 +0000 (UTC) (envelope-from leslie@eskk.nu) Received: from mx1.bjare.net (mx1.bjare.net [212.31.160.3]) by mx1.freebsd.org (Postfix) with ESMTP id 59F01A08 for ; Wed, 22 May 2013 17:54:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mx1.bjare.net (Postfix) with ESMTP id 692B95E551; Wed, 22 May 2013 19:54:40 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mx1.bjare.net X-Spam-Flag: NO X-Spam-Score: -0.968 X-Spam-Level: X-Spam-Status: No, score=-0.968 tagged_above=-999 required=5 tests=[AWL=0.030, BAYES_00=-2.599, RCVD_IN_PBL=0.905, RDNS_DYNAMIC=0.1, SPF_SOFTFAIL=0.596] Received: from mx1.bjare.net ([127.0.0.1]) by localhost (mx1.bjare.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id ZTkNY6cY2Bar; Wed, 22 May 2013 19:54:35 +0200 (CEST) X-BN-MX1: ja X-BN-MailInfo: BjareNet Received: from bljbsd01.no-ip.org (c224-156-70-80.bjare.net [80.70.156.224]) by mx1.bjare.net (Postfix) with ESMTP id 44B4C5E541; Wed, 22 May 2013 19:54:34 +0200 (CEST) Message-ID: <519D065A.3070806@eskk.nu> Date: Wed, 22 May 2013 19:54:34 +0200 From: Leslie Jensen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130516 Thunderbird/17.0.6 MIME-Version: 1.0 To: Ronald Klop Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach References: <519CA3A5.3020109@eskk.nu> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 17:54:42 -0000 2013-05-22 17:02, Ronald Klop skrev: > On Wed, 22 May 2013 12:53:25 +0200, Leslie Jensen wrote: > >> >> I've upgraded a machine with freebsd-update from 8.3 to 9.1. >> >> After the first restart I edited /etc/fstab in single user mode >> because the names on the disks had changed. But the zfs pool I have >> seem to have a problem and I'm sure on how to recover it. >> >> >> May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0. >> May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1. >> May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4. >> May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5. >> >> >> camcontrol devlist >> at scbus0 target 0 lun 0 (ada0,pass0) >> at scbus1 target 0 lun 0 (ada1,pass1) >> at scbus2 target 0 lun 0 (ada2,pass2) >> at scbus2 target 1 lun 0 (ada3,pass3) >> at scbus3 target 0 lun 0 (ada4,pass4) >> at scbus3 target 1 lun 0 (ada5,pass5) >> >> ada2 and ada3 are configured as raid/r0 holding the system. >> >> ada0, ada1, ada4 and ada5 should be a zfs pool with /home >> >> >> cat /etc/fstab >> >> # Device Mountpoint FStype Options Dump Pass# >> /dev/raid/r0s1b none swap sw 0 0 >> /dev/raid/r0s1a / ufs rw 1 1 >> /dev/raid/r0s1e /tmp ufs rw 2 2 >> /dev/raid/r0s1f /usr ufs rw 2 2 >> /dev/raid/r0s1d /var ufs rw 2 2 >> >> >> I'm not sure but shouldn't the mount point for /home also appear in >> fstab? >> >> Maybe I messed up during the manual editing of files during the update >> process? >> >> >> Can I recover or shall I destroy the pool and rebuild it? >> >> Thanks >> >> /Leslie > > Mounts from zfs do not have to be in fstab. You can set a mountpoint on > a zfs volume which kind of mounts itself. > Can you send more information about the 'Unable to attach to ada[0-4]' > messages? Maybe a complete dmesg (/var/run/dmesg.boot). > And how are these disks configured/partitioned? Output from 'gpart list > ada0' or 'gpart show ada0' and for the other disks. > Are you using stuff like glabel? > > Ronald. cat /var/run/dmesg.boot ----------------------------------------------- Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.1-RELEASE-p3 #0: Mon Apr 29 18:27:25 UTC 2013 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 link_elf_obj: symbol ata_controlcmd undefined KLD file atapicam.ko - could not finalize loading CPU: Intel(R) Pentium(R) 4 CPU 3.20GHz (3192.08-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0xf43 Family = f Model = 4 Stepping = 3 Features=0xbfebfbff Features2=0x649d AMD Features=0x20100800 TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 4084785152 (3895 MB) Event timer "LAPIC" quality 400 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 HTT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP/HT): APIC ID: 1 ioapic0: Changing APIC ID to 4 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: Power Button (fixed) acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, bfde0000 (3) failed cpu0: on acpi0 cpu1: on acpi0 attimer0: port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 atrtc0: port 0x70-0x73 irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 16 at device 1.0 on pci0 pci1: on pcib1 vgapci0: port 0xbe00-0xbeff mem 0xd0000000-0xdfffffff,0xfd9e0000-0xfd9effff irq 16 at device 0.0 on pci1 hdac0: mem 0xfd9fc000-0xfd9fffff irq 17 at device 0.1 on pci1 pcib2: irq 16 at device 28.0 on pci0 pci2: on pcib2 siis0: port 0xaf00-0xaf7f mem 0xfd6ff000-0xfd6ff07f,0xfd6f8000-0xfd6fbfff irq 16 at device 0.0 on pci2 siisch0: at channel 0 on siis0 siisch1: at channel 1 on siis0 pcib3: irq 17 at device 28.1 on pci0 pci3: on pcib3 bge0: mem 0xfddf0000-0xfddfffff irq 17 at device 0.0 on pci3 bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E miibus0: on bge0 brgphy0: PHY 1 on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge0: Ethernet address: 00:50:8d:7c:c2:60 pcib4: irq 16 at device 28.4 on pci0 pci4: on pcib4 bge1: mem 0xfdbf0000-0xfdbfffff irq 16 at device 0.0 on pci4 bge1: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E miibus1: on bge1 brgphy1: PHY 1 on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, auto-flow bge1: Ethernet address: 00:50:8d:7c:c2:61 uhci0: port 0xff00-0xff1f irq 23 at device 29.0 on pci0 usbus0 on uhci0 uhci1: port 0xfe00-0xfe1f irq 19 at device 29.1 on pci0 usbus1 on uhci1 uhci2: port 0xfd00-0xfd1f irq 18 at device 29.2 on pci0 usbus2 on uhci2 uhci3: port 0xfc00-0xfc1f irq 16 at device 29.3 on pci0 usbus3 on uhci3 ehci0: mem 0xfdfff000-0xfdfff3ff irq 23 at device 29.7 on pci0 usbus4: EHCI version 1.0 usbus4 on ehci0 pcib5: at device 30.0 on pci0 pci5: on pcib5 fwohci0: mem 0xfd8ff000-0xfd8ff7ff,0xfd8f8000-0xfd8fbfff irq 17 at device 2.0 on pci5 fwohci0: OHCI version 1.10 (ROM=1) fwohci0: No. of Isochronous channels is 4. fwohci0: EUI64 00:50:8d:00:00:72:25:9e fwohci0: invalid speed 7 (fixed to 3). fwohci0: Phy 1394a available S800, 3 ports. fwohci0: Link S800, max_rec 4096 bytes. firewire0: on fwohci0 fwe0: on firewire0 if_fwe0: Fake Ethernet address: 02:50:8d:72:25:9e fwe0: Ethernet address: 02:50:8d:72:25:9e fwip0: on firewire0 fwip0: Firewire address: 00:50:8d:00:00:72:25:9e @ 0xfffe00000000, S800, maxrec 4096 dcons_crom0: on firewire0 dcons_crom0: bus_addr 0x2a20000 fwohci0: Initiate bus reset fwohci0: fwohci_intr_core: BUS reset fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1, CYCLEMASTER mode isab0: at device 31.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfb00-0xfb0f mem 0xfdffe000-0xfdffe3ff at device 31.2 on pci0 ata0: at channel 0 on atapci0 ata1: at channel 1 on atapci0 pci0: at device 31.3 (no driver attached) acpi_tz0: on acpi0 acpi_tz0: _CRT value is absurd, ignored (255.0C) fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 orm0: at iomem 0xc0000-0xcffff,0xd0000-0xd3fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 atkbdc0: at port 0x60,0x64 on isa0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] ppc0: cannot reserve I/O port range ctl: CAM Target Layer loaded acpi_perf0: on cpu0 p4tcc0: on cpu0 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 102d0000102d device_attach: est1 attach returned 6 p4tcc1: on cpu1 (noperiph:siisch0:0:-1:-1): rescan already queued (noperiph:siisch1:0:-1:-1): rescan already queued firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me) firewire0: bus manager 0 fwohci0: phy int Timecounters tick every 1.000 msec hdacc0: at cad 0 on hdac0 hdaa0: at nid 1 on hdacc0 pcm0: at nid 3 on hdaa0 usbus0: 12Mbps Full Speed USB v1.0 usbus1: 12Mbps Full Speed USB v1.0 usbus2: 12Mbps Full Speed USB v1.0 usbus3: 12Mbps Full Speed USB v1.0 usbus4: 480Mbps High Speed USB v2.0 acpi_tz0: _CRT value is absurd, ignored (255.0C) acpi_tz0: failed to set new freq, disabling passive cooling ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 ugen2.1: at usbus2 uhub2: on usbus2 ugen3.1: at usbus3 uhub3: on usbus3 ugen4.1: at usbus4 uhub4: on usbus4 (aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 00 00 00 40 00 00 00 00 03 00 (aprobe2:ata0:0:0:0): CAM status: ATA Status Error (aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) (aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00 (aprobe2:ata0:0:0:0): Retrying command (aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 00 00 00 40 00 00 00 00 03 00 (aprobe2:ata0:0:0:0): CAM status: ATA Status Error (aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) (aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00 (aprobe2:ata0:0:0:0): Error 5, Retries exhausted (aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 00 00 00 40 00 00 00 00 03 00 (aprobe0:ata0:0:1:0): CAM status: ATA Status Error (aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) (aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00 (aprobe0:ata0:0:1:0): Retrying command (aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 00 00 00 40 00 00 00 00 03 00 (aprobe0:ata0:0:1:0): CAM status: ATA Status Error (aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) (aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00 (aprobe0:ata0:0:1:0): Error 5, Retries exhausted acpi_tz0: _CRT value is absurd, ignored (255.0C) uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered uhub2: 2 ports with 2 removable, self powered uhub3: 2 ports with 2 removable, self powered uhub4: 8 ports with 8 removable, self powered ada0 at siisch0 bus 0 scbus0 target 0 lun 0 ada0: ATA-7 SATA 1.x device ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes) ada0: Command Queueing enabled ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) ada0: Previously was known as ad4 ada1 at siisch1 bus 0 scbus1 target 0 lun 0 ada1: ATA-7 SATA 1.x device ada1: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad6 ada2 at ata0 bus 0 scbus2 target 0 lun 0 ada2: ATA-6 SATA 1.x device ada2: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) ada2: Previously was known as ad0 ada3 at ata0 bus 0 scbus2 target 1 lun 0 ada3: ATA-6 SATA 1.x device ada3: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) ada3: Previously was known as ad1 ada4 at ata1 bus 0 scbus3 target 0 lun 0 ada4: ATA-7 SATA 1.x device ada4: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) ada4: Previously was known as ad2 ada5 at ata1 bus 0 scbus3 target 1 lun 0 ada5: ATA-7 SATA 1.x device ada5: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) ada5: Previously was known as ad3 SMP: AP CPU #1 Launched! Timecounter "TSC-low" frequency 12469046 Hz quality 1000 GEOM_RAID: Intel-165267ff: Array Intel-165267ff created. GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created. GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE. GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from NONE to ACTIVE. GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE. GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from NONE to ACTIVE. GEOM_RAID: Intel-841a1a00: Array started. GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from STARTING to OPTIMAL. GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created. GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created. Root mount waiting for: GRAID-SiI GRAID-Intel ugen1.2: at usbus1 ukbd0: on usbus1 kbd2 at ukbd0 Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel Root mount waiting for: GRAID-SiI GRAID-Intel GEOM_RAID: Intel-165267ff: Force array start due to timeout. GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE. GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from NONE to ACTIVE. GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE. GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from NONE to ACTIVE. GEOM_RAID: Intel-165267ff: Array started. GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from STARTING to BROKEN. GEOM_RAID: SiI-100311082330: Force array start due to timeout. GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE. GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state changed from NONE to STALE. GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE. GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state changed from NONE to STALE. GEOM_RAID: SiI-100311082330: Array started. GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from STARTING to BROKEN. Trying to mount root from ufs:/dev/raid/r0s1a [rw]... ZFS filesystem version 5 ZFS storage pool version 28 ZFS WARNING: Unable to attach to ada0. ZFS WARNING: Unable to attach to ada1. ZFS WARNING: Unable to attach to ada4. ZFS WARNING: Unable to attach to ada5. ZFS WARNING: Unable to attach to ada0. ZFS WARNING: Unable to attach to ada1. ZFS WARNING: Unable to attach to ada4. ZFS WARNING: Unable to attach to ada5. ZFS WARNING: Unable to attach to ada0. ZFS WARNING: Unable to attach to ada1. ZFS WARNING: Unable to attach to ada4. ZFS WARNING: Unable to attach to ada5. ---------------------------------------------------- gpart list ------------------------------------------------ Geom name: raid/r0 modified: false state: OK fwheads: 255 fwsectors: 63 last: 72298495 first: 63 entries: 4 scheme: MBR Providers: 1. Name: raid/r0s1 Mediasize: 37013727744 (34G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 32256 Mode: r5w5e9 attrib: active rawtype: 165 length: 37013727744 offset: 32256 type: freebsd index: 1 end: 72292499 start: 63 Consumers: 1. Name: raid/r0 Mediasize: 37016829952 (34G) Sectorsize: 512 Mode: r5w5e14 Geom name: raid/r0s1 modified: false state: OK fwheads: 255 fwsectors: 63 last: 72292436 first: 0 entries: 8 scheme: BSD Providers: 1. Name: raid/r0s1a Mediasize: 1073741824 (1.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 32256 Mode: r1w1e1 rawtype: 7 length: 1073741824 offset: 0 type: freebsd-ufs index: 1 end: 2097151 start: 0 2. Name: raid/r0s1b Mediasize: 4294967296 (4.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 1073774080 Mode: r1w1e0 rawtype: 1 length: 4294967296 offset: 1073741824 type: freebsd-swap index: 2 end: 10485759 start: 2097152 3. Name: raid/r0s1d Mediasize: 5368709120 (5.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 1073774080 Mode: r1w1e1 rawtype: 7 length: 5368709120 offset: 5368709120 type: freebsd-ufs index: 4 end: 20971519 start: 10485760 4. Name: raid/r0s1e Mediasize: 2147483648 (2.0G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 2147515904 Mode: r1w1e1 rawtype: 7 length: 2147483648 offset: 10737418240 type: freebsd-ufs index: 5 end: 25165823 start: 20971520 5. Name: raid/r0s1f Mediasize: 24128825856 (22G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 32256 Mode: r1w1e1 rawtype: 7 length: 24128825856 offset: 12884901888 type: freebsd-ufs index: 6 end: 72292436 start: 25165824 Consumers: 1. Name: raid/r0s1 Mediasize: 37013727744 (34G) Sectorsize: 512 Stripesize: 0 Stripeoffset: 32256 Mode: r5w5e9 ---------------------------------------- gpart list ada0 gpart: No such geom: ada0. gpart list ada1 gpart: No such geom: ada1. gpart list ada4 gpart: No such geom: ada4. gpart list ada5 gpart: No such geom: ada5. gpart list ad0 gpart: No such geom: ad0. gpart list ad1 gpart: No such geom: ad1. gpart list ad4 gpart: No such geom: ad4. gpart list ad5 gpart: No such geom: ad5 -------------------------------------- gpart show (This is the system raid) ------------------------------------- => 63 72298433 raid/r0 MBR (34G) 63 72292437 1 freebsd [active] (34G) 72292500 5996 - free - (3M) => 0 72292437 raid/r0s1 BSD (34G) 0 2097152 1 freebsd-ufs (1.0G) 2097152 8388608 2 freebsd-swap (4.0G) 10485760 10485760 4 freebsd-ufs (5.0G) 20971520 4194304 5 freebsd-ufs (2.0G) 25165824 47126613 6 freebsd-ufs (22G) -------------------------------------------- gpart show ada0 gpart: No such geom: ada0 gpart show ada1 gpart: No such geom: ada1 gpart show ada4 gpart: No such geom: ada4 gpart show ada5 gpart: No such geom: ada5 I'm not sure about glabel, Sorry. Thanks /Leslie From owner-freebsd-fs@FreeBSD.ORG Wed May 22 18:01:02 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 382695EE for ; Wed, 22 May 2013 18:01:02 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qe0-f45.google.com (mail-qe0-f45.google.com [209.85.128.45]) by mx1.freebsd.org (Postfix) with ESMTP id F2AA8A83 for ; Wed, 22 May 2013 18:01:01 +0000 (UTC) Received: by mail-qe0-f45.google.com with SMTP id q19so1314173qeb.4 for ; Wed, 22 May 2013 11:01:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=giKXLYSquWvKLjNZ8iZpI+NfLpCI7b5Hi+3vFVGrn0c=; b=Mw8NXL6ti6Yfg77DS4eH/8AwqSXmQMDXhGBcXhRx4aRUAOjLW/OlSCUGbArxYt7FhW WAUGFgjPKF6I1qk7AIfS2/0pu9bPMiJuCHmYoCG79JdV69lE4GFwl4qy7ZBuu31wYCL7 DaFbDUl2Mpg6vJkbucJsldTvymNQjCAV8gD/FHe0SfvF0MU/IuGMZP603fZuTiW5KGQ9 78Epeeb7mRJ6PR9VLMBCu3kkt0XoUeU34BldsIS8zqRbKWiA6jXAjiXrY48dPJOeVany q+Tp7eBKfjTu5AZtgeCpT3WLFoA7GOjCsyiriBKq6jrAxQ1rJFqUUwNoQRINgdmdYFRS hBgg== MIME-Version: 1.0 X-Received: by 10.224.38.133 with SMTP id b5mr8139394qae.78.1369245247721; Wed, 22 May 2013 10:54:07 -0700 (PDT) Received: by 10.49.0.66 with HTTP; Wed, 22 May 2013 10:54:07 -0700 (PDT) In-Reply-To: <519D0428.8060800@eskk.nu> References: <519CA3A5.3020109@eskk.nu> <519D0428.8060800@eskk.nu> Date: Wed, 22 May 2013 10:54:07 -0700 Message-ID: Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach From: Freddie Cash To: Leslie Jensen Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 18:01:02 -0000 On Wed, May 22, 2013 at 10:45 AM, Leslie Jensen wrote: > zpool import in single user mode gives: > > zfs filesystem version 5 > zfs storage pool version 28 > ZFS WARNING: Unable to attach to ada0. > ZFS WARNING: Unable to attach to ada1. > ZFS WARNING: Unable to attach to ada4. > ZFS WARNING: Unable to attach to ada5. > > > > grep ada /var/log/messages > ------------------------------**-------------- > > May 22 19:34:03 bagnerbsd kernel: ada0 at siisch0 bus 0 scbus0 target 0 > lun 0 > May 22 19:34:03 bagnerbsd kernel: ada0: ATA-7 SATA 1.x > device > May 22 19:34:03 bagnerbsd kernel: ada0: 150.000MB/s transfers (SATA 1.x, > UDMA6, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada0: Command Queueing enabled > May 22 19:34:03 bagnerbsd kernel: ada0: 305245MB (625142448 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada0: Previously was known as ad4 > May 22 19:34:03 bagnerbsd kernel: ada1 at siisch1 bus 0 scbus1 target 0 > lun 0 > May 22 19:34:03 bagnerbsd kernel: ada1: ATA-7 SATA 1.x > device > May 22 19:34:03 bagnerbsd kernel: ada1: 150.000MB/s transfers (SATA 1.x, > UDMA6, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada1: Command Queueing enabled > May 22 19:34:03 bagnerbsd kernel: ada1: 305245MB (625142448 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada1: Previously was known as ad6 > May 22 19:34:03 bagnerbsd kernel: ada2 at ata0 bus 0 scbus2 target 0 lun 0 > May 22 19:34:03 bagnerbsd kernel: ada2: > ATA-6 SATA 1.x device > May 22 19:34:03 bagnerbsd kernel: ada2: 150.000MB/s transfers (SATA 1.x, > UDMA5, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada2: 35304MB (72303840 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada2: Previously was known as ad0 > May 22 19:34:03 bagnerbsd kernel: ada3 at ata0 bus 0 scbus2 target 1 lun 0 > May 22 19:34:03 bagnerbsd kernel: ada3: > ATA-6 SATA 1.x device > May 22 19:34:03 bagnerbsd kernel: ada3: 150.000MB/s transfers (SATA 1.x, > UDMA5, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada3: 35304MB (72303840 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada3: Previously was known as ad1 > May 22 19:34:03 bagnerbsd kernel: ada4 at ata1 bus 0 scbus3 target 0 lun 0 > May 22 19:34:03 bagnerbsd kernel: ada4: ATA-7 SATA 1.x > device > May 22 19:34:03 bagnerbsd kernel: ada4: 150.000MB/s transfers (SATA 1.x, > UDMA5, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada4: 305245MB (625142448 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada4: Previously was known as ad2 > May 22 19:34:03 bagnerbsd kernel: ada5 at ata1 bus 0 scbus3 target 1 lun 0 > May 22 19:34:03 bagnerbsd kernel: ada5: ATA-7 SATA 1.x > device > May 22 19:34:03 bagnerbsd kernel: ada5: 150.000MB/s transfers (SATA 1.x, > UDMA5, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada5: 305245MB (625142448 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada5: Previously was known as ad3 > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk ada2 > state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk > Volume0:0-ada2 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk ada3 > state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk > Volume0:1-ada3 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada0 > state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk > Volume0:1-ada0 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada1 > state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk > Volume0:3-ada1 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada4 > state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk SiI > Raid5 Set:0-ada4 state changed from NONE to STALE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada5 > state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk SiI > Raid5 Set:1-ada5 state changed from NONE to STALE. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5 > ------------------------------**---------------- > > I'm using whole disks. The pool was created back in 2010 and I have forgot > the details on how I did but I think I just created the out of the four > disks. > > The disks might have been renumbered I'm not sure. It would be logical > because the disks ada2 and ada3 had to bee changed in fstab as I wrote > before. > > Thanks > What happens if you rename /boot/zfs/zpool.cache, and run "zpool import -d /dev "? -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Wed May 22 18:04:28 2013 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id C5BEC757 for ; Wed, 22 May 2013 18:04:28 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 193CDAB8 for ; Wed, 22 May 2013 18:04:27 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id VAA13579; Wed, 22 May 2013 21:04:22 +0300 (EEST) (envelope-from avg@FreeBSD.org) Message-ID: <519D08A6.8020401@FreeBSD.org> Date: Wed, 22 May 2013 21:04:22 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130517 Thunderbird/17.0.6 MIME-Version: 1.0 To: Leslie Jensen Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach References: <519CA3A5.3020109@eskk.nu> <519D0428.8060800@eskk.nu> In-Reply-To: <519D0428.8060800@eskk.nu> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 18:04:28 -0000 on 22/05/2013 20:45 Leslie Jensen said the following: > > > 2013-05-22 17:33, Freddie Cash skrev: [snip] >> On Wed, May 22, 2013 at 3:53 AM, Leslie Jensen > > wrote: [snip] >> ada2 and ada3 are configured as raid/r0 holding the system. >> >> ada0, ada1, ada4 and ada5 should be a zfs pool with /home If you look at the messages below you can see that ada0, ada1, ada4 and ada5 are also claimed by the raid driver. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada0 state > changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk > Volume0:1-ada0 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk ada1 state > changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk > Volume0:3-ada1 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada4 state > changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 > Set:0-ada4 state changed from NONE to STALE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk ada5 state > changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 > Set:1-ada5 state changed from NONE to STALE. Apparently they used to be in raid configurations and the metadata was never properly cleared. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Wed May 22 18:06:24 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 0DB0A8D3 for ; Wed, 22 May 2013 18:06:24 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-qc0-x22f.google.com (mail-qc0-x22f.google.com [IPv6:2607:f8b0:400d:c01::22f]) by mx1.freebsd.org (Postfix) with ESMTP id C8C35AD5 for ; Wed, 22 May 2013 18:06:23 +0000 (UTC) Received: by mail-qc0-f175.google.com with SMTP id a1so1224865qcx.6 for ; Wed, 22 May 2013 11:06:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=U5NSRqfioN0KTLePWY5HPMxK6SkwbpNY14pRY5tm5FU=; b=mk9lHZP8jw5ICaANFx+b51ldekXmSPAqV8bC2sO983TZw/l5Eg/H2DKXlinFOOpxp1 ZtN7yWvMqaDnt2jelWFCKrcegGhy2gPLK1eAJCun1OrrhI10UjmbxT6C1HCJin3q2O11 URdObZUZyJeVr3F7fwzggLqop+wvyG9zY49mwBBQkdvFa6/S5HB0wFAJxEK3vNzjDe0u zKTIRdZ3fQNVwmjQsZ30EUK43XY0Yn7TmrOjx+1+EEiqhKo4XVMeuxGJ0YoVi2zn9TcA ga7h6yRP/zGbTFVZnW5rK/NRcbi+1929IwRGKScOjSoh5yVzCy5J4mHKTpu5ugGZayRH JE8g== MIME-Version: 1.0 X-Received: by 10.224.105.7 with SMTP id r7mr8314641qao.72.1369245983315; Wed, 22 May 2013 11:06:23 -0700 (PDT) Received: by 10.49.0.66 with HTTP; Wed, 22 May 2013 11:06:23 -0700 (PDT) In-Reply-To: <519D065A.3070806@eskk.nu> References: <519CA3A5.3020109@eskk.nu> <519D065A.3070806@eskk.nu> Date: Wed, 22 May 2013 11:06:23 -0700 Message-ID: Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach From: Freddie Cash To: Leslie Jensen Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: FreeBSD Filesystems , Ronald Klop X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 18:06:24 -0000 Do you have RAID arrays on both the Intel and the Sil controllers? Or just the Intel? I think you may be getting bit by the "graid now in kernel, old metadata being read, graid taking control of disks it shouldn't be" issue. You need to manually clear the RAID metadata from all your non-RAID (ie, used for ZFS) disks. See the graid man page for details. And the -stable mailing list archives. Once you've cleared the RAID metadata from the ZFS disks, ZFS will be able to pick them up correctly. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Wed May 22 18:07:00 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D69D894F for ; Wed, 22 May 2013 18:07:00 +0000 (UTC) (envelope-from leslie@eskk.nu) Received: from mx1.bjare.net (mx1.bjare.net [212.31.160.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4F231ADF for ; Wed, 22 May 2013 18:07:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mx1.bjare.net (Postfix) with ESMTP id 6AFE45E551; Wed, 22 May 2013 20:06:59 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mx1.bjare.net X-Spam-Flag: NO X-Spam-Score: -0.968 X-Spam-Level: X-Spam-Status: No, score=-0.968 tagged_above=-999 required=5 tests=[AWL=0.030, BAYES_00=-2.599, RCVD_IN_PBL=0.905, RDNS_DYNAMIC=0.1, SPF_SOFTFAIL=0.596] Received: from mx1.bjare.net ([127.0.0.1]) by localhost (mx1.bjare.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id LBz4pceVSJcS; Wed, 22 May 2013 20:06:54 +0200 (CEST) X-BN-MX1: ja X-BN-MailInfo: BjareNet Received: from bljbsd01.no-ip.org (c224-156-70-80.bjare.net [80.70.156.224]) by mx1.bjare.net (Postfix) with ESMTP id 6056A5E4F5; Wed, 22 May 2013 20:06:54 +0200 (CEST) Message-ID: <519D093E.8020304@eskk.nu> Date: Wed, 22 May 2013 20:06:54 +0200 From: Leslie Jensen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130516 Thunderbird/17.0.6 MIME-Version: 1.0 To: Freddie Cash Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach References: <519CA3A5.3020109@eskk.nu> <519D0428.8060800@eskk.nu> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 18:07:00 -0000 2013-05-22 19:54, Freddie Cash skrev: > On Wed, May 22, 2013 at 10:45 AM, Leslie Jensen > wrote: > > zpool import in single user mode gives: > > zfs filesystem version 5 > zfs storage pool version 28 > ZFS WARNING: Unable to attach to ada0. > ZFS WARNING: Unable to attach to ada1. > ZFS WARNING: Unable to attach to ada4. > ZFS WARNING: Unable to attach to ada5. > > > > grep ada /var/log/messages > ------------------------------__-------------- > > May 22 19:34:03 bagnerbsd kernel: ada0 at siisch0 bus 0 scbus0 > target 0 lun 0 > May 22 19:34:03 bagnerbsd kernel: ada0: ATA-7 > SATA 1.x device > May 22 19:34:03 bagnerbsd kernel: ada0: 150.000MB/s transfers (SATA > 1.x, UDMA6, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada0: Command Queueing enabled > May 22 19:34:03 bagnerbsd kernel: ada0: 305245MB (625142448 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada0: Previously was known as ad4 > May 22 19:34:03 bagnerbsd kernel: ada1 at siisch1 bus 0 scbus1 > target 0 lun 0 > May 22 19:34:03 bagnerbsd kernel: ada1: ATA-7 > SATA 1.x device > May 22 19:34:03 bagnerbsd kernel: ada1: 150.000MB/s transfers (SATA > 1.x, UDMA6, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada1: Command Queueing enabled > May 22 19:34:03 bagnerbsd kernel: ada1: 305245MB (625142448 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada1: Previously was known as ad6 > May 22 19:34:03 bagnerbsd kernel: ada2 at ata0 bus 0 scbus2 target 0 > lun 0 > May 22 19:34:03 bagnerbsd kernel: ada2: 35.06K35> ATA-6 SATA 1.x device > May 22 19:34:03 bagnerbsd kernel: ada2: 150.000MB/s transfers (SATA > 1.x, UDMA5, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada2: 35304MB (72303840 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada2: Previously was known as ad0 > May 22 19:34:03 bagnerbsd kernel: ada3 at ata0 bus 0 scbus2 target 1 > lun 0 > May 22 19:34:03 bagnerbsd kernel: ada3: 35.06K35> ATA-6 SATA 1.x device > May 22 19:34:03 bagnerbsd kernel: ada3: 150.000MB/s transfers (SATA > 1.x, UDMA5, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada3: 35304MB (72303840 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada3: Previously was known as ad1 > May 22 19:34:03 bagnerbsd kernel: ada4 at ata1 bus 0 scbus3 target 0 > lun 0 > May 22 19:34:03 bagnerbsd kernel: ada4: ATA-7 > SATA 1.x device > May 22 19:34:03 bagnerbsd kernel: ada4: 150.000MB/s transfers (SATA > 1.x, UDMA5, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada4: 305245MB (625142448 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada4: Previously was known as ad2 > May 22 19:34:03 bagnerbsd kernel: ada5 at ata1 bus 0 scbus3 target 1 > lun 0 > May 22 19:34:03 bagnerbsd kernel: ada5: ATA-7 > SATA 1.x device > May 22 19:34:03 bagnerbsd kernel: ada5: 150.000MB/s transfers (SATA > 1.x, UDMA5, PIO 8192bytes) > May 22 19:34:03 bagnerbsd kernel: ada5: 305245MB (625142448 512 byte > sectors: 16H 63S/T 16383C) > May 22 19:34:03 bagnerbsd kernel: ada5: Previously was known as ad3 > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk > ada2 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk > Volume0:0-ada2 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Disk > ada3 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-841a1a00: Subdisk > Volume0:1-ada3 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk > ada0 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk > Volume0:1-ada0 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Disk > ada1 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: Intel-165267ff: Subdisk > Volume0:3-ada1 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk > ada4 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: > Subdisk SiI Raid5 Set:0-ada4 state changed from NONE to STALE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: Disk > ada5 state changed from NONE to ACTIVE. > May 22 19:34:03 bagnerbsd kernel: GEOM_RAID: SiI-100311082330: > Subdisk SiI Raid5 Set:1-ada5 state changed from NONE to STALE. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada0. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada1. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada4. > May 22 19:34:03 bagnerbsd kernel: ZFS WARNING: Unable to attach to ada5 > ------------------------------__---------------- > > I'm using whole disks. The pool was created back in 2010 and I have > forgot the details on how I did but I think I just created the out > of the four disks. > > The disks might have been renumbered I'm not sure. It would be > logical because the disks ada2 and ada3 had to bee changed in fstab > as I wrote before. > > Thanks > > > What happens if you rename /boot/zfs/zpool.cache, and run "zpool import > -d /dev "? > > -- > Freddie Cash > fjwcash@gmail.com This is what happens zpool import -d /dev storage cannot import 'storage': a pool with that name is already created/imported, and no additional pools with that name were found /Leslie From owner-freebsd-fs@FreeBSD.ORG Wed May 22 18:20:06 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 348C3136 for ; Wed, 22 May 2013 18:20:06 +0000 (UTC) (envelope-from leslie@eskk.nu) Received: from mx1.bjare.net (mx1.bjare.net [212.31.160.3]) by mx1.freebsd.org (Postfix) with ESMTP id E7B76C26 for ; Wed, 22 May 2013 18:20:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mx1.bjare.net (Postfix) with ESMTP id 220785E551; Wed, 22 May 2013 20:20:05 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mx1.bjare.net X-Spam-Flag: NO X-Spam-Score: -0.969 X-Spam-Level: X-Spam-Status: No, score=-0.969 tagged_above=-999 required=5 tests=[AWL=0.029, BAYES_00=-2.599, RCVD_IN_PBL=0.905, RDNS_DYNAMIC=0.1, SPF_SOFTFAIL=0.596] Received: from mx1.bjare.net ([127.0.0.1]) by localhost (mx1.bjare.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 6Il5Ekvmvn8d; Wed, 22 May 2013 20:20:00 +0200 (CEST) X-BN-MX1: ja X-BN-MailInfo: BjareNet Received: from bljbsd01.no-ip.org (c224-156-70-80.bjare.net [80.70.156.224]) by mx1.bjare.net (Postfix) with ESMTP id 05C7C5E53F; Wed, 22 May 2013 20:19:59 +0200 (CEST) Message-ID: <519D0C4F.8070403@eskk.nu> Date: Wed, 22 May 2013 20:19:59 +0200 From: Leslie Jensen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130516 Thunderbird/17.0.6 MIME-Version: 1.0 To: Freddie Cash Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach References: <519CA3A5.3020109@eskk.nu> <519D065A.3070806@eskk.nu> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Filesystems , Ronald Klop X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 18:20:06 -0000 2013-05-22 20:06, Freddie Cash skrev: > Do you have RAID arrays on both the Intel and the Sil controllers? Or > just the Intel? I think you may be getting bit by the "graid now in > kernel, old metadata being read, graid taking control of disks it > shouldn't be" issue. > > You need to manually clear the RAID metadata from all your non-RAID (ie, > used for ZFS) disks. > > See the graid man page for details. And the -stable mailing list archives. > > Once you've cleared the RAID metadata from the ZFS disks, ZFS will be > able to pick them up correctly. > > -- > Freddie Cash > fjwcash@gmail.com I'll look into it. It should only be ada2 and ada2 that are in a raid configuration, if I remember correctly ;-) /Leslie From owner-freebsd-fs@FreeBSD.ORG Wed May 22 18:30:45 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 5EE5C656 for ; Wed, 22 May 2013 18:30:45 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from qmta12.emeryville.ca.mail.comcast.net (qmta12.emeryville.ca.mail.comcast.net [IPv6:2001:558:fe2d:44:76:96:27:227]) by mx1.freebsd.org (Postfix) with ESMTP id 30712D4F for ; Wed, 22 May 2013 18:30:45 +0000 (UTC) Received: from omta13.emeryville.ca.mail.comcast.net ([76.96.30.52]) by qmta12.emeryville.ca.mail.comcast.net with comcast id f5Xj1l00517UAYk016Wk3M; Wed, 22 May 2013 18:30:44 +0000 Received: from jdc.koitsu.org ([67.180.84.87]) by omta13.emeryville.ca.mail.comcast.net with comcast id f6Wj1l0011t3BNj8Z6WjzZ; Wed, 22 May 2013 18:30:43 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id D39FC73A33; Wed, 22 May 2013 11:30:42 -0700 (PDT) Date: Wed, 22 May 2013 11:30:42 -0700 From: Jeremy Chadwick To: Leslie Jensen Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach Message-ID: <20130522183042.GA1294@icarus.home.lan> References: <519CA3A5.3020109@eskk.nu> <519D065A.3070806@eskk.nu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <519D065A.3070806@eskk.nu> User-Agent: Mutt/1.5.21 (2010-09-15) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20121106; t=1369247444; bh=jk9kpVoVSmhlMKAb6YAOHMgelJ20H0L17lso86mHGj4=; h=Received:Received:Received:Date:From:To:Subject:Message-ID: MIME-Version:Content-Type; b=FfUtJElX78z3ekpkOVihNS9rgoqFgSGCVZS8B5VoH8eRtBPk5r31QyLAUI+I+IVZq 7qitJ0AcCBjrudKEH2kmgQSKLGDkc6k4Z8OYtpTmkEDHmTBwkVYV6XtI3yejtZ+UT2 k5ZkPOZCbq8dIoCSCkGX5Ym9BfT/MfU6BNyAaz2y9sHk9v/tVZpnGo7OIh4kkXZH+g J5UKoyRoQ1dDoKo9Q6qD3nSFIR4AZgsYmFLanq9PjsMmUjB1mjL4WXpGjXmRbOgBLT mjo/IoLFBmSVPdmMLeRUglIWoeOE2JYzoL8ENqa+IjDU3D/duHqO9oNUI0h3gug9Mr 6wtGk4S5zvoVA== Cc: freebsd-fs@freebsd.org, Ronald Klop X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 18:30:45 -0000 On Wed, May 22, 2013 at 07:54:34PM +0200, Leslie Jensen wrote: > > > 2013-05-22 17:02, Ronald Klop skrev: > >On Wed, 22 May 2013 12:53:25 +0200, Leslie Jensen wrote: > > > >> > >>I've upgraded a machine with freebsd-update from 8.3 to 9.1. > >> > >>After the first restart I edited /etc/fstab in single user mode > >>because the names on the disks had changed. But the zfs pool I have > >>seem to have a problem and I'm sure on how to recover it. > >> > >> > >>May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0. > >>May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1. > >>May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4. > >>May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5. > >> > >> > >>camcontrol devlist > >> at scbus0 target 0 lun 0 (ada0,pass0) > >> at scbus1 target 0 lun 0 (ada1,pass1) > >> at scbus2 target 0 lun 0 (ada2,pass2) > >> at scbus2 target 1 lun 0 (ada3,pass3) > >> at scbus3 target 0 lun 0 (ada4,pass4) > >> at scbus3 target 1 lun 0 (ada5,pass5) > >> > >>ada2 and ada3 are configured as raid/r0 holding the system. > >> > >>ada0, ada1, ada4 and ada5 should be a zfs pool with /home > >> > >> > >>cat /etc/fstab > >> > >># Device Mountpoint FStype Options Dump Pass# > >>/dev/raid/r0s1b none swap sw 0 0 > >>/dev/raid/r0s1a / ufs rw 1 1 > >>/dev/raid/r0s1e /tmp ufs rw 2 2 > >>/dev/raid/r0s1f /usr ufs rw 2 2 > >>/dev/raid/r0s1d /var ufs rw 2 2 > >> > >> > >>I'm not sure but shouldn't the mount point for /home also appear in > >>fstab? > >> > >>Maybe I messed up during the manual editing of files during the update > >>process? > >> > >> > >>Can I recover or shall I destroy the pool and rebuild it? > >> > >>Thanks > >> > >>/Leslie > > > >Mounts from zfs do not have to be in fstab. You can set a mountpoint on > >a zfs volume which kind of mounts itself. > >Can you send more information about the 'Unable to attach to ada[0-4]' > >messages? Maybe a complete dmesg (/var/run/dmesg.boot). > >And how are these disks configured/partitioned? Output from 'gpart list > >ada0' or 'gpart show ada0' and for the other disks. > >Are you using stuff like glabel? > > > >Ronald. > > > cat /var/run/dmesg.boot > ----------------------------------------------- > Copyright (c) 1992-2012 The FreeBSD Project. > Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 > The Regents of the University of California. All rights reserved. > FreeBSD is a registered trademark of The FreeBSD Foundation. > FreeBSD 9.1-RELEASE-p3 #0: Mon Apr 29 18:27:25 UTC 2013 > root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 > link_elf_obj: symbol ata_controlcmd undefined > KLD file atapicam.ko - could not finalize loading > CPU: Intel(R) Pentium(R) 4 CPU 3.20GHz (3192.08-MHz K8-class CPU) > Origin = "GenuineIntel" Id = 0xf43 Family = f Model = 4 Stepping = 3 > > Features=0xbfebfbff > Features2=0x649d > AMD Features=0x20100800 > TSC: P-state invariant > real memory = 4294967296 (4096 MB) > avail memory = 4084785152 (3895 MB) > Event timer "LAPIC" quality 400 > ACPI APIC Table: > FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs > FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 HTT threads > cpu0 (BSP): APIC ID: 0 > cpu1 (AP/HT): APIC ID: 1 > ioapic0: Changing APIC ID to 4 > ioapic0 irqs 0-23 on motherboard > kbd1 at kbdmux0 > acpi0: on motherboard > acpi0: Power Button (fixed) > acpi0: reservation of 0, a0000 (3) failed > acpi0: reservation of 100000, bfde0000 (3) failed > cpu0: on acpi0 > cpu1: on acpi0 > attimer0: port 0x40-0x43 irq 0 on acpi0 > Timecounter "i8254" frequency 1193182 Hz quality 0 > Event timer "i8254" frequency 1193182 Hz quality 100 > atrtc0: port 0x70-0x73 irq 8 on acpi0 > Event timer "RTC" frequency 32768 Hz quality 0 > Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 > acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 > acpi_button0: on acpi0 > pcib0: port 0xcf8-0xcff on acpi0 > pci0: on pcib0 > pcib1: irq 16 at device 1.0 on pci0 > pci1: on pcib1 > vgapci0: port 0xbe00-0xbeff mem > 0xd0000000-0xdfffffff,0xfd9e0000-0xfd9effff irq 16 at device 0.0 on > pci1 > hdac0: mem 0xfd9fc000-0xfd9fffff irq 17 > at device 0.1 on pci1 > pcib2: irq 16 at device 28.0 on pci0 > pci2: on pcib2 > siis0: port 0xaf00-0xaf7f mem > 0xfd6ff000-0xfd6ff07f,0xfd6f8000-0xfd6fbfff irq 16 at device 0.0 on > pci2 > siisch0: at channel 0 on siis0 > siisch1: at channel 1 on siis0 > pcib3: irq 17 at device 28.1 on pci0 > pci3: on pcib3 > bge0: 0x004101> mem 0xfddf0000-0xfddfffff irq 17 at device 0.0 on pci3 > bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E > miibus0: on bge0 > brgphy0: PHY 1 on miibus0 > brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, > 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, > auto-flow > bge0: Ethernet address: 00:50:8d:7c:c2:60 > pcib4: irq 16 at device 28.4 on pci0 > pci4: on pcib4 > bge1: 0x004101> mem 0xfdbf0000-0xfdbfffff irq 16 at device 0.0 on pci4 > bge1: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E > miibus1: on bge1 > brgphy1: PHY 1 on miibus1 > brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, > 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, > auto-flow > bge1: Ethernet address: 00:50:8d:7c:c2:61 > uhci0: port 0xff00-0xff1f > irq 23 at device 29.0 on pci0 > usbus0 on uhci0 > uhci1: port 0xfe00-0xfe1f > irq 19 at device 29.1 on pci0 > usbus1 on uhci1 > uhci2: port 0xfd00-0xfd1f > irq 18 at device 29.2 on pci0 > usbus2 on uhci2 > uhci3: port 0xfc00-0xfc1f > irq 16 at device 29.3 on pci0 > usbus3 on uhci3 > ehci0: mem > 0xfdfff000-0xfdfff3ff irq 23 at device 29.7 on pci0 > usbus4: EHCI version 1.0 > usbus4 on ehci0 > pcib5: at device 30.0 on pci0 > pci5: on pcib5 > fwohci0: mem > 0xfd8ff000-0xfd8ff7ff,0xfd8f8000-0xfd8fbfff irq 17 at device 2.0 on > pci5 > fwohci0: OHCI version 1.10 (ROM=1) > fwohci0: No. of Isochronous channels is 4. > fwohci0: EUI64 00:50:8d:00:00:72:25:9e > fwohci0: invalid speed 7 (fixed to 3). > fwohci0: Phy 1394a available S800, 3 ports. > fwohci0: Link S800, max_rec 4096 bytes. > firewire0: on fwohci0 > fwe0: on firewire0 > if_fwe0: Fake Ethernet address: 02:50:8d:72:25:9e > fwe0: Ethernet address: 02:50:8d:72:25:9e > fwip0: on firewire0 > fwip0: Firewire address: 00:50:8d:00:00:72:25:9e @ 0xfffe00000000, > S800, maxrec 4096 > dcons_crom0: on firewire0 > dcons_crom0: bus_addr 0x2a20000 > fwohci0: Initiate bus reset > fwohci0: fwohci_intr_core: BUS reset > fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1, > CYCLEMASTER mode > isab0: at device 31.0 on pci0 > isa0: on isab0 > atapci0: port > 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfb00-0xfb0f mem > 0xfdffe000-0xfdffe3ff at device 31.2 on pci0 > ata0: at channel 0 on atapci0 > ata1: at channel 1 on atapci0 > pci0: at device 31.3 (no driver attached) > acpi_tz0: on acpi0 > acpi_tz0: _CRT value is absurd, ignored (255.0C) > fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 > orm0: at iomem 0xc0000-0xcffff,0xd0000-0xd3fff on isa0 > sc0: at flags 0x100 on isa0 > sc0: VGA <16 virtual consoles, flags=0x300> > vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 > atkbdc0: at port 0x60,0x64 on isa0 > atkbd0: irq 1 on atkbdc0 > kbd0 at atkbd0 > atkbd0: [GIANT-LOCKED] > ppc0: cannot reserve I/O port range > ctl: CAM Target Layer loaded > acpi_perf0: on cpu0 > p4tcc0: on cpu0 > est1: on cpu1 > est: CPU supports Enhanced Speedstep, but is not recognized. > est: cpu_vendor GenuineIntel, msr 102d0000102d > device_attach: est1 attach returned 6 > p4tcc1: on cpu1 > (noperiph:siisch0:0:-1:-1): rescan already queued > (noperiph:siisch1:0:-1:-1): rescan already queued > firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me) > firewire0: bus manager 0 > fwohci0: phy int > Timecounters tick every 1.000 msec > hdacc0: at cad 0 on hdac0 > hdaa0: at nid 1 on hdacc0 > pcm0: at nid 3 on hdaa0 > usbus0: 12Mbps Full Speed USB v1.0 > usbus1: 12Mbps Full Speed USB v1.0 > usbus2: 12Mbps Full Speed USB v1.0 > usbus3: 12Mbps Full Speed USB v1.0 > usbus4: 480Mbps High Speed USB v2.0 > acpi_tz0: _CRT value is absurd, ignored (255.0C) > acpi_tz0: failed to set new freq, disabling passive cooling > ugen0.1: at usbus0 > uhub0: on usbus0 > ugen1.1: at usbus1 > uhub1: on usbus1 > ugen2.1: at usbus2 > uhub2: on usbus2 > ugen3.1: at usbus3 > uhub3: on usbus3 > ugen4.1: at usbus4 > uhub4: on usbus4 > (aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 > 00 00 00 40 00 00 00 00 03 00 > (aprobe2:ata0:0:0:0): CAM status: ATA Status Error > (aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) > (aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00 > (aprobe2:ata0:0:0:0): Retrying command > (aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 > 00 00 00 40 00 00 00 00 03 00 > (aprobe2:ata0:0:0:0): CAM status: ATA Status Error > (aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) > (aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00 > (aprobe2:ata0:0:0:0): Error 5, Retries exhausted > (aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 > 00 00 00 40 00 00 00 00 03 00 > (aprobe0:ata0:0:1:0): CAM status: ATA Status Error > (aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) > (aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00 > (aprobe0:ata0:0:1:0): Retrying command > (aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 > 00 00 00 40 00 00 00 00 03 00 > (aprobe0:ata0:0:1:0): CAM status: ATA Status Error > (aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) > (aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00 > (aprobe0:ata0:0:1:0): Error 5, Retries exhausted > acpi_tz0: _CRT value is absurd, ignored (255.0C) > uhub0: 2 ports with 2 removable, self powered > uhub1: 2 ports with 2 removable, self powered > uhub2: 2 ports with 2 removable, self powered > uhub3: 2 ports with 2 removable, self powered > uhub4: 8 ports with 8 removable, self powered > ada0 at siisch0 bus 0 scbus0 target 0 lun 0 > ada0: ATA-7 SATA 1.x device > ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes) > ada0: Command Queueing enabled > ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) > ada0: Previously was known as ad4 > ada1 at siisch1 bus 0 scbus1 target 0 lun 0 > ada1: ATA-7 SATA 1.x device > ada1: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes) > ada1: Command Queueing enabled > ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) > ada1: Previously was known as ad6 > ada2 at ata0 bus 0 scbus2 target 0 lun 0 > ada2: ATA-6 SATA 1.x device > ada2: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) > ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) > ada2: Previously was known as ad0 > ada3 at ata0 bus 0 scbus2 target 1 lun 0 > ada3: ATA-6 SATA 1.x device > ada3: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) > ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) > ada3: Previously was known as ad1 > ada4 at ata1 bus 0 scbus3 target 0 lun 0 > ada4: ATA-7 SATA 1.x device > ada4: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) > ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) > ada4: Previously was known as ad2 > ada5 at ata1 bus 0 scbus3 target 1 lun 0 > ada5: ATA-7 SATA 1.x device > ada5: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) > ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) > ada5: Previously was known as ad3 > SMP: AP CPU #1 Launched! > Timecounter "TSC-low" frequency 12469046 Hz quality 1000 > GEOM_RAID: Intel-165267ff: Array Intel-165267ff created. > GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created. > GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from > NONE to ACTIVE. > GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from > NONE to ACTIVE. > GEOM_RAID: Intel-841a1a00: Array started. > GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from > STARTING to OPTIMAL. > GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created. > GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created. > Root mount waiting for: GRAID-SiI GRAID-Intel > ugen1.2: at usbus1 > ukbd0: 2> on usbus1 > kbd2 at ukbd0 > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > Root mount waiting for: GRAID-SiI GRAID-Intel > GEOM_RAID: Intel-165267ff: Force array start due to timeout. > GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from > NONE to ACTIVE. > GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from > NONE to ACTIVE. > GEOM_RAID: Intel-165267ff: Array started. > GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from > STARTING to BROKEN. > GEOM_RAID: SiI-100311082330: Force array start due to timeout. > GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE. > GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state > changed from NONE to STALE. > GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE. > GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state > changed from NONE to STALE. > GEOM_RAID: SiI-100311082330: Array started. > GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from > STARTING to BROKEN. > Trying to mount root from ufs:/dev/raid/r0s1a [rw]... > ZFS filesystem version 5 > ZFS storage pool version 28 > ZFS WARNING: Unable to attach to ada0. > ZFS WARNING: Unable to attach to ada1. > ZFS WARNING: Unable to attach to ada4. > ZFS WARNING: Unable to attach to ada5. > ZFS WARNING: Unable to attach to ada0. > ZFS WARNING: Unable to attach to ada1. > ZFS WARNING: Unable to attach to ada4. > ZFS WARNING: Unable to attach to ada5. > ZFS WARNING: Unable to attach to ada0. > ZFS WARNING: Unable to attach to ada1. > ZFS WARNING: Unable to attach to ada4. > ZFS WARNING: Unable to attach to ada5. > ---------------------------------------------------- > > > > gpart list > ------------------------------------------------ > > Geom name: raid/r0 > modified: false > state: OK > fwheads: 255 > fwsectors: 63 > last: 72298495 > first: 63 > entries: 4 > scheme: MBR > Providers: > 1. Name: raid/r0s1 > Mediasize: 37013727744 (34G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 32256 > Mode: r5w5e9 > attrib: active > rawtype: 165 > length: 37013727744 > offset: 32256 > type: freebsd > index: 1 > end: 72292499 > start: 63 > Consumers: > 1. Name: raid/r0 > Mediasize: 37016829952 (34G) > Sectorsize: 512 > Mode: r5w5e14 > > Geom name: raid/r0s1 > modified: false > state: OK > fwheads: 255 > fwsectors: 63 > last: 72292436 > first: 0 > entries: 8 > scheme: BSD > Providers: > 1. Name: raid/r0s1a > Mediasize: 1073741824 (1.0G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 32256 > Mode: r1w1e1 > rawtype: 7 > length: 1073741824 > offset: 0 > type: freebsd-ufs > index: 1 > end: 2097151 > start: 0 > 2. Name: raid/r0s1b > Mediasize: 4294967296 (4.0G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 1073774080 > Mode: r1w1e0 > rawtype: 1 > length: 4294967296 > offset: 1073741824 > type: freebsd-swap > index: 2 > end: 10485759 > start: 2097152 > 3. Name: raid/r0s1d > Mediasize: 5368709120 (5.0G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 1073774080 > Mode: r1w1e1 > rawtype: 7 > length: 5368709120 > offset: 5368709120 > type: freebsd-ufs > index: 4 > end: 20971519 > start: 10485760 > 4. Name: raid/r0s1e > Mediasize: 2147483648 (2.0G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 2147515904 > Mode: r1w1e1 > rawtype: 7 > length: 2147483648 > offset: 10737418240 > type: freebsd-ufs > index: 5 > end: 25165823 > start: 20971520 > 5. Name: raid/r0s1f > Mediasize: 24128825856 (22G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 32256 > Mode: r1w1e1 > rawtype: 7 > length: 24128825856 > offset: 12884901888 > type: freebsd-ufs > index: 6 > end: 72292436 > start: 25165824 > Consumers: > 1. Name: raid/r0s1 > Mediasize: 37013727744 (34G) > Sectorsize: 512 > Stripesize: 0 > Stripeoffset: 32256 > Mode: r5w5e9 > > ---------------------------------------- > > gpart list ada0 > gpart: No such geom: ada0. > > gpart list ada1 > gpart: No such geom: ada1. > > gpart list ada4 > gpart: No such geom: ada4. > > gpart list ada5 > gpart: No such geom: ada5. > > gpart list ad0 > gpart: No such geom: ad0. > > gpart list ad1 > gpart: No such geom: ad1. > > gpart list ad4 > gpart: No such geom: ad4. > > gpart list ad5 > gpart: No such geom: ad5 > > -------------------------------------- > > > gpart show (This is the system raid) > > ------------------------------------- > > => 63 72298433 raid/r0 MBR (34G) > 63 72292437 1 freebsd [active] (34G) > 72292500 5996 - free - (3M) > > => 0 72292437 raid/r0s1 BSD (34G) > 0 2097152 1 freebsd-ufs (1.0G) > 2097152 8388608 2 freebsd-swap (4.0G) > 10485760 10485760 4 freebsd-ufs (5.0G) > 20971520 4194304 5 freebsd-ufs (2.0G) > 25165824 47126613 6 freebsd-ufs (22G) > > -------------------------------------------- > > gpart show ada0 > gpart: No such geom: ada0 > > gpart show ada1 > gpart: No such geom: ada1 > > gpart show ada4 > gpart: No such geom: ada4 > > gpart show ada5 > gpart: No such geom: ada5 > > > > I'm not sure about glabel, Sorry. I think I see what the issue is here. This is going to be a long Email. You have 6 disks: ada0 -- attached to siisch0 (Silicon Image 3132) ada1 -- attached to siisch1 (Silicon Image 3132) ada2 -- attached to ata0 (Intel ICH7, non-RAID) as master ada3 -- attached to ata0 (Intel ICH7, non-RAID) as slave ada4 -- attached to ata1 (Intel ICH7, non-RAID) as master ada5 -- attached to ata1 (Intel ICH7, non-RAID) as slave However, **BEFORE** ZFS starts, we can clearly see this (take the time to read it slowly and carefully) > GEOM_RAID: Intel-165267ff: Array Intel-165267ff created. > GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created. > GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-841a1a00: Array started. > GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from STARTING to OPTIMAL. > GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created. > GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created. > Root mount waiting for: GRAID-SiI GRAID-Intel > --- > --- this message repeats a very long time due to a problem > --- I'm about to try and explain > --- > GEOM_RAID: Intel-165267ff: Force array start due to timeout. > GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from NONE to ACTIVE. > GEOM_RAID: Intel-165267ff: Array started. > GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from STARTING to BROKEN. > GEOM_RAID: SiI-100311082330: Force array start due to timeout. > GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE. > GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state changed from NONE to STALE. > GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE. > GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state changed from NONE to STALE. > GEOM_RAID: SiI-100311082330: Array started. > GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from STARTING to BROKEN. > Trying to mount root from ufs:/dev/raid/r0s1a [rw]... This shows ***THREE*** graid(8) arrays being probed during disk tasting for RAID metadata. Remember two things: 1) graid(8) metadata tasting happens ***BEFORE*** ZFS metadata tasting. 2) The metadata is stored *on the disks themselves*. - Intel-165267ff -- disk ada0 -- disk ada1 - Intel-841a1a00 -- disk ada2 -- disk ada3 - SiI-100311082330 (used to be part of a RAID-5 set) -- disk ada4 -- disk ada5 All of these disks become "locked" by graid because of the discovery of metadata on them. Plain and simple: 1) ada0 and ada1 were previously used on an Intel controller that had BIOS-level RAID capability, and (presumably you) never cleared the metadata from the disks before making use of ZFS, 2) ada4 and ada5 were previously used on a Silicon Image controller that had RAID capability, and (presumably you) never cleared the metadata from the disks before making use of ZFS. Your next question is going to be "how to do I do that?" There are many ways to do this, and the one I'm going to tell you is the one that will work reliably without having to ""fight"" with GEOM or graid(8). So hear me loud and clear. 1. Boot a FreeBSD installation CD/DVD/USB stick 2. Escape to the loader prompt 3. Type: set kern.geom.raid.enable="0" 4. Type: boot 5. Go to a "Fixit" shell (the one that has all the useful utilities) 6. Zero the first and last 8MBytes or so (I'm not going to be accurate) of all the disks whose RAID controller metadata you want to remove. Apologies for the caps, but I need to make this clear: BE AWARE THIS WILL DESTROY PARTITION TABLES AND OVERWRITE/NUKE ANY OTHER KIND OF METADATA (INCLUDING ZFS), SO IF YOU HAVE DATA ON THESE DRIVES THAT YOU CARE ABOUT (SOMEHOW?), YOU HAD BETTER GET THE DATA OFF FIRST. FOR ZFS YOU WILL ALSO NEED TO RE-CREATE THE POOL AFTER THIS (AND IF THEY ARE 4K SECTOR DISKS YOU WILL NEED TO USE THE GNOP METHOD TO ENSURE PROPER ALIGNMENT; SEE IVAN VORAS' BLOG). The procedure to do that, for your disks (given their capacities) is this: > ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) > ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) > ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) > ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) > ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) > ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) sysctl kern.geom.debugflags=16 zpool import -a -- I am hoping your pool get imported successfully at this point. -- This might not even be needed depending upon what the kernel -- does during its earlier boot. zpool destroy {poolname} -- Yes, you really do want to do this. This is how you will get -- ZFS to "unlock" the disks that you're about to nuke metadata -- on. If you have multiple pools, you need to do this for -- all of them. dd if=/dev/zero of=/dev/ada0 bs=65536 count=128 dd if=/dev/zero of=/dev/ada0 bs=65536 count=128 oseek=4880000 dd if=/dev/zero of=/dev/ada1 bs=65536 count=128 dd if=/dev/zero of=/dev/ada1 bs=65536 count=128 oseek=4880000 dd if=/dev/zero of=/dev/ada4 bs=65536 count=128 dd if=/dev/zero of=/dev/ada4 bs=65536 count=128 oseek=4880000 dd if=/dev/zero of=/dev/ada5 bs=65536 count=128 dd if=/dev/zero of=/dev/ada5 bs=65536 count=128 oseek=4880000 On the commands with "oseek", you will probably receive an error that says something about "short end of device" or what not; that is perfectly fine. After this, I strongly recommend rebooting the system ("reboot") and booting off of your standard hard disks. Your system should start in multi-user, but obviously there will be no ZFS pools. At this point, recreate your ZFS pools as you did originally, and you should be good to go going forward. -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Wed May 22 20:02:28 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A5A3D919 for ; Wed, 22 May 2013 20:02:28 +0000 (UTC) (envelope-from leslie@eskk.nu) Received: from mx1.bjare.net (mx1.bjare.net [212.31.160.3]) by mx1.freebsd.org (Postfix) with ESMTP id A2641870 for ; Wed, 22 May 2013 20:02:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mx1.bjare.net (Postfix) with ESMTP id 66BD15E531; Wed, 22 May 2013 22:02:26 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at mx1.bjare.net X-Spam-Flag: NO X-Spam-Score: -0.271 X-Spam-Level: X-Spam-Status: No, score=-0.271 tagged_above=-999 required=5 tests=[AWL=-0.669, BAYES_00=-2.599, RCVD_IN_PBL=0.905, RDNS_DYNAMIC=0.1, SPF_SOFTFAIL=0.596, WEIRD_QUOTING=1.396] Received: from mx1.bjare.net ([127.0.0.1]) by localhost (mx1.bjare.net [127.0.0.1]) (amavisd-new, port 10024) with LMTP id 27FSF+yZtbDx; Wed, 22 May 2013 22:02:21 +0200 (CEST) X-BN-MX1: ja X-BN-MailInfo: BjareNet Received: from bljbsd01.no-ip.org (c224-156-70-80.bjare.net [80.70.156.224]) by mx1.bjare.net (Postfix) with ESMTP id E8A9F5E514; Wed, 22 May 2013 22:02:14 +0200 (CEST) Message-ID: <519D2446.7050803@eskk.nu> Date: Wed, 22 May 2013 22:02:14 +0200 From: Leslie Jensen User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130516 Thunderbird/17.0.6 MIME-Version: 1.0 To: Jeremy Chadwick Subject: Re: Upgrading from 8.3 to 9.1 zfs pool can't attach References: <519CA3A5.3020109@eskk.nu> <519D065A.3070806@eskk.nu> <20130522183042.GA1294@icarus.home.lan> In-Reply-To: <20130522183042.GA1294@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Ronald Klop X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 May 2013 20:02:28 -0000 2013-05-22 20:30, Jeremy Chadwick skrev: > On Wed, May 22, 2013 at 07:54:34PM +0200, Leslie Jensen wrote: >> >> >> 2013-05-22 17:02, Ronald Klop skrev: >>> On Wed, 22 May 2013 12:53:25 +0200, Leslie Jensen wrote: >>> >>>> >>>> I've upgraded a machine with freebsd-update from 8.3 to 9.1. >>>> >>>> After the first restart I edited /etc/fstab in single user mode >>>> because the names on the disks had changed. But the zfs pool I have >>>> seem to have a problem and I'm sure on how to recover it. >>>> >>>> >>>> May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada0. >>>> May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada1. >>>> May 22 12:00:39 kernel: ZFS WARNING: Unable to attach to ada4. >>>> May 22 12:00:40 kernel: ZFS WARNING: Unable to attach to ada5. >>>> >>>> >>>> camcontrol devlist >>>> at scbus0 target 0 lun 0 (ada0,pass0) >>>> at scbus1 target 0 lun 0 (ada1,pass1) >>>> at scbus2 target 0 lun 0 (ada2,pass2) >>>> at scbus2 target 1 lun 0 (ada3,pass3) >>>> at scbus3 target 0 lun 0 (ada4,pass4) >>>> at scbus3 target 1 lun 0 (ada5,pass5) >>>> >>>> ada2 and ada3 are configured as raid/r0 holding the system. >>>> >>>> ada0, ada1, ada4 and ada5 should be a zfs pool with /home >>>> >>>> >>>> cat /etc/fstab >>>> >>>> # Device Mountpoint FStype Options Dump Pass# >>>> /dev/raid/r0s1b none swap sw 0 0 >>>> /dev/raid/r0s1a / ufs rw 1 1 >>>> /dev/raid/r0s1e /tmp ufs rw 2 2 >>>> /dev/raid/r0s1f /usr ufs rw 2 2 >>>> /dev/raid/r0s1d /var ufs rw 2 2 >>>> >>>> >>>> I'm not sure but shouldn't the mount point for /home also appear in >>>> fstab? >>>> >>>> Maybe I messed up during the manual editing of files during the update >>>> process? >>>> >>>> >>>> Can I recover or shall I destroy the pool and rebuild it? >>>> >>>> Thanks >>>> >>>> /Leslie >>> >>> Mounts from zfs do not have to be in fstab. You can set a mountpoint on >>> a zfs volume which kind of mounts itself. >>> Can you send more information about the 'Unable to attach to ada[0-4]' >>> messages? Maybe a complete dmesg (/var/run/dmesg.boot). >>> And how are these disks configured/partitioned? Output from 'gpart list >>> ada0' or 'gpart show ada0' and for the other disks. >>> Are you using stuff like glabel? >>> >>> Ronald. >> >> >> cat /var/run/dmesg.boot >> ----------------------------------------------- >> Copyright (c) 1992-2012 The FreeBSD Project. >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 >> The Regents of the University of California. All rights reserved. >> FreeBSD is a registered trademark of The FreeBSD Foundation. >> FreeBSD 9.1-RELEASE-p3 #0: Mon Apr 29 18:27:25 UTC 2013 >> root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 >> link_elf_obj: symbol ata_controlcmd undefined >> KLD file atapicam.ko - could not finalize loading >> CPU: Intel(R) Pentium(R) 4 CPU 3.20GHz (3192.08-MHz K8-class CPU) >> Origin = "GenuineIntel" Id = 0xf43 Family = f Model = 4 Stepping = 3 >> >> Features=0xbfebfbff >> Features2=0x649d >> AMD Features=0x20100800 >> TSC: P-state invariant >> real memory = 4294967296 (4096 MB) >> avail memory = 4084785152 (3895 MB) >> Event timer "LAPIC" quality 400 >> ACPI APIC Table: >> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs >> FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 HTT threads >> cpu0 (BSP): APIC ID: 0 >> cpu1 (AP/HT): APIC ID: 1 >> ioapic0: Changing APIC ID to 4 >> ioapic0 irqs 0-23 on motherboard >> kbd1 at kbdmux0 >> acpi0: on motherboard >> acpi0: Power Button (fixed) >> acpi0: reservation of 0, a0000 (3) failed >> acpi0: reservation of 100000, bfde0000 (3) failed >> cpu0: on acpi0 >> cpu1: on acpi0 >> attimer0: port 0x40-0x43 irq 0 on acpi0 >> Timecounter "i8254" frequency 1193182 Hz quality 0 >> Event timer "i8254" frequency 1193182 Hz quality 100 >> atrtc0: port 0x70-0x73 irq 8 on acpi0 >> Event timer "RTC" frequency 32768 Hz quality 0 >> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 >> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0 >> acpi_button0: on acpi0 >> pcib0: port 0xcf8-0xcff on acpi0 >> pci0: on pcib0 >> pcib1: irq 16 at device 1.0 on pci0 >> pci1: on pcib1 >> vgapci0: port 0xbe00-0xbeff mem >> 0xd0000000-0xdfffffff,0xfd9e0000-0xfd9effff irq 16 at device 0.0 on >> pci1 >> hdac0: mem 0xfd9fc000-0xfd9fffff irq 17 >> at device 0.1 on pci1 >> pcib2: irq 16 at device 28.0 on pci0 >> pci2: on pcib2 >> siis0: port 0xaf00-0xaf7f mem >> 0xfd6ff000-0xfd6ff07f,0xfd6f8000-0xfd6fbfff irq 16 at device 0.0 on >> pci2 >> siisch0: at channel 0 on siis0 >> siisch1: at channel 1 on siis0 >> pcib3: irq 17 at device 28.1 on pci0 >> pci3: on pcib3 >> bge0: > 0x004101> mem 0xfddf0000-0xfddfffff irq 17 at device 0.0 on pci3 >> bge0: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E >> miibus0: on bge0 >> brgphy0: PHY 1 on miibus0 >> brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, >> 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, >> auto-flow >> bge0: Ethernet address: 00:50:8d:7c:c2:60 >> pcib4: irq 16 at device 28.4 on pci0 >> pci4: on pcib4 >> bge1: > 0x004101> mem 0xfdbf0000-0xfdbfffff irq 16 at device 0.0 on pci4 >> bge1: CHIP ID 0x00004101; ASIC REV 0x04; CHIP REV 0x41; PCI-E >> miibus1: on bge1 >> brgphy1: PHY 1 on miibus1 >> brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, >> 1000baseT-master, 1000baseT-FDX, 1000baseT-FDX-master, auto, >> auto-flow >> bge1: Ethernet address: 00:50:8d:7c:c2:61 >> uhci0: port 0xff00-0xff1f >> irq 23 at device 29.0 on pci0 >> usbus0 on uhci0 >> uhci1: port 0xfe00-0xfe1f >> irq 19 at device 29.1 on pci0 >> usbus1 on uhci1 >> uhci2: port 0xfd00-0xfd1f >> irq 18 at device 29.2 on pci0 >> usbus2 on uhci2 >> uhci3: port 0xfc00-0xfc1f >> irq 16 at device 29.3 on pci0 >> usbus3 on uhci3 >> ehci0: mem >> 0xfdfff000-0xfdfff3ff irq 23 at device 29.7 on pci0 >> usbus4: EHCI version 1.0 >> usbus4 on ehci0 >> pcib5: at device 30.0 on pci0 >> pci5: on pcib5 >> fwohci0: mem >> 0xfd8ff000-0xfd8ff7ff,0xfd8f8000-0xfd8fbfff irq 17 at device 2.0 on >> pci5 >> fwohci0: OHCI version 1.10 (ROM=1) >> fwohci0: No. of Isochronous channels is 4. >> fwohci0: EUI64 00:50:8d:00:00:72:25:9e >> fwohci0: invalid speed 7 (fixed to 3). >> fwohci0: Phy 1394a available S800, 3 ports. >> fwohci0: Link S800, max_rec 4096 bytes. >> firewire0: on fwohci0 >> fwe0: on firewire0 >> if_fwe0: Fake Ethernet address: 02:50:8d:72:25:9e >> fwe0: Ethernet address: 02:50:8d:72:25:9e >> fwip0: on firewire0 >> fwip0: Firewire address: 00:50:8d:00:00:72:25:9e @ 0xfffe00000000, >> S800, maxrec 4096 >> dcons_crom0: on firewire0 >> dcons_crom0: bus_addr 0x2a20000 >> fwohci0: Initiate bus reset >> fwohci0: fwohci_intr_core: BUS reset >> fwohci0: fwohci_intr_core: node_id=0x00000000, SelfID Count=1, >> CYCLEMASTER mode >> isab0: at device 31.0 on pci0 >> isa0: on isab0 >> atapci0: port >> 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xfb00-0xfb0f mem >> 0xfdffe000-0xfdffe3ff at device 31.2 on pci0 >> ata0: at channel 0 on atapci0 >> ata1: at channel 1 on atapci0 >> pci0: at device 31.3 (no driver attached) >> acpi_tz0: on acpi0 >> acpi_tz0: _CRT value is absurd, ignored (255.0C) >> fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 >> orm0: at iomem 0xc0000-0xcffff,0xd0000-0xd3fff on isa0 >> sc0: at flags 0x100 on isa0 >> sc0: VGA <16 virtual consoles, flags=0x300> >> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 >> atkbdc0: at port 0x60,0x64 on isa0 >> atkbd0: irq 1 on atkbdc0 >> kbd0 at atkbd0 >> atkbd0: [GIANT-LOCKED] >> ppc0: cannot reserve I/O port range >> ctl: CAM Target Layer loaded >> acpi_perf0: on cpu0 >> p4tcc0: on cpu0 >> est1: on cpu1 >> est: CPU supports Enhanced Speedstep, but is not recognized. >> est: cpu_vendor GenuineIntel, msr 102d0000102d >> device_attach: est1 attach returned 6 >> p4tcc1: on cpu1 >> (noperiph:siisch0:0:-1:-1): rescan already queued >> (noperiph:siisch1:0:-1:-1): rescan already queued >> firewire0: 1 nodes, maxhop <= 0 cable IRM irm(0) (me) >> firewire0: bus manager 0 >> fwohci0: phy int >> Timecounters tick every 1.000 msec >> hdacc0: at cad 0 on hdac0 >> hdaa0: at nid 1 on hdacc0 >> pcm0: at nid 3 on hdaa0 >> usbus0: 12Mbps Full Speed USB v1.0 >> usbus1: 12Mbps Full Speed USB v1.0 >> usbus2: 12Mbps Full Speed USB v1.0 >> usbus3: 12Mbps Full Speed USB v1.0 >> usbus4: 480Mbps High Speed USB v2.0 >> acpi_tz0: _CRT value is absurd, ignored (255.0C) >> acpi_tz0: failed to set new freq, disabling passive cooling >> ugen0.1: at usbus0 >> uhub0: on usbus0 >> ugen1.1: at usbus1 >> uhub1: on usbus1 >> ugen2.1: at usbus2 >> uhub2: on usbus2 >> ugen3.1: at usbus3 >> uhub3: on usbus3 >> ugen4.1: at usbus4 >> uhub4: on usbus4 >> (aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 >> 00 00 00 40 00 00 00 00 03 00 >> (aprobe2:ata0:0:0:0): CAM status: ATA Status Error >> (aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) >> (aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00 >> (aprobe2:ata0:0:0:0): Retrying command >> (aprobe2:ata0:0:0:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 >> 00 00 00 40 00 00 00 00 03 00 >> (aprobe2:ata0:0:0:0): CAM status: ATA Status Error >> (aprobe2:ata0:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) >> (aprobe2:ata0:0:0:0): RES: 51 04 00 00 00 00 00 00 00 03 00 >> (aprobe2:ata0:0:0:0): Error 5, Retries exhausted >> (aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 >> 00 00 00 40 00 00 00 00 03 00 >> (aprobe0:ata0:0:1:0): CAM status: ATA Status Error >> (aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) >> (aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00 >> (aprobe0:ata0:0:1:0): Retrying command >> (aprobe0:ata0:0:1:0): SETFEATURES DISABLE SATA FEATURE. ACB: ef 90 >> 00 00 00 40 00 00 00 00 03 00 >> (aprobe0:ata0:0:1:0): CAM status: ATA Status Error >> (aprobe0:ata0:0:1:0): ATA status: 51 (DRDY SERV ERR), error: 04 (ABRT ) >> (aprobe0:ata0:0:1:0): RES: 51 04 00 00 00 00 00 00 00 03 00 >> (aprobe0:ata0:0:1:0): Error 5, Retries exhausted >> acpi_tz0: _CRT value is absurd, ignored (255.0C) >> uhub0: 2 ports with 2 removable, self powered >> uhub1: 2 ports with 2 removable, self powered >> uhub2: 2 ports with 2 removable, self powered >> uhub3: 2 ports with 2 removable, self powered >> uhub4: 8 ports with 8 removable, self powered >> ada0 at siisch0 bus 0 scbus0 target 0 lun 0 >> ada0: ATA-7 SATA 1.x device >> ada0: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes) >> ada0: Command Queueing enabled >> ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) >> ada0: Previously was known as ad4 >> ada1 at siisch1 bus 0 scbus1 target 0 lun 0 >> ada1: ATA-7 SATA 1.x device >> ada1: 150.000MB/s transfers (SATA 1.x, UDMA6, PIO 8192bytes) >> ada1: Command Queueing enabled >> ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) >> ada1: Previously was known as ad6 >> ada2 at ata0 bus 0 scbus2 target 0 lun 0 >> ada2: ATA-6 SATA 1.x device >> ada2: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) >> ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) >> ada2: Previously was known as ad0 >> ada3 at ata0 bus 0 scbus2 target 1 lun 0 >> ada3: ATA-6 SATA 1.x device >> ada3: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) >> ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) >> ada3: Previously was known as ad1 >> ada4 at ata1 bus 0 scbus3 target 0 lun 0 >> ada4: ATA-7 SATA 1.x device >> ada4: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) >> ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) >> ada4: Previously was known as ad2 >> ada5 at ata1 bus 0 scbus3 target 1 lun 0 >> ada5: ATA-7 SATA 1.x device >> ada5: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes) >> ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) >> ada5: Previously was known as ad3 >> SMP: AP CPU #1 Launched! >> Timecounter "TSC-low" frequency 12469046 Hz quality 1000 >> GEOM_RAID: Intel-165267ff: Array Intel-165267ff created. >> GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created. >> GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from >> NONE to ACTIVE. >> GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from >> NONE to ACTIVE. >> GEOM_RAID: Intel-841a1a00: Array started. >> GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from >> STARTING to OPTIMAL. >> GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created. >> GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created. >> Root mount waiting for: GRAID-SiI GRAID-Intel >> ugen1.2: at usbus1 >> ukbd0: > 2> on usbus1 >> kbd2 at ukbd0 >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> Root mount waiting for: GRAID-SiI GRAID-Intel >> GEOM_RAID: Intel-165267ff: Force array start due to timeout. >> GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from >> NONE to ACTIVE. >> GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from >> NONE to ACTIVE. >> GEOM_RAID: Intel-165267ff: Array started. >> GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from >> STARTING to BROKEN. >> GEOM_RAID: SiI-100311082330: Force array start due to timeout. >> GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE. >> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state >> changed from NONE to STALE. >> GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE. >> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state >> changed from NONE to STALE. >> GEOM_RAID: SiI-100311082330: Array started. >> GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from >> STARTING to BROKEN. >> Trying to mount root from ufs:/dev/raid/r0s1a [rw]... >> ZFS filesystem version 5 >> ZFS storage pool version 28 >> ZFS WARNING: Unable to attach to ada0. >> ZFS WARNING: Unable to attach to ada1. >> ZFS WARNING: Unable to attach to ada4. >> ZFS WARNING: Unable to attach to ada5. >> ZFS WARNING: Unable to attach to ada0. >> ZFS WARNING: Unable to attach to ada1. >> ZFS WARNING: Unable to attach to ada4. >> ZFS WARNING: Unable to attach to ada5. >> ZFS WARNING: Unable to attach to ada0. >> ZFS WARNING: Unable to attach to ada1. >> ZFS WARNING: Unable to attach to ada4. >> ZFS WARNING: Unable to attach to ada5. >> ---------------------------------------------------- >> >> >> >> gpart list >> ------------------------------------------------ >> >> Geom name: raid/r0 >> modified: false >> state: OK >> fwheads: 255 >> fwsectors: 63 >> last: 72298495 >> first: 63 >> entries: 4 >> scheme: MBR >> Providers: >> 1. Name: raid/r0s1 >> Mediasize: 37013727744 (34G) >> Sectorsize: 512 >> Stripesize: 0 >> Stripeoffset: 32256 >> Mode: r5w5e9 >> attrib: active >> rawtype: 165 >> length: 37013727744 >> offset: 32256 >> type: freebsd >> index: 1 >> end: 72292499 >> start: 63 >> Consumers: >> 1. Name: raid/r0 >> Mediasize: 37016829952 (34G) >> Sectorsize: 512 >> Mode: r5w5e14 >> >> Geom name: raid/r0s1 >> modified: false >> state: OK >> fwheads: 255 >> fwsectors: 63 >> last: 72292436 >> first: 0 >> entries: 8 >> scheme: BSD >> Providers: >> 1. Name: raid/r0s1a >> Mediasize: 1073741824 (1.0G) >> Sectorsize: 512 >> Stripesize: 0 >> Stripeoffset: 32256 >> Mode: r1w1e1 >> rawtype: 7 >> length: 1073741824 >> offset: 0 >> type: freebsd-ufs >> index: 1 >> end: 2097151 >> start: 0 >> 2. Name: raid/r0s1b >> Mediasize: 4294967296 (4.0G) >> Sectorsize: 512 >> Stripesize: 0 >> Stripeoffset: 1073774080 >> Mode: r1w1e0 >> rawtype: 1 >> length: 4294967296 >> offset: 1073741824 >> type: freebsd-swap >> index: 2 >> end: 10485759 >> start: 2097152 >> 3. Name: raid/r0s1d >> Mediasize: 5368709120 (5.0G) >> Sectorsize: 512 >> Stripesize: 0 >> Stripeoffset: 1073774080 >> Mode: r1w1e1 >> rawtype: 7 >> length: 5368709120 >> offset: 5368709120 >> type: freebsd-ufs >> index: 4 >> end: 20971519 >> start: 10485760 >> 4. Name: raid/r0s1e >> Mediasize: 2147483648 (2.0G) >> Sectorsize: 512 >> Stripesize: 0 >> Stripeoffset: 2147515904 >> Mode: r1w1e1 >> rawtype: 7 >> length: 2147483648 >> offset: 10737418240 >> type: freebsd-ufs >> index: 5 >> end: 25165823 >> start: 20971520 >> 5. Name: raid/r0s1f >> Mediasize: 24128825856 (22G) >> Sectorsize: 512 >> Stripesize: 0 >> Stripeoffset: 32256 >> Mode: r1w1e1 >> rawtype: 7 >> length: 24128825856 >> offset: 12884901888 >> type: freebsd-ufs >> index: 6 >> end: 72292436 >> start: 25165824 >> Consumers: >> 1. Name: raid/r0s1 >> Mediasize: 37013727744 (34G) >> Sectorsize: 512 >> Stripesize: 0 >> Stripeoffset: 32256 >> Mode: r5w5e9 >> >> ---------------------------------------- >> >> gpart list ada0 >> gpart: No such geom: ada0. >> >> gpart list ada1 >> gpart: No such geom: ada1. >> >> gpart list ada4 >> gpart: No such geom: ada4. >> >> gpart list ada5 >> gpart: No such geom: ada5. >> >> gpart list ad0 >> gpart: No such geom: ad0. >> >> gpart list ad1 >> gpart: No such geom: ad1. >> >> gpart list ad4 >> gpart: No such geom: ad4. >> >> gpart list ad5 >> gpart: No such geom: ad5 >> >> -------------------------------------- >> >> >> gpart show (This is the system raid) >> >> ------------------------------------- >> >> => 63 72298433 raid/r0 MBR (34G) >> 63 72292437 1 freebsd [active] (34G) >> 72292500 5996 - free - (3M) >> >> => 0 72292437 raid/r0s1 BSD (34G) >> 0 2097152 1 freebsd-ufs (1.0G) >> 2097152 8388608 2 freebsd-swap (4.0G) >> 10485760 10485760 4 freebsd-ufs (5.0G) >> 20971520 4194304 5 freebsd-ufs (2.0G) >> 25165824 47126613 6 freebsd-ufs (22G) >> >> -------------------------------------------- >> >> gpart show ada0 >> gpart: No such geom: ada0 >> >> gpart show ada1 >> gpart: No such geom: ada1 >> >> gpart show ada4 >> gpart: No such geom: ada4 >> >> gpart show ada5 >> gpart: No such geom: ada5 >> >> >> >> I'm not sure about glabel, Sorry. > > I think I see what the issue is here. This is going to be a long Email. > > You have 6 disks: > > ada0 -- attached to siisch0 (Silicon Image 3132) > ada1 -- attached to siisch1 (Silicon Image 3132) > ada2 -- attached to ata0 (Intel ICH7, non-RAID) as master > ada3 -- attached to ata0 (Intel ICH7, non-RAID) as slave > ada4 -- attached to ata1 (Intel ICH7, non-RAID) as master > ada5 -- attached to ata1 (Intel ICH7, non-RAID) as slave > > However, **BEFORE** ZFS starts, we can clearly see this (take the time > to read it slowly and carefully) > >> GEOM_RAID: Intel-165267ff: Array Intel-165267ff created. >> GEOM_RAID: Intel-841a1a00: Array Intel-841a1a00 created. >> GEOM_RAID: Intel-841a1a00: Disk ada2 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:0-ada2 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-841a1a00: Disk ada3 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-841a1a00: Subdisk Volume0:1-ada3 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-841a1a00: Array started. >> GEOM_RAID: Intel-841a1a00: Volume Volume0 state changed from STARTING to OPTIMAL. >> GEOM_RAID: Intel-841a1a00: Provider raid/r0 for volume Volume0 created. >> GEOM_RAID: SiI-100311082330: Array SiI-100311082330 created. >> Root mount waiting for: GRAID-SiI GRAID-Intel >> --- >> --- this message repeats a very long time due to a problem >> --- I'm about to try and explain >> --- >> GEOM_RAID: Intel-165267ff: Force array start due to timeout. >> GEOM_RAID: Intel-165267ff: Disk ada0 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-165267ff: Subdisk Volume0:1-ada0 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-165267ff: Disk ada1 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-165267ff: Subdisk Volume0:3-ada1 state changed from NONE to ACTIVE. >> GEOM_RAID: Intel-165267ff: Array started. >> GEOM_RAID: Intel-165267ff: Volume Volume0 state changed from STARTING to BROKEN. >> GEOM_RAID: SiI-100311082330: Force array start due to timeout. >> GEOM_RAID: SiI-100311082330: Disk ada4 state changed from NONE to ACTIVE. >> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:0-ada4 state changed from NONE to STALE. >> GEOM_RAID: SiI-100311082330: Disk ada5 state changed from NONE to ACTIVE. >> GEOM_RAID: SiI-100311082330: Subdisk SiI Raid5 Set:1-ada5 state changed from NONE to STALE. >> GEOM_RAID: SiI-100311082330: Array started. >> GEOM_RAID: SiI-100311082330: Volume SiI Raid5 Set state changed from STARTING to BROKEN. >> Trying to mount root from ufs:/dev/raid/r0s1a [rw]... > > This shows ***THREE*** graid(8) arrays being probed during disk tasting > for RAID metadata. > > Remember two things: > > 1) graid(8) metadata tasting happens ***BEFORE*** ZFS metadata tasting. > 2) The metadata is stored *on the disks themselves*. > > - Intel-165267ff > -- disk ada0 > -- disk ada1 > - Intel-841a1a00 > -- disk ada2 > -- disk ada3 > - SiI-100311082330 (used to be part of a RAID-5 set) > -- disk ada4 > -- disk ada5 > > All of these disks become "locked" by graid because of the discovery of > metadata on them. > > Plain and simple: > > 1) ada0 and ada1 were previously used on an Intel controller that had > BIOS-level RAID capability, and (presumably you) never cleared the > metadata from the disks before making use of ZFS, > > 2) ada4 and ada5 were previously used on a Silicon Image controller that > had RAID capability, and (presumably you) never cleared the metadata > from the disks before making use of ZFS. > > Your next question is going to be "how to do I do that?" > > There are many ways to do this, and the one I'm going to tell you is the > one that will work reliably without having to ""fight"" with GEOM or > graid(8). So hear me loud and clear. > > 1. Boot a FreeBSD installation CD/DVD/USB stick > 2. Escape to the loader prompt > 3. Type: set kern.geom.raid.enable="0" > 4. Type: boot > 5. Go to a "Fixit" shell (the one that has all the useful utilities) > 6. Zero the first and last 8MBytes or so (I'm not going to be accurate) > of all the disks whose RAID controller metadata you want to remove. > > Apologies for the caps, but I need to make this clear: > > BE AWARE THIS WILL DESTROY PARTITION TABLES AND OVERWRITE/NUKE ANY OTHER > KIND OF METADATA (INCLUDING ZFS), SO IF YOU HAVE DATA ON THESE DRIVES > THAT YOU CARE ABOUT (SOMEHOW?), YOU HAD BETTER GET THE DATA OFF FIRST. > > FOR ZFS YOU WILL ALSO NEED TO RE-CREATE THE POOL AFTER THIS (AND IF THEY > ARE 4K SECTOR DISKS YOU WILL NEED TO USE THE GNOP METHOD TO ENSURE > PROPER ALIGNMENT; SEE IVAN VORAS' BLOG). > > The procedure to do that, for your disks (given their capacities) is > this: > >> ada0: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) >> ada1: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) >> ada2: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) >> ada3: 35304MB (72303840 512 byte sectors: 16H 63S/T 16383C) >> ada4: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) >> ada5: 305245MB (625142448 512 byte sectors: 16H 63S/T 16383C) > > sysctl kern.geom.debugflags=16 > > zpool import -a > -- I am hoping your pool get imported successfully at this point. > -- This might not even be needed depending upon what the kernel > -- does during its earlier boot. > > zpool destroy {poolname} > -- Yes, you really do want to do this. This is how you will get > -- ZFS to "unlock" the disks that you're about to nuke metadata > -- on. If you have multiple pools, you need to do this for > -- all of them. > > dd if=/dev/zero of=/dev/ada0 bs=65536 count=128 > dd if=/dev/zero of=/dev/ada0 bs=65536 count=128 oseek=4880000 > dd if=/dev/zero of=/dev/ada1 bs=65536 count=128 > dd if=/dev/zero of=/dev/ada1 bs=65536 count=128 oseek=4880000 > dd if=/dev/zero of=/dev/ada4 bs=65536 count=128 > dd if=/dev/zero of=/dev/ada4 bs=65536 count=128 oseek=4880000 > dd if=/dev/zero of=/dev/ada5 bs=65536 count=128 > dd if=/dev/zero of=/dev/ada5 bs=65536 count=128 oseek=4880000 > > On the commands with "oseek", you will probably receive an error that > says something about "short end of device" or what not; that is > perfectly fine. > > After this, I strongly recommend rebooting the system ("reboot") > and booting off of your standard hard disks. Your system should > start in multi-user, but obviously there will be no ZFS pools. > > At this point, recreate your ZFS pools as you did originally, and you > should be good to go going forward. > Thank you very much Jeremy. Yes you are right, the machine was a Windows home Server and I changed it to BSD :-) In that process I rearranged the disks because they where set up in a strange way, and YES they where already in a RAID configuration when I reused and reconnected the disks to have a more "correct" setup. The machine has been working for more that two years but when I started upgrading the Motherboard battery was out and that of course reset the BIOS. I do not remember how I configured the disks then (which I will document now so that I'll not make the same mistake once more) Luckily the system RAID is working so I can try to fix the problem as you describe. I might even get new and larger disks in the same time, and maybe even a genuine SATA III controller. /Leslie From owner-freebsd-fs@FreeBSD.ORG Thu May 23 06:10:00 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 0F3A7442 for ; Thu, 23 May 2013 06:10:00 +0000 (UTC) (envelope-from ajit.jain@cloudbyte.com) Received: from mail-oa0-f54.google.com (mail-oa0-f54.google.com [209.85.219.54]) by mx1.freebsd.org (Postfix) with ESMTP id CC29161D for ; Thu, 23 May 2013 06:09:59 +0000 (UTC) Received: by mail-oa0-f54.google.com with SMTP id o17so3909061oag.13 for ; Wed, 22 May 2013 23:09:52 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=kxJ9CHNH6+dake9NWUUsqLviQKtLDnokbwnnmDTU7P8=; b=cU7ttuploMqoemvkC1zNLGcVqxjLzic7g8biTaO0V8Ch2m63r2Jb6HZwpYJ9HSIfEo aN1q4eximfgCfnOiNwlid79YAIGNz2Hzx2c2xgmEQ2L8AbLCAq/KU+LWEjyIzL8ChvmX 2+JUvzJXhJ3oLuMbBuCyOZCW6c/p5yg2QXam1AM8WTdxC9Rhy5IUk4N6CcHRY3xi4DsE 0lIlasLGy+gyMr5fUj5jFSVNZCv2z1VT9wS1Vbrt+YlYbe21cKFObV5A8CCbBb0YhJPP i6BIHfiFDZyEBd6ZbtVY+lSLaWS8HOZUQD9sv7SN/iGLJy0JALprfkdIebMcuJOtPpyA ga+w== X-Received: by 10.182.225.199 with SMTP id rm7mr7398682obc.20.1369289392740; Wed, 22 May 2013 23:09:52 -0700 (PDT) MIME-Version: 1.0 Received: by 10.76.151.134 with HTTP; Wed, 22 May 2013 23:09:32 -0700 (PDT) In-Reply-To: <93D0677B373A452BAF58C8EA6823783D@multiplay.co.uk> References: <60316751643743738AB83DABC6A5934B@multiplay.co.uk> <20130429105143.GA1492@icarus.home.lan> <3AD1AB31003D49B2BF2EA7DD411B38A2@multiplay.co.uk> <9681E07546D348168052D4FC5365B4CD@multiplay.co.uk> <3E9CA9334E6F433A8F135ACD5C237340@multiplay.co.uk> <93D0677B373A452BAF58C8EA6823783D@multiplay.co.uk> From: Ajit Jain Date: Thu, 23 May 2013 11:39:32 +0530 Message-ID: Subject: Re: seeing data corruption with zfs trim functionality To: Steven Hartland X-Gm-Message-State: ALoCoQlYnYig6d90RoBeMNi6gNIkHmou9vJVoeOZkdA31HVDdco5YiDXJA7SA4OpDk37Pmn5K/Yr Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 May 2013 06:10:00 -0000 Hi Steven, FW version on the setup is P15. I will upgrade the FW to P16, but I think my best bet will be to update code base to 9 stable as unlike you, I was seeing corruption for all three delete methods. thanks ajit On Sat, May 18, 2013 at 4:15 AM, Steven Hartland wrote: > ----- Original Message ----- From: "Steven Hartland" < > killing@multiplay.co.uk> > > >> After initially seeing not issues, our overnight monitoring started >> moaning >> big time on the test box. So we checked and there was zpool corruption as >> well >> as a missing boot loader and a corrupt GPT, so I believe we have >> reproduced >> your issue. >> >> After recovering the machine I created 3 pools on 3 different disks each >> running a different delete_method. >> >> We then re-ran the tests which resulted in the pool running with >> delete_method >> WS16 being so broken it had suspended IO. A reboot resulted in it once >> again >> reporting no partition table via gpart. >> >> A third test run again produced a corrupt pool for WS16. >> >> I've conducted a preliminary review of the CAM WS16 code path along with >> SBC-3 >> spec which didn't identify any obvious issues. >> >> Given we're both using LSI 2008 based controllers it could be FW issue >> specific >> to WS16 but that's just speculation atm, so I'll continue to investigate. >> >> If you could re-test you end without using WS16 to see if you can >> reproduce the >> problem with either UNMAP or ATA_TRIM that would be a very useful data >> point. >> > > After much playing I narrow down a test case of one delete which was > causing > disc corruption for us (deleted the partition table instead of data in > the middle of the disk). > > The conclusion is LSI 2008 HBA with FW below P13 will eat the data on your > SATA > disks if you use WS16 due to the following bug:- > SCGCQ00230159 (DFCT) - Write same command to a SATA drive that doesn't > support > SCT write same may write wrong region. > > After updating here to P16, which we would generally be running, but test > box > was new and hadnt updated yet the corruption issue is no longer > reproducable. > > So Ajit please check your FW version, I'm hoping to here your on something > below P13, P12 possibly? > > If so then this is your issue, to fix simply update to P16 and the problem > should be gone. > > > Regards > Steve > > > ==============================**================== > This e.mail is private and confidential between Multiplay (UK) Ltd. and > the person or entity to whom it is addressed. In the event of misdirection, > the recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > > From owner-freebsd-fs@FreeBSD.ORG Thu May 23 09:34:06 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7DB684E2 for ; Thu, 23 May 2013 09:34:06 +0000 (UTC) (envelope-from prvs=1855369f91=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id C90FD7C for ; Thu, 23 May 2013 09:34:03 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50003951118.msg for ; Thu, 23 May 2013 10:33:15 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 23 May 2013 10:33:15 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1855369f91=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-fs@freebsd.org Message-ID: <35ABA7AAEB7F4D86A1ED54C4C47FEB49@multiplay.co.uk> From: "Steven Hartland" To: "Ajit Jain" References: <60316751643743738AB83DABC6A5934B@multiplay.co.uk> <20130429105143.GA1492@icarus.home.lan> <3AD1AB31003D49B2BF2EA7DD411B38A2@multiplay.co.uk> <9681E07546D348168052D4FC5365B4CD@multiplay.co.uk> <3E9CA9334E6F433A8F135ACD5C237340@multiplay.co.uk> <93D0677B373A452BAF58C8EA6823783D@multiplay.co.uk> Subject: Re: seeing data corruption with zfs trim functionality Date: Thu, 23 May 2013 10:33:11 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0730_01CE57A0.ECC7BD20" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 May 2013 09:34:06 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0730_01CE57A0.ECC7BD20 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit I've attacked the two patch sets I'm looking to MFC to stable-9, one adds BIO_DELETE CAM changes and the other is ZFS TRIM support. They should both apply cleanly to stable-9, if you could test with those on your machine and let me know. Regards Steve ----- Original Message ----- From: "Ajit Jain" > Hi Steven, > > FW version on the setup is P15. > I will upgrade the FW to P16, but I think my > best bet will be to update code base to 9 stable as unlike you, > I was seeing corruption for all three delete methods. > > thanks > ajit > > On Sat, May 18, 2013 at 4:15 AM, Steven Hartland wrote: > >> ----- Original Message ----- From: "Steven Hartland" < >> killing@multiplay.co.uk> >> >> >>> After initially seeing not issues, our overnight monitoring started >>> moaning >>> big time on the test box. So we checked and there was zpool corruption as >>> well >>> as a missing boot loader and a corrupt GPT, so I believe we have >>> reproduced >>> your issue. >>> >>> After recovering the machine I created 3 pools on 3 different disks each >>> running a different delete_method. >>> >>> We then re-ran the tests which resulted in the pool running with >>> delete_method >>> WS16 being so broken it had suspended IO. A reboot resulted in it once >>> again >>> reporting no partition table via gpart. >>> >>> A third test run again produced a corrupt pool for WS16. >>> >>> I've conducted a preliminary review of the CAM WS16 code path along with >>> SBC-3 >>> spec which didn't identify any obvious issues. >>> >>> Given we're both using LSI 2008 based controllers it could be FW issue >>> specific >>> to WS16 but that's just speculation atm, so I'll continue to investigate. >>> >>> If you could re-test you end without using WS16 to see if you can >>> reproduce the >>> problem with either UNMAP or ATA_TRIM that would be a very useful data >>> point. >>> >> >> After much playing I narrow down a test case of one delete which was >> causing >> disc corruption for us (deleted the partition table instead of data in >> the middle of the disk). >> >> The conclusion is LSI 2008 HBA with FW below P13 will eat the data on your >> SATA >> disks if you use WS16 due to the following bug:- >> SCGCQ00230159 (DFCT) - Write same command to a SATA drive that doesn't >> support >> SCT write same may write wrong region. >> >> After updating here to P16, which we would generally be running, but test >> box >> was new and hadnt updated yet the corruption issue is no longer >> reproducable. >> >> So Ajit please check your FW version, I'm hoping to here your on something >> below P13, P12 possibly? >> >> If so then this is your issue, to fix simply update to P16 and the problem >> should be gone. >> >> >> Regards >> Steve >> >> >> ==============================**================== >> This e.mail is private and confidential between Multiplay (UK) Ltd. and >> the person or entity to whom it is addressed. In the event of misdirection, >> the recipient is prohibited from using, copying, printing or otherwise >> disseminating it or any information contained in it. >> In the event of misdirection, illegible or incomplete transmission please >> telephone +44 845 868 1337 >> or return the E.mail to postmaster@multiplay.co.uk. >> >> > ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_0730_01CE57A0.ECC7BD20 Content-Type: application/octet-stream; name="mfc-bio_delete-bioqsort-stable-9.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="mfc-bio_delete-bioqsort-stable-9.patch" Enhanced BIO_DELETE support for CAM SCSI which adds ATA_TRIM support.=0A= =0A= Add the ability to enable / disable the sorting of BIO requests queue in = CAM,=0A= which is now disabled by default for non-rotating medium.=0A= =0A= MFC r246146 Format CDB output as 2 digit hex correcting the length=0A= MFC r248922 Adds the ability to enable / disable sorting of BIO requests=0A= MFC r248992 Added ATA Pass-Through support to CAM=0A= MFC r249929 Removed unneeded tests in dadeletemethodset=0A= MFC r249930 Added a sysctl to control the maximum size of a delete = request=0A= MFC r249931 Added Dataset Management defines to be used by TRIM=0A= MFC r249933 Added the ability to send ATA identify and TRIM commands via = SCSI=0A= MFC r249934 Updated TRIM calculations in cam/ata to be based off = ATA_DSM_* defines=0A= MFC r249937 Refactored scsi_xpt use of device_has_vpd=0A= MFC r249939 Added available delete methods discovery during device probe=0A= MFC r249941 Automatically disable BIO queue sorting for non-rotating = media=0A= MFC r250033 Correct comment typo's=0A= MFC r250179 Update probe flow so that devices with lbp can also disable = disksort=0A= MFC r250180 Fix probe in progress check in dareprobe=0A= MFC r250181 Check for ATA Information VPD before querying for ATA=0A= MFC r250183 Enable CAM SCSI to choice ATA TRIM during autodetection=0A= Index: sys=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys (revision 250577)=0A= +++ sys (working copy)=0A= =0A= Property changes on: sys=0A= ___________________________________________________________________=0A= Modified: svn:mergeinfo=0A= Merged = /head/sys:r246146,248922,248992,249930-249931,249933-249934,249937,249939= ,249941,250033,250179-250181,250183=0A= Index: sys/cam/cam.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/cam.c (revision 250577)=0A= +++ sys/cam/cam.c (working copy)=0A= @@ -110,8 +110,17 @@=0A= =0A= #ifdef _KERNEL=0A= SYSCTL_NODE(_kern, OID_AUTO, cam, CTLFLAG_RD, 0, "CAM Subsystem");=0A= +=0A= +#ifndef CAM_DEFAULT_SORT_IO_QUEUES=0A= +#define CAM_DEFAULT_SORT_IO_QUEUES 1=0A= #endif=0A= =0A= +int cam_sort_io_queues =3D CAM_DEFAULT_SORT_IO_QUEUES;=0A= +TUNABLE_INT("kern.cam.sort_io_queues", &cam_sort_io_queues);=0A= +SYSCTL_INT(_kern_cam, OID_AUTO, sort_io_queues, CTLFLAG_RWTUN,=0A= + &cam_sort_io_queues, 0, "Sort IO queues to try and optimise disk = access patterns");=0A= +#endif=0A= +=0A= void=0A= cam_strvis(u_int8_t *dst, const u_int8_t *src, int srclen, int dstlen)=0A= {=0A= Index: sys/cam/cam.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/cam.h (revision 250577)=0A= +++ sys/cam/cam.h (working copy)=0A= @@ -228,6 +228,9 @@=0A= =0A= extern const struct cam_status_entry cam_status_table[];=0A= extern const int num_cam_status_entries;=0A= +#ifdef _KERNEL=0A= +extern int cam_sort_io_queues;=0A= +#endif=0A= union ccb;=0A= =0A= #ifdef SYSCTL_DECL /* from sysctl.h */=0A= Index: sys/cam/scsi/scsi_all.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_all.c (revision 250577)=0A= +++ sys/cam/scsi/scsi_all.c (working copy)=0A= @@ -40,6 +40,9 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= +#include =0A= +#include =0A= #include =0A= #else=0A= #include =0A= @@ -53,8 +56,15 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= #include =0A= -#ifndef _KERNEL=0A= +=0A= +#ifdef _KERNEL=0A= +#include =0A= +#include =0A= +#include =0A= +#include =0A= +#else=0A= #include =0A= #include =0A= =0A= @@ -3136,7 +3146,7 @@=0A= *cdb_string =3D '\0';=0A= for (i =3D 0; i < cdb_len; i++)=0A= snprintf(cdb_string + strlen(cdb_string),=0A= - len - strlen(cdb_string), "%x ", cdb_ptr[i]);=0A= + len - strlen(cdb_string), "%02hhx ", cdb_ptr[i]);=0A= =0A= return(cdb_string);=0A= }=0A= @@ -5847,6 +5857,101 @@=0A= }=0A= =0A= void=0A= +scsi_ata_identify(struct ccb_scsiio *csio, u_int32_t retries,=0A= + void (*cbfcnp)(struct cam_periph *, union ccb *),=0A= + u_int8_t tag_action, u_int8_t *data_ptr,=0A= + u_int16_t dxfer_len, u_int8_t sense_len,=0A= + u_int32_t timeout)=0A= +{=0A= + scsi_ata_pass_16(csio,=0A= + retries,=0A= + cbfcnp,=0A= + /*flags*/CAM_DIR_IN,=0A= + tag_action,=0A= + /*protocol*/AP_PROTO_PIO_IN,=0A= + /*ata_flags*/AP_FLAG_TDIR_FROM_DEV|=0A= + AP_FLAG_BYT_BLOK_BYTES|AP_FLAG_TLEN_SECT_CNT,=0A= + /*features*/0,=0A= + /*sector_count*/dxfer_len,=0A= + /*lba*/0,=0A= + /*command*/ATA_ATA_IDENTIFY,=0A= + /*control*/0,=0A= + data_ptr,=0A= + dxfer_len,=0A= + sense_len,=0A= + timeout);=0A= +}=0A= +=0A= +void=0A= +scsi_ata_trim(struct ccb_scsiio *csio, u_int32_t retries,=0A= + void (*cbfcnp)(struct cam_periph *, union ccb *),=0A= + u_int8_t tag_action, u_int16_t block_count,=0A= + u_int8_t *data_ptr, u_int16_t dxfer_len, u_int8_t sense_len,=0A= + u_int32_t timeout)=0A= +{=0A= + scsi_ata_pass_16(csio,=0A= + retries,=0A= + cbfcnp,=0A= + /*flags*/CAM_DIR_OUT,=0A= + tag_action,=0A= + /*protocol*/AP_EXTEND|AP_PROTO_DMA,=0A= + /*ata_flags*/AP_FLAG_TLEN_SECT_CNT|AP_FLAG_BYT_BLOK_BLOCKS,=0A= + /*features*/ATA_DSM_TRIM,=0A= + /*sector_count*/block_count,=0A= + /*lba*/0,=0A= + /*command*/ATA_DATA_SET_MANAGEMENT,=0A= + /*control*/0,=0A= + data_ptr,=0A= + dxfer_len,=0A= + sense_len,=0A= + timeout);=0A= +}=0A= +=0A= +void=0A= +scsi_ata_pass_16(struct ccb_scsiio *csio, u_int32_t retries,=0A= + void (*cbfcnp)(struct cam_periph *, union ccb *),=0A= + u_int32_t flags, u_int8_t tag_action,=0A= + u_int8_t protocol, u_int8_t ata_flags, u_int16_t features,=0A= + u_int16_t sector_count, uint64_t lba, u_int8_t command,=0A= + u_int8_t control, u_int8_t *data_ptr, u_int16_t dxfer_len,=0A= + u_int8_t sense_len, u_int32_t timeout)=0A= +{=0A= + struct ata_pass_16 *ata_cmd;=0A= +=0A= + ata_cmd =3D (struct ata_pass_16 *)&csio->cdb_io.cdb_bytes;=0A= + ata_cmd->opcode =3D ATA_PASS_16;=0A= + ata_cmd->protocol =3D protocol;=0A= + ata_cmd->flags =3D ata_flags;=0A= + ata_cmd->features_ext =3D features >> 8;=0A= + ata_cmd->features =3D features;=0A= + ata_cmd->sector_count_ext =3D sector_count >> 8;=0A= + ata_cmd->sector_count =3D sector_count;=0A= + ata_cmd->lba_low =3D lba;=0A= + ata_cmd->lba_mid =3D lba >> 8;=0A= + ata_cmd->lba_high =3D lba >> 16;=0A= + ata_cmd->device =3D ATA_DEV_LBA;=0A= + if (protocol & AP_EXTEND) {=0A= + ata_cmd->lba_low_ext =3D lba >> 24;=0A= + ata_cmd->lba_mid_ext =3D lba >> 32;=0A= + ata_cmd->lba_high_ext =3D lba >> 40;=0A= + } else=0A= + ata_cmd->device |=3D (lba >> 24) & 0x0f;=0A= + ata_cmd->command =3D command;=0A= + ata_cmd->control =3D control;=0A= +=0A= + cam_fill_csio(csio,=0A= + retries,=0A= + cbfcnp,=0A= + flags,=0A= + tag_action,=0A= + data_ptr,=0A= + dxfer_len,=0A= + sense_len,=0A= + sizeof(*ata_cmd),=0A= + timeout);=0A= +}=0A= +=0A= +void=0A= scsi_unmap(struct ccb_scsiio *csio, u_int32_t retries,=0A= void (*cbfcnp)(struct cam_periph *, union ccb *),=0A= u_int8_t tag_action, u_int8_t byte2,=0A= @@ -6156,6 +6261,28 @@=0A= }=0A= =0A= #ifdef _KERNEL=0A= +int=0A= +scsi_vpd_supported_page(struct cam_periph *periph, uint8_t page_id)=0A= +{=0A= + struct cam_ed *device;=0A= + struct scsi_vpd_supported_pages *vpds;=0A= + int i, num_pages;=0A= +=0A= + device =3D periph->path->device;=0A= + vpds =3D (struct scsi_vpd_supported_pages *)device->supported_vpds;=0A= +=0A= + if (vpds !=3D NULL) {=0A= + num_pages =3D device->supported_vpds_len -=0A= + SVPD_SUPPORTED_PAGES_HDR_LEN;=0A= + for (i =3D 0; i < num_pages; i++) {=0A= + if (vpds->page_list[i] =3D=3D page_id)=0A= + return (1);=0A= + }=0A= + }=0A= +=0A= + return (0);=0A= +}=0A= +=0A= static void=0A= init_scsi_delay(void)=0A= {=0A= Index: sys/cam/scsi/scsi_all.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_all.h (revision 250577)=0A= +++ sys/cam/scsi/scsi_all.h (working copy)=0A= @@ -908,6 +908,19 @@=0A= struct ata_pass_12 {=0A= u_int8_t opcode;=0A= u_int8_t protocol;=0A= +#define AP_PROTO_HARD_RESET (0x00 << 1)=0A= +#define AP_PROTO_SRST (0x01 << 1)=0A= +#define AP_PROTO_NON_DATA (0x03 << 1)=0A= +#define AP_PROTO_PIO_IN (0x04 << 1)=0A= +#define AP_PROTO_PIO_OUT (0x05 << 1)=0A= +#define AP_PROTO_DMA (0x06 << 1)=0A= +#define AP_PROTO_DMA_QUEUED (0x07 << 1)=0A= +#define AP_PROTO_DEVICE_DIAG (0x08 << 1)=0A= +#define AP_PROTO_DEVICE_RESET (0x09 << 1)=0A= +#define AP_PROTO_UDMA_IN (0x10 << 1)=0A= +#define AP_PROTO_UDMA_OUT (0x11 << 1)=0A= +#define AP_PROTO_FPDMA (0x12 << 1)=0A= +#define AP_PROTO_RESP_INFO (0x15 << 1)=0A= #define AP_MULTI 0xe0=0A= u_int8_t flags;=0A= #define AP_T_LEN 0x03=0A= @@ -943,6 +956,15 @@=0A= u_int8_t protocol;=0A= #define AP_EXTEND 0x01=0A= u_int8_t flags;=0A= +#define AP_FLAG_TLEN_NO_DATA (0 << 0)=0A= +#define AP_FLAG_TLEN_FEAT (1 << 0)=0A= +#define AP_FLAG_TLEN_SECT_CNT (2 << 0)=0A= +#define AP_FLAG_TLEN_STPSIU (3 << 0)=0A= +#define AP_FLAG_BYT_BLOK_BYTES (0 << 2) =0A= +#define AP_FLAG_BYT_BLOK_BLOCKS (1 << 2) =0A= +#define AP_FLAG_TDIR_TO_DEV (0 << 3) =0A= +#define AP_FLAG_TDIR_FROM_DEV (1 << 3) =0A= +#define AP_FLAG_CHK_COND (1 << 5) =0A= u_int8_t features_ext;=0A= u_int8_t features;=0A= u_int8_t sector_count_ext;=0A= @@ -1064,7 +1086,7 @@=0A= =0A= /*=0A= * This length is the initial inquiry length used by the probe code, as = =0A= - * well as the legnth necessary for scsi_print_inquiry() to function =0A= + * well as the length necessary for scsi_print_inquiry() to function =0A= * correctly. If either use requires a different length in the future, =0A= * the two values should be de-coupled.=0A= */=0A= @@ -1407,6 +1429,91 @@=0A= uint8_t params[0];=0A= };=0A= =0A= +/*=0A= + * ATA Information VPD Page based on=0A= + * T10/2126-D Revision 04=0A= + */=0A= +#define SVPD_ATA_INFORMATION 0x89=0A= +=0A= +/*=0A= + * Block Device Characteristics VPD Page based on=0A= + * T10/1799-D Revision 31=0A= + */=0A= +struct scsi_vpd_block_characteristics=0A= +{=0A= + u_int8_t device;=0A= + u_int8_t page_code;=0A= +#define SVPD_BDC 0xB1=0A= + u_int8_t page_length[2];=0A= + u_int8_t medium_rotation_rate[2];=0A= +#define SVPD_BDC_RATE_NOT_REPORTED 0x00=0A= +#define SVPD_BDC_RATE_NONE_ROTATING 0x01=0A= + u_int8_t reserved1;=0A= + u_int8_t nominal_form_factor;=0A= +#define SVPD_BDC_FORM_NOT_REPORTED 0x00=0A= +#define SVPD_BDC_FORM_5_25INCH 0x01=0A= +#define SVPD_BDC_FORM_3_5INCH 0x02=0A= +#define SVPD_BDC_FORM_2_5INCH 0x03=0A= +#define SVPD_BDC_FORM_1_5INCH 0x04=0A= +#define SVPD_BDC_FORM_LESSTHAN_1_5INCH 0x05=0A= + u_int8_t reserved2[56];=0A= +};=0A= +=0A= +/*=0A= + * Logical Block Provisioning VPD Page based on=0A= + * T10/1799-D Revision 31=0A= + */=0A= +struct scsi_vpd_logical_block_prov=0A= +{=0A= + u_int8_t device;=0A= + u_int8_t page_code;=0A= +#define SVPD_LBP 0xB2=0A= + u_int8_t page_length[2];=0A= +#define SVPD_LBP_PL_BASIC 0x04=0A= + u_int8_t threshold_exponent;=0A= + u_int8_t flags;=0A= +#define SVPD_LBP_UNMAP 0x80=0A= +#define SVPD_LBP_WS16 0x40=0A= +#define SVPD_LBP_WS10 0x20=0A= +#define SVPD_LBP_RZ 0x04=0A= +#define SVPD_LBP_ANC_SUP 0x02=0A= +#define SVPD_LBP_DP 0x01=0A= + u_int8_t prov_type;=0A= +#define SVPD_LBP_RESOURCE 0x01=0A= +#define SVPD_LBP_THIN 0x02=0A= + u_int8_t reserved;=0A= + /*=0A= + * Provisioning Group Descriptor can be here if SVPD_LBP_DP is set=0A= + * Its size can be determined from page_length - 4=0A= + */=0A= +};=0A= +=0A= +/*=0A= + * Block Limits VDP Page based on=0A= + * T10/1799-D Revision 31=0A= + */=0A= +struct scsi_vpd_block_limits=0A= +{=0A= + u_int8_t device;=0A= + u_int8_t page_code;=0A= +#define SVPD_BLOCK_LIMITS 0xB0=0A= + u_int8_t page_length[2];=0A= +#define SVPD_BL_PL_BASIC 0x10=0A= +#define SVPD_BL_PL_TP 0x3C=0A= + u_int8_t reserved1;=0A= + u_int8_t max_cmp_write_len;=0A= + u_int8_t opt_txfer_len_grain[2];=0A= + u_int8_t max_txfer_len[4];=0A= + u_int8_t opt_txfer_len[4];=0A= + u_int8_t max_prefetch[4];=0A= + u_int8_t max_unmap_lba_cnt[4];=0A= + u_int8_t max_unmap_blk_cnt[4];=0A= + u_int8_t opt_unmap_grain[4];=0A= + u_int8_t unmap_grain_align[4];=0A= + u_int8_t max_write_same_length[8];=0A= + u_int8_t reserved2[20];=0A= +};=0A= +=0A= struct scsi_read_capacity=0A= {=0A= u_int8_t opcode;=0A= @@ -2180,6 +2287,8 @@=0A= char * scsi_sense_string(struct ccb_scsiio *csio,=0A= char *str, int str_len);=0A= void scsi_sense_print(struct ccb_scsiio *csio);=0A= +int scsi_vpd_supported_page(struct cam_periph *periph,=0A= + uint8_t page_id);=0A= #else /* _KERNEL */=0A= int scsi_command_string(struct cam_device *device,=0A= struct ccb_scsiio *csio, struct sbuf *sb);=0A= @@ -2370,6 +2479,26 @@=0A= u_int32_t dxfer_len, u_int8_t sense_len,=0A= u_int32_t timeout);=0A= =0A= +void scsi_ata_identify(struct ccb_scsiio *csio, u_int32_t retries,=0A= + void (*cbfcnp)(struct cam_periph *, union ccb *),=0A= + u_int8_t tag_action, u_int8_t *data_ptr,=0A= + u_int16_t dxfer_len, u_int8_t sense_len,=0A= + u_int32_t timeout);=0A= +=0A= +void scsi_ata_trim(struct ccb_scsiio *csio, u_int32_t retries,=0A= + void (*cbfcnp)(struct cam_periph *, union ccb *),=0A= + u_int8_t tag_action, u_int16_t block_count,=0A= + u_int8_t *data_ptr, u_int16_t dxfer_len,=0A= + u_int8_t sense_len, u_int32_t timeout);=0A= +=0A= +void scsi_ata_pass_16(struct ccb_scsiio *csio, u_int32_t retries,=0A= + void (*cbfcnp)(struct cam_periph *, union ccb *),=0A= + u_int32_t flags, u_int8_t tag_action,=0A= + u_int8_t protocol, u_int8_t ata_flags, u_int16_t features,=0A= + u_int16_t sector_count, uint64_t lba, u_int8_t command,=0A= + u_int8_t control, u_int8_t *data_ptr, u_int16_t dxfer_len,=0A= + u_int8_t sense_len, u_int32_t timeout);=0A= +=0A= void scsi_unmap(struct ccb_scsiio *csio, u_int32_t retries,=0A= void (*cbfcnp)(struct cam_periph *, union ccb *),=0A= u_int8_t tag_action, u_int8_t byte2,=0A= Index: sys/cam/scsi/scsi_xpt.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_xpt.c (revision 250577)=0A= +++ sys/cam/scsi/scsi_xpt.c (working copy)=0A= @@ -556,7 +556,6 @@=0A= static cam_status proberegister(struct cam_periph *periph,=0A= void *arg);=0A= static void probeschedule(struct cam_periph *probe_periph);=0A= -static int device_has_vpd(struct cam_ed *device, uint8_t page_id);=0A= static void probestart(struct cam_periph *periph, union ccb = *start_ccb);=0A= static void proberequestdefaultnegotiation(struct cam_periph *periph);=0A= static int proberequestbackoff(struct cam_periph *periph,=0A= @@ -708,21 +707,6 @@=0A= xpt_schedule(periph, CAM_PRIORITY_XPT);=0A= }=0A= =0A= -static int=0A= -device_has_vpd(struct cam_ed *device, uint8_t page_id)=0A= -{=0A= - int i, num_pages;=0A= - struct scsi_vpd_supported_pages *vpds;=0A= -=0A= - vpds =3D (struct scsi_vpd_supported_pages *)device->supported_vpds;=0A= - num_pages =3D device->supported_vpds_len - = SVPD_SUPPORTED_PAGES_HDR_LEN;=0A= - for (i =3D 0;i < num_pages;i++)=0A= - if (vpds->page_list[i] =3D=3D page_id)=0A= - return 1;=0A= -=0A= - return 0;=0A= -}=0A= -=0A= static void=0A= probestart(struct cam_periph *periph, union ccb *start_ccb)=0A= {=0A= @@ -910,11 +894,9 @@=0A= case PROBE_DEVICE_ID:=0A= {=0A= struct scsi_vpd_device_id *devid;=0A= - struct cam_ed *device;=0A= =0A= devid =3D NULL;=0A= - device =3D periph->path->device;=0A= - if (device_has_vpd(device, SVPD_DEVICE_ID))=0A= + if (scsi_vpd_supported_page(periph, SVPD_DEVICE_ID))=0A= devid =3D malloc(SVPD_DEVICE_ID_MAX_SIZE, M_CAMXPT,=0A= M_NOWAIT | M_ZERO);=0A= =0A= @@ -952,7 +934,7 @@=0A= device->serial_num_len =3D 0;=0A= }=0A= =0A= - if (device_has_vpd(device, SVPD_UNIT_SERIAL_NUMBER))=0A= + if (scsi_vpd_supported_page(periph, SVPD_UNIT_SERIAL_NUMBER))=0A= serial_buf =3D (struct scsi_vpd_unit_serial_number *)=0A= malloc(sizeof(*serial_buf), M_CAMXPT,=0A= M_NOWAIT|M_ZERO);=0A= Index: sys/cam/scsi/scsi_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/scsi/scsi_da.c (revision 250577)=0A= +++ sys/cam/scsi/scsi_da.c (working copy)=0A= @@ -44,6 +44,7 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= #include =0A= #include =0A= #endif /* _KERNEL */=0A= @@ -67,8 +68,12 @@=0A= =0A= #ifdef _KERNEL=0A= typedef enum {=0A= - DA_STATE_PROBE,=0A= - DA_STATE_PROBE2,=0A= + DA_STATE_PROBE_RC,=0A= + DA_STATE_PROBE_RC16,=0A= + DA_STATE_PROBE_LBP,=0A= + DA_STATE_PROBE_BLK_LIMITS,=0A= + DA_STATE_PROBE_BDC,=0A= + DA_STATE_PROBE_ATA,=0A= DA_STATE_NORMAL=0A= } da_state;=0A= =0A= @@ -96,29 +101,47 @@=0A= } da_quirks;=0A= =0A= typedef enum {=0A= - DA_CCB_PROBE =3D 0x01,=0A= - DA_CCB_PROBE2 =3D 0x02,=0A= - DA_CCB_BUFFER_IO =3D 0x03,=0A= - DA_CCB_WAITING =3D 0x04,=0A= - DA_CCB_DUMP =3D 0x05,=0A= - DA_CCB_DELETE =3D 0x06,=0A= - DA_CCB_TUR =3D 0x07,=0A= + DA_CCB_PROBE_RC =3D 0x01,=0A= + DA_CCB_PROBE_RC16 =3D 0x02,=0A= + DA_CCB_PROBE_LBP =3D 0x03,=0A= + DA_CCB_PROBE_BLK_LIMITS =3D 0x04,=0A= + DA_CCB_PROBE_BDC =3D 0x05,=0A= + DA_CCB_PROBE_ATA =3D 0x06,=0A= + DA_CCB_BUFFER_IO =3D 0x07,=0A= + DA_CCB_WAITING =3D 0x08,=0A= + DA_CCB_DUMP =3D 0x0A,=0A= + DA_CCB_DELETE =3D 0x0B,=0A= + DA_CCB_TUR =3D 0x0C,=0A= DA_CCB_TYPE_MASK =3D 0x0F,=0A= DA_CCB_RETRY_UA =3D 0x10=0A= } da_ccb_state;=0A= =0A= +/*=0A= + * Order here is important for method choice=0A= + *=0A= + * We prefer ATA_TRIM as tests run against a Sandforce 2281 SSD = attached to=0A= + * LSI 2008 (mps) controller (FW: v12, Drv: v14) resulted 20% quicker = deletes=0A= + * using ATA_TRIM than the corresponding UNMAP results for a real world = mysql=0A= + * import taking 5mins.=0A= + *=0A= + */=0A= typedef enum {=0A= DA_DELETE_NONE,=0A= DA_DELETE_DISABLE,=0A= + DA_DELETE_ATA_TRIM,=0A= + DA_DELETE_UNMAP,=0A= + DA_DELETE_WS16,=0A= + DA_DELETE_WS10,=0A= DA_DELETE_ZERO,=0A= - DA_DELETE_WS10,=0A= - DA_DELETE_WS16,=0A= - DA_DELETE_UNMAP,=0A= - DA_DELETE_MAX =3D DA_DELETE_UNMAP=0A= + DA_DELETE_MIN =3D DA_DELETE_ATA_TRIM,=0A= + DA_DELETE_MAX =3D DA_DELETE_ZERO=0A= } da_delete_methods;=0A= =0A= static const char *da_delete_method_names[] =3D=0A= - { "NONE", "DISABLE", "ZERO", "WS10", "WS16", "UNMAP" };=0A= + { "NONE", "DISABLE", "ATA_TRIM", "UNMAP", "WS16", "WS10", "ZERO" };=0A= +static const char *da_delete_method_desc[] =3D=0A= + { "NONE", "DISABLED", "ATA TRIM", "UNMAP", "WRITE SAME(16) with = UNMAP",=0A= + "WRITE SAME(10) with UNMAP", "ZERO" };=0A= =0A= /* Offsets into our private area for storing information */=0A= #define ccb_state ppriv_field0=0A= @@ -134,8 +157,18 @@=0A= u_int stripeoffset;=0A= };=0A= =0A= -#define UNMAP_MAX_RANGES 512=0A= +#define UNMAP_RANGE_MAX 0xffffffff=0A= +#define UNMAP_HEAD_SIZE 8=0A= +#define UNMAP_RANGE_SIZE 16=0A= +#define UNMAP_MAX_RANGES 2048 /* Protocol Max is 4095 */=0A= +#define UNMAP_BUF_SIZE ((UNMAP_MAX_RANGES * UNMAP_RANGE_SIZE) + \=0A= + UNMAP_HEAD_SIZE)=0A= =0A= +#define WS10_MAX_BLKS 0xffff=0A= +#define WS16_MAX_BLKS 0xffffffff=0A= +#define ATA_TRIM_MAX_RANGES ((UNMAP_BUF_SIZE / \=0A= + (ATA_DSM_RANGE_SIZE * ATA_DSM_BLK_SIZE)) * ATA_DSM_BLK_SIZE)=0A= +=0A= struct da_softc {=0A= struct bio_queue_head bio_queue;=0A= struct bio_queue_head delete_queue;=0A= @@ -145,15 +178,19 @@=0A= da_state state;=0A= da_flags flags; =0A= da_quirks quirks;=0A= + int sort_io_queue;=0A= int minimum_cmd_size;=0A= int error_inject;=0A= int ordered_tag_count;=0A= int outstanding_cmds;=0A= - int unmap_max_ranges;=0A= - int unmap_max_lba;=0A= + int trim_max_ranges;=0A= int delete_running;=0A= int tur;=0A= - da_delete_methods delete_method;=0A= + int delete_available; /* Delete methods possibly available */=0A= + uint32_t unmap_max_ranges;=0A= + uint32_t unmap_max_lba;=0A= + uint64_t ws_max_blks;=0A= + da_delete_methods delete_method;=0A= struct disk_params params;=0A= struct disk *disk;=0A= union ccb saved_ccb;=0A= @@ -162,11 +199,18 @@=0A= struct sysctl_oid *sysctl_tree;=0A= struct callout sendordered_c;=0A= uint64_t wwpn;=0A= - uint8_t unmap_buf[UNMAP_MAX_RANGES * 16 + 8];=0A= + uint8_t unmap_buf[UNMAP_BUF_SIZE];=0A= struct scsi_read_capacity_data_long rcaplong;=0A= struct callout mediapoll_c;=0A= };=0A= =0A= +#define dadeleteflag(softc, delete_method, enable) \=0A= + if (enable) { \=0A= + softc->delete_available |=3D (1 << delete_method); \=0A= + } else { \=0A= + softc->delete_available &=3D ~(1 << delete_method); \=0A= + }=0A= +=0A= struct da_quirk_entry {=0A= struct scsi_inquiry_pattern inq_pat;=0A= da_quirks quirks;=0A= @@ -869,6 +913,10 @@=0A= static int dadeletemethodsysctl(SYSCTL_HANDLER_ARGS);=0A= static int dadeletemethodset(struct da_softc *softc,=0A= da_delete_methods delete_method);=0A= +static void dadeletemethodchoose(struct da_softc *softc,=0A= + da_delete_methods default_method);=0A= +static void daprobedone(struct cam_periph *periph, union ccb *ccb);=0A= +=0A= static periph_ctor_t daregister;=0A= static periph_dtor_t dacleanup;=0A= static periph_start_t dastart;=0A= @@ -903,6 +951,8 @@=0A= #define DA_DEFAULT_SEND_ORDERED 1=0A= #endif=0A= =0A= +#define DA_SIO (softc->sort_io_queue >=3D 0 ? \=0A= + softc->sort_io_queue : cam_sort_io_queues)=0A= =0A= static int da_poll_period =3D DA_DEFAULT_POLL_PERIOD;=0A= static int da_retry_count =3D DA_DEFAULT_RETRY;=0A= @@ -1129,10 +1179,15 @@=0A= if (bp->bio_cmd =3D=3D BIO_DELETE) {=0A= if (bp->bio_bcount =3D=3D 0)=0A= biodone(bp);=0A= + else if (DA_SIO)=0A= + bioq_disksort(&softc->delete_queue, bp);=0A= else=0A= - bioq_disksort(&softc->delete_queue, bp);=0A= - } else=0A= + bioq_insert_tail(&softc->delete_queue, bp);=0A= + } else if (DA_SIO) {=0A= bioq_disksort(&softc->bio_queue, bp);=0A= + } else {=0A= + bioq_insert_tail(&softc->bio_queue, bp);=0A= + }=0A= =0A= /*=0A= * Schedule ourselves for performing the work.=0A= @@ -1487,6 +1542,9 @@=0A= OID_AUTO, "minimum_cmd_size", CTLTYPE_INT | CTLFLAG_RW,=0A= &softc->minimum_cmd_size, 0, dacmdsizesysctl, "I",=0A= "Minimum CDB size");=0A= + SYSCTL_ADD_INT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree),=0A= + OID_AUTO, "sort_io_queue", CTLFLAG_RW, &softc->sort_io_queue, 0,=0A= + "Sort IO queue to try and optimise disk access patterns");=0A= =0A= SYSCTL_ADD_INT(&softc->sysctl_ctx,=0A= SYSCTL_CHILDREN(softc->sysctl_tree),=0A= @@ -1574,6 +1632,85 @@=0A= return (0);=0A= }=0A= =0A= +static void=0A= +daprobedone(struct cam_periph *periph, union ccb *ccb)=0A= +{=0A= + struct da_softc *softc;=0A= +=0A= + softc =3D (struct da_softc *)periph->softc;=0A= +=0A= + dadeletemethodchoose(softc, DA_DELETE_NONE);=0A= +=0A= + if (bootverbose && (softc->flags & DA_FLAG_PROBED) =3D=3D 0) {=0A= + char buf[80];=0A= + int i, sep;=0A= +=0A= + snprintf(buf, sizeof(buf), "Delete methods: <");=0A= + sep =3D 0;=0A= + for (i =3D DA_DELETE_MIN; i <=3D DA_DELETE_MAX; i++) {=0A= + if (softc->delete_available & (1 << i)) {=0A= + if (sep) {=0A= + strlcat(buf, ",", sizeof(buf));=0A= + } else {=0A= + sep =3D 1;=0A= + }=0A= + strlcat(buf, da_delete_method_names[i],=0A= + sizeof(buf));=0A= + if (i =3D=3D softc->delete_method) {=0A= + strlcat(buf, "(*)", sizeof(buf));=0A= + }=0A= + }=0A= + }=0A= + if (sep =3D=3D 0) {=0A= + if (softc->delete_method =3D=3D DA_DELETE_NONE) =0A= + strlcat(buf, "NONE(*)", sizeof(buf));=0A= + else=0A= + strlcat(buf, "DISABLED(*)", sizeof(buf));=0A= + }=0A= + strlcat(buf, ">", sizeof(buf));=0A= + printf("%s%d: %s\n", periph->periph_name,=0A= + periph->unit_number, buf);=0A= + }=0A= +=0A= + /*=0A= + * Since our peripheral may be invalidated by an error=0A= + * above or an external event, we must release our CCB=0A= + * before releasing the probe lock on the peripheral.=0A= + * The peripheral will only go away once the last lock=0A= + * is removed, and we need it around for the CCB release=0A= + * operation.=0A= + */=0A= + xpt_release_ccb(ccb);=0A= + softc->state =3D DA_STATE_NORMAL;=0A= + daschedule(periph);=0A= + wakeup(&softc->disk->d_mediasize);=0A= + if ((softc->flags & DA_FLAG_PROBED) =3D=3D 0) {=0A= + softc->flags |=3D DA_FLAG_PROBED;=0A= + cam_periph_unhold(periph);=0A= + } else=0A= + cam_periph_release_locked(periph);=0A= +}=0A= +=0A= +static void=0A= +dadeletemethodchoose(struct da_softc *softc, da_delete_methods = default_method)=0A= +{=0A= + int i, delete_method;=0A= +=0A= + delete_method =3D default_method;=0A= +=0A= + /*=0A= + * Use the pre-defined order to choose the best=0A= + * performing delete.=0A= + */=0A= + for (i =3D DA_DELETE_MIN; i <=3D DA_DELETE_MAX; i++) {=0A= + if (softc->delete_available & (1 << i)) {=0A= + dadeletemethodset(softc, i);=0A= + return;=0A= + }=0A= + }=0A= + dadeletemethodset(softc, delete_method);=0A= +}=0A= +=0A= static int=0A= dadeletemethodsysctl(SYSCTL_HANDLER_ARGS)=0A= {=0A= @@ -1626,14 +1763,17 @@=0A= }=0A= =0A= LIST_INIT(&softc->pending_ccbs);=0A= - softc->state =3D DA_STATE_PROBE;=0A= + softc->state =3D DA_STATE_PROBE_RC;=0A= bioq_init(&softc->bio_queue);=0A= bioq_init(&softc->delete_queue);=0A= bioq_init(&softc->delete_run_queue);=0A= if (SID_IS_REMOVABLE(&cgd->inq_data))=0A= softc->flags |=3D DA_FLAG_PACK_REMOVABLE;=0A= softc->unmap_max_ranges =3D UNMAP_MAX_RANGES;=0A= - softc->unmap_max_lba =3D 1024*1024*2;=0A= + softc->unmap_max_lba =3D UNMAP_RANGE_MAX;=0A= + softc->ws_max_blks =3D WS16_MAX_BLKS;=0A= + softc->trim_max_ranges =3D ATA_TRIM_MAX_RANGES;=0A= + softc->sort_io_queue =3D -1;=0A= =0A= periph->softc =3D softc;=0A= =0A= @@ -1709,7 +1849,7 @@=0A= /* Predict whether device may support READ CAPACITY(16). */=0A= if (SID_ANSI_REV(&cgd->inq_data) >=3D SCSI_REV_SPC3) {=0A= softc->flags |=3D DA_FLAG_CAN_RC16;=0A= - softc->state =3D DA_STATE_PROBE2;=0A= + softc->state =3D DA_STATE_PROBE_RC16;=0A= }=0A= =0A= /*=0A= @@ -1809,6 +1949,7 @@=0A= =0A= CAM_DEBUG(periph->path, CAM_DEBUG_TRACE, ("dastart\n"));=0A= =0A= +skipstate:=0A= switch (softc->state) {=0A= case DA_STATE_NORMAL:=0A= {=0A= @@ -1833,14 +1974,37 @@=0A= if (!softc->delete_running &&=0A= (bp =3D bioq_first(&softc->delete_queue)) !=3D NULL) {=0A= uint64_t lba;=0A= - u_int count;=0A= + uint64_t count; /* forward compat with WS32 */=0A= =0A= + /*=0A= + * In each of the methods below, while its the caller's=0A= + * responsibility to ensure the request will fit into a=0A= + * single device request, we might have changed the delete=0A= + * method due to the device incorrectly advertising either=0A= + * its supported methods or limits.=0A= + * =0A= + * To prevent this causing further issues we validate the=0A= + * against the methods limits, and warn which would=0A= + * otherwise be unnecessary.=0A= + */=0A= +=0A= if (softc->delete_method =3D=3D DA_DELETE_UNMAP) {=0A= uint8_t *buf =3D softc->unmap_buf;=0A= uint64_t lastlba =3D (uint64_t)-1;=0A= - uint32_t lastcount =3D 0;=0A= - int blocks =3D 0, off, ranges =3D 0;=0A= + uint32_t lastcount =3D 0, c;=0A= + uint64_t totalcount =3D 0;=0A= + uint32_t off, ranges =3D 0;=0A= =0A= + /*=0A= + * Currently this doesn't take the UNMAP=0A= + * Granularity and Granularity Alignment=0A= + * fields into account.=0A= + *=0A= + * This could result in both unoptimal unmap=0A= + * requests as as well as UNMAP calls unmapping=0A= + * fewer LBA's than requested.=0A= + */=0A= +=0A= softc->delete_running =3D 1;=0A= bzero(softc->unmap_buf, sizeof(softc->unmap_buf));=0A= bp1 =3D bp;=0A= @@ -1853,22 +2017,44 @@=0A= =0A= /* Try to extend the previous range. */=0A= if (lba =3D=3D lastlba) {=0A= - lastcount +=3D count;=0A= - off =3D (ranges - 1) * 16 + 8;=0A= + c =3D min(count, softc->unmap_max_lba -=0A= + lastcount);=0A= + lastcount +=3D c;=0A= + off =3D ((ranges - 1) * UNMAP_RANGE_SIZE) +=0A= + UNMAP_HEAD_SIZE;=0A= scsi_ulto4b(lastcount, &buf[off + 8]);=0A= - } else if (count > 0) {=0A= - off =3D ranges * 16 + 8;=0A= + count -=3D c;=0A= + lba +=3Dc;=0A= + totalcount +=3D c;=0A= + }=0A= +=0A= + while (count > 0) {=0A= + c =3D min(count, softc->unmap_max_lba);=0A= + if (totalcount + c > softc->unmap_max_lba ||=0A= + ranges >=3D softc->unmap_max_ranges) {=0A= + xpt_print(periph->path,=0A= + "%s issuing short delete %ld > %ld"=0A= + "|| %d >=3D %d",=0A= + da_delete_method_desc[softc->delete_method],=0A= + totalcount + c, softc->unmap_max_lba,=0A= + ranges, softc->unmap_max_ranges);=0A= + break;=0A= + }=0A= + off =3D (ranges * UNMAP_RANGE_SIZE) +=0A= + UNMAP_HEAD_SIZE;=0A= scsi_u64to8b(lba, &buf[off + 0]);=0A= - scsi_ulto4b(count, &buf[off + 8]);=0A= - lastcount =3D count;=0A= + scsi_ulto4b(c, &buf[off + 8]);=0A= + lba +=3D c;=0A= + totalcount +=3D c;=0A= ranges++;=0A= + count -=3D c;=0A= + lastcount =3D c;=0A= }=0A= - blocks +=3D count;=0A= - lastlba =3D lba + count;=0A= + lastlba =3D lba;=0A= bp1 =3D bioq_first(&softc->delete_queue);=0A= if (bp1 =3D=3D NULL ||=0A= ranges >=3D softc->unmap_max_ranges ||=0A= - blocks + bp1->bio_bcount /=0A= + totalcount + bp1->bio_bcount /=0A= softc->params.secsize > softc->unmap_max_lba)=0A= break;=0A= } while (1);=0A= @@ -1886,9 +2072,87 @@=0A= da_default_timeout * 1000);=0A= start_ccb->ccb_h.ccb_state =3D DA_CCB_DELETE;=0A= goto out;=0A= + } else if (softc->delete_method =3D=3D DA_DELETE_ATA_TRIM) {=0A= + uint8_t *buf =3D softc->unmap_buf;=0A= + uint64_t lastlba =3D (uint64_t)-1;=0A= + uint32_t lastcount =3D 0, c, requestcount;=0A= + int ranges =3D 0, off, block_count;=0A= +=0A= + softc->delete_running =3D 1;=0A= + bzero(softc->unmap_buf, sizeof(softc->unmap_buf));=0A= + bp1 =3D bp;=0A= + do {=0A= + bioq_remove(&softc->delete_queue, bp1);=0A= + if (bp1 !=3D bp)=0A= + bioq_insert_tail(&softc->delete_run_queue, bp1);=0A= + lba =3D bp1->bio_pblkno;=0A= + count =3D bp1->bio_bcount / softc->params.secsize;=0A= + requestcount =3D count;=0A= +=0A= + /* Try to extend the previous range. */=0A= + if (lba =3D=3D lastlba) {=0A= + c =3D min(count, ATA_DSM_RANGE_MAX - lastcount);=0A= + lastcount +=3D c;=0A= + off =3D (ranges - 1) * 8;=0A= + buf[off + 6] =3D lastcount & 0xff;=0A= + buf[off + 7] =3D (lastcount >> 8) & 0xff;=0A= + count -=3D c;=0A= + lba +=3D c;=0A= + }=0A= +=0A= + while (count > 0) {=0A= + c =3D min(count, ATA_DSM_RANGE_MAX);=0A= + off =3D ranges * 8;=0A= +=0A= + buf[off + 0] =3D lba & 0xff;=0A= + buf[off + 1] =3D (lba >> 8) & 0xff;=0A= + buf[off + 2] =3D (lba >> 16) & 0xff;=0A= + buf[off + 3] =3D (lba >> 24) & 0xff;=0A= + buf[off + 4] =3D (lba >> 32) & 0xff;=0A= + buf[off + 5] =3D (lba >> 40) & 0xff;=0A= + buf[off + 6] =3D c & 0xff;=0A= + buf[off + 7] =3D (c >> 8) & 0xff;=0A= + lba +=3D c;=0A= + ranges++;=0A= + count -=3D c;=0A= + lastcount =3D c;=0A= + if (count !=3D 0 && ranges =3D=3D softc->trim_max_ranges) {=0A= + xpt_print(periph->path,=0A= + "%s issuing short delete %ld > %ld",=0A= + da_delete_method_desc[softc->delete_method],=0A= + requestcount,=0A= + (softc->trim_max_ranges - ranges) *=0A= + ATA_DSM_RANGE_MAX);=0A= + break;=0A= + }=0A= + }=0A= + lastlba =3D lba;=0A= + bp1 =3D bioq_first(&softc->delete_queue);=0A= + if (bp1 =3D=3D NULL ||=0A= + bp1->bio_bcount / softc->params.secsize >=0A= + (softc->trim_max_ranges - ranges) *=0A= + ATA_DSM_RANGE_MAX)=0A= + break;=0A= + } while (1);=0A= +=0A= + block_count =3D (ranges + ATA_DSM_BLK_RANGES - 1) /=0A= + ATA_DSM_BLK_RANGES;=0A= + scsi_ata_trim(&start_ccb->csio,=0A= + /*retries*/da_retry_count,=0A= + /*cbfcnp*/dadone,=0A= + /*tag_action*/MSG_SIMPLE_Q_TAG,=0A= + block_count,=0A= + /*data_ptr*/buf,=0A= + /*dxfer_len*/block_count * ATA_DSM_BLK_SIZE,=0A= + /*sense_len*/SSD_FULL_SIZE,=0A= + da_default_timeout * 1000);=0A= + start_ccb->ccb_h.ccb_state =3D DA_CCB_DELETE;=0A= + goto out;=0A= } else if (softc->delete_method =3D=3D DA_DELETE_ZERO ||=0A= softc->delete_method =3D=3D DA_DELETE_WS10 ||=0A= softc->delete_method =3D=3D DA_DELETE_WS16) {=0A= + uint64_t ws_max_blks;=0A= + ws_max_blks =3D softc->ws_max_blks / softc->params.secsize;=0A= softc->delete_running =3D 1;=0A= lba =3D bp->bio_pblkno;=0A= count =3D 0;=0A= @@ -1898,11 +2162,19 @@=0A= if (bp1 !=3D bp)=0A= bioq_insert_tail(&softc->delete_run_queue, bp1);=0A= count +=3D bp1->bio_bcount / softc->params.secsize;=0A= + if (count > ws_max_blks) {=0A= + count =3D min(count, ws_max_blks);=0A= + xpt_print(periph->path,=0A= + "%s issuing short delete %ld > %ld",=0A= + da_delete_method_desc[softc->delete_method],=0A= + count, ws_max_blks);=0A= + break;=0A= + }=0A= bp1 =3D bioq_first(&softc->delete_queue);=0A= if (bp1 =3D=3D NULL ||=0A= lba + count !=3D bp1->bio_pblkno ||=0A= count + bp1->bio_bcount /=0A= - softc->params.secsize > 0xffff)=0A= + softc->params.secsize > ws_max_blks)=0A= break;=0A= } while (1);=0A= =0A= @@ -2022,7 +2294,7 @@=0A= daschedule(periph);=0A= break;=0A= }=0A= - case DA_STATE_PROBE:=0A= + case DA_STATE_PROBE_RC:=0A= {=0A= struct scsi_read_capacity_data *rcap;=0A= =0A= @@ -2041,11 +2313,11 @@=0A= SSD_FULL_SIZE,=0A= /*timeout*/5000);=0A= start_ccb->ccb_h.ccb_bp =3D NULL;=0A= - start_ccb->ccb_h.ccb_state =3D DA_CCB_PROBE;=0A= + start_ccb->ccb_h.ccb_state =3D DA_CCB_PROBE_RC;=0A= xpt_action(start_ccb);=0A= break;=0A= }=0A= - case DA_STATE_PROBE2:=0A= + case DA_STATE_PROBE_RC16:=0A= {=0A= struct scsi_read_capacity_data_long *rcaplong;=0A= =0A= @@ -2067,11 +2339,151 @@=0A= /*sense_len*/ SSD_FULL_SIZE,=0A= /*timeout*/ da_default_timeout * 1000);=0A= start_ccb->ccb_h.ccb_bp =3D NULL;=0A= - start_ccb->ccb_h.ccb_state =3D DA_CCB_PROBE2;=0A= - xpt_action(start_ccb); =0A= + start_ccb->ccb_h.ccb_state =3D DA_CCB_PROBE_RC16;=0A= + xpt_action(start_ccb);=0A= break;=0A= }=0A= + case DA_STATE_PROBE_LBP:=0A= + {=0A= + struct scsi_vpd_logical_block_prov *lbp;=0A= +=0A= + if (!scsi_vpd_supported_page(periph, SVPD_LBP)) {=0A= + /*=0A= + * If we get here we don't support any SBC-3 delete=0A= + * methods with UNMAP as the Logical Block Provisioning=0A= + * VPD page support is required for devices which=0A= + * support it according to T10/1799-D Revision 31=0A= + * however older revisions of the spec don't mandate=0A= + * this so we currently don't remove these methods=0A= + * from the available set.=0A= + */=0A= + softc->state =3D DA_STATE_PROBE_BLK_LIMITS;=0A= + goto skipstate;=0A= + }=0A= +=0A= + lbp =3D (struct scsi_vpd_logical_block_prov *)=0A= + malloc(sizeof(*lbp), M_SCSIDA, M_NOWAIT|M_ZERO);=0A= +=0A= + if (lbp =3D=3D NULL) {=0A= + printf("dastart: Couldn't malloc lbp data\n");=0A= + /* da_free_periph??? */=0A= + break;=0A= + }=0A= +=0A= + scsi_inquiry(&start_ccb->csio,=0A= + /*retries*/da_retry_count,=0A= + /*cbfcnp*/dadone,=0A= + /*tag_action*/MSG_SIMPLE_Q_TAG,=0A= + /*inq_buf*/(u_int8_t *)lbp,=0A= + /*inq_len*/sizeof(*lbp),=0A= + /*evpd*/TRUE,=0A= + /*page_code*/SVPD_LBP,=0A= + /*sense_len*/SSD_MIN_SIZE,=0A= + /*timeout*/da_default_timeout * 1000);=0A= + start_ccb->ccb_h.ccb_bp =3D NULL;=0A= + start_ccb->ccb_h.ccb_state =3D DA_CCB_PROBE_LBP;=0A= + xpt_action(start_ccb);=0A= + break;=0A= }=0A= + case DA_STATE_PROBE_BLK_LIMITS:=0A= + {=0A= + struct scsi_vpd_block_limits *block_limits;=0A= +=0A= + if (!scsi_vpd_supported_page(periph, SVPD_BLOCK_LIMITS)) {=0A= + /* Not supported skip to next probe */=0A= + softc->state =3D DA_STATE_PROBE_BDC;=0A= + goto skipstate;=0A= + }=0A= +=0A= + block_limits =3D (struct scsi_vpd_block_limits *)=0A= + malloc(sizeof(*block_limits), M_SCSIDA, M_NOWAIT|M_ZERO);=0A= +=0A= + if (block_limits =3D=3D NULL) {=0A= + printf("dastart: Couldn't malloc block_limits data\n");=0A= + /* da_free_periph??? */=0A= + break;=0A= + }=0A= +=0A= + scsi_inquiry(&start_ccb->csio,=0A= + /*retries*/da_retry_count,=0A= + /*cbfcnp*/dadone,=0A= + /*tag_action*/MSG_SIMPLE_Q_TAG,=0A= + /*inq_buf*/(u_int8_t *)block_limits,=0A= + /*inq_len*/sizeof(*block_limits),=0A= + /*evpd*/TRUE,=0A= + /*page_code*/SVPD_BLOCK_LIMITS,=0A= + /*sense_len*/SSD_MIN_SIZE,=0A= + /*timeout*/da_default_timeout * 1000);=0A= + start_ccb->ccb_h.ccb_bp =3D NULL;=0A= + start_ccb->ccb_h.ccb_state =3D DA_CCB_PROBE_BLK_LIMITS;=0A= + xpt_action(start_ccb);=0A= + break;=0A= + }=0A= + case DA_STATE_PROBE_BDC:=0A= + {=0A= + struct scsi_vpd_block_characteristics *bdc;=0A= +=0A= + if (!scsi_vpd_supported_page(periph, SVPD_BDC)) {=0A= + softc->state =3D DA_STATE_PROBE_ATA;=0A= + goto skipstate;=0A= + }=0A= +=0A= + bdc =3D (struct scsi_vpd_block_characteristics *)=0A= + malloc(sizeof(*bdc), M_SCSIDA, M_NOWAIT|M_ZERO);=0A= +=0A= + if (bdc =3D=3D NULL) {=0A= + printf("dastart: Couldn't malloc bdc data\n");=0A= + /* da_free_periph??? */=0A= + break;=0A= + }=0A= +=0A= + scsi_inquiry(&start_ccb->csio,=0A= + /*retries*/da_retry_count,=0A= + /*cbfcnp*/dadone,=0A= + /*tag_action*/MSG_SIMPLE_Q_TAG,=0A= + /*inq_buf*/(u_int8_t *)bdc,=0A= + /*inq_len*/sizeof(*bdc),=0A= + /*evpd*/TRUE,=0A= + /*page_code*/SVPD_BDC,=0A= + /*sense_len*/SSD_MIN_SIZE,=0A= + /*timeout*/da_default_timeout * 1000);=0A= + start_ccb->ccb_h.ccb_bp =3D NULL;=0A= + start_ccb->ccb_h.ccb_state =3D DA_CCB_PROBE_BDC;=0A= + xpt_action(start_ccb);=0A= + break;=0A= + }=0A= + case DA_STATE_PROBE_ATA:=0A= + {=0A= + struct ata_params *ata_params;=0A= +=0A= + if (!scsi_vpd_supported_page(periph, SVPD_ATA_INFORMATION)) {=0A= + daprobedone(periph, start_ccb);=0A= + break;=0A= + }=0A= +=0A= + ata_params =3D (struct ata_params*)=0A= + malloc(sizeof(*ata_params), M_SCSIDA, M_NOWAIT|M_ZERO);=0A= +=0A= + if (ata_params =3D=3D NULL) {=0A= + printf("dastart: Couldn't malloc ata_params data\n");=0A= + /* da_free_periph??? */=0A= + break;=0A= + }=0A= +=0A= + scsi_ata_identify(&start_ccb->csio,=0A= + /*retries*/da_retry_count,=0A= + /*cbfcnp*/dadone,=0A= + /*tag_action*/MSG_SIMPLE_Q_TAG,=0A= + /*data_ptr*/(u_int8_t *)ata_params,=0A= + /*dxfer_len*/sizeof(*ata_params),=0A= + /*sense_len*/SSD_FULL_SIZE,=0A= + /*timeout*/da_default_timeout * 1000);=0A= + start_ccb->ccb_h.ccb_bp =3D NULL;=0A= + start_ccb->ccb_h.ccb_state =3D DA_CCB_PROBE_ATA;=0A= + xpt_action(start_ccb);=0A= + break;=0A= + }=0A= + }=0A= }=0A= =0A= static int=0A= @@ -2088,30 +2500,41 @@=0A= softc =3D (struct da_softc *)xpt_path_periph(ccb->ccb_h.path)->softc;=0A= =0A= if (ccb->ccb_h.ccb_state =3D=3D DA_CCB_DELETE) {=0A= - if (softc->delete_method =3D=3D DA_DELETE_UNMAP) {=0A= - xpt_print(ccb->ccb_h.path, "UNMAP is not supported, "=0A= - "switching to WRITE SAME(16) with UNMAP.\n");=0A= - dadeletemethodset(softc, DA_DELETE_WS16);=0A= - } else if (softc->delete_method =3D=3D DA_DELETE_WS16) {=0A= + da_delete_methods old_method =3D softc->delete_method;=0A= +=0A= + /*=0A= + * Typically there are two reasons for failure here=0A= + * 1. Delete method was detected as supported but isn't=0A= + * 2. Delete failed due to invalid params e.g. too big=0A= + *=0A= + * While we will attempt to choose an alternative delete method=0A= + * this may result in short deletes if the existing delete=0A= + * requests from geom are big for the new method choosen.=0A= + *=0A= + * This method assumes that the error which triggered this=0A= + * will not retry the io otherwise a panic will occur=0A= + */=0A= + dadeleteflag(softc, old_method, 0);=0A= + dadeletemethodchoose(softc, DA_DELETE_DISABLE);=0A= + if (softc->delete_method =3D=3D DA_DELETE_DISABLE)=0A= xpt_print(ccb->ccb_h.path,=0A= - "WRITE SAME(16) with UNMAP is not supported, "=0A= - "disabling BIO_DELETE.\n");=0A= - dadeletemethodset(softc, DA_DELETE_DISABLE);=0A= - } else if (softc->delete_method =3D=3D DA_DELETE_WS10) {=0A= + "%s failed, disabling BIO_DELETE\n",=0A= + da_delete_method_desc[old_method]);=0A= + else=0A= xpt_print(ccb->ccb_h.path,=0A= - "WRITE SAME(10) with UNMAP is not supported, "=0A= - "disabling BIO_DELETE.\n");=0A= - dadeletemethodset(softc, DA_DELETE_DISABLE);=0A= - } else if (softc->delete_method =3D=3D DA_DELETE_ZERO) {=0A= - xpt_print(ccb->ccb_h.path,=0A= - "WRITE SAME(10) is not supported, "=0A= - "disabling BIO_DELETE.\n");=0A= - dadeletemethodset(softc, DA_DELETE_DISABLE);=0A= - } else=0A= - dadeletemethodset(softc, DA_DELETE_DISABLE);=0A= - while ((bp =3D bioq_takefirst(&softc->delete_run_queue))=0A= - !=3D NULL)=0A= - bioq_disksort(&softc->delete_queue, bp);=0A= + "%s failed, switching to %s BIO_DELETE\n",=0A= + da_delete_method_desc[old_method],=0A= + da_delete_method_desc[softc->delete_method]);=0A= +=0A= + if (DA_SIO) {=0A= + while ((bp =3D bioq_takefirst(&softc->delete_run_queue))=0A= + !=3D NULL)=0A= + bioq_disksort(&softc->delete_queue, bp);=0A= + } else {=0A= + while ((bp =3D bioq_takefirst(&softc->delete_run_queue))=0A= + !=3D NULL)=0A= + bioq_insert_tail(&softc->delete_queue, bp);=0A= + }=0A= bioq_insert_tail(&softc->delete_queue,=0A= (struct bio *)ccb->ccb_h.ccb_bp);=0A= ccb->ccb_h.ccb_bp =3D NULL;=0A= @@ -2185,7 +2608,7 @@=0A= error =3D daerror(done_ccb, CAM_RETRY_SELTO, sf);=0A= if (error =3D=3D ERESTART) {=0A= /*=0A= - * A retry was scheuled, so=0A= + * A retry was scheduled, so=0A= * just return.=0A= */=0A= return;=0A= @@ -2281,16 +2704,18 @@=0A= biodone(bp);=0A= break;=0A= }=0A= - case DA_CCB_PROBE:=0A= - case DA_CCB_PROBE2:=0A= + case DA_CCB_PROBE_RC:=0A= + case DA_CCB_PROBE_RC16:=0A= {=0A= struct scsi_read_capacity_data *rdcap;=0A= struct scsi_read_capacity_data_long *rcaplong;=0A= char announce_buf[80];=0A= + int lbp;=0A= =0A= + lbp =3D 0;=0A= rdcap =3D NULL;=0A= rcaplong =3D NULL;=0A= - if (state =3D=3D DA_CCB_PROBE)=0A= + if (state =3D=3D DA_CCB_PROBE_RC)=0A= rdcap =3D(struct scsi_read_capacity_data *)csio->data_ptr;=0A= else=0A= rcaplong =3D (struct scsi_read_capacity_data_long *)=0A= @@ -2303,7 +2728,7 @@=0A= u_int lbppbe; /* LB per physical block exponent. */=0A= u_int lalba; /* Lowest aligned LBA. */=0A= =0A= - if (state =3D=3D DA_CCB_PROBE) {=0A= + if (state =3D=3D DA_CCB_PROBE_RC) {=0A= block_size =3D scsi_4btoul(rdcap->length);=0A= maxsector =3D scsi_4btoul(rdcap->addr);=0A= lbppbe =3D 0;=0A= @@ -2318,9 +2743,9 @@=0A= * with the short version of the command.=0A= */=0A= if (maxsector =3D=3D 0xffffffff) {=0A= - softc->state =3D DA_STATE_PROBE2;=0A= free(rdcap, M_SCSIDA);=0A= xpt_release_ccb(done_ccb);=0A= + softc->state =3D DA_STATE_PROBE_RC16;=0A= xpt_schedule(periph, priority);=0A= return;=0A= }=0A= @@ -2353,9 +2778,7 @@=0A= */=0A= dasetgeom(periph, block_size, maxsector,=0A= rcaplong, sizeof(*rcaplong));=0A= - if ((lalba & SRC16_LBPME_A)=0A= - && softc->delete_method =3D=3D DA_DELETE_NONE)=0A= - dadeletemethodset(softc, DA_DELETE_UNMAP);=0A= + lbp =3D (lalba & SRC16_LBPME_A);=0A= dp =3D &softc->params;=0A= snprintf(announce_buf, sizeof(announce_buf),=0A= "%juMB (%ju %u byte sectors: %dH %dS/T "=0A= @@ -2416,7 +2839,7 @@=0A= * If we tried READ CAPACITY(16) and failed,=0A= * fallback to READ CAPACITY(10).=0A= */=0A= - if ((state =3D=3D DA_CCB_PROBE2) &&=0A= + if ((state =3D=3D DA_CCB_PROBE_RC16) &&=0A= (softc->flags & DA_FLAG_CAN_RC16) &&=0A= (((csio->ccb_h.status & CAM_STATUS_MASK) =3D=3D=0A= CAM_REQ_INVALID) ||=0A= @@ -2424,9 +2847,9 @@=0A= (error_code =3D=3D SSD_CURRENT_ERROR) &&=0A= (sense_key =3D=3D SSD_KEY_ILLEGAL_REQUEST)))) {=0A= softc->flags &=3D ~DA_FLAG_CAN_RC16;=0A= - softc->state =3D DA_STATE_PROBE;=0A= free(rdcap, M_SCSIDA);=0A= xpt_release_ccb(done_ccb);=0A= + softc->state =3D DA_STATE_PROBE_RC;=0A= xpt_schedule(periph, priority);=0A= return;=0A= } else=0A= @@ -2482,30 +2905,250 @@=0A= taskqueue_enqueue(taskqueue_thread,=0A= &softc->sysctl_task);=0A= xpt_announce_periph(periph, announce_buf);=0A= +=0A= } else {=0A= xpt_print(periph->path, "fatal error, "=0A= "could not acquire reference count\n");=0A= }=0A= }=0A= - /*=0A= - * Since our peripheral may be invalidated by an error=0A= - * above or an external event, we must release our CCB=0A= - * before releasing the probe lock on the peripheral.=0A= - * The peripheral will only go away once the last lock=0A= - * is removed, and we need it around for the CCB release=0A= - * operation.=0A= - */=0A= +=0A= + /* Ensure re-probe doesn't see old delete. */=0A= + softc->delete_available =3D 0;=0A= + if (lbp) {=0A= + /*=0A= + * Based on older SBC-3 spec revisions=0A= + * any of the UNMAP methods "may" be=0A= + * available via LBP given this flag so=0A= + * we flag all of them as availble and=0A= + * then remove those which further=0A= + * probes confirm aren't available=0A= + * later.=0A= + *=0A= + * We could also check readcap(16) p_type=0A= + * flag to exclude one or more invalid=0A= + * write same (X) types here=0A= + */=0A= + dadeleteflag(softc, DA_DELETE_WS16, 1);=0A= + dadeleteflag(softc, DA_DELETE_WS10, 1);=0A= + dadeleteflag(softc, DA_DELETE_ZERO, 1);=0A= + dadeleteflag(softc, DA_DELETE_UNMAP, 1);=0A= +=0A= + xpt_release_ccb(done_ccb);=0A= + softc->state =3D DA_STATE_PROBE_LBP;=0A= + xpt_schedule(periph, priority);=0A= + return;=0A= + }=0A= +=0A= xpt_release_ccb(done_ccb);=0A= - softc->state =3D DA_STATE_NORMAL;=0A= - daschedule(periph);=0A= - wakeup(&softc->disk->d_mediasize);=0A= - if ((softc->flags & DA_FLAG_PROBED) =3D=3D 0) {=0A= - softc->flags |=3D DA_FLAG_PROBED;=0A= - cam_periph_unhold(periph);=0A= - } else=0A= - cam_periph_release_locked(periph);=0A= + softc->state =3D DA_STATE_PROBE_BDC;=0A= + xpt_schedule(periph, priority);=0A= return;=0A= }=0A= + case DA_CCB_PROBE_LBP:=0A= + {=0A= + struct scsi_vpd_logical_block_prov *lbp;=0A= +=0A= + lbp =3D (struct scsi_vpd_logical_block_prov *)csio->data_ptr;=0A= +=0A= + if ((csio->ccb_h.status & CAM_STATUS_MASK) =3D=3D CAM_REQ_CMP) {=0A= + /*=0A= + * T10/1799-D Revision 31 states at least one of these=0A= + * must be supported but we don't currently enforce this.=0A= + */=0A= + dadeleteflag(softc, DA_DELETE_WS16,=0A= + (lbp->flags & SVPD_LBP_WS16));=0A= + dadeleteflag(softc, DA_DELETE_WS10,=0A= + (lbp->flags & SVPD_LBP_WS10));=0A= + dadeleteflag(softc, DA_DELETE_ZERO,=0A= + (lbp->flags & SVPD_LBP_WS10));=0A= + dadeleteflag(softc, DA_DELETE_UNMAP,=0A= + (lbp->flags & SVPD_LBP_UNMAP));=0A= +=0A= + if (lbp->flags & SVPD_LBP_UNMAP) {=0A= + free(lbp, M_SCSIDA);=0A= + xpt_release_ccb(done_ccb);=0A= + softc->state =3D DA_STATE_PROBE_BLK_LIMITS;=0A= + xpt_schedule(periph, priority);=0A= + return;=0A= + }=0A= + } else {=0A= + int error;=0A= + error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= + SF_RETRY_UA|SF_NO_PRINT);=0A= + if (error =3D=3D ERESTART)=0A= + return;=0A= + else if (error !=3D 0) {=0A= + if ((done_ccb->ccb_h.status & CAM_DEV_QFRZN) !=3D 0) {=0A= + /* Don't wedge this device's queue */=0A= + cam_release_devq(done_ccb->ccb_h.path,=0A= + /*relsim_flags*/0,=0A= + /*reduction*/0,=0A= + /*timeout*/0,=0A= + /*getcount_only*/0);=0A= + }=0A= +=0A= + /*=0A= + * Failure indicates we don't support any SBC-3=0A= + * delete methods with UNMAP=0A= + */=0A= + }=0A= + }=0A= +=0A= + free(lbp, M_SCSIDA);=0A= + xpt_release_ccb(done_ccb);=0A= + softc->state =3D DA_STATE_PROBE_BDC;=0A= + xpt_schedule(periph, priority);=0A= + return;=0A= + }=0A= + case DA_CCB_PROBE_BLK_LIMITS:=0A= + {=0A= + struct scsi_vpd_block_limits *block_limits;=0A= +=0A= + block_limits =3D (struct scsi_vpd_block_limits *)csio->data_ptr;=0A= +=0A= + if ((csio->ccb_h.status & CAM_STATUS_MASK) =3D=3D CAM_REQ_CMP) {=0A= + uint32_t max_unmap_lba_cnt =3D scsi_4btoul(=0A= + block_limits->max_unmap_lba_cnt);=0A= + uint32_t max_unmap_blk_cnt =3D scsi_4btoul(=0A= + block_limits->max_unmap_blk_cnt);=0A= + uint64_t ws_max_blks =3D scsi_8btou64(=0A= + block_limits->max_write_same_length);=0A= + /*=0A= + * We should already support UNMAP but we check lba=0A= + * and block count to be sure=0A= + */=0A= + if (max_unmap_lba_cnt !=3D 0x00L &&=0A= + max_unmap_blk_cnt !=3D 0x00L) {=0A= + softc->unmap_max_lba =3D max_unmap_lba_cnt;=0A= + softc->unmap_max_ranges =3D min(max_unmap_blk_cnt,=0A= + UNMAP_MAX_RANGES);=0A= + } else {=0A= + /*=0A= + * Unexpected UNMAP limits which means the=0A= + * device doesn't actually support UNMAP=0A= + */=0A= + dadeleteflag(softc, DA_DELETE_UNMAP, 0);=0A= + }=0A= +=0A= + if (ws_max_blks !=3D 0x00L)=0A= + softc->ws_max_blks =3D ws_max_blks;=0A= + } else {=0A= + int error;=0A= + error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= + SF_RETRY_UA|SF_NO_PRINT);=0A= + if (error =3D=3D ERESTART)=0A= + return;=0A= + else if (error !=3D 0) {=0A= + if ((done_ccb->ccb_h.status & CAM_DEV_QFRZN) !=3D 0) {=0A= + /* Don't wedge this device's queue */=0A= + cam_release_devq(done_ccb->ccb_h.path,=0A= + /*relsim_flags*/0,=0A= + /*reduction*/0,=0A= + /*timeout*/0,=0A= + /*getcount_only*/0);=0A= + }=0A= +=0A= + /*=0A= + * Failure here doesn't mean UNMAP is not=0A= + * supported as this is an optional page.=0A= + */=0A= + softc->unmap_max_lba =3D 1;=0A= + softc->unmap_max_ranges =3D 1;=0A= + }=0A= + }=0A= +=0A= + free(block_limits, M_SCSIDA);=0A= + xpt_release_ccb(done_ccb);=0A= + softc->state =3D DA_STATE_PROBE_BDC;=0A= + xpt_schedule(periph, priority);=0A= + return;=0A= + }=0A= + case DA_CCB_PROBE_BDC:=0A= + {=0A= + struct scsi_vpd_block_characteristics *bdc;=0A= +=0A= + bdc =3D (struct scsi_vpd_block_characteristics *)csio->data_ptr;=0A= +=0A= + if ((csio->ccb_h.status & CAM_STATUS_MASK) =3D=3D CAM_REQ_CMP) {=0A= + /*=0A= + * Disable queue sorting for non-rotational media=0A= + * by default.=0A= + */=0A= + if (scsi_2btoul(bdc->medium_rotation_rate) =3D=3D=0A= + SVPD_BDC_RATE_NONE_ROTATING)=0A= + softc->sort_io_queue =3D 0;=0A= + } else {=0A= + int error;=0A= + error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= + SF_RETRY_UA|SF_NO_PRINT);=0A= + if (error =3D=3D ERESTART)=0A= + return;=0A= + else if (error !=3D 0) {=0A= + if ((done_ccb->ccb_h.status & CAM_DEV_QFRZN) !=3D 0) {=0A= + /* Don't wedge this device's queue */=0A= + cam_release_devq(done_ccb->ccb_h.path,=0A= + /*relsim_flags*/0,=0A= + /*reduction*/0,=0A= + /*timeout*/0,=0A= + /*getcount_only*/0);=0A= + }=0A= + }=0A= + }=0A= +=0A= + free(bdc, M_SCSIDA);=0A= + xpt_release_ccb(done_ccb);=0A= + softc->state =3D DA_STATE_PROBE_ATA;=0A= + xpt_schedule(periph, priority);=0A= + return;=0A= + }=0A= + case DA_CCB_PROBE_ATA:=0A= + {=0A= + int i;=0A= + struct ata_params *ata_params;=0A= + int16_t *ptr;=0A= +=0A= + ata_params =3D (struct ata_params *)csio->data_ptr;=0A= + ptr =3D (uint16_t *)ata_params;=0A= +=0A= + if ((csio->ccb_h.status & CAM_STATUS_MASK) =3D=3D CAM_REQ_CMP) {=0A= + for (i =3D 0; i < sizeof(*ata_params) / 2; i++)=0A= + ptr[i] =3D le16toh(ptr[i]);=0A= + if (ata_params->support_dsm & ATA_SUPPORT_DSM_TRIM) {=0A= + dadeleteflag(softc, DA_DELETE_ATA_TRIM, 1);=0A= + if (ata_params->max_dsm_blocks !=3D 0)=0A= + softc->trim_max_ranges =3D min(=0A= + softc->trim_max_ranges,=0A= + ata_params->max_dsm_blocks *=0A= + ATA_DSM_BLK_RANGES);=0A= + }=0A= + /*=0A= + * Disable queue sorting for non-rotational media=0A= + * by default.=0A= + */=0A= + if (ata_params->media_rotation_rate =3D=3D 1)=0A= + softc->sort_io_queue =3D 0;=0A= + } else {=0A= + int error;=0A= + error =3D daerror(done_ccb, CAM_RETRY_SELTO,=0A= + SF_RETRY_UA|SF_NO_PRINT);=0A= + if (error =3D=3D ERESTART)=0A= + return;=0A= + else if (error !=3D 0) {=0A= + if ((done_ccb->ccb_h.status & CAM_DEV_QFRZN) !=3D 0) {=0A= + /* Don't wedge this device's queue */=0A= + cam_release_devq(done_ccb->ccb_h.path,=0A= + /*relsim_flags*/0,=0A= + /*reduction*/0,=0A= + /*timeout*/0,=0A= + /*getcount_only*/0);=0A= + }=0A= + }=0A= + }=0A= +=0A= + free(ata_params, M_SCSIDA);=0A= + daprobedone(periph, done_ccb);=0A= + return;=0A= + }=0A= case DA_CCB_WAITING:=0A= {=0A= /* Caller will release the CCB */=0A= @@ -2549,7 +3192,7 @@=0A= softc =3D (struct da_softc *)periph->softc;=0A= =0A= /* Probe in progress; don't interfere. */=0A= - if ((softc->flags & DA_FLAG_PROBED) =3D=3D 0)=0A= + if (softc->state !=3D DA_STATE_NORMAL)=0A= return;=0A= =0A= status =3D cam_periph_acquire(periph);=0A= @@ -2557,9 +3200,9 @@=0A= ("dareprobe: cam_periph_acquire failed"));=0A= =0A= if (softc->flags & DA_FLAG_CAN_RC16)=0A= - softc->state =3D DA_STATE_PROBE2;=0A= + softc->state =3D DA_STATE_PROBE_RC16;=0A= else=0A= - softc->state =3D DA_STATE_PROBE;=0A= + softc->state =3D DA_STATE_PROBE_RC;=0A= =0A= xpt_schedule(periph, CAM_PRIORITY_DEV);=0A= }=0A= @@ -2781,10 +3424,6 @@=0A= softc->disk->d_fwheads =3D softc->params.heads;=0A= softc->disk->d_devstat->block_size =3D softc->params.secsize;=0A= softc->disk->d_devstat->flags &=3D ~DEVSTAT_BS_UNAVAILABLE;=0A= - if (softc->delete_method > DA_DELETE_DISABLE)=0A= - softc->disk->d_flags |=3D DISKFLAG_CANDELETE;=0A= - else=0A= - softc->disk->d_flags &=3D ~DISKFLAG_CANDELETE;=0A= }=0A= =0A= static void=0A= Index: sys/cam/ata/ata_da.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cam/ata/ata_da.c (revision 250577)=0A= +++ sys/cam/ata/ata_da.c (working copy)=0A= @@ -117,10 +117,10 @@=0A= };=0A= =0A= #define TRIM_MAX_BLOCKS 8=0A= -#define TRIM_MAX_RANGES (TRIM_MAX_BLOCKS * 64)=0A= +#define TRIM_MAX_RANGES (TRIM_MAX_BLOCKS * ATA_DSM_BLK_RANGES)=0A= #define TRIM_MAX_BIOS (TRIM_MAX_RANGES * 4)=0A= struct trim_request {=0A= - uint8_t data[TRIM_MAX_RANGES * 8];=0A= + uint8_t data[TRIM_MAX_RANGES * ATA_DSM_RANGE_SIZE];=0A= struct bio *bps[TRIM_MAX_BIOS];=0A= };=0A= =0A= @@ -130,6 +130,7 @@=0A= ada_state state;=0A= ada_flags flags; =0A= ada_quirks quirks;=0A= + int sort_io_queue;=0A= int ordered_tag_count;=0A= int outstanding_cmds;=0A= int trim_max_ranges;=0A= @@ -449,6 +450,8 @@=0A= softc->read_ahead : ada_read_ahead)=0A= #define ADA_WC (softc->write_cache >=3D 0 ? \=0A= softc->write_cache : ada_write_cache)=0A= +#define ADA_SIO (softc->sort_io_queue >=3D 0 ? \=0A= + softc->sort_io_queue : cam_sort_io_queues)=0A= =0A= /*=0A= * Most platforms map firmware geometry to actual, but some don't. If=0A= @@ -659,10 +662,17 @@=0A= * Place it in the queue of disk activities for this disk=0A= */=0A= if (bp->bio_cmd =3D=3D BIO_DELETE &&=0A= - (softc->flags & ADA_FLAG_CAN_TRIM))=0A= - bioq_disksort(&softc->trim_queue, bp);=0A= - else=0A= - bioq_disksort(&softc->bio_queue, bp);=0A= + (softc->flags & ADA_FLAG_CAN_TRIM)) {=0A= + if (ADA_SIO)=0A= + bioq_disksort(&softc->trim_queue, bp);=0A= + else=0A= + bioq_insert_tail(&softc->trim_queue, bp);=0A= + } else {=0A= + if (ADA_SIO)=0A= + bioq_disksort(&softc->bio_queue, bp);=0A= + else=0A= + bioq_insert_tail(&softc->bio_queue, bp);=0A= + }=0A= =0A= /*=0A= * Schedule ourselves for performing the work.=0A= @@ -1000,6 +1010,10 @@=0A= SYSCTL_ADD_INT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree),=0A= OID_AUTO, "write_cache", CTLFLAG_RW | CTLFLAG_MPSAFE,=0A= &softc->write_cache, 0, "Enable disk write cache.");=0A= + SYSCTL_ADD_INT(&softc->sysctl_ctx, SYSCTL_CHILDREN(softc->sysctl_tree),=0A= + OID_AUTO, "sort_io_queue", CTLFLAG_RW | CTLFLAG_MPSAFE,=0A= + &softc->sort_io_queue, 0,=0A= + "Sort IO queue to try and optimise disk access patterns");=0A= #ifdef ADA_TEST_FAILURE=0A= /*=0A= * Add a 'door bell' sysctl which allows one to set it from userland=0A= @@ -1086,8 +1100,8 @@=0A= softc->trim_max_ranges =3D TRIM_MAX_RANGES;=0A= if (cgd->ident_data.max_dsm_blocks !=3D 0) {=0A= softc->trim_max_ranges =3D=0A= - min(cgd->ident_data.max_dsm_blocks * 64,=0A= - softc->trim_max_ranges);=0A= + min(cgd->ident_data.max_dsm_blocks *=0A= + ATA_DSM_BLK_RANGES, softc->trim_max_ranges);=0A= }=0A= }=0A= if (cgd->ident_data.support.command2 & ATA_SUPPORT_CFA)=0A= @@ -1132,6 +1146,11 @@=0A= snprintf(announce_buf, sizeof(announce_buf),=0A= "kern.cam.ada.%d.write_cache", periph->unit_number);=0A= TUNABLE_INT_FETCH(announce_buf, &softc->write_cache);=0A= + /* Disable queue sorting for non-rotational media by default. */=0A= + if (cgd->ident_data.media_rotation_rate =3D=3D 1)=0A= + softc->sort_io_queue =3D 0;=0A= + else=0A= + softc->sort_io_queue =3D -1;=0A= adagetparams(periph, cgd);=0A= softc->disk =3D disk_alloc();=0A= softc->disk->d_devstat =3D devstat_new_entry(periph->periph_name,=0A= @@ -1162,10 +1181,12 @@=0A= softc->disk->d_flags =3D 0;=0A= if (softc->flags & ADA_FLAG_CAN_FLUSHCACHE)=0A= softc->disk->d_flags |=3D DISKFLAG_CANFLUSHCACHE;=0A= - if ((softc->flags & ADA_FLAG_CAN_TRIM) ||=0A= - ((softc->flags & ADA_FLAG_CAN_CFA) &&=0A= - !(softc->flags & ADA_FLAG_CAN_48BIT)))=0A= + if (softc->flags & ADA_FLAG_CAN_TRIM) {=0A= softc->disk->d_flags |=3D DISKFLAG_CANDELETE;=0A= + } else if ((softc->flags & ADA_FLAG_CAN_CFA) &&=0A= + !(softc->flags & ADA_FLAG_CAN_48BIT)) {=0A= + softc->disk->d_flags |=3D DISKFLAG_CANDELETE;=0A= + }=0A= strlcpy(softc->disk->d_descr, cgd->ident_data.model,=0A= MIN(sizeof(softc->disk->d_descr), sizeof(cgd->ident_data.model)));=0A= strlcpy(softc->disk->d_ident, cgd->ident_data.serial,=0A= @@ -1332,9 +1353,9 @@=0A= =0A= /* Try to extend the previous range. */=0A= if (lba =3D=3D lastlba) {=0A= - c =3D min(count, 0xffff - lastcount);=0A= + c =3D min(count, ATA_DSM_RANGE_MAX - lastcount);=0A= lastcount +=3D c;=0A= - off =3D (ranges - 1) * 8;=0A= + off =3D (ranges - 1) * ATA_DSM_RANGE_SIZE;=0A= req->data[off + 6] =3D lastcount & 0xff;=0A= req->data[off + 7] =3D=0A= (lastcount >> 8) & 0xff;=0A= @@ -1343,8 +1364,8 @@=0A= }=0A= =0A= while (count > 0) {=0A= - c =3D min(count, 0xffff);=0A= - off =3D ranges * 8;=0A= + c =3D min(count, ATA_DSM_RANGE_MAX);=0A= + off =3D ranges * ATA_DSM_RANGE_SIZE;=0A= req->data[off + 0] =3D lba & 0xff;=0A= req->data[off + 1] =3D (lba >> 8) & 0xff;=0A= req->data[off + 2] =3D (lba >> 16) & 0xff;=0A= @@ -1357,6 +1378,11 @@=0A= count -=3D c;=0A= lastcount =3D c;=0A= ranges++;=0A= + /*=0A= + * Its the caller's responsibility to ensure the=0A= + * request will fit so we don't need to check for=0A= + * overrun here=0A= + */=0A= }=0A= lastlba =3D lba;=0A= req->bps[bps++] =3D bp1;=0A= @@ -1364,7 +1390,8 @@=0A= if (bps >=3D TRIM_MAX_BIOS ||=0A= bp1 =3D=3D NULL ||=0A= bp1->bio_bcount / softc->params.secsize >=0A= - (softc->trim_max_ranges - ranges) * 0xffff)=0A= + (softc->trim_max_ranges - ranges) *=0A= + ATA_DSM_RANGE_MAX)=0A= break;=0A= } while (1);=0A= cam_fill_ataio(ataio,=0A= @@ -1373,10 +1400,12 @@=0A= CAM_DIR_OUT,=0A= 0,=0A= req->data,=0A= - ((ranges + 63) / 64) * 512,=0A= + ((ranges + ATA_DSM_BLK_RANGES - 1) /=0A= + ATA_DSM_BLK_RANGES) * ATA_DSM_BLK_SIZE,=0A= ada_default_timeout * 1000);=0A= ata_48bit_cmd(ataio, ATA_DATA_SET_MANAGEMENT,=0A= - ATA_DSM_TRIM, 0, (ranges + 63) / 64);=0A= + ATA_DSM_TRIM, 0, (ranges + ATA_DSM_BLK_RANGES -=0A= + 1) / ATA_DSM_BLK_RANGES);=0A= start_ccb->ccb_h.ccb_state =3D ADA_CCB_TRIM;=0A= goto out;=0A= }=0A= Index: sys/sys=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/sys (revision 250577)=0A= +++ sys/sys (working copy)=0A= =0A= Property changes on: sys/sys=0A= ___________________________________________________________________=0A= Modified: svn:mergeinfo=0A= Merged /head/sys/sys:r249931=0A= Index: sys/sys/ata.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/sys/ata.h (revision 250577)=0A= +++ sys/sys/ata.h (working copy)=0A= @@ -261,6 +261,12 @@=0A= /*255*/ u_int16_t integrity;=0A= } __packed;=0A= =0A= +/* ATA Dataset Management */=0A= +#define ATA_DSM_BLK_SIZE 512=0A= +#define ATA_DSM_BLK_RANGES 64=0A= +#define ATA_DSM_RANGE_SIZE 8=0A= +#define ATA_DSM_RANGE_MAX 65535=0A= +=0A= /*=0A= * ATA Device Register=0A= *=0A= Index: sys/geom/geom_dev.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/geom/geom_dev.c (revision 250577)=0A= +++ sys/geom/geom_dev.c (working copy)=0A= @@ -52,6 +52,7 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= #include =0A= #include =0A= #include =0A= @@ -93,6 +94,19 @@=0A= .attrchanged =3D g_dev_attrchanged=0A= };=0A= =0A= +/*=0A= + * We target 262144 (8 x 32768) sectors by default as this significantly=0A= + * increases the throughput on commonly used SSD's with a marginal=0A= + * increase in non-interruptible request latency.=0A= + */=0A= +static uint64_t g_dev_del_max_sectors =3D 262144;=0A= +SYSCTL_DECL(_kern_geom);=0A= +SYSCTL_NODE(_kern_geom, OID_AUTO, dev, CTLFLAG_RW, 0, "GEOM_DEV stuff");=0A= +SYSCTL_QUAD(_kern_geom_dev, OID_AUTO, delete_max_sectors, CTLFLAG_RW,=0A= + &g_dev_del_max_sectors, 0, "Maximum number of sectors in a single "=0A= + "delete request sent to the provider. Larger requests are chunked "=0A= + "so they can be interrupted. (0 =3D disable chunking)");=0A= +=0A= static void=0A= g_dev_destroy(void *arg, int flags __unused)=0A= {=0A= @@ -412,17 +426,20 @@=0A= }=0A= while (length > 0) { =0A= chunk =3D length;=0A= - if (chunk > 65536 * cp->provider->sectorsize)=0A= - chunk =3D 65536 * cp->provider->sectorsize;=0A= + if (g_dev_del_max_sectors !=3D 0 && chunk >=0A= + g_dev_del_max_sectors * cp->provider->sectorsize) {=0A= + chunk =3D g_dev_del_max_sectors *=0A= + cp->provider->sectorsize;=0A= + }=0A= error =3D g_delete_data(cp, offset, chunk);=0A= length -=3D chunk;=0A= offset +=3D chunk;=0A= if (error)=0A= break;=0A= /*=0A= - * Since the request size is unbounded, the service=0A= - * time is likewise. We make this ioctl interruptible=0A= - * by checking for signals for each bio.=0A= + * Since the request size can be large, the service=0A= + * time can be is likewise. We make this ioctl=0A= + * interruptible by checking for signals for each bio.=0A= */=0A= if (SIGPENDING(td))=0A= break;=0A= ------=_NextPart_000_0730_01CE57A0.ECC7BD20 Content-Type: application/octet-stream; name="mfc-zfs-trim-stable-9.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="mfc-zfs-trim-stable-9.patch" MFC r240868: Add TRIM support=0A= MFC r244155: Renamed zfs trim stats=0A= MFC r244187: Upgrade TRIM free request sizes optimisation =0A= MFC r244188: Added vfs.zfs.vdev.trim_on_init sysctl=0A= MFC r248572: Add TRIM support for L2ARC=0A= MFC r248574: Improve TXG handling in the TRIM module=0A= MFC r248575: TRIM cache devices based on time instead of TXGs=0A= MFC r248576: Names the ZFS TRIM thread=0A= MFC r248577: Optimisation of TRIM processing=0A= MFC r248602: Fix for building libzpool under i386=0A= MFC r249921: Enabled ZFS TRIM by default=0A= Index: cddl/lib=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- cddl/lib (revision 250526)=0A= +++ cddl/lib (working copy)=0A= =0A= Property changes on: cddl/lib=0A= ___________________________________________________________________=0A= Modified: svn:mergeinfo=0A= Merged /head/cddl/lib:r240868=0A= Index: cddl/lib/libzpool/Makefile=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- cddl/lib/libzpool/Makefile (revision 250526)=0A= +++ cddl/lib/libzpool/Makefile (working copy)=0A= @@ -26,7 +26,7 @@=0A= =0A= LIB=3D zpool=0A= =0A= -ZFS_COMMON_SRCS=3D ${ZFS_COMMON_OBJS:C/.o$/.c/} vdev_file.c=0A= +ZFS_COMMON_SRCS=3D ${ZFS_COMMON_OBJS:C/.o$/.c/} vdev_file.c trim_map.c=0A= ZFS_SHARED_SRCS=3D ${ZFS_SHARED_OBJS:C/.o$/.c/}=0A= KERNEL_SRCS=3D kernel.c taskq.c util.c=0A= LIST_SRCS=3D list.c=0A= Index: UPDATING=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- UPDATING (revision 250526)=0A= +++ UPDATING (working copy)=0A= @@ -11,6 +11,22 @@=0A= Items affecting the ports and packages system can be found in=0A= /usr/ports/UPDATING. Please read that file before running portupgrade.=0A= =0A= +20130512:=0A= + Added ZFS TRIM support which is enabled by default. To disable=0A= + ZFS TRIM support set vfs.zfs.trim.enabled=3D0 in loader.conf.=0A= +=0A= + Creating new ZFS pools and adding new devices to existing pools=0A= + first performs a full device level TRIM, which can take a significant=0A= + amount of time. Set the sysctl vfs.zfs.vdev.trim_on_init to 0 to=0A= + disable this behaviour.=0A= +=0A= + ZFS TRIM requires the underlying device support BIO_DELETE which=0A= + is currently provided by methods such as ATA TRIM and SCSI UNMAP=0A= + via CAM, which are typically supported by SSD's.=0A= +=0A= + Stats for ZFS TRIM can be monitored by looking at the sysctl's=0A= + under kstat.zfs.misc.zio_trim.=0A= +=0A= 20130430:=0A= The mergemaster command now uses the default MAKEOBJDIRPREFIX=0A= rather than creating it's own in the temporary directory in=0A= Index: sys=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys (revision 250526)=0A= +++ sys (working copy)=0A= =0A= Property changes on: sys=0A= ___________________________________________________________________=0A= Modified: svn:mergeinfo=0A= Merged = /head/sys:r240868,244155,244187-244188,248572,248574-248577,248602,249921=0A= Index: sys/modules=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/modules (revision 250526)=0A= +++ sys/modules (working copy)=0A= =0A= Property changes on: sys/modules=0A= ___________________________________________________________________=0A= Modified: svn:mergeinfo=0A= Merged /head/sys/modules:r240868=0A= Index: sys/modules/zfs/Makefile=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/modules/zfs/Makefile (revision 250526)=0A= +++ sys/modules/zfs/Makefile (working copy)=0A= @@ -72,6 +72,7 @@=0A= ZFS_SRCS=3D ${ZFS_OBJS:C/.o$/.c/}=0A= SRCS+=3D ${ZFS_SRCS}=0A= SRCS+=3D vdev_geom.c=0A= +SRCS+=3D trim_map.c=0A= =0A= # Use FreeBSD's namecache.=0A= CFLAGS+=3D-DFREEBSD_NAMECACHE=0A= Index: sys/cddl/contrib/opensolaris=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris (working copy)=0A= =0A= Property changes on: sys/cddl/contrib/opensolaris=0A= ___________________________________________________________________=0A= Modified: svn:mergeinfo=0A= Merged = /head/sys/cddl/contrib/opensolaris:r240868,244155,244187-244188,248572,24= 8574-248577,248602,249921=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (working copy)=0A= @@ -293,10 +293,11 @@=0A= c =3D vdev_mirror_child_select(zio);=0A= children =3D (c >=3D 0);=0A= } else {=0A= - ASSERT(zio->io_type =3D=3D ZIO_TYPE_WRITE);=0A= + ASSERT(zio->io_type =3D=3D ZIO_TYPE_WRITE ||=0A= + zio->io_type =3D=3D ZIO_TYPE_FREE);=0A= =0A= /*=0A= - * Writes go to all children.=0A= + * Writes and frees go to all children.=0A= */=0A= c =3D 0;=0A= children =3D mm->mm_children;=0A= @@ -377,6 +378,8 @@=0A= zio->io_error =3D vdev_mirror_worst_error(mm);=0A= }=0A= return;=0A= + } else if (zio->io_type =3D=3D ZIO_TYPE_FREE) {=0A= + return;=0A= }=0A= =0A= ASSERT(zio->io_type =3D=3D ZIO_TYPE_READ);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (working copy)=0A= @@ -83,6 +83,11 @@=0A= TUNABLE_INT("vfs.zfs.cache_flush_disable", &zfs_nocacheflush);=0A= SYSCTL_INT(_vfs_zfs, OID_AUTO, cache_flush_disable, CTLFLAG_RDTUN,=0A= &zfs_nocacheflush, 0, "Disable cache flush");=0A= +boolean_t zfs_trim_enabled =3D B_TRUE;=0A= +SYSCTL_DECL(_vfs_zfs_trim);=0A= +TUNABLE_INT("vfs.zfs.trim.enabled", &zfs_trim_enabled);=0A= +SYSCTL_INT(_vfs_zfs_trim, OID_AUTO, enabled, CTLFLAG_RDTUN, = &zfs_trim_enabled, 0,=0A= + "Enable ZFS TRIM");=0A= =0A= static kmem_cache_t *zil_lwb_cache;=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c (working = copy)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c (working = copy)=0A= @@ -27,7 +27,31 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= =0A= +/*=0A= + * Calculate the zio end, upgrading based on ashift which would be=0A= + * done by zio_vdev_io_start.=0A= + *=0A= + * This makes free range consolidation much more effective=0A= + * than it would otherwise be as well as ensuring that entire=0A= + * blocks are invalidated by writes.=0A= + */=0A= +#define TRIM_ZIO_END(vd, offset, size) (offset + \=0A= + P2ROUNDUP(size, 1ULL << vd->vdev_top->vdev_ashift))=0A= +=0A= +#define TRIM_MAP_SINC(tm, size) \=0A= + atomic_add_64(&(tm)->tm_bytes, (size))=0A= +=0A= +#define TRIM_MAP_SDEC(tm, size) \=0A= + atomic_add_64(&(tm)->tm_bytes, -(size))=0A= +=0A= +#define TRIM_MAP_QINC(tm) \=0A= + atomic_inc_64(&(tm)->tm_pending); \=0A= +=0A= +#define TRIM_MAP_QDEC(tm) \=0A= + atomic_dec_64(&(tm)->tm_pending);=0A= +=0A= typedef struct trim_map {=0A= list_t tm_head; /* List of segments sorted by txg. */=0A= avl_tree_t tm_queued_frees; /* AVL tree of segments waiting for TRIM. = */=0A= @@ -35,6 +59,8 @@=0A= avl_tree_t tm_inflight_writes; /* AVL tree of in-flight writes. */=0A= list_t tm_pending_writes; /* Writes blocked on in-flight frees. */=0A= kmutex_t tm_lock;=0A= + uint64_t tm_pending; /* Count of pending TRIMs. */=0A= + uint64_t tm_bytes; /* Total size in bytes of queued TRIMs. */=0A= } trim_map_t;=0A= =0A= typedef struct trim_seg {=0A= @@ -43,17 +69,47 @@=0A= uint64_t ts_start; /* Starting offset of this segment. */=0A= uint64_t ts_end; /* Ending offset (non-inclusive). */=0A= uint64_t ts_txg; /* Segment creation txg. */=0A= + hrtime_t ts_time; /* Segment creation time. */=0A= } trim_seg_t;=0A= =0A= -extern boolean_t zfs_notrim;=0A= +extern boolean_t zfs_trim_enabled;=0A= =0A= +static u_int trim_txg_delay =3D 32;=0A= +static u_int trim_timeout =3D 30;=0A= +static u_int trim_max_interval =3D 1;=0A= +/* Limit outstanding TRIMs to 2G (max size for a single TRIM request) */=0A= +static uint64_t trim_vdev_max_bytes =3D 2147483648;=0A= +/* Limit outstanding TRIMs to 64 (max ranges for a single TRIM request) = */ =0A= +static u_int trim_vdev_max_pending =3D 64;=0A= +=0A= SYSCTL_DECL(_vfs_zfs);=0A= -/* Delay TRIMs by that many TXGs. */=0A= -static int trim_txg_limit =3D 64;=0A= -TUNABLE_INT("vfs.zfs.trim_txg_limit", &trim_txg_limit);=0A= -SYSCTL_INT(_vfs_zfs, OID_AUTO, trim_txg_limit, CTLFLAG_RW, = &trim_txg_limit, 0,=0A= - "Delay TRIMs by that many TXGs.");=0A= +SYSCTL_NODE(_vfs_zfs, OID_AUTO, trim, CTLFLAG_RD, 0, "ZFS TRIM");=0A= =0A= +TUNABLE_INT("vfs.zfs.trim.txg_delay", &trim_txg_delay);=0A= +SYSCTL_UINT(_vfs_zfs_trim, OID_AUTO, txg_delay, CTLFLAG_RWTUN, = &trim_txg_delay,=0A= + 0, "Delay TRIMs by up to this many TXGs");=0A= +=0A= +TUNABLE_INT("vfs.zfs.trim.timeout", &trim_timeout);=0A= +SYSCTL_UINT(_vfs_zfs_trim, OID_AUTO, timeout, CTLFLAG_RWTUN, = &trim_timeout, 0,=0A= + "Delay TRIMs by up to this many seconds");=0A= +=0A= +TUNABLE_INT("vfs.zfs.trim.max_interval", &trim_max_interval);=0A= +SYSCTL_UINT(_vfs_zfs_trim, OID_AUTO, max_interval, CTLFLAG_RWTUN,=0A= + &trim_max_interval, 0,=0A= + "Maximum interval between TRIM queue processing (seconds)");=0A= +=0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +TUNABLE_QUAD("vfs.zfs.vdev.trim_max_bytes", &trim_vdev_max_bytes);=0A= +SYSCTL_QUAD(_vfs_zfs_vdev, OID_AUTO, trim_max_bytes, CTLFLAG_RWTUN,=0A= + &trim_vdev_max_bytes, 0,=0A= + "Maximum pending TRIM bytes for a vdev");=0A= +=0A= +TUNABLE_INT("vfs.zfs.vdev.trim_max_pending", &trim_vdev_max_pending);=0A= +SYSCTL_UINT(_vfs_zfs_vdev, OID_AUTO, trim_max_pending, CTLFLAG_RWTUN,=0A= + &trim_vdev_max_pending, 0,=0A= + "Maximum pending TRIM segments for a vdev");=0A= +=0A= +=0A= static void trim_map_vdev_commit_done(spa_t *spa, vdev_t *vd);=0A= =0A= static int=0A= @@ -101,7 +157,7 @@=0A= =0A= ASSERT(vd->vdev_ops->vdev_op_leaf);=0A= =0A= - if (zfs_notrim)=0A= + if (!zfs_trim_enabled)=0A= return;=0A= =0A= tm =3D kmem_zalloc(sizeof (*tm), KM_SLEEP);=0A= @@ -127,7 +183,7 @@=0A= =0A= ASSERT(vd->vdev_ops->vdev_op_leaf);=0A= =0A= - if (zfs_notrim)=0A= + if (!zfs_trim_enabled)=0A= return;=0A= =0A= tm =3D vd->vdev_trimmap;=0A= @@ -146,6 +202,8 @@=0A= avl_remove(&tm->tm_queued_frees, ts);=0A= list_remove(&tm->tm_head, ts);=0A= kmem_free(ts, sizeof (*ts));=0A= + TRIM_MAP_SDEC(tm, ts->ts_end - ts->ts_start);=0A= + TRIM_MAP_QDEC(tm);=0A= }=0A= mutex_exit(&tm->tm_lock);=0A= =0A= @@ -165,10 +223,12 @@=0A= avl_index_t where;=0A= trim_seg_t tsearch, *ts_before, *ts_after, *ts;=0A= boolean_t merge_before, merge_after;=0A= + hrtime_t time;=0A= =0A= ASSERT(MUTEX_HELD(&tm->tm_lock));=0A= VERIFY(start < end);=0A= =0A= + time =3D gethrtime();=0A= tsearch.ts_start =3D start;=0A= tsearch.ts_end =3D end;=0A= =0A= @@ -184,25 +244,36 @@=0A= ts_before =3D avl_nearest(&tm->tm_queued_frees, where, AVL_BEFORE);=0A= ts_after =3D avl_nearest(&tm->tm_queued_frees, where, AVL_AFTER);=0A= =0A= - merge_before =3D (ts_before !=3D NULL && ts_before->ts_end =3D=3D = start &&=0A= - ts_before->ts_txg =3D=3D txg);=0A= - merge_after =3D (ts_after !=3D NULL && ts_after->ts_start =3D=3D end &&=0A= - ts_after->ts_txg =3D=3D txg);=0A= + merge_before =3D (ts_before !=3D NULL && ts_before->ts_end =3D=3D = start);=0A= + merge_after =3D (ts_after !=3D NULL && ts_after->ts_start =3D=3D end);=0A= =0A= if (merge_before && merge_after) {=0A= + TRIM_MAP_SINC(tm, ts_after->ts_start - ts_before->ts_end);=0A= + TRIM_MAP_QDEC(tm);=0A= avl_remove(&tm->tm_queued_frees, ts_before);=0A= list_remove(&tm->tm_head, ts_before);=0A= ts_after->ts_start =3D ts_before->ts_start;=0A= + ts_after->ts_txg =3D txg;=0A= + ts_after->ts_time =3D time;=0A= kmem_free(ts_before, sizeof (*ts_before));=0A= } else if (merge_before) {=0A= + TRIM_MAP_SINC(tm, end - ts_before->ts_end);=0A= ts_before->ts_end =3D end;=0A= + ts_before->ts_txg =3D txg;=0A= + ts_before->ts_time =3D time;=0A= } else if (merge_after) {=0A= + TRIM_MAP_SINC(tm, ts_after->ts_start - start);=0A= ts_after->ts_start =3D start;=0A= + ts_after->ts_txg =3D txg;=0A= + ts_after->ts_time =3D time;=0A= } else {=0A= + TRIM_MAP_SINC(tm, end - start);=0A= + TRIM_MAP_QINC(tm);=0A= ts =3D kmem_alloc(sizeof (*ts), KM_SLEEP);=0A= ts->ts_start =3D start;=0A= ts->ts_end =3D end;=0A= ts->ts_txg =3D txg;=0A= + ts->ts_time =3D time;=0A= avl_insert(&tm->tm_queued_frees, ts, where);=0A= list_insert_tail(&tm->tm_head, ts);=0A= }=0A= @@ -220,14 +291,17 @@=0A= left_over =3D (ts->ts_start < start);=0A= right_over =3D (ts->ts_end > end);=0A= =0A= + TRIM_MAP_SDEC(tm, end - start);=0A= if (left_over && right_over) {=0A= nts =3D kmem_alloc(sizeof (*nts), KM_SLEEP);=0A= nts->ts_start =3D end;=0A= nts->ts_end =3D ts->ts_end;=0A= nts->ts_txg =3D ts->ts_txg;=0A= + nts->ts_time =3D ts->ts_time;=0A= ts->ts_end =3D start;=0A= avl_insert_here(&tm->tm_queued_frees, nts, ts, AVL_AFTER);=0A= list_insert_after(&tm->tm_head, ts, nts);=0A= + TRIM_MAP_QINC(tm);=0A= } else if (left_over) {=0A= ts->ts_end =3D start;=0A= } else if (right_over) {=0A= @@ -235,6 +309,7 @@=0A= } else {=0A= avl_remove(&tm->tm_queued_frees, ts);=0A= list_remove(&tm->tm_head, ts);=0A= + TRIM_MAP_QDEC(tm);=0A= kmem_free(ts, sizeof (*ts));=0A= }=0A= }=0A= @@ -261,17 +336,15 @@=0A= }=0A= =0A= void=0A= -trim_map_free(zio_t *zio)=0A= +trim_map_free(vdev_t *vd, uint64_t offset, uint64_t size, uint64_t txg)=0A= {=0A= - vdev_t *vd =3D zio->io_vd;=0A= trim_map_t *tm =3D vd->vdev_trimmap;=0A= =0A= - if (zfs_notrim || vd->vdev_notrim || tm =3D=3D NULL)=0A= + if (!zfs_trim_enabled || vd->vdev_notrim || tm =3D=3D NULL)=0A= return;=0A= =0A= mutex_enter(&tm->tm_lock);=0A= - trim_map_free_locked(tm, zio->io_offset, zio->io_offset + zio->io_size,=0A= - vd->vdev_spa->spa_syncing_txg);=0A= + trim_map_free_locked(tm, offset, TRIM_ZIO_END(vd, offset, size), txg);=0A= mutex_exit(&tm->tm_lock);=0A= }=0A= =0A= @@ -284,11 +357,11 @@=0A= boolean_t left_over, right_over;=0A= uint64_t start, end;=0A= =0A= - if (zfs_notrim || vd->vdev_notrim || tm =3D=3D NULL)=0A= + if (!zfs_trim_enabled || vd->vdev_notrim || tm =3D=3D NULL)=0A= return (B_TRUE);=0A= =0A= start =3D zio->io_offset;=0A= - end =3D start + zio->io_size;=0A= + end =3D TRIM_ZIO_END(zio->io_vd, start, zio->io_size);=0A= tsearch.ts_start =3D start;=0A= tsearch.ts_end =3D end;=0A= =0A= @@ -331,7 +404,7 @@=0A= * Don't check for vdev_notrim, since the write could have=0A= * started before vdev_notrim was set.=0A= */=0A= - if (zfs_notrim || tm =3D=3D NULL)=0A= + if (!zfs_trim_enabled || tm =3D=3D NULL)=0A= return;=0A= =0A= mutex_enter(&tm->tm_lock);=0A= @@ -348,19 +421,25 @@=0A= }=0A= =0A= /*=0A= - * Return the oldest segment (the one with the lowest txg) or false if=0A= - * the list is empty or the first element's txg is greater than txg = given=0A= - * as function argument.=0A= + * Return the oldest segment (the one with the lowest txg / time) or = NULL if:=0A= + * 1. The list is empty=0A= + * 2. The first element's txg is greater than txgsafe=0A= + * 3. The first element's txg is not greater than the txg argument and = the=0A= + * the first element's time is not greater than time argument=0A= */=0A= static trim_seg_t *=0A= -trim_map_first(trim_map_t *tm, uint64_t txg)=0A= +trim_map_first(trim_map_t *tm, uint64_t txg, uint64_t txgsafe, hrtime_t = time)=0A= {=0A= trim_seg_t *ts;=0A= =0A= ASSERT(MUTEX_HELD(&tm->tm_lock));=0A= + VERIFY(txgsafe >=3D txg);=0A= =0A= ts =3D list_head(&tm->tm_head);=0A= - if (ts !=3D NULL && ts->ts_txg <=3D txg)=0A= + if (ts !=3D NULL && ts->ts_txg <=3D txgsafe &&=0A= + (ts->ts_txg <=3D txg || ts->ts_time <=3D time ||=0A= + tm->tm_bytes > trim_vdev_max_bytes ||=0A= + tm->tm_pending > trim_vdev_max_pending))=0A= return (ts);=0A= return (NULL);=0A= }=0A= @@ -370,26 +449,37 @@=0A= {=0A= trim_map_t *tm =3D vd->vdev_trimmap;=0A= trim_seg_t *ts;=0A= - uint64_t start, size, txglimit;=0A= + uint64_t size, txgtarget, txgsafe;=0A= + hrtime_t timelimit;=0A= =0A= ASSERT(vd->vdev_ops->vdev_op_leaf);=0A= =0A= if (tm =3D=3D NULL)=0A= return;=0A= =0A= - txglimit =3D MIN(spa->spa_syncing_txg, spa_freeze_txg(spa)) -=0A= - trim_txg_limit;=0A= + timelimit =3D gethrtime() - trim_timeout * NANOSEC;=0A= + if (vd->vdev_isl2cache) {=0A= + txgsafe =3D UINT64_MAX;=0A= + txgtarget =3D UINT64_MAX;=0A= + } else {=0A= + txgsafe =3D MIN(spa_last_synced_txg(spa), spa_freeze_txg(spa));=0A= + if (txgsafe > trim_txg_delay)=0A= + txgtarget =3D txgsafe - trim_txg_delay;=0A= + else=0A= + txgtarget =3D 0;=0A= + }=0A= =0A= mutex_enter(&tm->tm_lock);=0A= - /*=0A= - * Loop until we send all frees up to the txglimit.=0A= - */=0A= - while ((ts =3D trim_map_first(tm, txglimit)) !=3D NULL) {=0A= + /* Loop until we have sent all outstanding free's */=0A= + while ((ts =3D trim_map_first(tm, txgtarget, txgsafe, timelimit))=0A= + !=3D NULL) {=0A= list_remove(&tm->tm_head, ts);=0A= avl_remove(&tm->tm_queued_frees, ts);=0A= avl_add(&tm->tm_inflight_frees, ts);=0A= - zio_nowait(zio_trim(zio, spa, vd, ts->ts_start,=0A= - ts->ts_end - ts->ts_start));=0A= + size =3D ts->ts_end - ts->ts_start;=0A= + zio_nowait(zio_trim(zio, spa, vd, ts->ts_start, size));=0A= + TRIM_MAP_SDEC(tm, size);=0A= + TRIM_MAP_QDEC(tm);=0A= }=0A= mutex_exit(&tm->tm_lock);=0A= }=0A= @@ -434,7 +524,7 @@=0A= {=0A= int c;=0A= =0A= - if (vd =3D=3D NULL || spa->spa_syncing_txg <=3D trim_txg_limit)=0A= + if (vd =3D=3D NULL)=0A= return;=0A= =0A= if (vd->vdev_ops->vdev_op_leaf) {=0A= @@ -467,6 +557,11 @@=0A= spa_t *spa =3D arg;=0A= zio_t *zio;=0A= =0A= +#ifdef _KERNEL=0A= + (void) snprintf(curthread->td_name, sizeof(curthread->td_name),=0A= + "trim %s", spa_name(spa));=0A= +#endif=0A= +=0A= for (;;) {=0A= mutex_enter(&spa->spa_trim_lock);=0A= if (spa->spa_trim_thread =3D=3D NULL) {=0A= @@ -475,7 +570,9 @@=0A= mutex_exit(&spa->spa_trim_lock);=0A= thread_exit();=0A= }=0A= - cv_wait(&spa->spa_trim_cv, &spa->spa_trim_lock);=0A= +=0A= + (void) cv_timedwait(&spa->spa_trim_cv, &spa->spa_trim_lock,=0A= + hz * trim_max_interval);=0A= mutex_exit(&spa->spa_trim_lock);=0A= =0A= zio =3D zio_root(spa, NULL, NULL, ZIO_FLAG_CANFAIL);=0A= @@ -492,7 +589,7 @@=0A= trim_thread_create(spa_t *spa)=0A= {=0A= =0A= - if (zfs_notrim)=0A= + if (!zfs_trim_enabled)=0A= return;=0A= =0A= mutex_init(&spa->spa_trim_lock, NULL, MUTEX_DEFAULT, NULL);=0A= @@ -507,7 +604,7 @@=0A= trim_thread_destroy(spa_t *spa)=0A= {=0A= =0A= - if (zfs_notrim)=0A= + if (!zfs_trim_enabled)=0A= return;=0A= if (spa->spa_trim_thread =3D=3D NULL)=0A= return;=0A= @@ -530,7 +627,7 @@=0A= trim_thread_wakeup(spa_t *spa)=0A= {=0A= =0A= - if (zfs_notrim)=0A= + if (!zfs_trim_enabled)=0A= return;=0A= if (spa->spa_trim_thread =3D=3D NULL)=0A= return;=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c (working = copy)=0A= @@ -397,7 +397,8 @@=0A= dsl_free_sync(zio_t *pio, dsl_pool_t *dp, uint64_t txg, const blkptr_t = *bpp)=0A= {=0A= ASSERT(dsl_pool_sync_context(dp));=0A= - zio_nowait(zio_free_sync(pio, dp->dp_spa, txg, bpp, pio->io_flags));=0A= + zio_nowait(zio_free_sync(pio, dp->dp_spa, txg, bpp, BP_GET_PSIZE(bpp),=0A= + pio->io_flags));=0A= }=0A= =0A= static uint64_t=0A= @@ -1364,7 +1365,7 @@=0A= }=0A= =0A= zio_nowait(zio_free_sync(scn->scn_zio_root, scn->scn_dp->dp_spa,=0A= - dmu_tx_get_txg(tx), bp, 0));=0A= + dmu_tx_get_txg(tx), bp, BP_GET_PSIZE(bp), 0));=0A= dsl_dir_diduse_space(tx->tx_pool->dp_free_dir, DD_USED_HEAD,=0A= -bp_get_dsize_sync(scn->scn_dp->dp_spa, bp),=0A= -BP_GET_PSIZE(bp), -BP_GET_UCSIZE(bp), tx);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c (working = copy)=0A= @@ -259,7 +259,9 @@=0A= size_t size;=0A= =0A= for (c =3D 0; c < rm->rm_firstdatacol; c++) {=0A= - zio_buf_free(rm->rm_col[c].rc_data, rm->rm_col[c].rc_size);=0A= + if (rm->rm_col[c].rc_data !=3D NULL)=0A= + zio_buf_free(rm->rm_col[c].rc_data,=0A= + rm->rm_col[c].rc_size);=0A= =0A= if (rm->rm_col[c].rc_gdata !=3D NULL)=0A= zio_buf_free(rm->rm_col[c].rc_gdata,=0A= @@ -504,14 +506,20 @@=0A= ASSERT3U(rm->rm_asize - asize, =3D=3D, rm->rm_nskip << unit_shift);=0A= ASSERT3U(rm->rm_nskip, <=3D, nparity);=0A= =0A= - for (c =3D 0; c < rm->rm_firstdatacol; c++)=0A= - rm->rm_col[c].rc_data =3D zio_buf_alloc(rm->rm_col[c].rc_size);=0A= + if (zio->io_type !=3D ZIO_TYPE_FREE) {=0A= + for (c =3D 0; c < rm->rm_firstdatacol; c++) {=0A= + rm->rm_col[c].rc_data =3D=0A= + zio_buf_alloc(rm->rm_col[c].rc_size);=0A= + }=0A= =0A= - rm->rm_col[c].rc_data =3D zio->io_data;=0A= + rm->rm_col[c].rc_data =3D zio->io_data;=0A= =0A= - for (c =3D c + 1; c < acols; c++)=0A= - rm->rm_col[c].rc_data =3D (char *)rm->rm_col[c - 1].rc_data +=0A= - rm->rm_col[c - 1].rc_size;=0A= + for (c =3D c + 1; c < acols; c++) {=0A= + rm->rm_col[c].rc_data =3D=0A= + (char *)rm->rm_col[c - 1].rc_data +=0A= + rm->rm_col[c - 1].rc_size;=0A= + }=0A= + }=0A= =0A= /*=0A= * If all data stored spans all columns, there's a danger that parity=0A= @@ -1536,6 +1544,18 @@=0A= =0A= ASSERT3U(rm->rm_asize, =3D=3D, vdev_psize_to_asize(vd, zio->io_size));=0A= =0A= + if (zio->io_type =3D=3D ZIO_TYPE_FREE) {=0A= + for (c =3D 0; c < rm->rm_cols; c++) {=0A= + rc =3D &rm->rm_col[c];=0A= + cvd =3D vd->vdev_child[rc->rc_devidx];=0A= + zio_nowait(zio_vdev_child_io(zio, NULL, cvd,=0A= + rc->rc_offset, rc->rc_data, rc->rc_size,=0A= + zio->io_type, zio->io_priority, 0,=0A= + vdev_raidz_child_done, rc));=0A= + }=0A= + return (ZIO_PIPELINE_CONTINUE);=0A= + }=0A= +=0A= if (zio->io_type =3D=3D ZIO_TYPE_WRITE) {=0A= vdev_raidz_generate_parity(rm);=0A= =0A= @@ -1918,6 +1938,8 @@=0A= zio->io_error =3D vdev_raidz_worst_error(rm);=0A= =0A= return;=0A= + } else if (zio->io_type =3D=3D ZIO_TYPE_FREE) {=0A= + return;=0A= }=0A= =0A= ASSERT(zio->io_type =3D=3D ZIO_TYPE_READ);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c (working copy)=0A= @@ -43,6 +43,7 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= =0A= SYSCTL_DECL(_vfs_zfs);=0A= SYSCTL_NODE(_vfs_zfs, OID_AUTO, vdev, CTLFLAG_RW, 0, "ZFS VDEV");=0A= @@ -1196,6 +1197,11 @@=0A= if (vd->vdev_ishole || vd->vdev_ops =3D=3D &vdev_missing_ops)=0A= return (0);=0A= =0A= + if (vd->vdev_ops->vdev_op_leaf) {=0A= + vd->vdev_notrim =3D B_FALSE;=0A= + trim_map_create(vd);=0A= + }=0A= +=0A= for (int c =3D 0; c < vd->vdev_children; c++) {=0A= if (vd->vdev_child[c]->vdev_state !=3D VDEV_STATE_HEALTHY) {=0A= vdev_set_state(vd, B_TRUE, VDEV_STATE_DEGRADED,=0A= @@ -1441,6 +1447,9 @@=0A= =0A= vdev_cache_purge(vd);=0A= =0A= + if (vd->vdev_ops->vdev_op_leaf)=0A= + trim_map_destroy(vd);=0A= +=0A= /*=0A= * We record the previous state before we close it, so that if we are=0A= * doing a reopen(), we don't generate FMA ereports if we notice that=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/trim_map.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/trim_map.h = (working copy)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/trim_map.h = (working copy)=0A= @@ -36,7 +36,7 @@=0A= =0A= extern void trim_map_create(vdev_t *vd);=0A= extern void trim_map_destroy(vdev_t *vd);=0A= -extern void trim_map_free(zio_t *zio);=0A= +extern void trim_map_free(vdev_t *vd, uint64_t offset, uint64_t size, = uint64_t txg);=0A= extern boolean_t trim_map_write_start(zio_t *zio);=0A= extern void trim_map_write_done(zio_t *zio);=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (working = copy)=0A= @@ -46,6 +46,7 @@=0A= } vdev_dtl_type_t;=0A= =0A= extern boolean_t zfs_nocacheflush;=0A= +extern boolean_t zfs_trim_enabled;=0A= =0A= extern int vdev_open(vdev_t *);=0A= extern void vdev_open_children(vdev_t *);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h = (working copy)=0A= @@ -221,6 +221,9 @@=0A= spa_proc_state_t spa_proc_state; /* see definition */=0A= struct proc *spa_proc; /* "zpool-poolname" process */=0A= uint64_t spa_did; /* if procp !=3D p0, did of t1 */=0A= + kthread_t *spa_trim_thread; /* thread sending TRIM I/Os */=0A= + kmutex_t spa_trim_lock; /* protects spa_trim_cv */=0A= + kcondvar_t spa_trim_cv; /* used to notify TRIM thread */=0A= boolean_t spa_autoreplace; /* autoreplace set in open */=0A= int spa_vdev_locks; /* locks grabbed */=0A= uint64_t spa_creation_version; /* version at pool creation */=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_impl.h = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_impl.h = (working copy)=0A= @@ -130,9 +130,9 @@=0A= =0A= ZIO_STAGE_READY =3D 1 << 16, /* RWFCI */=0A= =0A= - ZIO_STAGE_VDEV_IO_START =3D 1 << 17, /* RW--I */=0A= - ZIO_STAGE_VDEV_IO_DONE =3D 1 << 18, /* RW--- */=0A= - ZIO_STAGE_VDEV_IO_ASSESS =3D 1 << 19, /* RW--I */=0A= + ZIO_STAGE_VDEV_IO_START =3D 1 << 17, /* RWF-I */=0A= + ZIO_STAGE_VDEV_IO_DONE =3D 1 << 18, /* RWF-- */=0A= + ZIO_STAGE_VDEV_IO_ASSESS =3D 1 << 19, /* RWF-I */=0A= =0A= ZIO_STAGE_CHECKSUM_VERIFY =3D 1 << 20, /* R---- */=0A= =0A= @@ -214,7 +214,9 @@=0A= (ZIO_INTERLOCK_STAGES | \=0A= ZIO_STAGE_FREE_BP_INIT | \=0A= ZIO_STAGE_ISSUE_ASYNC | \=0A= - ZIO_STAGE_DVA_FREE)=0A= + ZIO_STAGE_DVA_FREE | \=0A= + ZIO_STAGE_VDEV_IO_START | \=0A= + ZIO_STAGE_VDEV_IO_ASSESS)=0A= =0A= #define ZIO_DDT_FREE_PIPELINE \=0A= (ZIO_INTERLOCK_STAGES | \=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (working copy)=0A= @@ -183,6 +183,7 @@=0A= uint64_t vdev_unspare; /* unspare when resilvering done */=0A= hrtime_t vdev_last_try; /* last reopen time */=0A= boolean_t vdev_nowritecache; /* true if flushwritecache failed */=0A= + boolean_t vdev_notrim; /* true if trim failed */=0A= boolean_t vdev_checkremove; /* temporary online test */=0A= boolean_t vdev_forcefault; /* force online fault */=0A= boolean_t vdev_splitting; /* split or repair in progress */=0A= @@ -198,6 +199,7 @@=0A= spa_aux_vdev_t *vdev_aux; /* for l2cache vdevs */=0A= zio_t *vdev_probe_zio; /* root of current probe */=0A= vdev_aux_t vdev_label_aux; /* on-disk aux state */=0A= + struct trim_map *vdev_trimmap;=0A= =0A= /*=0A= * For DTrace to work in userland (libzpool) context, these fields must=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h (working = copy)=0A= @@ -32,6 +32,7 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= #include =0A= #include =0A= =0A= @@ -137,7 +138,8 @@=0A= #define ZIO_PRIORITY_RESILVER (zio_priority_table[9])=0A= #define ZIO_PRIORITY_SCRUB (zio_priority_table[10])=0A= #define ZIO_PRIORITY_DDT_PREFETCH (zio_priority_table[11])=0A= -#define ZIO_PRIORITY_TABLE_SIZE 12=0A= +#define ZIO_PRIORITY_TRIM (zio_priority_table[12])=0A= +#define ZIO_PRIORITY_TABLE_SIZE 13=0A= =0A= #define ZIO_PIPELINE_CONTINUE 0x100=0A= #define ZIO_PIPELINE_STOP 0x101=0A= @@ -367,6 +369,39 @@=0A= list_node_t zl_child_node;=0A= } zio_link_t;=0A= =0A= +/*=0A= + * Used for TRIM kstat.=0A= + */=0A= +typedef struct zio_trim_stats {=0A= + /*=0A= + * Number of bytes successfully TRIMmed.=0A= + */=0A= + kstat_named_t bytes;=0A= +=0A= + /*=0A= + * Number of successful TRIM requests.=0A= + */=0A= + kstat_named_t success;=0A= +=0A= + /*=0A= + * Number of TRIM requests that failed because TRIM is not=0A= + * supported.=0A= + */=0A= + kstat_named_t unsupported;=0A= +=0A= + /*=0A= + * Number of TRIM requests that failed for other reasons.=0A= + */=0A= + kstat_named_t failed;=0A= +} zio_trim_stats_t;=0A= +=0A= +extern zio_trim_stats_t zio_trim_stats;=0A= +=0A= +#define ZIO_TRIM_STAT_INCR(stat, val) \=0A= + atomic_add_64(&zio_trim_stats.stat.value.ui64, (val));=0A= +#define ZIO_TRIM_STAT_BUMP(stat) \=0A= + ZIO_TRIM_STAT_INCR(stat, 1);=0A= +=0A= struct zio {=0A= /* Core information about this I/O */=0A= zbookmark_t io_bookmark;=0A= @@ -441,6 +476,8 @@=0A= /* FreeBSD only. */=0A= struct ostask io_task;=0A= #endif=0A= + avl_node_t io_trim_node;=0A= + list_node_t io_trim_link;=0A= };=0A= =0A= extern zio_t *zio_null(zio_t *pio, spa_t *spa, vdev_t *vd,=0A= @@ -472,8 +509,8 @@=0A= zio_done_func_t *done, void *priv, enum zio_flag flags);=0A= =0A= extern zio_t *zio_ioctl(zio_t *pio, spa_t *spa, vdev_t *vd, int cmd,=0A= - zio_done_func_t *done, void *priv, int priority,=0A= - enum zio_flag flags);=0A= + uint64_t offset, uint64_t size, zio_done_func_t *done, void *priv,=0A= + int priority, enum zio_flag flags);=0A= =0A= extern zio_t *zio_read_phys(zio_t *pio, vdev_t *vd, uint64_t offset,=0A= uint64_t size, void *data, int checksum,=0A= @@ -486,12 +523,14 @@=0A= boolean_t labels);=0A= =0A= extern zio_t *zio_free_sync(zio_t *pio, spa_t *spa, uint64_t txg,=0A= - const blkptr_t *bp, enum zio_flag flags);=0A= + const blkptr_t *bp, uint64_t size, enum zio_flag flags);=0A= =0A= extern int zio_alloc_zil(spa_t *spa, uint64_t txg, blkptr_t *new_bp,=0A= blkptr_t *old_bp, uint64_t size, boolean_t use_slog);=0A= extern void zio_free_zil(spa_t *spa, uint64_t txg, blkptr_t *bp);=0A= extern void zio_flush(zio_t *zio, vdev_t *vd);=0A= +extern zio_t *zio_trim(zio_t *zio, spa_t *spa, vdev_t *vd, uint64_t = offset,=0A= + uint64_t size);=0A= extern void zio_shrink(zio_t *zio, uint64_t size);=0A= =0A= extern int zio_wait(zio_t *zio);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working = copy)=0A= @@ -49,14 +49,17 @@=0A= =0A= DECLARE_GEOM_CLASS(zfs_vdev_class, zfs_vdev);=0A= =0A= -/*=0A= - * Don't send BIO_FLUSH.=0A= - */=0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +/* Don't send BIO_FLUSH. */=0A= static int vdev_geom_bio_flush_disable =3D 0;=0A= TUNABLE_INT("vfs.zfs.vdev.bio_flush_disable", = &vdev_geom_bio_flush_disable);=0A= -SYSCTL_DECL(_vfs_zfs_vdev);=0A= SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, bio_flush_disable, CTLFLAG_RW,=0A= &vdev_geom_bio_flush_disable, 0, "Disable BIO_FLUSH");=0A= +/* Don't send BIO_DELETE. */=0A= +static int vdev_geom_bio_delete_disable =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.bio_delete_disable", = &vdev_geom_bio_delete_disable);=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, bio_delete_disable, CTLFLAG_RW,=0A= + &vdev_geom_bio_delete_disable, 0, "Disable BIO_DELETE");=0A= =0A= static void=0A= vdev_geom_orphan(struct g_consumer *cp)=0A= @@ -663,8 +666,8 @@=0A= *ashift =3D highbit(MAX(pp->sectorsize, SPA_MINBLOCKSIZE)) - 1;=0A= =0A= /*=0A= - * Clear the nowritecache bit, so that on a vdev_reopen() we will=0A= - * try again.=0A= + * Clear the nowritecache settings, so that on a vdev_reopen()=0A= + * we will try again.=0A= */=0A= vd->vdev_nowritecache =3D B_FALSE;=0A= =0A= @@ -710,6 +713,15 @@=0A= */=0A= vd->vdev_nowritecache =3D B_TRUE;=0A= }=0A= + if (bp->bio_cmd =3D=3D BIO_DELETE && bp->bio_error =3D=3D ENOTSUP) {=0A= + /*=0A= + * If we get ENOTSUP, we know that no future=0A= + * attempts will ever succeed. In this case we=0A= + * set a persistent bit so that we don't bother=0A= + * with the ioctl in the future.=0A= + */=0A= + vd->vdev_notrim =3D B_TRUE;=0A= + }=0A= if (zio->io_error =3D=3D EIO && !vd->vdev_remove_wanted) {=0A= /*=0A= * If provider's error is set we assume it is being=0A= @@ -752,18 +764,22 @@=0A= }=0A= =0A= switch (zio->io_cmd) {=0A= -=0A= case DKIOCFLUSHWRITECACHE:=0A= -=0A= if (zfs_nocacheflush || vdev_geom_bio_flush_disable)=0A= break;=0A= -=0A= if (vd->vdev_nowritecache) {=0A= zio->io_error =3D ENOTSUP;=0A= break;=0A= }=0A= -=0A= goto sendreq;=0A= + case DKIOCTRIM:=0A= + if (vdev_geom_bio_delete_disable)=0A= + break;=0A= + if (vd->vdev_notrim) {=0A= + zio->io_error =3D ENOTSUP;=0A= + break;=0A= + }=0A= + goto sendreq;=0A= default:=0A= zio->io_error =3D ENOTSUP;=0A= }=0A= @@ -787,11 +803,21 @@=0A= bp->bio_length =3D zio->io_size;=0A= break;=0A= case ZIO_TYPE_IOCTL:=0A= - bp->bio_cmd =3D BIO_FLUSH;=0A= - bp->bio_flags |=3D BIO_ORDERED;=0A= - bp->bio_data =3D NULL;=0A= - bp->bio_offset =3D cp->provider->mediasize;=0A= - bp->bio_length =3D 0;=0A= + switch (zio->io_cmd) {=0A= + case DKIOCFLUSHWRITECACHE:=0A= + bp->bio_cmd =3D BIO_FLUSH;=0A= + bp->bio_flags |=3D BIO_ORDERED;=0A= + bp->bio_data =3D NULL;=0A= + bp->bio_offset =3D cp->provider->mediasize;=0A= + bp->bio_length =3D 0;=0A= + break;=0A= + case DKIOCTRIM:=0A= + bp->bio_cmd =3D BIO_DELETE;=0A= + bp->bio_data =3D NULL;=0A= + bp->bio_offset =3D zio->io_offset;=0A= + bp->bio_length =3D zio->io_size;=0A= + break;=0A= + }=0A= break;=0A= }=0A= bp->bio_done =3D vdev_geom_io_intr;=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c (working copy)=0A= @@ -67,6 +67,7 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= =0A= #ifdef _KERNEL=0A= #include =0A= @@ -1001,6 +1002,11 @@=0A= spa_create_zio_taskqs(spa);=0A= }=0A= =0A= + /*=0A= + * Start TRIM thread.=0A= + */=0A= + trim_thread_create(spa);=0A= +=0A= list_create(&spa->spa_config_dirty_list, sizeof (vdev_t),=0A= offsetof(vdev_t, vdev_config_dirty_node));=0A= list_create(&spa->spa_state_dirty_list, sizeof (vdev_t),=0A= @@ -1029,6 +1035,12 @@=0A= ASSERT(spa->spa_async_zio_root =3D=3D NULL);=0A= ASSERT(spa->spa_state !=3D POOL_STATE_UNINITIALIZED);=0A= =0A= + /*=0A= + * Stop TRIM thread in case spa_unload() wasn't called directly=0A= + * before spa_deactivate().=0A= + */=0A= + trim_thread_destroy(spa);=0A= +=0A= txg_list_destroy(&spa->spa_vdev_txg_list);=0A= =0A= list_destroy(&spa->spa_config_dirty_list);=0A= @@ -1145,6 +1157,11 @@=0A= ASSERT(MUTEX_HELD(&spa_namespace_lock));=0A= =0A= /*=0A= + * Stop TRIM thread.=0A= + */=0A= + trim_thread_destroy(spa);=0A= +=0A= + /*=0A= * Stop async tasks.=0A= */=0A= spa_async_suspend(spa);=0A= @@ -5875,7 +5892,7 @@=0A= zio_t *zio =3D arg;=0A= =0A= zio_nowait(zio_free_sync(zio, zio->io_spa, dmu_tx_get_txg(tx), bp,=0A= - zio->io_flags));=0A= + BP_GET_PSIZE(bp), zio->io_flags));=0A= return (0);=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy)=0A= @@ -130,6 +130,7 @@=0A= #endif=0A= #include =0A= #include =0A= +#include =0A= #include =0A= #include =0A= =0A= @@ -1691,6 +1692,8 @@=0A= }=0A= =0A= if (l2hdr !=3D NULL) {=0A= + trim_map_free(l2hdr->b_dev->l2ad_vdev, l2hdr->b_daddr,=0A= + hdr->b_size, 0);=0A= list_remove(l2hdr->b_dev->l2ad_buflist, hdr);=0A= ARCSTAT_INCR(arcstat_l2_size, -hdr->b_size);=0A= kmem_free(l2hdr, sizeof (l2arc_buf_hdr_t));=0A= @@ -3528,6 +3531,8 @@=0A= buf->b_private =3D NULL;=0A= =0A= if (l2hdr) {=0A= + trim_map_free(l2hdr->b_dev->l2ad_vdev, l2hdr->b_daddr,=0A= + hdr->b_size, 0);=0A= list_remove(l2hdr->b_dev->l2ad_buflist, hdr);=0A= kmem_free(l2hdr, sizeof (l2arc_buf_hdr_t));=0A= ARCSTAT_INCR(arcstat_l2_size, -buf_size);=0A= @@ -4442,6 +4447,8 @@=0A= list_remove(buflist, ab);=0A= abl2 =3D ab->b_l2hdr;=0A= ab->b_l2hdr =3D NULL;=0A= + trim_map_free(abl2->b_dev->l2ad_vdev, abl2->b_daddr,=0A= + ab->b_size, 0);=0A= kmem_free(abl2, sizeof (l2arc_buf_hdr_t));=0A= ARCSTAT_INCR(arcstat_l2_size, -ab->b_size);=0A= }=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (working copy)=0A= @@ -35,6 +35,7 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= =0A= SYSCTL_DECL(_vfs_zfs);=0A= SYSCTL_NODE(_vfs_zfs, OID_AUTO, zio, CTLFLAG_RW, 0, "ZFS ZIO");=0A= @@ -43,6 +44,19 @@=0A= SYSCTL_INT(_vfs_zfs_zio, OID_AUTO, use_uma, CTLFLAG_RDTUN, = &zio_use_uma, 0,=0A= "Use uma(9) for ZIO allocations");=0A= =0A= +zio_trim_stats_t zio_trim_stats =3D {=0A= + { "bytes", KSTAT_DATA_UINT64,=0A= + "Number of bytes successfully TRIMmed" },=0A= + { "success", KSTAT_DATA_UINT64,=0A= + "Number of successful TRIM requests" },=0A= + { "unsupported", KSTAT_DATA_UINT64,=0A= + "Number of TRIM requests that failed because TRIM is not supported" = },=0A= + { "failed", KSTAT_DATA_UINT64,=0A= + "Number of TRIM requests that failed for reasons other than not = supported" },=0A= +};=0A= +=0A= +static kstat_t *zio_trim_ksp;=0A= +=0A= /*=0A= * = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= * I/O priority table=0A= @@ -61,6 +75,7 @@=0A= 10, /* ZIO_PRIORITY_RESILVER */=0A= 20, /* ZIO_PRIORITY_SCRUB */=0A= 2, /* ZIO_PRIORITY_DDT_PREFETCH */=0A= + 30, /* ZIO_PRIORITY_TRIM */=0A= };=0A= =0A= /*=0A= @@ -209,6 +224,16 @@=0A= zfs_mg_alloc_failures =3D 8;=0A= =0A= zio_inject_init();=0A= +=0A= + zio_trim_ksp =3D kstat_create("zfs", 0, "zio_trim", "misc",=0A= + KSTAT_TYPE_NAMED,=0A= + sizeof(zio_trim_stats) / sizeof(kstat_named_t),=0A= + KSTAT_FLAG_VIRTUAL);=0A= +=0A= + if (zio_trim_ksp !=3D NULL) {=0A= + zio_trim_ksp->ks_data =3D &zio_trim_stats;=0A= + kstat_install(zio_trim_ksp);=0A= + }=0A= }=0A= =0A= void=0A= @@ -236,6 +261,11 @@=0A= kmem_cache_destroy(zio_cache);=0A= =0A= zio_inject_fini();=0A= +=0A= + if (zio_trim_ksp !=3D NULL) {=0A= + kstat_delete(zio_trim_ksp);=0A= + zio_trim_ksp =3D NULL;=0A= + }=0A= }=0A= =0A= /*=0A= @@ -543,7 +573,7 @@=0A= {=0A= zio_t *zio;=0A= =0A= - ASSERT3U(size, <=3D, SPA_MAXBLOCKSIZE);=0A= + ASSERT3U(type =3D=3D ZIO_TYPE_FREE || size, <=3D, SPA_MAXBLOCKSIZE);=0A= ASSERT(P2PHASE(size, SPA_MINBLOCKSIZE) =3D=3D 0);=0A= ASSERT(P2PHASE(offset, SPA_MINBLOCKSIZE) =3D=3D 0);=0A= =0A= @@ -730,7 +760,7 @@=0A= =0A= zio_t *=0A= zio_free_sync(zio_t *pio, spa_t *spa, uint64_t txg, const blkptr_t *bp,=0A= - enum zio_flag flags)=0A= + uint64_t size, enum zio_flag flags)=0A= {=0A= zio_t *zio;=0A= =0A= @@ -743,7 +773,7 @@=0A= =0A= metaslab_check_free(spa, bp);=0A= =0A= - zio =3D zio_create(pio, spa, txg, bp, NULL, BP_GET_PSIZE(bp),=0A= + zio =3D zio_create(pio, spa, txg, bp, NULL, size,=0A= NULL, NULL, ZIO_TYPE_FREE, ZIO_PRIORITY_FREE, flags,=0A= NULL, 0, NULL, ZIO_STAGE_OPEN, ZIO_FREE_PIPELINE);=0A= =0A= @@ -780,15 +810,16 @@=0A= }=0A= =0A= zio_t *=0A= -zio_ioctl(zio_t *pio, spa_t *spa, vdev_t *vd, int cmd,=0A= - zio_done_func_t *done, void *private, int priority, enum zio_flag = flags)=0A= +zio_ioctl(zio_t *pio, spa_t *spa, vdev_t *vd, int cmd, uint64_t offset,=0A= + uint64_t size, zio_done_func_t *done, void *private, int priority,=0A= + enum zio_flag flags)=0A= {=0A= zio_t *zio;=0A= int c;=0A= =0A= if (vd->vdev_children =3D=3D 0) {=0A= - zio =3D zio_create(pio, spa, 0, NULL, NULL, 0, done, private,=0A= - ZIO_TYPE_IOCTL, priority, flags, vd, 0, NULL,=0A= + zio =3D zio_create(pio, spa, 0, NULL, NULL, size, done, private,=0A= + ZIO_TYPE_IOCTL, priority, flags, vd, offset, NULL,=0A= ZIO_STAGE_OPEN, ZIO_IOCTL_PIPELINE);=0A= =0A= zio->io_cmd =3D cmd;=0A= @@ -797,7 +828,7 @@=0A= =0A= for (c =3D 0; c < vd->vdev_children; c++)=0A= zio_nowait(zio_ioctl(zio, spa, vd->vdev_child[c], cmd,=0A= - done, private, priority, flags));=0A= + offset, size, done, private, priority, flags));=0A= }=0A= =0A= return (zio);=0A= @@ -922,11 +953,22 @@=0A= void=0A= zio_flush(zio_t *zio, vdev_t *vd)=0A= {=0A= - zio_nowait(zio_ioctl(zio, zio->io_spa, vd, DKIOCFLUSHWRITECACHE,=0A= + zio_nowait(zio_ioctl(zio, zio->io_spa, vd, DKIOCFLUSHWRITECACHE, 0, 0,=0A= NULL, NULL, ZIO_PRIORITY_NOW,=0A= ZIO_FLAG_CANFAIL | ZIO_FLAG_DONT_PROPAGATE | ZIO_FLAG_DONT_RETRY));=0A= }=0A= =0A= +zio_t *=0A= +zio_trim(zio_t *zio, spa_t *spa, vdev_t *vd, uint64_t offset, uint64_t = size)=0A= +{=0A= +=0A= + ASSERT(vd->vdev_ops->vdev_op_leaf);=0A= +=0A= + return zio_ioctl(zio, spa, vd, DKIOCTRIM, offset, size,=0A= + NULL, NULL, ZIO_PRIORITY_TRIM,=0A= + ZIO_FLAG_CANFAIL | ZIO_FLAG_DONT_PROPAGATE | ZIO_FLAG_DONT_RETRY);=0A= +}=0A= +=0A= void=0A= zio_shrink(zio_t *zio, uint64_t size)=0A= {=0A= @@ -1549,6 +1591,7 @@=0A= zio_free_gang(zio_t *pio, blkptr_t *bp, zio_gang_node_t *gn, void *data)=0A= {=0A= return (zio_free_sync(pio, pio->io_spa, pio->io_txg, bp,=0A= + BP_IS_GANG(bp) ? SPA_GANGBLOCKSIZE : BP_GET_PSIZE(bp),=0A= ZIO_GANG_CHILD_FLAGS(pio)));=0A= }=0A= =0A= @@ -1681,7 +1724,7 @@=0A= }=0A= }=0A= =0A= - if (gn =3D=3D gio->io_gang_tree)=0A= + if (gn =3D=3D gio->io_gang_tree && gio->io_data !=3D NULL)=0A= ASSERT3P((char *)gio->io_data + gio->io_size, =3D=3D, data);=0A= =0A= if (zio !=3D pio)=0A= @@ -2403,7 +2446,7 @@=0A= =0A= /*=0A= * = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= - * Read and write to physical devices=0A= + * Read, write and delete to physical devices=0A= * = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= */=0A= static int=0A= @@ -2426,6 +2469,11 @@=0A= return (vdev_mirror_ops.vdev_op_io_start(zio));=0A= }=0A= =0A= + if (vd->vdev_ops->vdev_op_leaf && zio->io_type =3D=3D ZIO_TYPE_FREE) {=0A= + trim_map_free(vd, zio->io_offset, zio->io_size, zio->io_txg);=0A= + return (ZIO_PIPELINE_CONTINUE);=0A= + }=0A= +=0A= /*=0A= * We keep track of time-sensitive I/Os so that the scan thread=0A= * can quickly react to certain workloads. In particular, we care=0A= @@ -2450,18 +2498,22 @@=0A= =0A= if (P2PHASE(zio->io_size, align) !=3D 0) {=0A= uint64_t asize =3D P2ROUNDUP(zio->io_size, align);=0A= - char *abuf =3D zio_buf_alloc(asize);=0A= + char *abuf =3D NULL;=0A= + if (zio->io_type =3D=3D ZIO_TYPE_READ ||=0A= + zio->io_type =3D=3D ZIO_TYPE_WRITE)=0A= + abuf =3D zio_buf_alloc(asize);=0A= ASSERT(vd =3D=3D vd->vdev_top);=0A= if (zio->io_type =3D=3D ZIO_TYPE_WRITE) {=0A= bcopy(zio->io_data, abuf, zio->io_size);=0A= bzero(abuf + zio->io_size, asize - zio->io_size);=0A= }=0A= - zio_push_transform(zio, abuf, asize, asize, zio_subblock);=0A= + zio_push_transform(zio, abuf, asize, abuf ? asize : 0,=0A= + zio_subblock);=0A= }=0A= =0A= ASSERT(P2PHASE(zio->io_offset, align) =3D=3D 0);=0A= ASSERT(P2PHASE(zio->io_size, align) =3D=3D 0);=0A= - VERIFY(zio->io_type !=3D ZIO_TYPE_WRITE || spa_writeable(spa));=0A= + VERIFY(zio->io_type =3D=3D ZIO_TYPE_READ || spa_writeable(spa));=0A= =0A= /*=0A= * If this is a repair I/O, and there's no self-healing involved --=0A= @@ -2501,6 +2553,11 @@=0A= }=0A= }=0A= =0A= + if (vd->vdev_ops->vdev_op_leaf && zio->io_type =3D=3D ZIO_TYPE_WRITE) {=0A= + if (!trim_map_write_start(zio))=0A= + return (ZIO_PIPELINE_STOP);=0A= + }=0A= +=0A= return (vd->vdev_ops->vdev_op_io_start(zio));=0A= }=0A= =0A= @@ -2514,10 +2571,17 @@=0A= if (zio_wait_for_children(zio, ZIO_CHILD_VDEV, ZIO_WAIT_DONE))=0A= return (ZIO_PIPELINE_STOP);=0A= =0A= - ASSERT(zio->io_type =3D=3D ZIO_TYPE_READ || zio->io_type =3D=3D = ZIO_TYPE_WRITE);=0A= + ASSERT(zio->io_type =3D=3D ZIO_TYPE_READ ||=0A= + zio->io_type =3D=3D ZIO_TYPE_WRITE || zio->io_type =3D=3D = ZIO_TYPE_FREE);=0A= =0A= - if (vd !=3D NULL && vd->vdev_ops->vdev_op_leaf) {=0A= + if (vd !=3D NULL && vd->vdev_ops->vdev_op_leaf &&=0A= + zio->io_type =3D=3D ZIO_TYPE_WRITE) {=0A= + trim_map_write_done(zio);=0A= + }=0A= =0A= + if (vd !=3D NULL && vd->vdev_ops->vdev_op_leaf &&=0A= + (zio->io_type =3D=3D ZIO_TYPE_READ || zio->io_type =3D=3D = ZIO_TYPE_WRITE)) {=0A= +=0A= vdev_queue_io_done(zio);=0A= =0A= if (zio->io_type =3D=3D ZIO_TYPE_WRITE)=0A= @@ -2592,6 +2656,20 @@=0A= if (zio_injection_enabled && zio->io_error =3D=3D 0)=0A= zio->io_error =3D zio_handle_fault_injection(zio, EIO);=0A= =0A= + if (zio->io_type =3D=3D ZIO_TYPE_IOCTL && zio->io_cmd =3D=3D DKIOCTRIM)=0A= + switch (zio->io_error) {=0A= + case 0:=0A= + ZIO_TRIM_STAT_INCR(bytes, zio->io_size);=0A= + ZIO_TRIM_STAT_BUMP(success);=0A= + break;=0A= + case EOPNOTSUPP:=0A= + ZIO_TRIM_STAT_BUMP(unsupported);=0A= + break;=0A= + default:=0A= + ZIO_TRIM_STAT_BUMP(failed);=0A= + break;=0A= + }=0A= +=0A= /*=0A= * If the I/O failed, determine whether we should attempt to retry it.=0A= *=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c (working = copy)=0A= @@ -145,8 +145,14 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= #include =0A= =0A= +static boolean_t vdev_trim_on_init =3D B_TRUE;=0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, trim_on_init, CTLFLAG_RW,=0A= + &vdev_trim_on_init, 0, "Enable/disable full vdev trim on = initialisation");=0A= +=0A= /*=0A= * Basic routines to read and write from a vdev label.=0A= * Used throughout the rest of this file.=0A= @@ -718,6 +724,16 @@=0A= }=0A= =0A= /*=0A= + * TRIM the whole thing so that we start with a clean slate.=0A= + * It's just an optimization, so we don't care if it fails.=0A= + * Don't TRIM if removing so that we don't interfere with zpool=0A= + * disaster recovery.=0A= + */=0A= + if (zfs_trim_enabled && vdev_trim_on_init && (reason =3D=3D = VDEV_LABEL_CREATE ||=0A= + reason =3D=3D VDEV_LABEL_SPARE || reason =3D=3D = VDEV_LABEL_L2CACHE))=0A= + zio_wait(zio_trim(NULL, spa, vd, 0, vd->vdev_psize));=0A= +=0A= + /*=0A= * Initialize its label.=0A= */=0A= vp =3D zio_buf_alloc(sizeof (vdev_phys_t));=0A= @@ -1282,5 +1298,10 @@=0A= * to disk to ensure that all odd-label updates are committed to=0A= * stable storage before the next transaction group begins.=0A= */=0A= - return (vdev_label_sync_list(spa, 1, txg, flags));=0A= + if ((error =3D vdev_label_sync_list(spa, 1, txg, flags)) !=3D 0)=0A= + return (error);=0A= +=0A= + trim_thread_wakeup(spa);=0A= +=0A= + return (0);=0A= }=0A= Index: sys/cddl/compat/opensolaris/kern/opensolaris_kstat.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/compat/opensolaris/kern/opensolaris_kstat.c (revision = 250526)=0A= +++ sys/cddl/compat/opensolaris/kern/opensolaris_kstat.c (working copy)=0A= @@ -118,7 +118,7 @@=0A= SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx,=0A= SYSCTL_CHILDREN(ksp->ks_sysctl_root), OID_AUTO, ksent->name,=0A= CTLTYPE_U64 | CTLFLAG_RD, ksent, sizeof(*ksent),=0A= - kstat_sysctl, "QU", "");=0A= + kstat_sysctl, "QU", ksent->desc);=0A= }=0A= }=0A= =0A= Index: sys/cddl/compat/opensolaris/sys/dkio.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/compat/opensolaris/sys/dkio.h (revision 250526)=0A= +++ sys/cddl/compat/opensolaris/sys/dkio.h (working copy)=0A= @@ -75,6 +75,8 @@=0A= */=0A= #define DKIOCFLUSHWRITECACHE (DKIOC|34) /* flush cache to phys medium */=0A= =0A= +#define DKIOCTRIM (DKIOC|35) /* TRIM a block */=0A= +=0A= struct dk_callback {=0A= void (*dkc_callback)(void *dkc_cookie, int error);=0A= void *dkc_cookie;=0A= Index: sys/cddl/compat/opensolaris/sys/time.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/compat/opensolaris/sys/time.h (revision 250526)=0A= +++ sys/cddl/compat/opensolaris/sys/time.h (working copy)=0A= @@ -35,6 +35,7 @@=0A= #define MILLISEC 1000=0A= #define MICROSEC 1000000=0A= #define NANOSEC 1000000000=0A= +#define TIME_MAX LLONG_MAX=0A= =0A= typedef longlong_t hrtime_t;=0A= =0A= Index: sys/cddl/compat/opensolaris/sys/kstat.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/compat/opensolaris/sys/kstat.h (revision 250526)=0A= +++ sys/cddl/compat/opensolaris/sys/kstat.h (working copy)=0A= @@ -53,6 +53,8 @@=0A= #define KSTAT_DATA_INT64 3=0A= #define KSTAT_DATA_UINT64 4=0A= uchar_t data_type;=0A= +#define KSTAT_DESCLEN 128=0A= + char desc[KSTAT_DESCLEN];=0A= union {=0A= uint64_t ui64;=0A= } value;=0A= ------=_NextPart_000_0730_01CE57A0.ECC7BD20-- From owner-freebsd-fs@FreeBSD.ORG Thu May 23 10:33:30 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 9B60C52A for ; Thu, 23 May 2013 10:33:30 +0000 (UTC) (envelope-from ajit.jain@cloudbyte.com) Received: from mail-ob0-x235.google.com (mail-ob0-x235.google.com [IPv6:2607:f8b0:4003:c01::235]) by mx1.freebsd.org (Postfix) with ESMTP id 6267432F for ; Thu, 23 May 2013 10:33:30 +0000 (UTC) Received: by mail-ob0-f181.google.com with SMTP id dn14so3665361obc.26 for ; Thu, 23 May 2013 03:33:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:x-gm-message-state; bh=XO/sXeDpp4bJdamgqY3IN0fwwhiCrpeRFdd63vzn7C8=; b=Tq1YgKWz/csIsIj89Xufgae4xrX9FTzSRb2YZWda1XOOD+gClL1mXytpOjSISg4vLh /QdDpDtKAu5tpK7N4g5pKMaWq+0GyxfdApdC0z8PUbJy0e/ZaPsxXsSEjEjZFyrgSwl6 QFkTW0X/YwYXQnTyzZXAx0KPxZVk+tUbKdSXxbstWrcjSc5aMzwAWiB2nlqTDIz8jh70 eP0Rh3Tlst2galUXnU1htK0QiAgsh5HuNd8nbRXWVJ2R6gdc5eIzTChqOogTTrfO4dRW b2pKxt0p4GdgsedoL1SX+M1EABFTZuyb+5JvwRzdPuDXi2WH7inb0Y0jkxGQ7pQcAKDf TcQw== X-Received: by 10.182.96.66 with SMTP id dq2mr7676026obb.34.1369305209914; Thu, 23 May 2013 03:33:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.76.151.134 with HTTP; Thu, 23 May 2013 03:33:09 -0700 (PDT) In-Reply-To: <35ABA7AAEB7F4D86A1ED54C4C47FEB49@multiplay.co.uk> References: <60316751643743738AB83DABC6A5934B@multiplay.co.uk> <20130429105143.GA1492@icarus.home.lan> <3AD1AB31003D49B2BF2EA7DD411B38A2@multiplay.co.uk> <9681E07546D348168052D4FC5365B4CD@multiplay.co.uk> <3E9CA9334E6F433A8F135ACD5C237340@multiplay.co.uk> <93D0677B373A452BAF58C8EA6823783D@multiplay.co.uk> <35ABA7AAEB7F4D86A1ED54C4C47FEB49@multiplay.co.uk> From: Ajit Jain Date: Thu, 23 May 2013 16:03:09 +0530 Message-ID: Subject: Re: seeing data corruption with zfs trim functionality To: Steven Hartland X-Gm-Message-State: ALoCoQkprGU6Cnp3jJa74GZUJpqYweU2GMIWXM9oyknRnQW8TMPpwqciIBSUnWlAL7VSVO70DCUS Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 May 2013 10:33:30 -0000 Sure Steven, I'll apply the patches and update ASAP. thanks ajit On Thu, May 23, 2013 at 3:03 PM, Steven Hartland wrote: > I've attacked the two patch sets I'm looking to MFC to stable-9, one > adds BIO_DELETE CAM changes and the other is ZFS TRIM support. > > They should both apply cleanly to stable-9, if you could test with > those on your machine and let me know. > > Regards > Steve > > ----- Original Message ----- From: "Ajit Jain" > > > Hi Steven, >> >> FW version on the setup is P15. >> I will upgrade the FW to P16, but I think my >> best bet will be to update code base to 9 stable as unlike you, >> I was seeing corruption for all three delete methods. >> >> thanks >> ajit >> >> On Sat, May 18, 2013 at 4:15 AM, Steven Hartland > >**wrote: >> >> ----- Original Message ----- From: "Steven Hartland" < >>> killing@multiplay.co.uk> >>> >>> >>> After initially seeing not issues, our overnight monitoring started >>>> moaning >>>> big time on the test box. So we checked and there was zpool corruption >>>> as >>>> well >>>> as a missing boot loader and a corrupt GPT, so I believe we have >>>> reproduced >>>> your issue. >>>> >>>> After recovering the machine I created 3 pools on 3 different disks each >>>> running a different delete_method. >>>> >>>> We then re-ran the tests which resulted in the pool running with >>>> delete_method >>>> WS16 being so broken it had suspended IO. A reboot resulted in it once >>>> again >>>> reporting no partition table via gpart. >>>> >>>> A third test run again produced a corrupt pool for WS16. >>>> >>>> I've conducted a preliminary review of the CAM WS16 code path along with >>>> SBC-3 >>>> spec which didn't identify any obvious issues. >>>> >>>> Given we're both using LSI 2008 based controllers it could be FW issue >>>> specific >>>> to WS16 but that's just speculation atm, so I'll continue to >>>> investigate. >>>> >>>> If you could re-test you end without using WS16 to see if you can >>>> reproduce the >>>> problem with either UNMAP or ATA_TRIM that would be a very useful data >>>> point. >>>> >>>> >>> After much playing I narrow down a test case of one delete which was >>> causing >>> disc corruption for us (deleted the partition table instead of data in >>> the middle of the disk). >>> >>> The conclusion is LSI 2008 HBA with FW below P13 will eat the data on >>> your >>> SATA >>> disks if you use WS16 due to the following bug:- >>> SCGCQ00230159 (DFCT) - Write same command to a SATA drive that doesn't >>> support >>> SCT write same may write wrong region. >>> >>> After updating here to P16, which we would generally be running, but test >>> box >>> was new and hadnt updated yet the corruption issue is no longer >>> reproducable. >>> >>> So Ajit please check your FW version, I'm hoping to here your on >>> something >>> below P13, P12 possibly? >>> >>> If so then this is your issue, to fix simply update to P16 and the >>> problem >>> should be gone. >>> >>> >>> Regards >>> Steve >>> >>> >>> ==============================****================== >>> >>> This e.mail is private and confidential between Multiplay (UK) Ltd. and >>> the person or entity to whom it is addressed. In the event of >>> misdirection, >>> the recipient is prohibited from using, copying, printing or otherwise >>> disseminating it or any information contained in it. >>> In the event of misdirection, illegible or incomplete transmission please >>> telephone +44 845 868 1337 >>> or return the E.mail to postmaster@multiplay.co.uk. >>> >>> >>> >> > ==============================**================== > This e.mail is private and confidential between Multiplay (UK) Ltd. and > the person or entity to whom it is addressed. In the event of misdirection, > the recipient is prohibited from using, copying, printing or otherwise > disseminating it or any information contained in it. > In the event of misdirection, illegible or incomplete transmission please > telephone +44 845 868 1337 > or return the E.mail to postmaster@multiplay.co.uk. > From owner-freebsd-fs@FreeBSD.ORG Thu May 23 14:10:03 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7AC23166 for ; Thu, 23 May 2013 14:10:03 +0000 (UTC) (envelope-from tevans.uk@googlemail.com) Received: from mail-la0-x230.google.com (mail-la0-x230.google.com [IPv6:2a00:1450:4010:c03::230]) by mx1.freebsd.org (Postfix) with ESMTP id 0897E37E for ; Thu, 23 May 2013 14:10:02 +0000 (UTC) Received: by mail-la0-f48.google.com with SMTP id fs12so3237710lab.7 for ; Thu, 23 May 2013 07:10:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=WFKbf4nFkXDCEdosrKEu9/hhta5RZnceEdgzh2TXNqk=; b=Im0AxEXB82vb0sG02puWg7qz0mAV/D7V4DMwp8DZzfjsN1CE4NyewjpZkHANuNgfHe RpCosSENXVp0iTzVTq7fsWLgj4rFr/hoPO38rSJ6J0iT/sQskDgDJek1p6r8wGSKsPvu bgHpoYpajYWz7sFThBnaZQmXUbOeSpzBZcW6qylakfbztkIA4sTUhv5B9X0AmVbhK7XQ KA9w1y9UYzMOQis6PkQ8X6gdAuqSlZHFq7JfqtHARfIcpvO7NhO9CpD4yt2wSeiNHZ/4 5FX3bf56/6zdTO2tvlST1gvdva6MCLUC/rl+rNeL+98kGK2NFy/34ydsFhvxhttxN6is VPZg== MIME-Version: 1.0 X-Received: by 10.112.156.194 with SMTP id wg2mr3818011lbb.116.1369318201917; Thu, 23 May 2013 07:10:01 -0700 (PDT) Received: by 10.112.202.137 with HTTP; Thu, 23 May 2013 07:10:01 -0700 (PDT) In-Reply-To: References: Date: Thu, 23 May 2013 15:10:01 +0100 Message-ID: Subject: Re: ZFS TRIM support MFC From: Tom Evans To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD FS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 May 2013 14:10:03 -0000 On Tue, Mar 19, 2013 at 2:57 PM, Steven Hartland wrote: > ----- Original Message ----- From: "Tom Evans" > > >> I was wondering if there was an anticipated date when the ZFS TRIM >> support would be MFC'd to a stable branch? > > > That's likely something that I'll do. I've just finished a round of > patches which are in review ATM and will hopefully get committed > in the next week or two. > > After those have had time to bed down in HEAD, another few weeks, > then would be the first point at which this should be MFC'ed IMO. > > That said we've been running TRIM on 8.3 for months in production > so if your happy with back porting the changes yourself you should > have no problem doing so. If you happen to be on 8 then I could > provide you with our current patch set. > Hi Steven Just wondering if an MFC of these TRIM features to 9 is any closer. Cheers Tom From owner-freebsd-fs@FreeBSD.ORG Thu May 23 14:26:36 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id D638AB5E for ; Thu, 23 May 2013 14:26:36 +0000 (UTC) (envelope-from prvs=1855369f91=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 7CECA6DC for ; Thu, 23 May 2013 14:26:36 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50003955162.msg for ; Thu, 23 May 2013 15:26:34 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Thu, 23 May 2013 15:26:34 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1855369f91=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-fs@freebsd.org Message-ID: <598739E4EB374056A2D529F818A209B9@multiplay.co.uk> From: "Steven Hartland" To: "Tom Evans" References: Subject: Re: ZFS TRIM support MFC Date: Thu, 23 May 2013 15:26:29 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="utf-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: FreeBSD FS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 May 2013 14:26:36 -0000 ----- Original Message ----- From: "Tom Evans" > Hi Steven > > Just wondering if an MFC of these TRIM features to 9 is any closer. Yes, I've created the MFC patches for stable 8 & 9 but we're just looking to close out an issue reported by Ajit Jain thread "seeing data corruption with zfs trim functionality" on freebsd-fs@ My current thinking is this is due to either a LSI FW issue, related to a know WS16 bug or potentially a merge problem, as Ajit wasn't using our code base directly. I posted the two main patch sets I'm looking to commit to that thread this morning, your welcome to test and let us know how you get on. If you are using LSI mps controllers, either ensure your using P16 FW or don't manually select Write Same delete methods for your disks to avoid the know FW bug which will cause corruption. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. From owner-freebsd-fs@FreeBSD.ORG Thu May 23 22:26:55 2013 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 09B35ECE for ; Thu, 23 May 2013 22:26:55 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) by mx1.freebsd.org (Postfix) with ESMTP id 903EC60E for ; Thu, 23 May 2013 22:26:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.5/8.14.5) with ESMTP id r4NMQnF2036862 for ; Fri, 24 May 2013 02:26:50 +0400 (MSK) (envelope-from marck@rinet.ru) Date: Fri, 24 May 2013 02:26:49 +0400 (MSK) From: Dmitry Morozovsky To: freebsd-fs@FreeBSD.org Subject: Proposed MFC to hastctl: compact 'status' and introduce 'list' command (fwd) Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (woozle.rinet.ru [0.0.0.0]); Fri, 24 May 2013 02:26:50 +0400 (MSK) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 May 2013 22:26:55 -0000 Dear colleagues, I think some of you do not follow -stable@ -- hence my question here too -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ ---------- Forwarded message ---------- Date: Fri, 24 May 2013 00:54:56 From: Dmitry Morozovsky To: freebsd-stable@freebsd.org Subject: Proposed MFC to hastctl: compact 'status' and introduce 'list' command Dear colleagues, is there any objection for MFCing the change introduced in http://svnweb.freebsd.org/changeset/base/248291 (the most major change: compacting output of `hastctl status' to one-liner per provider; old output is retained as `list' command) to at least stable/9 ? The reason I'm asking is that it could lead to changes in hast-related scripts which one use in production. If no objections are received I'm (with the generous support from trociny, thank you Mikolaj!) tend to merge it after, say, 2 weeks. Thanks! -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Fri May 24 00:04:15 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 594647AD for ; Fri, 24 May 2013 00:04:15 +0000 (UTC) (envelope-from prvs=1856dc80c3=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id AD40F9C5 for ; Fri, 24 May 2013 00:04:13 +0000 (UTC) Received: from r2d2 ([82.69.141.170]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50003962037.msg for ; Fri, 24 May 2013 01:04:10 +0100 X-Spam-Processed: mail1.multiplay.co.uk, Fri, 24 May 2013 01:04:10 +0100 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 82.69.141.170 X-Return-Path: prvs=1856dc80c3=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-fs@freebsd.org Message-ID: <9219DF6C2DFD422998FF9B5526A06EF1@multiplay.co.uk> From: "Steven Hartland" To: "Steven Hartland" , "Ajit Jain" References: <60316751643743738AB83DABC6A5934B@multiplay.co.uk> <20130429105143.GA1492@icarus.home.lan> <3AD1AB31003D49B2BF2EA7DD411B38A2@multiplay.co.uk> <9681E07546D348168052D4FC5365B4CD@multiplay.co.uk> <3E9CA9334E6F433A8F135ACD5C237340@multiplay.co.uk> <93D0677B373A452BAF58C8EA6823783D@multiplay.co.uk> <35ABA7AAEB7F4D86A1ED54C4C47FEB49@multiplay.co.uk> Subject: Re: seeing data corruption with zfs trim functionality Date: Fri, 24 May 2013 01:04:06 +0100 MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_NextPart_000_0391_01CE581A.96C6FBA0" X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 May 2013 00:04:15 -0000 This is a multi-part message in MIME format. ------=_NextPart_000_0391_01CE581A.96C6FBA0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit Updated trim patch attached, this time created with --show-copies-as-adds so you get all the files. /me shakes his fist at svn!! Thanks to Michael Moll for the push in the right direction on that :) Regards Steve ----- Original Message ----- From: "Steven Hartland" > I've attacked the two patch sets I'm looking to MFC to stable-9, one > adds BIO_DELETE CAM changes and the other is ZFS TRIM support. > > They should both apply cleanly to stable-9, if you could test with > those on your machine and let me know. ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk. ------=_NextPart_000_0391_01CE581A.96C6FBA0 Content-Type: application/octet-stream; name="mfc-zfs-misc-stable-9-v2.patch" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="mfc-zfs-misc-stable-9-v2.patch" Index: cddl/lib=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- cddl/lib (revision 250526)=0A= +++ cddl/lib (working copy)=0A= =0A= Property changes on: cddl/lib=0A= ___________________________________________________________________=0A= Modified: svn:mergeinfo=0A= Merged /head/cddl/lib:r240868=0A= Index: cddl/lib/libzpool/Makefile=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- cddl/lib/libzpool/Makefile (revision 250526)=0A= +++ cddl/lib/libzpool/Makefile (working copy)=0A= @@ -26,7 +26,7 @@=0A= =0A= LIB=3D zpool=0A= =0A= -ZFS_COMMON_SRCS=3D ${ZFS_COMMON_OBJS:C/.o$/.c/} vdev_file.c=0A= +ZFS_COMMON_SRCS=3D ${ZFS_COMMON_OBJS:C/.o$/.c/} vdev_file.c trim_map.c=0A= ZFS_SHARED_SRCS=3D ${ZFS_SHARED_OBJS:C/.o$/.c/}=0A= KERNEL_SRCS=3D kernel.c taskq.c util.c=0A= LIST_SRCS=3D list.c=0A= Index: UPDATING=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- UPDATING (revision 250526)=0A= +++ UPDATING (working copy)=0A= @@ -11,6 +11,22 @@=0A= Items affecting the ports and packages system can be found in=0A= /usr/ports/UPDATING. Please read that file before running portupgrade.=0A= =0A= +20130512:=0A= + Added ZFS TRIM support which is enabled by default. To disable=0A= + ZFS TRIM support set vfs.zfs.trim.enabled=3D0 in loader.conf.=0A= +=0A= + Creating new ZFS pools and adding new devices to existing pools=0A= + first performs a full device level TRIM, which can take a significant=0A= + amount of time. Set the sysctl vfs.zfs.vdev.trim_on_init to 0 to=0A= + disable this behaviour.=0A= +=0A= + ZFS TRIM requires the underlying device support BIO_DELETE which=0A= + is currently provided by methods such as ATA TRIM and SCSI UNMAP=0A= + via CAM, which are typically supported by SSD's.=0A= +=0A= + Stats for ZFS TRIM can be monitored by looking at the sysctl's=0A= + under kstat.zfs.misc.zio_trim.=0A= +=0A= 20130430:=0A= The mergemaster command now uses the default MAKEOBJDIRPREFIX=0A= rather than creating it's own in the temporary directory in=0A= Index: sys=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys (revision 250526)=0A= +++ sys (working copy)=0A= =0A= Property changes on: sys=0A= ___________________________________________________________________=0A= Modified: svn:mergeinfo=0A= Merged = /head/sys:r240868,244155,244187-244188,248572,248574-248577,248602,249921=0A= Index: sys/modules=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/modules (revision 250526)=0A= +++ sys/modules (working copy)=0A= =0A= Property changes on: sys/modules=0A= ___________________________________________________________________=0A= Modified: svn:mergeinfo=0A= Merged /head/sys/modules:r240868=0A= Index: sys/modules/zfs/Makefile=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/modules/zfs/Makefile (revision 250526)=0A= +++ sys/modules/zfs/Makefile (working copy)=0A= @@ -72,6 +72,7 @@=0A= ZFS_SRCS=3D ${ZFS_OBJS:C/.o$/.c/}=0A= SRCS+=3D ${ZFS_SRCS}=0A= SRCS+=3D vdev_geom.c=0A= +SRCS+=3D trim_map.c=0A= =0A= # Use FreeBSD's namecache.=0A= CFLAGS+=3D-DFREEBSD_NAMECACHE=0A= Index: sys/cddl/contrib/opensolaris=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris (working copy)=0A= =0A= Property changes on: sys/cddl/contrib/opensolaris=0A= ___________________________________________________________________=0A= Modified: svn:mergeinfo=0A= Merged = /head/sys/cddl/contrib/opensolaris:r240868,244155,244187-244188,248572,24= 8574-248577,248602,249921=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_mirror.c = (working copy)=0A= @@ -293,10 +293,11 @@=0A= c =3D vdev_mirror_child_select(zio);=0A= children =3D (c >=3D 0);=0A= } else {=0A= - ASSERT(zio->io_type =3D=3D ZIO_TYPE_WRITE);=0A= + ASSERT(zio->io_type =3D=3D ZIO_TYPE_WRITE ||=0A= + zio->io_type =3D=3D ZIO_TYPE_FREE);=0A= =0A= /*=0A= - * Writes go to all children.=0A= + * Writes and frees go to all children.=0A= */=0A= c =3D 0;=0A= children =3D mm->mm_children;=0A= @@ -377,6 +378,8 @@=0A= zio->io_error =3D vdev_mirror_worst_error(mm);=0A= }=0A= return;=0A= + } else if (zio->io_type =3D=3D ZIO_TYPE_FREE) {=0A= + return;=0A= }=0A= =0A= ASSERT(zio->io_type =3D=3D ZIO_TYPE_READ);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zil.c (working copy)=0A= @@ -83,6 +83,11 @@=0A= TUNABLE_INT("vfs.zfs.cache_flush_disable", &zfs_nocacheflush);=0A= SYSCTL_INT(_vfs_zfs, OID_AUTO, cache_flush_disable, CTLFLAG_RDTUN,=0A= &zfs_nocacheflush, 0, "Disable cache flush");=0A= +boolean_t zfs_trim_enabled =3D B_TRUE;=0A= +SYSCTL_DECL(_vfs_zfs_trim);=0A= +TUNABLE_INT("vfs.zfs.trim.enabled", &zfs_trim_enabled);=0A= +SYSCTL_INT(_vfs_zfs_trim, OID_AUTO, enabled, CTLFLAG_RDTUN, = &zfs_trim_enabled, 0,=0A= + "Enable ZFS TRIM");=0A= =0A= static kmem_cache_t *zil_lwb_cache;=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c (revision = 0)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c (working = copy)=0A= @@ -0,0 +1,638 @@=0A= +/*=0A= + * CDDL HEADER START=0A= + *=0A= + * The contents of this file are subject to the terms of the=0A= + * Common Development and Distribution License (the "License").=0A= + * You may not use this file except in compliance with the License.=0A= + *=0A= + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE=0A= + * or http://www.opensolaris.org/os/licensing.=0A= + * See the License for the specific language governing permissions=0A= + * and limitations under the License.=0A= + *=0A= + * When distributing Covered Code, include this CDDL HEADER in each=0A= + * file and include the License file at usr/src/OPENSOLARIS.LICENSE.=0A= + * If applicable, add the following below this CDDL HEADER, with the=0A= + * fields enclosed by brackets "[]" replaced with your own identifying=0A= + * information: Portions Copyright [yyyy] [name of copyright owner]=0A= + *=0A= + * CDDL HEADER END=0A= + */=0A= +/*=0A= + * Copyright (c) 2012 Pawel Jakub Dawidek .=0A= + * All rights reserved.=0A= + */=0A= +=0A= +#include =0A= +#include =0A= +#include =0A= +#include =0A= +#include =0A= +=0A= +/*=0A= + * Calculate the zio end, upgrading based on ashift which would be=0A= + * done by zio_vdev_io_start.=0A= + *=0A= + * This makes free range consolidation much more effective=0A= + * than it would otherwise be as well as ensuring that entire=0A= + * blocks are invalidated by writes.=0A= + */=0A= +#define TRIM_ZIO_END(vd, offset, size) (offset + \=0A= + P2ROUNDUP(size, 1ULL << vd->vdev_top->vdev_ashift))=0A= +=0A= +#define TRIM_MAP_SINC(tm, size) \=0A= + atomic_add_64(&(tm)->tm_bytes, (size))=0A= +=0A= +#define TRIM_MAP_SDEC(tm, size) \=0A= + atomic_add_64(&(tm)->tm_bytes, -(size))=0A= +=0A= +#define TRIM_MAP_QINC(tm) \=0A= + atomic_inc_64(&(tm)->tm_pending); \=0A= +=0A= +#define TRIM_MAP_QDEC(tm) \=0A= + atomic_dec_64(&(tm)->tm_pending);=0A= +=0A= +typedef struct trim_map {=0A= + list_t tm_head; /* List of segments sorted by txg. */=0A= + avl_tree_t tm_queued_frees; /* AVL tree of segments waiting for TRIM. = */=0A= + avl_tree_t tm_inflight_frees; /* AVL tree of in-flight TRIMs. */=0A= + avl_tree_t tm_inflight_writes; /* AVL tree of in-flight writes. */=0A= + list_t tm_pending_writes; /* Writes blocked on in-flight frees. */=0A= + kmutex_t tm_lock;=0A= + uint64_t tm_pending; /* Count of pending TRIMs. */=0A= + uint64_t tm_bytes; /* Total size in bytes of queued TRIMs. */=0A= +} trim_map_t;=0A= +=0A= +typedef struct trim_seg {=0A= + avl_node_t ts_node; /* AVL node. */=0A= + list_node_t ts_next; /* List element. */=0A= + uint64_t ts_start; /* Starting offset of this segment. */=0A= + uint64_t ts_end; /* Ending offset (non-inclusive). */=0A= + uint64_t ts_txg; /* Segment creation txg. */=0A= + hrtime_t ts_time; /* Segment creation time. */=0A= +} trim_seg_t;=0A= +=0A= +extern boolean_t zfs_trim_enabled;=0A= +=0A= +static u_int trim_txg_delay =3D 32;=0A= +static u_int trim_timeout =3D 30;=0A= +static u_int trim_max_interval =3D 1;=0A= +/* Limit outstanding TRIMs to 2G (max size for a single TRIM request) */=0A= +static uint64_t trim_vdev_max_bytes =3D 2147483648;=0A= +/* Limit outstanding TRIMs to 64 (max ranges for a single TRIM request) = */ =0A= +static u_int trim_vdev_max_pending =3D 64;=0A= +=0A= +SYSCTL_DECL(_vfs_zfs);=0A= +SYSCTL_NODE(_vfs_zfs, OID_AUTO, trim, CTLFLAG_RD, 0, "ZFS TRIM");=0A= +=0A= +TUNABLE_INT("vfs.zfs.trim.txg_delay", &trim_txg_delay);=0A= +SYSCTL_UINT(_vfs_zfs_trim, OID_AUTO, txg_delay, CTLFLAG_RWTUN, = &trim_txg_delay,=0A= + 0, "Delay TRIMs by up to this many TXGs");=0A= +=0A= +TUNABLE_INT("vfs.zfs.trim.timeout", &trim_timeout);=0A= +SYSCTL_UINT(_vfs_zfs_trim, OID_AUTO, timeout, CTLFLAG_RWTUN, = &trim_timeout, 0,=0A= + "Delay TRIMs by up to this many seconds");=0A= +=0A= +TUNABLE_INT("vfs.zfs.trim.max_interval", &trim_max_interval);=0A= +SYSCTL_UINT(_vfs_zfs_trim, OID_AUTO, max_interval, CTLFLAG_RWTUN,=0A= + &trim_max_interval, 0,=0A= + "Maximum interval between TRIM queue processing (seconds)");=0A= +=0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +TUNABLE_QUAD("vfs.zfs.vdev.trim_max_bytes", &trim_vdev_max_bytes);=0A= +SYSCTL_QUAD(_vfs_zfs_vdev, OID_AUTO, trim_max_bytes, CTLFLAG_RWTUN,=0A= + &trim_vdev_max_bytes, 0,=0A= + "Maximum pending TRIM bytes for a vdev");=0A= +=0A= +TUNABLE_INT("vfs.zfs.vdev.trim_max_pending", &trim_vdev_max_pending);=0A= +SYSCTL_UINT(_vfs_zfs_vdev, OID_AUTO, trim_max_pending, CTLFLAG_RWTUN,=0A= + &trim_vdev_max_pending, 0,=0A= + "Maximum pending TRIM segments for a vdev");=0A= +=0A= +=0A= +static void trim_map_vdev_commit_done(spa_t *spa, vdev_t *vd);=0A= +=0A= +static int=0A= +trim_map_seg_compare(const void *x1, const void *x2)=0A= +{=0A= + const trim_seg_t *s1 =3D x1;=0A= + const trim_seg_t *s2 =3D x2;=0A= +=0A= + if (s1->ts_start < s2->ts_start) {=0A= + if (s1->ts_end > s2->ts_start)=0A= + return (0);=0A= + return (-1);=0A= + }=0A= + if (s1->ts_start > s2->ts_start) {=0A= + if (s1->ts_start < s2->ts_end)=0A= + return (0);=0A= + return (1);=0A= + }=0A= + return (0);=0A= +}=0A= +=0A= +static int=0A= +trim_map_zio_compare(const void *x1, const void *x2)=0A= +{=0A= + const zio_t *z1 =3D x1;=0A= + const zio_t *z2 =3D x2;=0A= +=0A= + if (z1->io_offset < z2->io_offset) {=0A= + if (z1->io_offset + z1->io_size > z2->io_offset)=0A= + return (0);=0A= + return (-1);=0A= + }=0A= + if (z1->io_offset > z2->io_offset) {=0A= + if (z1->io_offset < z2->io_offset + z2->io_size)=0A= + return (0);=0A= + return (1);=0A= + }=0A= + return (0);=0A= +}=0A= +=0A= +void=0A= +trim_map_create(vdev_t *vd)=0A= +{=0A= + trim_map_t *tm;=0A= +=0A= + ASSERT(vd->vdev_ops->vdev_op_leaf);=0A= +=0A= + if (!zfs_trim_enabled)=0A= + return;=0A= +=0A= + tm =3D kmem_zalloc(sizeof (*tm), KM_SLEEP);=0A= + mutex_init(&tm->tm_lock, NULL, MUTEX_DEFAULT, NULL);=0A= + list_create(&tm->tm_head, sizeof (trim_seg_t),=0A= + offsetof(trim_seg_t, ts_next));=0A= + list_create(&tm->tm_pending_writes, sizeof (zio_t),=0A= + offsetof(zio_t, io_trim_link));=0A= + avl_create(&tm->tm_queued_frees, trim_map_seg_compare,=0A= + sizeof (trim_seg_t), offsetof(trim_seg_t, ts_node));=0A= + avl_create(&tm->tm_inflight_frees, trim_map_seg_compare,=0A= + sizeof (trim_seg_t), offsetof(trim_seg_t, ts_node));=0A= + avl_create(&tm->tm_inflight_writes, trim_map_zio_compare,=0A= + sizeof (zio_t), offsetof(zio_t, io_trim_node));=0A= + vd->vdev_trimmap =3D tm;=0A= +}=0A= +=0A= +void=0A= +trim_map_destroy(vdev_t *vd)=0A= +{=0A= + trim_map_t *tm;=0A= + trim_seg_t *ts;=0A= +=0A= + ASSERT(vd->vdev_ops->vdev_op_leaf);=0A= +=0A= + if (!zfs_trim_enabled)=0A= + return;=0A= +=0A= + tm =3D vd->vdev_trimmap;=0A= + if (tm =3D=3D NULL)=0A= + return;=0A= +=0A= + /*=0A= + * We may have been called before trim_map_vdev_commit_done()=0A= + * had a chance to run, so do it now to prune the remaining=0A= + * inflight frees.=0A= + */=0A= + trim_map_vdev_commit_done(vd->vdev_spa, vd);=0A= +=0A= + mutex_enter(&tm->tm_lock);=0A= + while ((ts =3D list_head(&tm->tm_head)) !=3D NULL) {=0A= + avl_remove(&tm->tm_queued_frees, ts);=0A= + list_remove(&tm->tm_head, ts);=0A= + kmem_free(ts, sizeof (*ts));=0A= + TRIM_MAP_SDEC(tm, ts->ts_end - ts->ts_start);=0A= + TRIM_MAP_QDEC(tm);=0A= + }=0A= + mutex_exit(&tm->tm_lock);=0A= +=0A= + avl_destroy(&tm->tm_queued_frees);=0A= + avl_destroy(&tm->tm_inflight_frees);=0A= + avl_destroy(&tm->tm_inflight_writes);=0A= + list_destroy(&tm->tm_pending_writes);=0A= + list_destroy(&tm->tm_head);=0A= + mutex_destroy(&tm->tm_lock);=0A= + kmem_free(tm, sizeof (*tm));=0A= + vd->vdev_trimmap =3D NULL;=0A= +}=0A= +=0A= +static void=0A= +trim_map_segment_add(trim_map_t *tm, uint64_t start, uint64_t end, = uint64_t txg)=0A= +{=0A= + avl_index_t where;=0A= + trim_seg_t tsearch, *ts_before, *ts_after, *ts;=0A= + boolean_t merge_before, merge_after;=0A= + hrtime_t time;=0A= +=0A= + ASSERT(MUTEX_HELD(&tm->tm_lock));=0A= + VERIFY(start < end);=0A= +=0A= + time =3D gethrtime();=0A= + tsearch.ts_start =3D start;=0A= + tsearch.ts_end =3D end;=0A= +=0A= + ts =3D avl_find(&tm->tm_queued_frees, &tsearch, &where);=0A= + if (ts !=3D NULL) {=0A= + if (start < ts->ts_start)=0A= + trim_map_segment_add(tm, start, ts->ts_start, txg);=0A= + if (end > ts->ts_end)=0A= + trim_map_segment_add(tm, ts->ts_end, end, txg);=0A= + return;=0A= + }=0A= +=0A= + ts_before =3D avl_nearest(&tm->tm_queued_frees, where, AVL_BEFORE);=0A= + ts_after =3D avl_nearest(&tm->tm_queued_frees, where, AVL_AFTER);=0A= +=0A= + merge_before =3D (ts_before !=3D NULL && ts_before->ts_end =3D=3D = start);=0A= + merge_after =3D (ts_after !=3D NULL && ts_after->ts_start =3D=3D end);=0A= +=0A= + if (merge_before && merge_after) {=0A= + TRIM_MAP_SINC(tm, ts_after->ts_start - ts_before->ts_end);=0A= + TRIM_MAP_QDEC(tm);=0A= + avl_remove(&tm->tm_queued_frees, ts_before);=0A= + list_remove(&tm->tm_head, ts_before);=0A= + ts_after->ts_start =3D ts_before->ts_start;=0A= + ts_after->ts_txg =3D txg;=0A= + ts_after->ts_time =3D time;=0A= + kmem_free(ts_before, sizeof (*ts_before));=0A= + } else if (merge_before) {=0A= + TRIM_MAP_SINC(tm, end - ts_before->ts_end);=0A= + ts_before->ts_end =3D end;=0A= + ts_before->ts_txg =3D txg;=0A= + ts_before->ts_time =3D time;=0A= + } else if (merge_after) {=0A= + TRIM_MAP_SINC(tm, ts_after->ts_start - start);=0A= + ts_after->ts_start =3D start;=0A= + ts_after->ts_txg =3D txg;=0A= + ts_after->ts_time =3D time;=0A= + } else {=0A= + TRIM_MAP_SINC(tm, end - start);=0A= + TRIM_MAP_QINC(tm);=0A= + ts =3D kmem_alloc(sizeof (*ts), KM_SLEEP);=0A= + ts->ts_start =3D start;=0A= + ts->ts_end =3D end;=0A= + ts->ts_txg =3D txg;=0A= + ts->ts_time =3D time;=0A= + avl_insert(&tm->tm_queued_frees, ts, where);=0A= + list_insert_tail(&tm->tm_head, ts);=0A= + }=0A= +}=0A= +=0A= +static void=0A= +trim_map_segment_remove(trim_map_t *tm, trim_seg_t *ts, uint64_t start,=0A= + uint64_t end)=0A= +{=0A= + trim_seg_t *nts;=0A= + boolean_t left_over, right_over;=0A= +=0A= + ASSERT(MUTEX_HELD(&tm->tm_lock));=0A= +=0A= + left_over =3D (ts->ts_start < start);=0A= + right_over =3D (ts->ts_end > end);=0A= +=0A= + TRIM_MAP_SDEC(tm, end - start);=0A= + if (left_over && right_over) {=0A= + nts =3D kmem_alloc(sizeof (*nts), KM_SLEEP);=0A= + nts->ts_start =3D end;=0A= + nts->ts_end =3D ts->ts_end;=0A= + nts->ts_txg =3D ts->ts_txg;=0A= + nts->ts_time =3D ts->ts_time;=0A= + ts->ts_end =3D start;=0A= + avl_insert_here(&tm->tm_queued_frees, nts, ts, AVL_AFTER);=0A= + list_insert_after(&tm->tm_head, ts, nts);=0A= + TRIM_MAP_QINC(tm);=0A= + } else if (left_over) {=0A= + ts->ts_end =3D start;=0A= + } else if (right_over) {=0A= + ts->ts_start =3D end;=0A= + } else {=0A= + avl_remove(&tm->tm_queued_frees, ts);=0A= + list_remove(&tm->tm_head, ts);=0A= + TRIM_MAP_QDEC(tm);=0A= + kmem_free(ts, sizeof (*ts));=0A= + }=0A= +}=0A= +=0A= +static void=0A= +trim_map_free_locked(trim_map_t *tm, uint64_t start, uint64_t end, = uint64_t txg)=0A= +{=0A= + zio_t zsearch, *zs;=0A= +=0A= + ASSERT(MUTEX_HELD(&tm->tm_lock));=0A= +=0A= + zsearch.io_offset =3D start;=0A= + zsearch.io_size =3D end - start;=0A= +=0A= + zs =3D avl_find(&tm->tm_inflight_writes, &zsearch, NULL);=0A= + if (zs =3D=3D NULL) {=0A= + trim_map_segment_add(tm, start, end, txg);=0A= + return;=0A= + }=0A= + if (start < zs->io_offset)=0A= + trim_map_free_locked(tm, start, zs->io_offset, txg);=0A= + if (zs->io_offset + zs->io_size < end)=0A= + trim_map_free_locked(tm, zs->io_offset + zs->io_size, end, txg);=0A= +}=0A= +=0A= +void=0A= +trim_map_free(vdev_t *vd, uint64_t offset, uint64_t size, uint64_t txg)=0A= +{=0A= + trim_map_t *tm =3D vd->vdev_trimmap;=0A= +=0A= + if (!zfs_trim_enabled || vd->vdev_notrim || tm =3D=3D NULL)=0A= + return;=0A= +=0A= + mutex_enter(&tm->tm_lock);=0A= + trim_map_free_locked(tm, offset, TRIM_ZIO_END(vd, offset, size), txg);=0A= + mutex_exit(&tm->tm_lock);=0A= +}=0A= +=0A= +boolean_t=0A= +trim_map_write_start(zio_t *zio)=0A= +{=0A= + vdev_t *vd =3D zio->io_vd;=0A= + trim_map_t *tm =3D vd->vdev_trimmap;=0A= + trim_seg_t tsearch, *ts;=0A= + boolean_t left_over, right_over;=0A= + uint64_t start, end;=0A= +=0A= + if (!zfs_trim_enabled || vd->vdev_notrim || tm =3D=3D NULL)=0A= + return (B_TRUE);=0A= +=0A= + start =3D zio->io_offset;=0A= + end =3D TRIM_ZIO_END(zio->io_vd, start, zio->io_size);=0A= + tsearch.ts_start =3D start;=0A= + tsearch.ts_end =3D end;=0A= +=0A= + mutex_enter(&tm->tm_lock);=0A= +=0A= + /*=0A= + * Checking for colliding in-flight frees.=0A= + */=0A= + ts =3D avl_find(&tm->tm_inflight_frees, &tsearch, NULL);=0A= + if (ts !=3D NULL) {=0A= + list_insert_tail(&tm->tm_pending_writes, zio);=0A= + mutex_exit(&tm->tm_lock);=0A= + return (B_FALSE);=0A= + }=0A= +=0A= + ts =3D avl_find(&tm->tm_queued_frees, &tsearch, NULL);=0A= + if (ts !=3D NULL) {=0A= + /*=0A= + * Loop until all overlapping segments are removed.=0A= + */=0A= + do {=0A= + trim_map_segment_remove(tm, ts, start, end);=0A= + ts =3D avl_find(&tm->tm_queued_frees, &tsearch, NULL);=0A= + } while (ts !=3D NULL);=0A= + }=0A= + avl_add(&tm->tm_inflight_writes, zio);=0A= +=0A= + mutex_exit(&tm->tm_lock);=0A= +=0A= + return (B_TRUE);=0A= +}=0A= +=0A= +void=0A= +trim_map_write_done(zio_t *zio)=0A= +{=0A= + vdev_t *vd =3D zio->io_vd;=0A= + trim_map_t *tm =3D vd->vdev_trimmap;=0A= +=0A= + /*=0A= + * Don't check for vdev_notrim, since the write could have=0A= + * started before vdev_notrim was set.=0A= + */=0A= + if (!zfs_trim_enabled || tm =3D=3D NULL)=0A= + return;=0A= +=0A= + mutex_enter(&tm->tm_lock);=0A= + /*=0A= + * Don't fail if the write isn't in the tree, since the write=0A= + * could have started after vdev_notrim was set.=0A= + */=0A= + if (zio->io_trim_node.avl_child[0] ||=0A= + zio->io_trim_node.avl_child[1] ||=0A= + AVL_XPARENT(&zio->io_trim_node) ||=0A= + tm->tm_inflight_writes.avl_root =3D=3D &zio->io_trim_node)=0A= + avl_remove(&tm->tm_inflight_writes, zio);=0A= + mutex_exit(&tm->tm_lock);=0A= +}=0A= +=0A= +/*=0A= + * Return the oldest segment (the one with the lowest txg / time) or = NULL if:=0A= + * 1. The list is empty=0A= + * 2. The first element's txg is greater than txgsafe=0A= + * 3. The first element's txg is not greater than the txg argument and = the=0A= + * the first element's time is not greater than time argument=0A= + */=0A= +static trim_seg_t *=0A= +trim_map_first(trim_map_t *tm, uint64_t txg, uint64_t txgsafe, hrtime_t = time)=0A= +{=0A= + trim_seg_t *ts;=0A= +=0A= + ASSERT(MUTEX_HELD(&tm->tm_lock));=0A= + VERIFY(txgsafe >=3D txg);=0A= +=0A= + ts =3D list_head(&tm->tm_head);=0A= + if (ts !=3D NULL && ts->ts_txg <=3D txgsafe &&=0A= + (ts->ts_txg <=3D txg || ts->ts_time <=3D time ||=0A= + tm->tm_bytes > trim_vdev_max_bytes ||=0A= + tm->tm_pending > trim_vdev_max_pending))=0A= + return (ts);=0A= + return (NULL);=0A= +}=0A= +=0A= +static void=0A= +trim_map_vdev_commit(spa_t *spa, zio_t *zio, vdev_t *vd)=0A= +{=0A= + trim_map_t *tm =3D vd->vdev_trimmap;=0A= + trim_seg_t *ts;=0A= + uint64_t size, txgtarget, txgsafe;=0A= + hrtime_t timelimit;=0A= +=0A= + ASSERT(vd->vdev_ops->vdev_op_leaf);=0A= +=0A= + if (tm =3D=3D NULL)=0A= + return;=0A= +=0A= + timelimit =3D gethrtime() - trim_timeout * NANOSEC;=0A= + if (vd->vdev_isl2cache) {=0A= + txgsafe =3D UINT64_MAX;=0A= + txgtarget =3D UINT64_MAX;=0A= + } else {=0A= + txgsafe =3D MIN(spa_last_synced_txg(spa), spa_freeze_txg(spa));=0A= + if (txgsafe > trim_txg_delay)=0A= + txgtarget =3D txgsafe - trim_txg_delay;=0A= + else=0A= + txgtarget =3D 0;=0A= + }=0A= +=0A= + mutex_enter(&tm->tm_lock);=0A= + /* Loop until we have sent all outstanding free's */=0A= + while ((ts =3D trim_map_first(tm, txgtarget, txgsafe, timelimit))=0A= + !=3D NULL) {=0A= + list_remove(&tm->tm_head, ts);=0A= + avl_remove(&tm->tm_queued_frees, ts);=0A= + avl_add(&tm->tm_inflight_frees, ts);=0A= + size =3D ts->ts_end - ts->ts_start;=0A= + zio_nowait(zio_trim(zio, spa, vd, ts->ts_start, size));=0A= + TRIM_MAP_SDEC(tm, size);=0A= + TRIM_MAP_QDEC(tm);=0A= + }=0A= + mutex_exit(&tm->tm_lock);=0A= +}=0A= +=0A= +static void=0A= +trim_map_vdev_commit_done(spa_t *spa, vdev_t *vd)=0A= +{=0A= + trim_map_t *tm =3D vd->vdev_trimmap;=0A= + trim_seg_t *ts;=0A= + list_t pending_writes;=0A= + zio_t *zio;=0A= + uint64_t start, size;=0A= + void *cookie;=0A= +=0A= + ASSERT(vd->vdev_ops->vdev_op_leaf);=0A= +=0A= + if (tm =3D=3D NULL)=0A= + return;=0A= +=0A= + mutex_enter(&tm->tm_lock);=0A= + if (!avl_is_empty(&tm->tm_inflight_frees)) {=0A= + cookie =3D NULL;=0A= + while ((ts =3D avl_destroy_nodes(&tm->tm_inflight_frees,=0A= + &cookie)) !=3D NULL) {=0A= + kmem_free(ts, sizeof (*ts));=0A= + }=0A= + }=0A= + list_create(&pending_writes, sizeof (zio_t), offsetof(zio_t,=0A= + io_trim_link));=0A= + list_move_tail(&pending_writes, &tm->tm_pending_writes);=0A= + mutex_exit(&tm->tm_lock);=0A= +=0A= + while ((zio =3D list_remove_head(&pending_writes)) !=3D NULL) {=0A= + zio_vdev_io_reissue(zio);=0A= + zio_execute(zio);=0A= + }=0A= + list_destroy(&pending_writes);=0A= +}=0A= +=0A= +static void=0A= +trim_map_commit(spa_t *spa, zio_t *zio, vdev_t *vd)=0A= +{=0A= + int c;=0A= +=0A= + if (vd =3D=3D NULL)=0A= + return;=0A= +=0A= + if (vd->vdev_ops->vdev_op_leaf) {=0A= + trim_map_vdev_commit(spa, zio, vd);=0A= + } else {=0A= + for (c =3D 0; c < vd->vdev_children; c++)=0A= + trim_map_commit(spa, zio, vd->vdev_child[c]);=0A= + }=0A= +}=0A= +=0A= +static void=0A= +trim_map_commit_done(spa_t *spa, vdev_t *vd)=0A= +{=0A= + int c;=0A= +=0A= + if (vd =3D=3D NULL)=0A= + return;=0A= +=0A= + if (vd->vdev_ops->vdev_op_leaf) {=0A= + trim_map_vdev_commit_done(spa, vd);=0A= + } else {=0A= + for (c =3D 0; c < vd->vdev_children; c++)=0A= + trim_map_commit_done(spa, vd->vdev_child[c]);=0A= + }=0A= +}=0A= +=0A= +static void=0A= +trim_thread(void *arg)=0A= +{=0A= + spa_t *spa =3D arg;=0A= + zio_t *zio;=0A= +=0A= +#ifdef _KERNEL=0A= + (void) snprintf(curthread->td_name, sizeof(curthread->td_name),=0A= + "trim %s", spa_name(spa));=0A= +#endif=0A= +=0A= + for (;;) {=0A= + mutex_enter(&spa->spa_trim_lock);=0A= + if (spa->spa_trim_thread =3D=3D NULL) {=0A= + spa->spa_trim_thread =3D curthread;=0A= + cv_signal(&spa->spa_trim_cv);=0A= + mutex_exit(&spa->spa_trim_lock);=0A= + thread_exit();=0A= + }=0A= +=0A= + (void) cv_timedwait(&spa->spa_trim_cv, &spa->spa_trim_lock,=0A= + hz * trim_max_interval);=0A= + mutex_exit(&spa->spa_trim_lock);=0A= +=0A= + zio =3D zio_root(spa, NULL, NULL, ZIO_FLAG_CANFAIL);=0A= +=0A= + spa_config_enter(spa, SCL_STATE, FTAG, RW_READER);=0A= + trim_map_commit(spa, zio, spa->spa_root_vdev);=0A= + (void) zio_wait(zio);=0A= + trim_map_commit_done(spa, spa->spa_root_vdev);=0A= + spa_config_exit(spa, SCL_STATE, FTAG);=0A= + }=0A= +}=0A= +=0A= +void=0A= +trim_thread_create(spa_t *spa)=0A= +{=0A= +=0A= + if (!zfs_trim_enabled)=0A= + return;=0A= +=0A= + mutex_init(&spa->spa_trim_lock, NULL, MUTEX_DEFAULT, NULL);=0A= + cv_init(&spa->spa_trim_cv, NULL, CV_DEFAULT, NULL);=0A= + mutex_enter(&spa->spa_trim_lock);=0A= + spa->spa_trim_thread =3D thread_create(NULL, 0, trim_thread, spa, 0, = &p0,=0A= + TS_RUN, minclsyspri);=0A= + mutex_exit(&spa->spa_trim_lock);=0A= +}=0A= +=0A= +void=0A= +trim_thread_destroy(spa_t *spa)=0A= +{=0A= +=0A= + if (!zfs_trim_enabled)=0A= + return;=0A= + if (spa->spa_trim_thread =3D=3D NULL)=0A= + return;=0A= +=0A= + mutex_enter(&spa->spa_trim_lock);=0A= + /* Setting spa_trim_thread to NULL tells the thread to stop. */=0A= + spa->spa_trim_thread =3D NULL;=0A= + cv_signal(&spa->spa_trim_cv);=0A= + /* The thread will set it back to !=3D NULL on exit. */=0A= + while (spa->spa_trim_thread =3D=3D NULL)=0A= + cv_wait(&spa->spa_trim_cv, &spa->spa_trim_lock);=0A= + spa->spa_trim_thread =3D NULL;=0A= + mutex_exit(&spa->spa_trim_lock);=0A= +=0A= + cv_destroy(&spa->spa_trim_cv);=0A= + mutex_destroy(&spa->spa_trim_lock);=0A= +}=0A= +=0A= +void=0A= +trim_thread_wakeup(spa_t *spa)=0A= +{=0A= +=0A= + if (!zfs_trim_enabled)=0A= + return;=0A= + if (spa->spa_trim_thread =3D=3D NULL)=0A= + return;=0A= +=0A= + mutex_enter(&spa->spa_trim_lock);=0A= + cv_signal(&spa->spa_trim_cv);=0A= + mutex_exit(&spa->spa_trim_lock);=0A= +}=0A= =0A= Property changes on: = sys/cddl/contrib/opensolaris/uts/common/fs/zfs/trim_map.c=0A= ___________________________________________________________________=0A= Added: svn:mime-type=0A= ## -0,0 +1 ##=0A= +text/plain=0A= \ No newline at end of property=0A= Added: svn:keywords=0A= ## -0,0 +1 ##=0A= +FreeBSD=3D%H=0A= \ No newline at end of property=0A= Added: svn:eol-style=0A= ## -0,0 +1 ##=0A= +native=0A= \ No newline at end of property=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_scan.c (working = copy)=0A= @@ -397,7 +397,8 @@=0A= dsl_free_sync(zio_t *pio, dsl_pool_t *dp, uint64_t txg, const blkptr_t = *bpp)=0A= {=0A= ASSERT(dsl_pool_sync_context(dp));=0A= - zio_nowait(zio_free_sync(pio, dp->dp_spa, txg, bpp, pio->io_flags));=0A= + zio_nowait(zio_free_sync(pio, dp->dp_spa, txg, bpp, BP_GET_PSIZE(bpp),=0A= + pio->io_flags));=0A= }=0A= =0A= static uint64_t=0A= @@ -1364,7 +1365,7 @@=0A= }=0A= =0A= zio_nowait(zio_free_sync(scn->scn_zio_root, scn->scn_dp->dp_spa,=0A= - dmu_tx_get_txg(tx), bp, 0));=0A= + dmu_tx_get_txg(tx), bp, BP_GET_PSIZE(bp), 0));=0A= dsl_dir_diduse_space(tx->tx_pool->dp_free_dir, DD_USED_HEAD,=0A= -bp_get_dsize_sync(scn->scn_dp->dp_spa, bp),=0A= -BP_GET_PSIZE(bp), -BP_GET_UCSIZE(bp), tx);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c (working = copy)=0A= @@ -259,7 +259,9 @@=0A= size_t size;=0A= =0A= for (c =3D 0; c < rm->rm_firstdatacol; c++) {=0A= - zio_buf_free(rm->rm_col[c].rc_data, rm->rm_col[c].rc_size);=0A= + if (rm->rm_col[c].rc_data !=3D NULL)=0A= + zio_buf_free(rm->rm_col[c].rc_data,=0A= + rm->rm_col[c].rc_size);=0A= =0A= if (rm->rm_col[c].rc_gdata !=3D NULL)=0A= zio_buf_free(rm->rm_col[c].rc_gdata,=0A= @@ -504,14 +506,20 @@=0A= ASSERT3U(rm->rm_asize - asize, =3D=3D, rm->rm_nskip << unit_shift);=0A= ASSERT3U(rm->rm_nskip, <=3D, nparity);=0A= =0A= - for (c =3D 0; c < rm->rm_firstdatacol; c++)=0A= - rm->rm_col[c].rc_data =3D zio_buf_alloc(rm->rm_col[c].rc_size);=0A= + if (zio->io_type !=3D ZIO_TYPE_FREE) {=0A= + for (c =3D 0; c < rm->rm_firstdatacol; c++) {=0A= + rm->rm_col[c].rc_data =3D=0A= + zio_buf_alloc(rm->rm_col[c].rc_size);=0A= + }=0A= =0A= - rm->rm_col[c].rc_data =3D zio->io_data;=0A= + rm->rm_col[c].rc_data =3D zio->io_data;=0A= =0A= - for (c =3D c + 1; c < acols; c++)=0A= - rm->rm_col[c].rc_data =3D (char *)rm->rm_col[c - 1].rc_data +=0A= - rm->rm_col[c - 1].rc_size;=0A= + for (c =3D c + 1; c < acols; c++) {=0A= + rm->rm_col[c].rc_data =3D=0A= + (char *)rm->rm_col[c - 1].rc_data +=0A= + rm->rm_col[c - 1].rc_size;=0A= + }=0A= + }=0A= =0A= /*=0A= * If all data stored spans all columns, there's a danger that parity=0A= @@ -1536,6 +1544,18 @@=0A= =0A= ASSERT3U(rm->rm_asize, =3D=3D, vdev_psize_to_asize(vd, zio->io_size));=0A= =0A= + if (zio->io_type =3D=3D ZIO_TYPE_FREE) {=0A= + for (c =3D 0; c < rm->rm_cols; c++) {=0A= + rc =3D &rm->rm_col[c];=0A= + cvd =3D vd->vdev_child[rc->rc_devidx];=0A= + zio_nowait(zio_vdev_child_io(zio, NULL, cvd,=0A= + rc->rc_offset, rc->rc_data, rc->rc_size,=0A= + zio->io_type, zio->io_priority, 0,=0A= + vdev_raidz_child_done, rc));=0A= + }=0A= + return (ZIO_PIPELINE_CONTINUE);=0A= + }=0A= +=0A= if (zio->io_type =3D=3D ZIO_TYPE_WRITE) {=0A= vdev_raidz_generate_parity(rm);=0A= =0A= @@ -1918,6 +1938,8 @@=0A= zio->io_error =3D vdev_raidz_worst_error(rm);=0A= =0A= return;=0A= + } else if (zio->io_type =3D=3D ZIO_TYPE_FREE) {=0A= + return;=0A= }=0A= =0A= ASSERT(zio->io_type =3D=3D ZIO_TYPE_READ);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c (working copy)=0A= @@ -43,6 +43,7 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= =0A= SYSCTL_DECL(_vfs_zfs);=0A= SYSCTL_NODE(_vfs_zfs, OID_AUTO, vdev, CTLFLAG_RW, 0, "ZFS VDEV");=0A= @@ -1196,6 +1197,11 @@=0A= if (vd->vdev_ishole || vd->vdev_ops =3D=3D &vdev_missing_ops)=0A= return (0);=0A= =0A= + if (vd->vdev_ops->vdev_op_leaf) {=0A= + vd->vdev_notrim =3D B_FALSE;=0A= + trim_map_create(vd);=0A= + }=0A= +=0A= for (int c =3D 0; c < vd->vdev_children; c++) {=0A= if (vd->vdev_child[c]->vdev_state !=3D VDEV_STATE_HEALTHY) {=0A= vdev_set_state(vd, B_TRUE, VDEV_STATE_DEGRADED,=0A= @@ -1441,6 +1447,9 @@=0A= =0A= vdev_cache_purge(vd);=0A= =0A= + if (vd->vdev_ops->vdev_op_leaf)=0A= + trim_map_destroy(vd);=0A= +=0A= /*=0A= * We record the previous state before we close it, so that if we are=0A= * doing a reopen(), we don't generate FMA ereports if we notice that=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/trim_map.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/trim_map.h = (revision 0)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/trim_map.h = (working copy)=0A= @@ -0,0 +1,51 @@=0A= +/*=0A= + * CDDL HEADER START=0A= + *=0A= + * The contents of this file are subject to the terms of the=0A= + * Common Development and Distribution License (the "License").=0A= + * You may not use this file except in compliance with the License.=0A= + *=0A= + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE=0A= + * or http://www.opensolaris.org/os/licensing.=0A= + * See the License for the specific language governing permissions=0A= + * and limitations under the License.=0A= + *=0A= + * When distributing Covered Code, include this CDDL HEADER in each=0A= + * file and include the License file at usr/src/OPENSOLARIS.LICENSE.=0A= + * If applicable, add the following below this CDDL HEADER, with the=0A= + * fields enclosed by brackets "[]" replaced with your own identifying=0A= + * information: Portions Copyright [yyyy] [name of copyright owner]=0A= + *=0A= + * CDDL HEADER END=0A= + */=0A= +/*=0A= + * Copyright (c) 2012 Pawel Jakub Dawidek .=0A= + * All rights reserved.=0A= + */=0A= +=0A= +#ifndef _SYS_TRIM_MAP_H=0A= +#define _SYS_TRIM_MAP_H=0A= +=0A= +#include =0A= +#include =0A= +#include =0A= +=0A= +#ifdef __cplusplus=0A= +extern "C" {=0A= +#endif=0A= +=0A= +extern void trim_map_create(vdev_t *vd);=0A= +extern void trim_map_destroy(vdev_t *vd);=0A= +extern void trim_map_free(vdev_t *vd, uint64_t offset, uint64_t size, = uint64_t txg);=0A= +extern boolean_t trim_map_write_start(zio_t *zio);=0A= +extern void trim_map_write_done(zio_t *zio);=0A= +=0A= +extern void trim_thread_create(spa_t *spa);=0A= +extern void trim_thread_destroy(spa_t *spa);=0A= +extern void trim_thread_wakeup(spa_t *spa);=0A= +=0A= +#ifdef __cplusplus=0A= +}=0A= +#endif=0A= +=0A= +#endif /* _SYS_TRIM_MAP_H */=0A= =0A= Property changes on: = sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/trim_map.h=0A= ___________________________________________________________________=0A= Added: svn:mime-type=0A= ## -0,0 +1 ##=0A= +text/plain=0A= \ No newline at end of property=0A= Added: svn:keywords=0A= ## -0,0 +1 ##=0A= +FreeBSD=3D%H=0A= \ No newline at end of property=0A= Added: svn:eol-style=0A= ## -0,0 +1 ##=0A= +native=0A= \ No newline at end of property=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev.h (working = copy)=0A= @@ -46,6 +46,7 @@=0A= } vdev_dtl_type_t;=0A= =0A= extern boolean_t zfs_nocacheflush;=0A= +extern boolean_t zfs_trim_enabled;=0A= =0A= extern int vdev_open(vdev_t *);=0A= extern void vdev_open_children(vdev_t *);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/spa_impl.h = (working copy)=0A= @@ -221,6 +221,9 @@=0A= spa_proc_state_t spa_proc_state; /* see definition */=0A= struct proc *spa_proc; /* "zpool-poolname" process */=0A= uint64_t spa_did; /* if procp !=3D p0, did of t1 */=0A= + kthread_t *spa_trim_thread; /* thread sending TRIM I/Os */=0A= + kmutex_t spa_trim_lock; /* protects spa_trim_cv */=0A= + kcondvar_t spa_trim_cv; /* used to notify TRIM thread */=0A= boolean_t spa_autoreplace; /* autoreplace set in open */=0A= int spa_vdev_locks; /* locks grabbed */=0A= uint64_t spa_creation_version; /* version at pool creation */=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_impl.h = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio_impl.h = (working copy)=0A= @@ -130,9 +130,9 @@=0A= =0A= ZIO_STAGE_READY =3D 1 << 16, /* RWFCI */=0A= =0A= - ZIO_STAGE_VDEV_IO_START =3D 1 << 17, /* RW--I */=0A= - ZIO_STAGE_VDEV_IO_DONE =3D 1 << 18, /* RW--- */=0A= - ZIO_STAGE_VDEV_IO_ASSESS =3D 1 << 19, /* RW--I */=0A= + ZIO_STAGE_VDEV_IO_START =3D 1 << 17, /* RWF-I */=0A= + ZIO_STAGE_VDEV_IO_DONE =3D 1 << 18, /* RWF-- */=0A= + ZIO_STAGE_VDEV_IO_ASSESS =3D 1 << 19, /* RWF-I */=0A= =0A= ZIO_STAGE_CHECKSUM_VERIFY =3D 1 << 20, /* R---- */=0A= =0A= @@ -214,7 +214,9 @@=0A= (ZIO_INTERLOCK_STAGES | \=0A= ZIO_STAGE_FREE_BP_INIT | \=0A= ZIO_STAGE_ISSUE_ASYNC | \=0A= - ZIO_STAGE_DVA_FREE)=0A= + ZIO_STAGE_DVA_FREE | \=0A= + ZIO_STAGE_VDEV_IO_START | \=0A= + ZIO_STAGE_VDEV_IO_ASSESS)=0A= =0A= #define ZIO_DDT_FREE_PIPELINE \=0A= (ZIO_INTERLOCK_STAGES | \=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/vdev_impl.h = (working copy)=0A= @@ -183,6 +183,7 @@=0A= uint64_t vdev_unspare; /* unspare when resilvering done */=0A= hrtime_t vdev_last_try; /* last reopen time */=0A= boolean_t vdev_nowritecache; /* true if flushwritecache failed */=0A= + boolean_t vdev_notrim; /* true if trim failed */=0A= boolean_t vdev_checkremove; /* temporary online test */=0A= boolean_t vdev_forcefault; /* force online fault */=0A= boolean_t vdev_splitting; /* split or repair in progress */=0A= @@ -198,6 +199,7 @@=0A= spa_aux_vdev_t *vdev_aux; /* for l2cache vdevs */=0A= zio_t *vdev_probe_zio; /* root of current probe */=0A= vdev_aux_t vdev_label_aux; /* on-disk aux state */=0A= + struct trim_map *vdev_trimmap;=0A= =0A= /*=0A= * For DTrace to work in userland (libzpool) context, these fields must=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/zio.h (working = copy)=0A= @@ -32,6 +32,7 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= #include =0A= #include =0A= =0A= @@ -137,7 +138,8 @@=0A= #define ZIO_PRIORITY_RESILVER (zio_priority_table[9])=0A= #define ZIO_PRIORITY_SCRUB (zio_priority_table[10])=0A= #define ZIO_PRIORITY_DDT_PREFETCH (zio_priority_table[11])=0A= -#define ZIO_PRIORITY_TABLE_SIZE 12=0A= +#define ZIO_PRIORITY_TRIM (zio_priority_table[12])=0A= +#define ZIO_PRIORITY_TABLE_SIZE 13=0A= =0A= #define ZIO_PIPELINE_CONTINUE 0x100=0A= #define ZIO_PIPELINE_STOP 0x101=0A= @@ -367,6 +369,39 @@=0A= list_node_t zl_child_node;=0A= } zio_link_t;=0A= =0A= +/*=0A= + * Used for TRIM kstat.=0A= + */=0A= +typedef struct zio_trim_stats {=0A= + /*=0A= + * Number of bytes successfully TRIMmed.=0A= + */=0A= + kstat_named_t bytes;=0A= +=0A= + /*=0A= + * Number of successful TRIM requests.=0A= + */=0A= + kstat_named_t success;=0A= +=0A= + /*=0A= + * Number of TRIM requests that failed because TRIM is not=0A= + * supported.=0A= + */=0A= + kstat_named_t unsupported;=0A= +=0A= + /*=0A= + * Number of TRIM requests that failed for other reasons.=0A= + */=0A= + kstat_named_t failed;=0A= +} zio_trim_stats_t;=0A= +=0A= +extern zio_trim_stats_t zio_trim_stats;=0A= +=0A= +#define ZIO_TRIM_STAT_INCR(stat, val) \=0A= + atomic_add_64(&zio_trim_stats.stat.value.ui64, (val));=0A= +#define ZIO_TRIM_STAT_BUMP(stat) \=0A= + ZIO_TRIM_STAT_INCR(stat, 1);=0A= +=0A= struct zio {=0A= /* Core information about this I/O */=0A= zbookmark_t io_bookmark;=0A= @@ -441,6 +476,8 @@=0A= /* FreeBSD only. */=0A= struct ostask io_task;=0A= #endif=0A= + avl_node_t io_trim_node;=0A= + list_node_t io_trim_link;=0A= };=0A= =0A= extern zio_t *zio_null(zio_t *pio, spa_t *spa, vdev_t *vd,=0A= @@ -472,8 +509,8 @@=0A= zio_done_func_t *done, void *priv, enum zio_flag flags);=0A= =0A= extern zio_t *zio_ioctl(zio_t *pio, spa_t *spa, vdev_t *vd, int cmd,=0A= - zio_done_func_t *done, void *priv, int priority,=0A= - enum zio_flag flags);=0A= + uint64_t offset, uint64_t size, zio_done_func_t *done, void *priv,=0A= + int priority, enum zio_flag flags);=0A= =0A= extern zio_t *zio_read_phys(zio_t *pio, vdev_t *vd, uint64_t offset,=0A= uint64_t size, void *data, int checksum,=0A= @@ -486,12 +523,14 @@=0A= boolean_t labels);=0A= =0A= extern zio_t *zio_free_sync(zio_t *pio, spa_t *spa, uint64_t txg,=0A= - const blkptr_t *bp, enum zio_flag flags);=0A= + const blkptr_t *bp, uint64_t size, enum zio_flag flags);=0A= =0A= extern int zio_alloc_zil(spa_t *spa, uint64_t txg, blkptr_t *new_bp,=0A= blkptr_t *old_bp, uint64_t size, boolean_t use_slog);=0A= extern void zio_free_zil(spa_t *spa, uint64_t txg, blkptr_t *bp);=0A= extern void zio_flush(zio_t *zio, vdev_t *vd);=0A= +extern zio_t *zio_trim(zio_t *zio, spa_t *spa, vdev_t *vd, uint64_t = offset,=0A= + uint64_t size);=0A= extern void zio_shrink(zio_t *zio, uint64_t size);=0A= =0A= extern int zio_wait(zio_t *zio);=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c (working = copy)=0A= @@ -49,14 +49,17 @@=0A= =0A= DECLARE_GEOM_CLASS(zfs_vdev_class, zfs_vdev);=0A= =0A= -/*=0A= - * Don't send BIO_FLUSH.=0A= - */=0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +/* Don't send BIO_FLUSH. */=0A= static int vdev_geom_bio_flush_disable =3D 0;=0A= TUNABLE_INT("vfs.zfs.vdev.bio_flush_disable", = &vdev_geom_bio_flush_disable);=0A= -SYSCTL_DECL(_vfs_zfs_vdev);=0A= SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, bio_flush_disable, CTLFLAG_RW,=0A= &vdev_geom_bio_flush_disable, 0, "Disable BIO_FLUSH");=0A= +/* Don't send BIO_DELETE. */=0A= +static int vdev_geom_bio_delete_disable =3D 0;=0A= +TUNABLE_INT("vfs.zfs.vdev.bio_delete_disable", = &vdev_geom_bio_delete_disable);=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, bio_delete_disable, CTLFLAG_RW,=0A= + &vdev_geom_bio_delete_disable, 0, "Disable BIO_DELETE");=0A= =0A= static void=0A= vdev_geom_orphan(struct g_consumer *cp)=0A= @@ -663,8 +666,8 @@=0A= *ashift =3D highbit(MAX(pp->sectorsize, SPA_MINBLOCKSIZE)) - 1;=0A= =0A= /*=0A= - * Clear the nowritecache bit, so that on a vdev_reopen() we will=0A= - * try again.=0A= + * Clear the nowritecache settings, so that on a vdev_reopen()=0A= + * we will try again.=0A= */=0A= vd->vdev_nowritecache =3D B_FALSE;=0A= =0A= @@ -710,6 +713,15 @@=0A= */=0A= vd->vdev_nowritecache =3D B_TRUE;=0A= }=0A= + if (bp->bio_cmd =3D=3D BIO_DELETE && bp->bio_error =3D=3D ENOTSUP) {=0A= + /*=0A= + * If we get ENOTSUP, we know that no future=0A= + * attempts will ever succeed. In this case we=0A= + * set a persistent bit so that we don't bother=0A= + * with the ioctl in the future.=0A= + */=0A= + vd->vdev_notrim =3D B_TRUE;=0A= + }=0A= if (zio->io_error =3D=3D EIO && !vd->vdev_remove_wanted) {=0A= /*=0A= * If provider's error is set we assume it is being=0A= @@ -752,18 +764,22 @@=0A= }=0A= =0A= switch (zio->io_cmd) {=0A= -=0A= case DKIOCFLUSHWRITECACHE:=0A= -=0A= if (zfs_nocacheflush || vdev_geom_bio_flush_disable)=0A= break;=0A= -=0A= if (vd->vdev_nowritecache) {=0A= zio->io_error =3D ENOTSUP;=0A= break;=0A= }=0A= -=0A= goto sendreq;=0A= + case DKIOCTRIM:=0A= + if (vdev_geom_bio_delete_disable)=0A= + break;=0A= + if (vd->vdev_notrim) {=0A= + zio->io_error =3D ENOTSUP;=0A= + break;=0A= + }=0A= + goto sendreq;=0A= default:=0A= zio->io_error =3D ENOTSUP;=0A= }=0A= @@ -787,11 +803,21 @@=0A= bp->bio_length =3D zio->io_size;=0A= break;=0A= case ZIO_TYPE_IOCTL:=0A= - bp->bio_cmd =3D BIO_FLUSH;=0A= - bp->bio_flags |=3D BIO_ORDERED;=0A= - bp->bio_data =3D NULL;=0A= - bp->bio_offset =3D cp->provider->mediasize;=0A= - bp->bio_length =3D 0;=0A= + switch (zio->io_cmd) {=0A= + case DKIOCFLUSHWRITECACHE:=0A= + bp->bio_cmd =3D BIO_FLUSH;=0A= + bp->bio_flags |=3D BIO_ORDERED;=0A= + bp->bio_data =3D NULL;=0A= + bp->bio_offset =3D cp->provider->mediasize;=0A= + bp->bio_length =3D 0;=0A= + break;=0A= + case DKIOCTRIM:=0A= + bp->bio_cmd =3D BIO_DELETE;=0A= + bp->bio_data =3D NULL;=0A= + bp->bio_offset =3D zio->io_offset;=0A= + bp->bio_length =3D zio->io_size;=0A= + break;=0A= + }=0A= break;=0A= }=0A= bp->bio_done =3D vdev_geom_io_intr;=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c (working copy)=0A= @@ -67,6 +67,7 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= =0A= #ifdef _KERNEL=0A= #include =0A= @@ -1001,6 +1002,11 @@=0A= spa_create_zio_taskqs(spa);=0A= }=0A= =0A= + /*=0A= + * Start TRIM thread.=0A= + */=0A= + trim_thread_create(spa);=0A= +=0A= list_create(&spa->spa_config_dirty_list, sizeof (vdev_t),=0A= offsetof(vdev_t, vdev_config_dirty_node));=0A= list_create(&spa->spa_state_dirty_list, sizeof (vdev_t),=0A= @@ -1029,6 +1035,12 @@=0A= ASSERT(spa->spa_async_zio_root =3D=3D NULL);=0A= ASSERT(spa->spa_state !=3D POOL_STATE_UNINITIALIZED);=0A= =0A= + /*=0A= + * Stop TRIM thread in case spa_unload() wasn't called directly=0A= + * before spa_deactivate().=0A= + */=0A= + trim_thread_destroy(spa);=0A= +=0A= txg_list_destroy(&spa->spa_vdev_txg_list);=0A= =0A= list_destroy(&spa->spa_config_dirty_list);=0A= @@ -1145,6 +1157,11 @@=0A= ASSERT(MUTEX_HELD(&spa_namespace_lock));=0A= =0A= /*=0A= + * Stop TRIM thread.=0A= + */=0A= + trim_thread_destroy(spa);=0A= +=0A= + /*=0A= * Stop async tasks.=0A= */=0A= spa_async_suspend(spa);=0A= @@ -5875,7 +5892,7 @@=0A= zio_t *zio =3D arg;=0A= =0A= zio_nowait(zio_free_sync(zio, zio->io_spa, dmu_tx_get_txg(tx), bp,=0A= - zio->io_flags));=0A= + BP_GET_PSIZE(bp), zio->io_flags));=0A= return (0);=0A= }=0A= =0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c (working copy)=0A= @@ -130,6 +130,7 @@=0A= #endif=0A= #include =0A= #include =0A= +#include =0A= #include =0A= #include =0A= =0A= @@ -1691,6 +1692,8 @@=0A= }=0A= =0A= if (l2hdr !=3D NULL) {=0A= + trim_map_free(l2hdr->b_dev->l2ad_vdev, l2hdr->b_daddr,=0A= + hdr->b_size, 0);=0A= list_remove(l2hdr->b_dev->l2ad_buflist, hdr);=0A= ARCSTAT_INCR(arcstat_l2_size, -hdr->b_size);=0A= kmem_free(l2hdr, sizeof (l2arc_buf_hdr_t));=0A= @@ -3528,6 +3531,8 @@=0A= buf->b_private =3D NULL;=0A= =0A= if (l2hdr) {=0A= + trim_map_free(l2hdr->b_dev->l2ad_vdev, l2hdr->b_daddr,=0A= + hdr->b_size, 0);=0A= list_remove(l2hdr->b_dev->l2ad_buflist, hdr);=0A= kmem_free(l2hdr, sizeof (l2arc_buf_hdr_t));=0A= ARCSTAT_INCR(arcstat_l2_size, -buf_size);=0A= @@ -4442,6 +4447,8 @@=0A= list_remove(buflist, ab);=0A= abl2 =3D ab->b_l2hdr;=0A= ab->b_l2hdr =3D NULL;=0A= + trim_map_free(abl2->b_dev->l2ad_vdev, abl2->b_daddr,=0A= + ab->b_size, 0);=0A= kmem_free(abl2, sizeof (l2arc_buf_hdr_t));=0A= ARCSTAT_INCR(arcstat_l2_size, -ab->b_size);=0A= }=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (revision = 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c (working copy)=0A= @@ -35,6 +35,7 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= =0A= SYSCTL_DECL(_vfs_zfs);=0A= SYSCTL_NODE(_vfs_zfs, OID_AUTO, zio, CTLFLAG_RW, 0, "ZFS ZIO");=0A= @@ -43,6 +44,19 @@=0A= SYSCTL_INT(_vfs_zfs_zio, OID_AUTO, use_uma, CTLFLAG_RDTUN, = &zio_use_uma, 0,=0A= "Use uma(9) for ZIO allocations");=0A= =0A= +zio_trim_stats_t zio_trim_stats =3D {=0A= + { "bytes", KSTAT_DATA_UINT64,=0A= + "Number of bytes successfully TRIMmed" },=0A= + { "success", KSTAT_DATA_UINT64,=0A= + "Number of successful TRIM requests" },=0A= + { "unsupported", KSTAT_DATA_UINT64,=0A= + "Number of TRIM requests that failed because TRIM is not supported" = },=0A= + { "failed", KSTAT_DATA_UINT64,=0A= + "Number of TRIM requests that failed for reasons other than not = supported" },=0A= +};=0A= +=0A= +static kstat_t *zio_trim_ksp;=0A= +=0A= /*=0A= * = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= * I/O priority table=0A= @@ -61,6 +75,7 @@=0A= 10, /* ZIO_PRIORITY_RESILVER */=0A= 20, /* ZIO_PRIORITY_SCRUB */=0A= 2, /* ZIO_PRIORITY_DDT_PREFETCH */=0A= + 30, /* ZIO_PRIORITY_TRIM */=0A= };=0A= =0A= /*=0A= @@ -209,6 +224,16 @@=0A= zfs_mg_alloc_failures =3D 8;=0A= =0A= zio_inject_init();=0A= +=0A= + zio_trim_ksp =3D kstat_create("zfs", 0, "zio_trim", "misc",=0A= + KSTAT_TYPE_NAMED,=0A= + sizeof(zio_trim_stats) / sizeof(kstat_named_t),=0A= + KSTAT_FLAG_VIRTUAL);=0A= +=0A= + if (zio_trim_ksp !=3D NULL) {=0A= + zio_trim_ksp->ks_data =3D &zio_trim_stats;=0A= + kstat_install(zio_trim_ksp);=0A= + }=0A= }=0A= =0A= void=0A= @@ -236,6 +261,11 @@=0A= kmem_cache_destroy(zio_cache);=0A= =0A= zio_inject_fini();=0A= +=0A= + if (zio_trim_ksp !=3D NULL) {=0A= + kstat_delete(zio_trim_ksp);=0A= + zio_trim_ksp =3D NULL;=0A= + }=0A= }=0A= =0A= /*=0A= @@ -543,7 +573,7 @@=0A= {=0A= zio_t *zio;=0A= =0A= - ASSERT3U(size, <=3D, SPA_MAXBLOCKSIZE);=0A= + ASSERT3U(type =3D=3D ZIO_TYPE_FREE || size, <=3D, SPA_MAXBLOCKSIZE);=0A= ASSERT(P2PHASE(size, SPA_MINBLOCKSIZE) =3D=3D 0);=0A= ASSERT(P2PHASE(offset, SPA_MINBLOCKSIZE) =3D=3D 0);=0A= =0A= @@ -730,7 +760,7 @@=0A= =0A= zio_t *=0A= zio_free_sync(zio_t *pio, spa_t *spa, uint64_t txg, const blkptr_t *bp,=0A= - enum zio_flag flags)=0A= + uint64_t size, enum zio_flag flags)=0A= {=0A= zio_t *zio;=0A= =0A= @@ -743,7 +773,7 @@=0A= =0A= metaslab_check_free(spa, bp);=0A= =0A= - zio =3D zio_create(pio, spa, txg, bp, NULL, BP_GET_PSIZE(bp),=0A= + zio =3D zio_create(pio, spa, txg, bp, NULL, size,=0A= NULL, NULL, ZIO_TYPE_FREE, ZIO_PRIORITY_FREE, flags,=0A= NULL, 0, NULL, ZIO_STAGE_OPEN, ZIO_FREE_PIPELINE);=0A= =0A= @@ -780,15 +810,16 @@=0A= }=0A= =0A= zio_t *=0A= -zio_ioctl(zio_t *pio, spa_t *spa, vdev_t *vd, int cmd,=0A= - zio_done_func_t *done, void *private, int priority, enum zio_flag = flags)=0A= +zio_ioctl(zio_t *pio, spa_t *spa, vdev_t *vd, int cmd, uint64_t offset,=0A= + uint64_t size, zio_done_func_t *done, void *private, int priority,=0A= + enum zio_flag flags)=0A= {=0A= zio_t *zio;=0A= int c;=0A= =0A= if (vd->vdev_children =3D=3D 0) {=0A= - zio =3D zio_create(pio, spa, 0, NULL, NULL, 0, done, private,=0A= - ZIO_TYPE_IOCTL, priority, flags, vd, 0, NULL,=0A= + zio =3D zio_create(pio, spa, 0, NULL, NULL, size, done, private,=0A= + ZIO_TYPE_IOCTL, priority, flags, vd, offset, NULL,=0A= ZIO_STAGE_OPEN, ZIO_IOCTL_PIPELINE);=0A= =0A= zio->io_cmd =3D cmd;=0A= @@ -797,7 +828,7 @@=0A= =0A= for (c =3D 0; c < vd->vdev_children; c++)=0A= zio_nowait(zio_ioctl(zio, spa, vd->vdev_child[c], cmd,=0A= - done, private, priority, flags));=0A= + offset, size, done, private, priority, flags));=0A= }=0A= =0A= return (zio);=0A= @@ -922,11 +953,22 @@=0A= void=0A= zio_flush(zio_t *zio, vdev_t *vd)=0A= {=0A= - zio_nowait(zio_ioctl(zio, zio->io_spa, vd, DKIOCFLUSHWRITECACHE,=0A= + zio_nowait(zio_ioctl(zio, zio->io_spa, vd, DKIOCFLUSHWRITECACHE, 0, 0,=0A= NULL, NULL, ZIO_PRIORITY_NOW,=0A= ZIO_FLAG_CANFAIL | ZIO_FLAG_DONT_PROPAGATE | ZIO_FLAG_DONT_RETRY));=0A= }=0A= =0A= +zio_t *=0A= +zio_trim(zio_t *zio, spa_t *spa, vdev_t *vd, uint64_t offset, uint64_t = size)=0A= +{=0A= +=0A= + ASSERT(vd->vdev_ops->vdev_op_leaf);=0A= +=0A= + return zio_ioctl(zio, spa, vd, DKIOCTRIM, offset, size,=0A= + NULL, NULL, ZIO_PRIORITY_TRIM,=0A= + ZIO_FLAG_CANFAIL | ZIO_FLAG_DONT_PROPAGATE | ZIO_FLAG_DONT_RETRY);=0A= +}=0A= +=0A= void=0A= zio_shrink(zio_t *zio, uint64_t size)=0A= {=0A= @@ -1549,6 +1591,7 @@=0A= zio_free_gang(zio_t *pio, blkptr_t *bp, zio_gang_node_t *gn, void *data)=0A= {=0A= return (zio_free_sync(pio, pio->io_spa, pio->io_txg, bp,=0A= + BP_IS_GANG(bp) ? SPA_GANGBLOCKSIZE : BP_GET_PSIZE(bp),=0A= ZIO_GANG_CHILD_FLAGS(pio)));=0A= }=0A= =0A= @@ -1681,7 +1724,7 @@=0A= }=0A= }=0A= =0A= - if (gn =3D=3D gio->io_gang_tree)=0A= + if (gn =3D=3D gio->io_gang_tree && gio->io_data !=3D NULL)=0A= ASSERT3P((char *)gio->io_data + gio->io_size, =3D=3D, data);=0A= =0A= if (zio !=3D pio)=0A= @@ -2403,7 +2446,7 @@=0A= =0A= /*=0A= * = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= - * Read and write to physical devices=0A= + * Read, write and delete to physical devices=0A= * = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= */=0A= static int=0A= @@ -2426,6 +2469,11 @@=0A= return (vdev_mirror_ops.vdev_op_io_start(zio));=0A= }=0A= =0A= + if (vd->vdev_ops->vdev_op_leaf && zio->io_type =3D=3D ZIO_TYPE_FREE) {=0A= + trim_map_free(vd, zio->io_offset, zio->io_size, zio->io_txg);=0A= + return (ZIO_PIPELINE_CONTINUE);=0A= + }=0A= +=0A= /*=0A= * We keep track of time-sensitive I/Os so that the scan thread=0A= * can quickly react to certain workloads. In particular, we care=0A= @@ -2450,18 +2498,22 @@=0A= =0A= if (P2PHASE(zio->io_size, align) !=3D 0) {=0A= uint64_t asize =3D P2ROUNDUP(zio->io_size, align);=0A= - char *abuf =3D zio_buf_alloc(asize);=0A= + char *abuf =3D NULL;=0A= + if (zio->io_type =3D=3D ZIO_TYPE_READ ||=0A= + zio->io_type =3D=3D ZIO_TYPE_WRITE)=0A= + abuf =3D zio_buf_alloc(asize);=0A= ASSERT(vd =3D=3D vd->vdev_top);=0A= if (zio->io_type =3D=3D ZIO_TYPE_WRITE) {=0A= bcopy(zio->io_data, abuf, zio->io_size);=0A= bzero(abuf + zio->io_size, asize - zio->io_size);=0A= }=0A= - zio_push_transform(zio, abuf, asize, asize, zio_subblock);=0A= + zio_push_transform(zio, abuf, asize, abuf ? asize : 0,=0A= + zio_subblock);=0A= }=0A= =0A= ASSERT(P2PHASE(zio->io_offset, align) =3D=3D 0);=0A= ASSERT(P2PHASE(zio->io_size, align) =3D=3D 0);=0A= - VERIFY(zio->io_type !=3D ZIO_TYPE_WRITE || spa_writeable(spa));=0A= + VERIFY(zio->io_type =3D=3D ZIO_TYPE_READ || spa_writeable(spa));=0A= =0A= /*=0A= * If this is a repair I/O, and there's no self-healing involved --=0A= @@ -2501,6 +2553,11 @@=0A= }=0A= }=0A= =0A= + if (vd->vdev_ops->vdev_op_leaf && zio->io_type =3D=3D ZIO_TYPE_WRITE) {=0A= + if (!trim_map_write_start(zio))=0A= + return (ZIO_PIPELINE_STOP);=0A= + }=0A= +=0A= return (vd->vdev_ops->vdev_op_io_start(zio));=0A= }=0A= =0A= @@ -2514,10 +2571,17 @@=0A= if (zio_wait_for_children(zio, ZIO_CHILD_VDEV, ZIO_WAIT_DONE))=0A= return (ZIO_PIPELINE_STOP);=0A= =0A= - ASSERT(zio->io_type =3D=3D ZIO_TYPE_READ || zio->io_type =3D=3D = ZIO_TYPE_WRITE);=0A= + ASSERT(zio->io_type =3D=3D ZIO_TYPE_READ ||=0A= + zio->io_type =3D=3D ZIO_TYPE_WRITE || zio->io_type =3D=3D = ZIO_TYPE_FREE);=0A= =0A= - if (vd !=3D NULL && vd->vdev_ops->vdev_op_leaf) {=0A= + if (vd !=3D NULL && vd->vdev_ops->vdev_op_leaf &&=0A= + zio->io_type =3D=3D ZIO_TYPE_WRITE) {=0A= + trim_map_write_done(zio);=0A= + }=0A= =0A= + if (vd !=3D NULL && vd->vdev_ops->vdev_op_leaf &&=0A= + (zio->io_type =3D=3D ZIO_TYPE_READ || zio->io_type =3D=3D = ZIO_TYPE_WRITE)) {=0A= +=0A= vdev_queue_io_done(zio);=0A= =0A= if (zio->io_type =3D=3D ZIO_TYPE_WRITE)=0A= @@ -2592,6 +2656,20 @@=0A= if (zio_injection_enabled && zio->io_error =3D=3D 0)=0A= zio->io_error =3D zio_handle_fault_injection(zio, EIO);=0A= =0A= + if (zio->io_type =3D=3D ZIO_TYPE_IOCTL && zio->io_cmd =3D=3D DKIOCTRIM)=0A= + switch (zio->io_error) {=0A= + case 0:=0A= + ZIO_TRIM_STAT_INCR(bytes, zio->io_size);=0A= + ZIO_TRIM_STAT_BUMP(success);=0A= + break;=0A= + case EOPNOTSUPP:=0A= + ZIO_TRIM_STAT_BUMP(unsupported);=0A= + break;=0A= + default:=0A= + ZIO_TRIM_STAT_BUMP(failed);=0A= + break;=0A= + }=0A= +=0A= /*=0A= * If the I/O failed, determine whether we should attempt to retry it.=0A= *=0A= Index: sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c = (revision 250526)=0A= +++ sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_label.c (working = copy)=0A= @@ -145,8 +145,14 @@=0A= #include =0A= #include =0A= #include =0A= +#include =0A= #include =0A= =0A= +static boolean_t vdev_trim_on_init =3D B_TRUE;=0A= +SYSCTL_DECL(_vfs_zfs_vdev);=0A= +SYSCTL_INT(_vfs_zfs_vdev, OID_AUTO, trim_on_init, CTLFLAG_RW,=0A= + &vdev_trim_on_init, 0, "Enable/disable full vdev trim on = initialisation");=0A= +=0A= /*=0A= * Basic routines to read and write from a vdev label.=0A= * Used throughout the rest of this file.=0A= @@ -718,6 +724,16 @@=0A= }=0A= =0A= /*=0A= + * TRIM the whole thing so that we start with a clean slate.=0A= + * It's just an optimization, so we don't care if it fails.=0A= + * Don't TRIM if removing so that we don't interfere with zpool=0A= + * disaster recovery.=0A= + */=0A= + if (zfs_trim_enabled && vdev_trim_on_init && (reason =3D=3D = VDEV_LABEL_CREATE ||=0A= + reason =3D=3D VDEV_LABEL_SPARE || reason =3D=3D = VDEV_LABEL_L2CACHE))=0A= + zio_wait(zio_trim(NULL, spa, vd, 0, vd->vdev_psize));=0A= +=0A= + /*=0A= * Initialize its label.=0A= */=0A= vp =3D zio_buf_alloc(sizeof (vdev_phys_t));=0A= @@ -1282,5 +1298,10 @@=0A= * to disk to ensure that all odd-label updates are committed to=0A= * stable storage before the next transaction group begins.=0A= */=0A= - return (vdev_label_sync_list(spa, 1, txg, flags));=0A= + if ((error =3D vdev_label_sync_list(spa, 1, txg, flags)) !=3D 0)=0A= + return (error);=0A= +=0A= + trim_thread_wakeup(spa);=0A= +=0A= + return (0);=0A= }=0A= Index: sys/cddl/compat/opensolaris/kern/opensolaris_kstat.c=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/compat/opensolaris/kern/opensolaris_kstat.c (revision = 250526)=0A= +++ sys/cddl/compat/opensolaris/kern/opensolaris_kstat.c (working copy)=0A= @@ -118,7 +118,7 @@=0A= SYSCTL_ADD_PROC(&ksp->ks_sysctl_ctx,=0A= SYSCTL_CHILDREN(ksp->ks_sysctl_root), OID_AUTO, ksent->name,=0A= CTLTYPE_U64 | CTLFLAG_RD, ksent, sizeof(*ksent),=0A= - kstat_sysctl, "QU", "");=0A= + kstat_sysctl, "QU", ksent->desc);=0A= }=0A= }=0A= =0A= Index: sys/cddl/compat/opensolaris/sys/dkio.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/compat/opensolaris/sys/dkio.h (revision 250526)=0A= +++ sys/cddl/compat/opensolaris/sys/dkio.h (working copy)=0A= @@ -75,6 +75,8 @@=0A= */=0A= #define DKIOCFLUSHWRITECACHE (DKIOC|34) /* flush cache to phys medium */=0A= =0A= +#define DKIOCTRIM (DKIOC|35) /* TRIM a block */=0A= +=0A= struct dk_callback {=0A= void (*dkc_callback)(void *dkc_cookie, int error);=0A= void *dkc_cookie;=0A= Index: sys/cddl/compat/opensolaris/sys/time.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/compat/opensolaris/sys/time.h (revision 250526)=0A= +++ sys/cddl/compat/opensolaris/sys/time.h (working copy)=0A= @@ -35,6 +35,7 @@=0A= #define MILLISEC 1000=0A= #define MICROSEC 1000000=0A= #define NANOSEC 1000000000=0A= +#define TIME_MAX LLONG_MAX=0A= =0A= typedef longlong_t hrtime_t;=0A= =0A= Index: sys/cddl/compat/opensolaris/sys/kstat.h=0A= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A= --- sys/cddl/compat/opensolaris/sys/kstat.h (revision 250526)=0A= +++ sys/cddl/compat/opensolaris/sys/kstat.h (working copy)=0A= @@ -53,6 +53,8 @@=0A= #define KSTAT_DATA_INT64 3=0A= #define KSTAT_DATA_UINT64 4=0A= uchar_t data_type;=0A= +#define KSTAT_DESCLEN 128=0A= + char desc[KSTAT_DESCLEN];=0A= union {=0A= uint64_t ui64;=0A= } value;=0A= ------=_NextPart_000_0391_01CE581A.96C6FBA0-- From owner-freebsd-fs@FreeBSD.ORG Fri May 24 13:28:25 2013 Return-Path: Delivered-To: freebsd-fs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 1B36E3D5; Fri, 24 May 2013 13:28:25 +0000 (UTC) (envelope-from linimon@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id E941BE1B; Fri, 24 May 2013 13:28:24 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4ODSOIv074151; Fri, 24 May 2013 13:28:24 GMT (envelope-from linimon@freefall.freebsd.org) Received: (from linimon@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4ODSOgl074150; Fri, 24 May 2013 13:28:24 GMT (envelope-from linimon) Date: Fri, 24 May 2013 13:28:24 GMT Message-Id: <201305241328.r4ODSOgl074150@freefall.freebsd.org> To: linimon@FreeBSD.org, freebsd-bugs@FreeBSD.org, freebsd-fs@FreeBSD.org From: linimon@FreeBSD.org Subject: Re: kern/178854: [ufs] FreeBSD kernel crash in UFS X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 May 2013 13:28:25 -0000 Old Synopsis: FreeBSD kernel crash in UFS New Synopsis: [ufs] FreeBSD kernel crash in UFS Responsible-Changed-From-To: freebsd-bugs->freebsd-fs Responsible-Changed-By: linimon Responsible-Changed-When: Fri May 24 13:28:04 UTC 2013 Responsible-Changed-Why: Over to maintainer(s). http://www.freebsd.org/cgi/query-pr.cgi?pr=178854 From owner-freebsd-fs@FreeBSD.ORG Fri May 24 17:56:32 2013 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id C9F872BF for ; Fri, 24 May 2013 17:56:32 +0000 (UTC) (envelope-from steve@mouf.net) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) by mx1.freebsd.org (Postfix) with ESMTP id 9225162F for ; Fri, 24 May 2013 17:56:32 +0000 (UTC) Received: from mouf.net (www@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id r4OHuLZq014935 for ; Fri, 24 May 2013 17:56:26 GMT (envelope-from steve@mouf.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=mouf.net; s=mail; t=1369418191; bh=kJmmlIaZ/1RYxSyPEOLb8B5xxAFj/i+5o4jkbwg7Z0k=; h=Date:Subject:From:To; b=FSJ4Rz5lNvTFT7R9nK8q/bgzJdS7+blrptwh6v6fOHOFZ66bl1jnUhQ850/M31hHK g1zCuAwTdoLTMSU3EuvvH7WUpKT2FPozt9H0uX4XqesfxhH+NgCS+QrWavpSvjEaQ9 vzRee+CEarhmxXirvnjxTtq38T+3+YRTw5WrwAig= Received: from 64.128.208.27 (SquirrelMail authenticated user swills) by mouf.net with HTTP; Fri, 24 May 2013 13:56:26 -0400 Message-ID: <8ea8b9c8074fd122f78c5eaa3b289805.squirrel@mouf.net> Date: Fri, 24 May 2013 13:56:26 -0400 Subject: dev entries for cloned zvol don't show up until after reboot From: "Steve Wills" To: fs@freebsd.org User-Agent: SquirrelMail/1.4.22 MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Fri, 24 May 2013 17:56:26 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=none autolearn=unavailable version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on mouf.net X-Virus-Scanned: clamav-milter 0.97.7 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 24 May 2013 17:56:32 -0000 Hi, I've noticed that if I make zvol, create a snapshot of it, then clone that, the /dev/zvol/* entries for it don't show up until after I reboot. This is on r250925. Is this a known bug? Thanks, Steve