From owner-freebsd-fs@freebsd.org Mon Oct 2 19:16:13 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C6EA2E25349 for ; Mon, 2 Oct 2017 19:16:13 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wr0-x22d.google.com (mail-wr0-x22d.google.com [IPv6:2a00:1450:400c:c0c::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 58CB374355 for ; Mon, 2 Oct 2017 19:16:13 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wr0-x22d.google.com with SMTP id b21so4767757wrg.7 for ; Mon, 02 Oct 2017 12:16:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=3UwtB7WqZlN1pNLFVvcoI3qM0qz8MSfx16i5RsyGC9A=; b=dDPvmJ49isQHSea7TnVKHScHUS6I+LY81Eu4cDm/sO9djAC6A8+PSyHA216dEV7cuH LxzW9CGrCnCjK7OPy7Bd1uSeiAn29jN2K2qhZ82IUcabNsknrNxWpb8a8jQkhGAGWPQu wgYoCVJgLB8dvkHLWBBO6bJNLGtKj/fd6eG3pNz7Fhkfr9dKkoeuvBINZRwTp7e61/bO H+0nVOM64Jy1f1kULs9jj3M0ug7vggvekAnlsdfD/gnuGAwL+GGyDCkBX2codiCH1ibd Uy2+jpTJnkXuFsxE0gLdHi5nQqfYwM99CRHMZ0+kgJ9shC1c+68N5CUVL+OCPSEOOt+k e49Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=3UwtB7WqZlN1pNLFVvcoI3qM0qz8MSfx16i5RsyGC9A=; b=rkEigBC6bMkVj5BtesmA7qkMphMntXnSbjeW/LWFhYTwmKri/nVo6PT9S99Xi2GLhw WVnQsHh0UK1mRgIvSARO03Gxc/yx+PRc6dxf2+mn7EsIA+zgiHEaaojuYJ0JT4KUht9+ vjT2ST88JtQnqr1KMBZVWdiR77p71CLZl0WPntHjB6+MS85YfdBCBNpTJRe3nIC1Scda Nd/enp16m2Zc2TuWIlYnMJ9dYtcQAMDP/zFA2/dZA1SXjUM0r8iDQsYvRIxJc5/U+fbA iMX0DPVtSWH3a8LOQ4DeOXi7/IrkzXqK6IsbdCmrrgWdWVHSMelN+VX/yA4q1B9m/8Ag aVUw== X-Gm-Message-State: AMCzsaU7s9W++dEMOp/vxKyYpRzzYPXJxHQodzf4gTeCJBR4AmnRwatT CXo97RwvAUq80G/y+u+Q6MJI9dtg X-Google-Smtp-Source: AOwi7QCx0qxmJQ2z3icbzYCWRlLkOq7TuYK4On73GfYUDTIBlwWkVcztQCzAA+uUaWCovhpHLiTGhA== X-Received: by 10.223.192.138 with SMTP id d10mr5005143wrf.6.1506971771746; Mon, 02 Oct 2017 12:16:11 -0700 (PDT) Received: from bens-mac.home (LFbn-MAR-1-445-220.w2-15.abo.wanadoo.fr. [2.15.38.220]) by smtp.gmail.com with ESMTPSA id w18sm8919756wra.61.2017.10.02.12.16.10 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 02 Oct 2017 12:16:11 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: ZFS stalled after some mirror disks were lost From: Ben RUBSON In-Reply-To: Date: Mon, 2 Oct 2017 21:16:10 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <4A0E9EB8-57EA-4E76-9D7E-3E344B2037D2@gmail.com> To: Freebsd fs X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 02 Oct 2017 19:16:13 -0000 > On 02 Oct 2017, at 20:44, Adam Vande More = wrote: >=20 >> On Mon, Oct 2, 2017 at 1:12 PM, Ben RUBSON = wrote: >> Hi, >>=20 >> On a FreeBSD 11 server, the following online/healthy zpool : >>=20 >> home >> mirror-0 >> label/local1 >> label/local2 >> label/iscsi1 >> label/iscsi2 >> mirror-1 >> label/local3 >> label/local4 >> label/iscsi3 >> label/iscsi4 >> cache >> label/local5 >> label/local6 >>=20 >> A sustained read throughput of 180 MB/s, 45 MB/s on each iscsi disk >> according to "zpool iostat", nothing on local disks (strange but I >> noticed that IOs always prefer iscsi disks to local disks). >> No write IOs. >>=20 >> Let's disconnect all iSCSI disks : >> iscsictl -Ra >>=20 >> Expected behavior : >> IO activity flawlessly continue on local disks. >=20 > Perhaps I'm misunderstanding your setup, but my expected behavior = would be exactly what you see.=20 Unfortunately, what I see is the following quoted : >> What happened : >> All IOs stalled, server only answers to IOs are made to its zroot = pool. >> All commands related to the iSCSI disks (iscsictl), or to ZFS = (zfs/zpool), >> don't return. (and I would have expected the IO activity to flawlessly continue)