From owner-freebsd-fs@freebsd.org Sun Sep 3 11:24:50 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CAEF0E067EF for ; Sun, 3 Sep 2017 11:24:50 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from smtp.digiware.nl (unknown [IPv6:2001:4cb8:90:ffff::3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 92B2668348 for ; Sun, 3 Sep 2017 11:24:50 +0000 (UTC) (envelope-from wjw@digiware.nl) Received: from router.digiware.nl (localhost.digiware.nl [127.0.0.1]) by smtp.digiware.nl (Postfix) with ESMTP id 85A5A224D0; Sun, 3 Sep 2017 13:24:33 +0200 (CEST) X-Virus-Scanned: amavisd-new at digiware.com Received: from smtp.digiware.nl ([127.0.0.1]) by router.digiware.nl (router.digiware.nl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2nhC7zSMfR4U; Sun, 3 Sep 2017 13:24:32 +0200 (CEST) Received: from [192.168.10.67] (opteron [192.168.10.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.digiware.nl (Postfix) with ESMTPSA id 9CADD224CD; Sun, 3 Sep 2017 13:24:32 +0200 (CEST) Subject: FreeBSSD: [Bug 221997] net/ceph: Luminous (12.2.0) release for Ceph References: To: Ceph Development , FreeBSD Filesystems From: Willem Jan Withagen X-Forwarded-Message-Id: Message-ID: Date: Sun, 3 Sep 2017 13:24:32 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: nl Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Sep 2017 11:24:50 -0000 Hi all, Today the FreeBSD luminous release got accepted in ports. So in a few days packages should be available. --WjW -------- Forwarded Message -------- Subject: [Bug 221997] net/ceph: Luminous (12.2.0) release for Ceph Date: Sun, 03 Sep 2017 08:34:28 +0000 From: bugzilla-noreply@freebsd.org To: wjw@digiware.nl https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=221997 Kurt Jaeger changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-ports-bugs@FreeBSD. |pi@FreeBSD.org |org | Status|New |Closed Resolution|--- |FIXED --- Comment #7 from Kurt Jaeger --- Committed, thanks! -- You are receiving this mail because: You reported the bug. From owner-freebsd-fs@freebsd.org Mon Sep 4 04:56:07 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1B382E13891 for ; Mon, 4 Sep 2017 04:56:07 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EF8EA64F8F; Mon, 4 Sep 2017 04:56:06 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (106-68-122-71.dyn.iinet.net.au [106.68.122.71]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id v844trGw074130 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 3 Sep 2017 21:55:56 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: Tips on remote debugging for filesystem code To: Aijaz Baig , Gary Palmer Cc: Bakul Shah , Kirk McKusick , freebsd-fs@freebsd.org References: <201708250251.v7P2pwcl029687@chez.mckusick.com> <20170827195428.49400124AE9F@mail.bitblocks.com> <20170828132647.GA60563@in-addr.com> From: Julian Elischer Message-ID: Date: Mon, 4 Sep 2017 12:55:47 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Sep 2017 04:56:07 -0000 On 2/9/17 8:40 pm, Aijaz Baig wrote: > I was finally able to deploy a FreeBSD-Current VM on Bhyve but even > with that setup I face a similar issue > > For instance I put a breakpoint on 'ffs_read' and let the debugged > VM continue but the control is neither returned to (k)gdb nor does > the VM actually run. It appears hung. are you sure it's not stopped at the breakpoint in ddb on the console? > > Is there something within the ffs/ufs code that I need to be aware > of that it doesn't facilitate this sort of debugging? Or is it > because Bhyve is being run on FreeBSD which is itself run on another > hypervisor? Will that affect this? > > On Mon, Aug 28, 2017 at 7:03 PM, Aijaz Baig > wrote: > > HI Gary > > Just checked that. Indeed that was the case. In addition I was > running FreeBSD on top of vmware hypervisor and had not allowed > hardware acceleration features pass through to the FreeBSD VM. > It is now working! Need to explore more on getting it to connect > to GDB > > Perhaps you'll see a thread soon for it (Has anything worked > perfectly for the first time ever?? ;) )! > > On Mon, Aug 28, 2017 at 6:56 PM, Gary Palmer > > wrote: > > On Mon, Aug 28, 2017 at 01:12:09PM +0530, Aijaz Baig wrote: > > Thanks a lot bakul. Will certain look into this > > > > Nevertheless I got hold of a PowerEdge 320 system which > DOES have a POPCNT > > instruction which shows support for V-x support via EPT > but I still get the > > same error there > > > > sh /usr/share/examples/bhyve/vmrun.sh -c 1 -m 2048M -t > tap0 -d guest.img -i > > -I FreeBSD-10.3-RELEASE-amd64-bootonly.iso fbsd10 > > Launching virtual machine "fbsd10" ... > > vm_create: Device not configured > > > > dmesg: > > vmx_init: processor does not support VMX operation > > > > cat /var/run/dmesg.boot | grep -i popcnt > > > > > Features2=0xffba2203 > > > > > Features2=0xffba2203 > > > > Any ideas? > > Hi, > > Check the BIOS for virtualisation or other related options.  > Some BIOSes have > the ability to turn those features off, even if the > processor normally > supports them > > Regards, > > Gary > > > > > -- > > Best Regards, > Aijaz Baig > > > > > -- > > Best Regards, > Aijaz Baig From owner-freebsd-fs@freebsd.org Mon Sep 4 13:55:17 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0471EE08E31 for ; Mon, 4 Sep 2017 13:55:17 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: from mail-it0-x22d.google.com (mail-it0-x22d.google.com [IPv6:2607:f8b0:4001:c0b::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AED8974A6B; Mon, 4 Sep 2017 13:55:16 +0000 (UTC) (envelope-from aijazbaig1@gmail.com) Received: by mail-it0-x22d.google.com with SMTP id j17so3043349iti.1; Mon, 04 Sep 2017 06:55:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=9WB6VLUNPw8Nk8HJeYB37Wf6xbU2mlGjyHWIzCxgFFM=; b=pVL1Ld2jWbpaI/WapjP/FCxZyBXqtiv/q5oC4QB7O8Z5QqTArlqWfqTkjnlxiVJe1k uAkGoq+TaVw6K/b4Uw9pvvRjDW8TN/gaosDl6zVCxf+ZJpcmTcjxLSW9d4wZmbjxHU8l yOrCHoGaW5+E3ZeTDSvNzBaTpeYMO0066nf0Rakhxg19p9aO2eDhfUdloDeUR9IoiWQ7 /mnL2iVph4BuKcq6iUUkXdYdo4p7FNJpDM5aTisfQO33SXD+faBz8pCquo1qwpLJj+So 9OwBOuUFhhBAQJhImOjxV06mB3Idykuaf7MKlRXAkJwUSDa/mSrlgwYJl9LeV917vJTQ S1sQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=9WB6VLUNPw8Nk8HJeYB37Wf6xbU2mlGjyHWIzCxgFFM=; b=RAYL4OXCHbYbJkR6O81F8zbY1DomnbwWQfG+EMGu2DwTHwiMEX81ishyKDdyDP5FhX wY9seshxnMFo4fuW1uqQfyUfrIXE4vin/8Z+U7St6hhDIs0Weq7sBI9xn2M9SxGKdCf1 fmUmwKbP49zNzK1+CPZlJBMh4UJv2A8XIZilNfPsE2vO6tKRHtq9EQCt8ouhqR2ccN5r FRlnSEq+wP0EaON5ySma2IGzGdBs/adFkEMem8ceYhY4xldyxnWJbR474EYBDGJ3k/ZB AU5B+v4KFG/7lJ9Kt83/mR7Ff5RuWmxYEG4BrMaIXiQVKbF6ao9+du5+SPWeIAGuwCkr +UjQ== X-Gm-Message-State: AHPjjUgfPJy8LnltNHeWx5krSUeFeb6FubqJQZRpF3pAbujJfH+TYnTD 93vzfjnIClT8s5lHzO1MaIFV18V9LQ== X-Google-Smtp-Source: ADKCNb7+JvKMnZCfMbdOT1PzZ/CxzkySHZOVVexXnPgqak9qBPe1CNtKprwV1pXsRR6CDDXQGS9466UhP3cUFkY5z7M= X-Received: by 10.36.176.5 with SMTP id d5mr890412itf.150.1504533315558; Mon, 04 Sep 2017 06:55:15 -0700 (PDT) MIME-Version: 1.0 Received: by 10.2.22.66 with HTTP; Mon, 4 Sep 2017 06:55:14 -0700 (PDT) In-Reply-To: References: <201708250251.v7P2pwcl029687@chez.mckusick.com> <20170827195428.49400124AE9F@mail.bitblocks.com> <20170828132647.GA60563@in-addr.com> From: Aijaz Baig Date: Mon, 4 Sep 2017 19:25:14 +0530 Message-ID: Subject: Re: Tips on remote debugging for filesystem code To: Julian Elischer Cc: Gary Palmer , Bakul Shah , Kirk McKusick , freebsd-fs@freebsd.org Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Sep 2017 13:55:17 -0000 Hi Julian Yes it just hangs. Yes it's not like it just hangs every time. It's more intermittent. This time I was stepping through 'kern_openat', really slowly as I am new to this, checking out the structures and the likes and finally this hit when I was within vn_open_cred. So yes the total time between attaching to the stub within the VM and this happening was close to 3 hours but nonetheless I do face this eventually is there something I should watch out for, I mean is there something time bound in some way for any of this code? Is this the reason why the VM becomes unresponsive if I step really slowly through the stack trace? On Mon, Sep 4, 2017 at 10:25 AM, Julian Elischer wrote: > On 2/9/17 8:40 pm, Aijaz Baig wrote: > > I was finally able to deploy a FreeBSD-Current VM on Bhyve but even with > that setup I face a similar issue > > For instance I put a breakpoint on 'ffs_read' and let the debugged VM > continue but the control is neither returned to (k)gdb nor does the VM > actually run. It appears hung. > > are you sure it's not stopped at the breakpoint in ddb on the console? > > > Is there something within the ffs/ufs code that I need to be aware of that > it doesn't facilitate this sort of debugging? Or is it because Bhyve is > being run on FreeBSD which is itself run on another hypervisor? Will that > affect this? > > On Mon, Aug 28, 2017 at 7:03 PM, Aijaz Baig wrote: > >> HI Gary >> >> Just checked that. Indeed that was the case. In addition I was running >> FreeBSD on top of vmware hypervisor and had not allowed hardware >> acceleration features pass through to the FreeBSD VM. It is now working! >> Need to explore more on getting it to connect to GDB >> >> Perhaps you'll see a thread soon for it (Has anything worked perfectly >> for the first time ever?? ;) )! >> >> On Mon, Aug 28, 2017 at 6:56 PM, Gary Palmer wrote: >> >>> On Mon, Aug 28, 2017 at 01:12:09PM +0530, Aijaz Baig wrote: >>> > Thanks a lot bakul. Will certain look into this >>> > >>> > Nevertheless I got hold of a PowerEdge 320 system which DOES have a >>> POPCNT >>> > instruction which shows support for V-x support via EPT but I still >>> get the >>> > same error there >>> > >>> > sh /usr/share/examples/bhyve/vmrun.sh -c 1 -m 2048M -t tap0 -d >>> guest.img -i >>> > -I FreeBSD-10.3-RELEASE-amd64-bootonly.iso fbsd10 >>> > Launching virtual machine "fbsd10" ... >>> > vm_create: Device not configured >>> > >>> > dmesg: >>> > vmx_init: processor does not support VMX operation >>> > >>> > cat /var/run/dmesg.boot | grep -i popcnt >>> > >>> > Features2=0xffba2203>> SE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> >>> > >>> > Features2=0xffba2203>> SE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV> >>> > >>> > Any ideas? >>> >>> Hi, >>> >>> Check the BIOS for virtualisation or other related options. Some BIOSes >>> have >>> the ability to turn those features off, even if the processor normally >>> supports them >>> >>> Regards, >>> >>> Gary >>> >> >> >> >> -- >> >> Best Regards, >> Aijaz Baig >> > > > > -- > > Best Regards, > Aijaz Baig > > > -- Best Regards, Aijaz Baig From owner-freebsd-fs@freebsd.org Mon Sep 4 17:03:05 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4C457E11EAE for ; Mon, 4 Sep 2017 17:03:05 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 2B0297E741; Mon, 4 Sep 2017 17:03:04 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (106-68-122-71.dyn.iinet.net.au [106.68.122.71]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id v84H2tQN076840 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Mon, 4 Sep 2017 10:02:58 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: Tips on remote debugging for filesystem code To: Aijaz Baig Cc: Gary Palmer , Bakul Shah , Kirk McKusick , freebsd-fs@freebsd.org References: <201708250251.v7P2pwcl029687@chez.mckusick.com> <20170827195428.49400124AE9F@mail.bitblocks.com> <20170828132647.GA60563@in-addr.com> From: Julian Elischer Message-ID: <4c7a54e4-a9fb-cf8b-fb1b-4a24e3cffc7c@freebsd.org> Date: Tue, 5 Sep 2017 01:02:49 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Sep 2017 17:03:05 -0000 On 4/9/17 9:55 pm, Aijaz Baig wrote: > Hi Julian > > Yes it just hangs. Yes it's not like it just hangs every time. It's > more intermittent. This time I was stepping through 'kern_openat', > really slowly as I am new to this, checking out the structures and > the likes and finally this hit when I was within vn_open_cred. So > yes the total time between attaching to the stub within the VM and > this happening was close to 3 hours but nonetheless I do face this > eventually > > is there something I should watch out for, I mean is there something > time bound in some way for any of this code? Is this the reason why > the VM becomes unresponsive if I step really slowly through the > stack trace? I haven't used it for  a year or so but I used to occasionally have a problem when doing things that would disable interrupts.  I found I had to put a breakpoint on ht eother side of them and 'c'ontinue over them or it would get stuck.. > > On Mon, Sep 4, 2017 at 10:25 AM, Julian Elischer > wrote: > > On 2/9/17 8:40 pm, Aijaz Baig wrote: >> I was finally able to deploy a FreeBSD-Current VM on Bhyve but >> even with that setup I face a similar issue >> >> For instance I put a breakpoint on 'ffs_read' and let the >> debugged VM continue but the control is neither returned to >> (k)gdb nor does the VM actually run. It appears hung. > are you sure it's not stopped at the breakpoint in ddb on the > console? > >> >> Is there something within the ffs/ufs code that I need to be >> aware of that it doesn't facilitate this sort of debugging? Or >> is it because Bhyve is being run on FreeBSD which is itself run >> on another hypervisor? Will that affect this? >> >> On Mon, Aug 28, 2017 at 7:03 PM, Aijaz Baig >> > wrote: >> >> HI Gary >> >> Just checked that. Indeed that was the case. In addition I >> was running FreeBSD on top of vmware hypervisor and had not >> allowed hardware acceleration features pass through to the >> FreeBSD VM. It is now working! Need to explore more on >> getting it to connect to GDB >> >> Perhaps you'll see a thread soon for it (Has anything >> worked perfectly for the first time ever?? ;) )! >> >> On Mon, Aug 28, 2017 at 6:56 PM, Gary Palmer >> > wrote: >> >> On Mon, Aug 28, 2017 at 01:12:09PM +0530, Aijaz Baig wrote: >> > Thanks a lot bakul. Will certain look into this >> > >> > Nevertheless I got hold of a PowerEdge 320 system >> which DOES have a POPCNT >> > instruction which shows support for V-x support via >> EPT but I still get the >> > same error there >> > >> > sh /usr/share/examples/bhyve/vmrun.sh -c 1 -m 2048M >> -t tap0 -d guest.img -i >> > -I FreeBSD-10.3-RELEASE-amd64-bootonly.iso fbsd10 >> > Launching virtual machine "fbsd10" ... >> > vm_create: Device not configured >> > >> > dmesg: >> > vmx_init: processor does not support VMX operation >> > >> > cat /var/run/dmesg.boot | grep -i popcnt >> > >> > >> Features2=0xffba2203 >> > >> > >> Features2=0xffba2203 >> > >> > Any ideas? >> >> Hi, >> >> Check the BIOS for virtualisation or other related >> options.  Some BIOSes have >> the ability to turn those features off, even if the >> processor normally >> supports them >> >> Regards, >> >> Gary >> >> >> >> >> -- >> >> Best Regards, >> Aijaz Baig >> >> >> >> >> -- >> >> Best Regards, >> Aijaz Baig > > > > > > -- > > Best Regards, > Aijaz Baig From owner-freebsd-fs@freebsd.org Mon Sep 4 17:12:53 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4E0AE12654 for ; Mon, 4 Sep 2017 17:12:53 +0000 (UTC) (envelope-from bsd@vink.pl) Received: from mail-qk0-x231.google.com (mail-qk0-x231.google.com [IPv6:2607:f8b0:400d:c09::231]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9FC947EBAF for ; Mon, 4 Sep 2017 17:12:53 +0000 (UTC) (envelope-from bsd@vink.pl) Received: by mail-qk0-x231.google.com with SMTP id a128so3751926qkc.5 for ; Mon, 04 Sep 2017 10:12:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vink-pl.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=azghGHSQNpRrjJrGaajKEC2y7A/smi1DE+uo69sJa7M=; b=szW9ZlYLEp5Kaed5KBXK45hHzpDgjbjk7X8a5k8vCX/gB+WgKSn3bIBI2pi9Tw+EZm HfJs8EU2bB3KX24rXuaZAeTJUMhb7XPvPjC572WrmYwRgi5B/Ay4grRM2ZugJ8F7F3K2 PsupeFJUQGdgFgK+L7FTzXyI0rUpbY50Q9F+rgpUwLPH251E263Tzv4M3Xqqkzw82z3W wF9eXv5X0KWXCPZ+z/MC0w6on0nNaAnAuNnOtzP0MANGnFpT9Tz6JjnLMWpFagxJSGaA DLteKQHt5w+6ureLtpPBUM0oVmdHzq/fKswjng/WOrqlnictvCfE2QlELfI/bWbJVOco jXZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=azghGHSQNpRrjJrGaajKEC2y7A/smi1DE+uo69sJa7M=; b=EuZ5OyqixzVt+camjTrpYxA3o1ehO5ELVqOueBZCaViIp+WpkE+JuMpiJtenrQqSQd yWDPugBt2xQIQCCGHuMNDtgYHC1qE9uwxCE7yffCcYzu1UFRxvFDJWmJxrHIwgYmM9Jc HBQC5KgO5uulp/tLZ61MjhtnQe5et9lHr0OSsHZKmXXCBa/ZFuS8b+HnMkzu18BgpoAq QDN8HTutyVI6CC0XSoBJsR08soTGcDlYpO/TK5Dr4XZ3Q9Yl+XuIF+SD3Q5hXrTO+IMv 9h4Esx9SRGha9NiBOSU9/fxeivQX85eE2ZYry+b09hkT1VAJ5StfjXW+IVIVHCMTXS06 BcPw== X-Gm-Message-State: AHPjjUhIVU5Bx2ZjthHzZUzrxZA5sQS5yykb1LT1Jrm/bB3Zla9Q+hFA 4rrk/7fclqI4JvjUK5g= X-Received: by 10.55.87.135 with SMTP id l129mr1632503qkb.282.1504545172024; Mon, 04 Sep 2017 10:12:52 -0700 (PDT) Received: from mail-qt0-f170.google.com (mail-qt0-f170.google.com. [209.85.216.170]) by smtp.gmail.com with ESMTPSA id i51sm5415146qta.75.2017.09.04.10.12.51 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Sep 2017 10:12:51 -0700 (PDT) Received: by mail-qt0-f170.google.com with SMTP id m35so4098526qte.1 for ; Mon, 04 Sep 2017 10:12:51 -0700 (PDT) X-Google-Smtp-Source: ADKCNb67Y0NvBn8s1gz21jhu4CbbvfJPEeiLLAXHiYou2lPSn7Hdhy1o0ynSTBVPEHWIFop5jFy/s9c0Ht0GOHJmuwg= X-Received: by 10.200.42.243 with SMTP id c48mr1762375qta.106.1504545171366; Mon, 04 Sep 2017 10:12:51 -0700 (PDT) MIME-Version: 1.0 Received: by 10.12.156.197 with HTTP; Mon, 4 Sep 2017 10:12:50 -0700 (PDT) In-Reply-To: References: From: Wiktor Niesiobedzki Date: Mon, 4 Sep 2017 19:12:50 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: Resolving errors with ZVOL-s To: freebsd-fs Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Sep 2017 17:12:54 -0000 Hi, I can follow up on my issue - the same problem just happened on the second ZVOL that I've created: # zpool status -v pool: tank state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://illumos.org/msg/ZFS-8000-8A scan: scrub repaired 0 in 5h27m with 0 errors on Sat Sep 2 15:30:59 2017 config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 14 mirror-0 ONLINE 0 0 28 gpt/tank1.eli ONLINE 0 0 28 gpt/tank2.eli ONLINE 0 0 28 errors: Permanent errors have been detected in the following files: tank/docker-big:<0x1> <0x5095>:<0x1> I suspect that these errors might be related to my recent upgrade to 11.1. Until 19 of August I was running 11.0. I consider rolling back to 11.0 right now. For reference: # zfs get all tank/docker-big NAME PROPERTY VALUE SOURCE tank/docker-big type volume - tank/docker-big creation Sat Sep 2 10:09 2017 - tank/docker-big used 100G - tank/docker-big available 747G - tank/docker-big referenced 10.5G - tank/docker-big compressratio 4.58x - tank/docker-big reservation none default tank/docker-big volsize 100G local tank/docker-big volblocksize 128K - tank/docker-big checksum skein inherited from tank tank/docker-big compression lz4 inherited from tank tank/docker-big readonly off default tank/docker-big copies 1 default tank/docker-big refreservation 100G local tank/docker-big primarycache all default tank/docker-big secondarycache all default tank/docker-big usedbysnapshots 0 - tank/docker-big usedbydataset 10.5G - tank/docker-big usedbychildren 0 - tank/docker-big usedbyrefreservation 89.7G - tank/docker-big logbias latency default tank/docker-big dedup off default tank/docker-big mlslabel - tank/docker-big sync standard default tank/docker-big refcompressratio 4.58x - tank/docker-big written 10.5G - tank/docker-big logicalused 47.8G - tank/docker-big logicalreferenced 47.8G - tank/docker-big volmode dev local tank/docker-big snapshot_limit none default tank/docker-big snapshot_count none default tank/docker-big redundant_metadata all default tank/docker-big com.sun:auto-snapshot false local Any ideas what should I try before rolling back? Cheers, Wiktor 2017-09-02 19:17 GMT+02:00 Wiktor Niesiobedzki : > Hi, > > I have recently encountered errors on my ZFS Pool on my 11.1-R: > $ uname -a > FreeBSD kadlubek 11.1-RELEASE-p1 FreeBSD 11.1-RELEASE-p1 #0: Wed Aug 9 > 11:55:48 UTC 2017 root@amd64-builder.daemonology > .net:/usr/obj/usr/src/sys/GENERIC amd64 > > # zpool status -v tank > pool: tank > state: ONLINE > status: One or more devices has experienced an error resulting in data > corruption. Applications may be affected. > action: Restore the file in question if possible. Otherwise restore the > entire pool from backup. > see: http://illumos.org/msg/ZFS-8000-8A > scan: scrub repaired 0 in 5h27m with 0 errors on Sat Sep 2 15:30:59 2017 > config: > > NAME STATE READ WRITE CKSUM > tank ONLINE 0 0 98 > mirror-0 ONLINE 0 0 196 > gpt/tank1.eli ONLINE 0 0 196 > gpt/tank2.eli ONLINE 0 0 196 > > errors: Permanent errors have been detected in the following files: > > dkr-test:<0x1> > > dkr-test is ZVOL that I use within bhyve and indeed - within bhyve I have > noticed I/O errors on this volume. This ZVOL did not have any snapshots. > > Following the advice mentioned in action I tried to restore the ZVOL: > # zfs desroy tank/dkr-test > > But still errors are mentioned in zpool status: > errors: Permanent errors have been detected in the following files: > > <0x5095>:<0x1> > > I can't find any reference to this dataset in zdb: > # zdb -d tank | grep 5095 > # zdb -d tank | grep 20629 > > > I tried also getting statistics about metadata in this pool: > # zdb -b tank > > Traversing all blocks to verify nothing leaked ... > > loading space map for vdev 0 of 1, metaslab 159 of 174 ... > No leaks (block sum matches space maps exactly) > > bp count: 24426601 > ganged count: 0 > bp logical: 1983127334912 avg: 81187 > bp physical: 1817897247232 avg: 74422 compression: > 1.09 > bp allocated: 1820446928896 avg: 74527 compression: > 1.09 > bp deduped: 0 ref>1: 0 deduplication: 1.00 > SPA allocated: 1820446928896 used: 60.90% > > additional, non-pointer bps of type 0: 57981 > Dittoed blocks on same vdev: 296490 > > And zdb got stuck using 100% CPU > > And now to my questions: > 1. Do I interpret correctly, that this situation is probably due to error > during write, and both copies of the block got checksum mismatching their > data? And if it is a hardware problem, it is probably something other than > disk? (No, I don't use ECC RAM) > > 2. Is there any way to remove offending dataset and clean the pool of the > errors? > > 3. Is my metadata OK? Or should I restore entire pool from backup? > > 4. I tried also running zdb -bc tank, but this resulted in kernel panic. I > might try to get the stack trace once I get physical access to machine next > week. Also - checksum verification slows down process from 1000MB/s to less > than 1MB/s. Is this expected? > > 5. When I work with zdb (as as above) should I try to limit writes to the > pool (e.g. by unmounting the datasets)? > > Cheers, > > Wiktor Niesiobedzki > > PS. Sorry for previous partial message. > > From owner-freebsd-fs@freebsd.org Mon Sep 4 21:14:49 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C7784E1EC7B for ; Mon, 4 Sep 2017 21:14:49 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: from mail-wm0-x243.google.com (mail-wm0-x243.google.com [IPv6:2a00:1450:400c:c09::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 58E4A2D58 for ; Mon, 4 Sep 2017 21:14:49 +0000 (UTC) (envelope-from ben.rubson@gmail.com) Received: by mail-wm0-x243.google.com with SMTP id x189so1323595wmg.4 for ; Mon, 04 Sep 2017 14:14:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=7O6FyPuyiUpKOGw/DUUy3Y9JX5IEhV+4gYbYe0wGcRA=; b=FfGhM1ZofziEI/uhjluFzc7Hg7+MZaSGj4Z/Y0B6eE1cP+mX661bIW+JtNtt+HKm/L FZpsNbDkGoFmZXUZLKgbI5t/rdL493viljv0dv7IibNPLidLj/davy0fY7/rFFJ5IaAG s7GpCvsAh6ebSRm1k0nvdPWPdE4nXQE1bLLeP+g7lWf94XkpaI3x7fCdp+gHOttgDpSe Mu/lacNc0z+qyH9ogMQjJPcxCUu4JK2Z4IZmK9j/sKAuG84a9o1ZqGnOGveU7hrPerx2 SbuuTNp7FYjXj917x9BWbFpVmPfGWxYO96PtpO1JZbaEF3aV8S0vG58lDcy1FjI4O4st 8lSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date :content-transfer-encoding:message-id:references:to; bh=7O6FyPuyiUpKOGw/DUUy3Y9JX5IEhV+4gYbYe0wGcRA=; b=kzJv0BQ2DjfmqNb+cT5HbJX9P91Yaqv6+cShQDIXsP+/3yTYoyDn9t23KwA/BZNOue lcHqQqy+zYyZFMmJmY/rkMBrYZAOfPmLLSsJ+JkB0OyoAGcdMhDgmm9ee/O15pH6KfED y+c8iLJW8QXevkp4SvXPpQj7N1rs8PZbwxnzi4+Ak9KeeWXRjlouFh6HOTPPq9JZ7d/F LwebjKvPBWe6e54l7tmpJV74tWOi7dWAqXAIEJ4CmdFEoLXb4sGVXGqFiXl7UZez+Yiz n2ySa7DfgIp8qR0D+rQz2Q2CZwv4NoTjTOTVqvLZdnC/jucNYKCxsOMh4CJ6a6DzGYwC GjXQ== X-Gm-Message-State: AHPjjUgsZzyb1W3QYUx58p9WKqFFLQcVy6AEuG/qZIN2MNwiRiWj7vIn 3lhUjZKz+hwQzl3XFXQ= X-Google-Smtp-Source: ADKCNb6S25kDEFnez154ZrZ1FHERQYiDrffGPaXKiuoDWullGV0Oy7TvVqgiAgf9aeqSjAJpWPsW6A== X-Received: by 10.28.30.140 with SMTP id e134mr1088447wme.11.1504559687269; Mon, 04 Sep 2017 14:14:47 -0700 (PDT) Received: from 6.10.20.172.rev.sfr.net (15.87.136.77.rev.sfr.net. [77.136.87.15]) by smtp.gmail.com with ESMTPSA id o191sm1183995wmd.35.2017.09.04.14.14.46 for (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 04 Sep 2017 14:14:46 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: umount() taking minutes for FUSE filesystems From: Ben RUBSON In-Reply-To: Date: Mon, 4 Sep 2017 23:14:47 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> References: <87bmn44ruu.fsf@vostro.rath.org> <87o9qyrbs8.fsf@vostro.rath.org> To: Freebsd fs X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Sep 2017 21:14:49 -0000 I managed to reproduce the issue. unmount takes exactly 60 seconds, as if a timeout was running. # procstat -kk $! COMM TDNAME KSTACK =20 printcap - mi_switch+0xd2 sleepq_catch_signals+0xb7 = sleepq_timedwait_sig+0x10 _sleep+0x26f fdisp_wait_answ+0x171 = fuse_vfsop_unmount+0xf5 dounmount+0x9b6 sys_unmount+0x41b = amd64_syscall+0x4ce Xfast_syscall+0xfb=20 # uname -sr FreeBSD 11.0-RELEASE-p9 Ben > On 29 Aug 2017, at 17:02, Conrad Meyer wrote: >=20 > Hey Nikolaus, >=20 > A first cut debug tactic might be running 'procstat -kk printcap process>' to see where it is spending time in the kernel. I > haven't had time to try and reproduce this myself, sorry. >=20 > Best, > Conrad >=20 > On Tue, Aug 29, 2017 at 1:57 AM, Nikolaus Rath = wrote: >> *ping* >>=20 >> Anyone able to help? >>=20 >> Thanks, >> -Nikolaus >>=20 >> On Aug 24 2017, Nikolaus Rath wrote: >>> Hello, >>>=20 >>> It seems that in some situations, the unmount() system call takes >>> minutes to unmount fuse filesystems. To reproduce, download a Git >>> snapshot of libfuse 3 from https://github.com/libfuse/libfuse and = do: >>>=20 >>> $ md build; cd build >>> $ meson .. >>> $ mesonconf -D buildtype=3Ddebug # optional >>> $ ninja >>>=20 >>> Then execute examples/printcap. This will take very long, and all = the >>> time is spent in the unmount(mountpoint, MNT_FORCE) call in >>> mount_bsd.c:fuse_kern_unmount(). >>>=20 >>> Does anyone have an idea what might be causing this and how to fix = it? >>>=20 >>> My hunch is that it has something to do with libfuse no longer >>> responding to requests at this point. But I'm at my wits end as to = why >>> and what to do about it.... >>>=20 >>> Best, >>> -Nikolaus >>>=20 >>> -- >>> GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F >>>=20 >>> =C2=BBTime flies like an arrow, fruit flies like a = Banana.=C2=AB >>> _______________________________________________ >>> freebsd-fs@freebsd.org mailing list >>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>> To unsubscribe, send any mail to = "freebsd-fs-unsubscribe@freebsd.org" >>=20 >>=20 >> -- >> GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F >>=20 >> =C2=BBTime flies like an arrow, fruit flies like a = Banana.=C2=AB >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Tue Sep 5 08:48:45 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1A77FE2110D for ; Tue, 5 Sep 2017 08:48:45 +0000 (UTC) (envelope-from se@freebsd.org) Received: from mailout02.t-online.de (mailout02.t-online.de [194.25.134.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D17033784 for ; Tue, 5 Sep 2017 08:48:44 +0000 (UTC) (envelope-from se@freebsd.org) Received: from fwd15.aul.t-online.de (fwd15.aul.t-online.de [172.20.27.63]) by mailout02.t-online.de (Postfix) with SMTP id B9D8D41A25DA; Tue, 5 Sep 2017 10:42:12 +0200 (CEST) Received: from Stefans-MBP-2.fritz.box (EBWrTrZ-oh18N0daKClnKNg22RfgboAQf7wYnzNXQuVJLUnFUnWshLWsVU7nTVRQIx@[84.154.110.22]) by fwd15.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1dp9Qt-04byrY0; Tue, 5 Sep 2017 10:42:07 +0200 Subject: Re: umount() taking minutes for FUSE filesystems To: Ben RUBSON , Freebsd fs References: <87bmn44ruu.fsf@vostro.rath.org> <87o9qyrbs8.fsf@vostro.rath.org> <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> From: Stefan Esser Message-ID: <29de6425-9f92-3bd8-f446-1c9dded33b15@freebsd.org> Date: Tue, 5 Sep 2017 10:42:07 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> Content-Type: text/plain; charset=windows-1252 Content-Language: en-GB Content-Transfer-Encoding: 7bit X-ID: EBWrTrZ-oh18N0daKClnKNg22RfgboAQf7wYnzNXQuVJLUnFUnWshLWsVU7nTVRQIx X-TOI-MSGID: a3cfa107-44e9-44cb-80fb-a3810c671637 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Sep 2017 08:48:45 -0000 Am 04.09.17 um 23:14 schrieb Ben RUBSON: > I managed to reproduce the issue. > unmount takes exactly 60 seconds, as if a timeout was running. > > # procstat -kk $! > COMM TDNAME KSTACK > printcap - mi_switch+0xd2 sleepq_catch_signals+0xb7 sleepq_timedwait_sig+0x10 _sleep+0x26f fdisp_wait_answ+0x171 fuse_vfsop_unmount+0xf5 dounmount+0x9b6 sys_unmount+0x41b amd64_syscall+0x4ce Xfast_syscall+0xfb > > # uname -sr > FreeBSD 11.0-RELEASE-p9 I have given the exact position of this 60 second msleep() in multiple mails before. It is in fuse_ipc.c, the particular msleep with "fu_ans" (line 333 in -CURRENT). I did not try to diagnose, why this particular umount() takes so long, while others are fast, but it is obvious that the kernel module does wait for a signal at the end of some IPC and the signal is either lost or never sent. There is a check for a dead connection, just before the msleep() and the connection is considered alive at that point (and should be, to support the umount() result being reported). I did not have time to look into this during the previous week and won't during this week, but it should not be too hard to see, what's going on. A starting point could be to compare this test with those that perform the unmount without delay. Regards, STefan From owner-freebsd-fs@freebsd.org Tue Sep 5 09:38:29 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A6535E22D44 for ; Tue, 5 Sep 2017 09:38:29 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 77D0567A39 for ; Tue, 5 Sep 2017 09:38:28 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 5C58E20A80 for ; Tue, 5 Sep 2017 05:38:22 -0400 (EDT) Received: from frontend2 ([10.202.2.161]) by compute1.internal (MEProxy); Tue, 05 Sep 2017 05:38:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=YGIyLkd60lZgI3/Ottz41vomlnBvRiEB+kNkGEjm+Y0=; b=C3USkgcj ZC4YGrz2G6og6STJEHRnHP7u6iYKKT1O/7JGf66wXTKBxiHtfKvQMRHDKrrKlQro SBgvfzS8WaeGoV2ZQsmOK0W1QOwGeis2zcjPDFGpgEBL+I+dWjQPs3Nb40saZMNc WqPlPNne8WAt0WMs4oNGToHxyquJxBbu4n1DMrXovr9xxlbAajCQ/i0xKRhi3V/d cgKeeKRwB3VZIHlI2hKCFAF53qGz7PFHBEzRZv4K6kdrcSSOpCbLAZqqK2s+qE+T FTz8RtStD6w8JHHIyE8NWFA/W++6KvDc988TYiwL8yR5HR8MrBlBroX9IbH4RuIH AkcWI0KSnvh2sA== X-ME-Sender: X-Sasl-enc: I09yrkcvKO3YMZs1Lo/nYJrEd4gCzPS9tKYryPkemk2A 1504604302 Received: from ebox.rath.org (ebox.rath.org [45.79.69.51]) by mail.messagingengine.com (Postfix) with ESMTPA id 1ADE02418B for ; Tue, 5 Sep 2017 05:38:22 -0400 (EDT) Received: from thinkpad.rath.org (thinkpad [192.168.12.2]) by ebox.rath.org (Postfix) with ESMTPS id DCECC11C for ; Tue, 5 Sep 2017 09:38:20 +0000 (UTC) Received: by thinkpad.rath.org (Postfix, from userid 1000) id 6EA8BBFCB5; Tue, 5 Sep 2017 11:38:19 +0200 (CEST) From: Nikolaus Rath To: freebsd-fs@freebsd.org Subject: Re: umount() taking minutes for FUSE filesystems References: <87bmn44ruu.fsf@vostro.rath.org> <87o9qyrbs8.fsf@vostro.rath.org> <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> <29de6425-9f92-3bd8-f446-1c9dded33b15@freebsd.org> Mail-Copies-To: never Mail-Followup-To: freebsd-fs@freebsd.org Date: Tue, 05 Sep 2017 11:38:18 +0200 In-Reply-To: <29de6425-9f92-3bd8-f446-1c9dded33b15@freebsd.org> (Stefan Esser's message of "Tue, 5 Sep 2017 10:42:07 +0200") Message-ID: <87k21dzdrp.fsf@thinkpad.rath.org> User-Agent: Gnus/5.130014 (Ma Gnus v0.14) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Sep 2017 09:38:29 -0000 On Sep 05 2017, Stefan Esser wrote: > Am 04.09.17 um 23:14 schrieb Ben RUBSON: >> I managed to reproduce the issue. >> unmount takes exactly 60 seconds, as if a timeout was running. >>=20 >> # procstat -kk $! >> COMM TDNAME KSTACK=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20= =20=20=20=20=20=20=20 >> printcap - mi_switch+0xd2 sleepq_catch_signals+0xb7 >> sleepq_timedwait_sig+0x10 _sleep+0x26f fdisp_wait_answ+0x171 >> fuse_vfsop_unmount+0xf5 dounmount+0x9b6 sys_unmount+0x41b >> amd64_syscall+0x4ce Xfast_syscall+0xfb >>=20 >> # uname -sr >> FreeBSD 11.0-RELEASE-p9 > > I have given the exact position of this 60 second msleep() in multiple > mails before. It is in fuse_ipc.c, the particular msleep with "fu_ans" > (line 333 in -CURRENT). > > I did not try to diagnose, why this particular umount() takes so long, > while others are fast, but it is obvious that the kernel module does > wait for a signal at the end of some IPC and the signal is either lost > or never sent. There is a check for a dead connection, just before the > msleep() and the connection is considered alive at that point (and > should be, to support the umount() result being reported). > > I did not have time to look into this during the previous week and > won't during this week, but it should not be too hard to see, what's > going on. A starting point could be to compare this test with those > that perform the unmount without delay. Probably the crucial difference is that the test that takes long exits its main loop on its own and then informs the FUSE kernel module about that, while the other tests terminate the main loop because the kernel module tells them to do so. Best, Nikolaus --=20 GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2= =AB From owner-freebsd-fs@freebsd.org Tue Sep 5 18:22:47 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE0C8E1700F for ; Tue, 5 Sep 2017 18:22:47 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [46.17.166.21]) by mx1.freebsd.org (Postfix) with ESMTP id BAB4B6355C for ; Tue, 5 Sep 2017 18:22:45 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (higson.cam.lispworks.com [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.15.2/8.15.2) with ESMTP id v85IBmMo041059; Tue, 5 Sep 2017 19:11:48 +0100 (BST) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (localhost.localdomain [127.0.0.1]) by higson.cam.lispworks.com (8.14.4) id v85IBm4n005444; Tue, 5 Sep 2017 19:11:48 +0100 Received: (from martin@localhost) by higson.cam.lispworks.com (8.14.4/8.14.4/Submit) id v85IBmbO005440; Tue, 5 Sep 2017 19:11:48 +0100 Date: Tue, 5 Sep 2017 19:11:48 +0100 Message-Id: <201709051811.v85IBmbO005440@higson.cam.lispworks.com> From: Martin Simmons To: freebsd-fs@freebsd.org In-reply-to: <87k21dzdrp.fsf@thinkpad.rath.org> (message from Nikolaus Rath on Tue, 05 Sep 2017 11:38:18 +0200) Subject: Re: umount() taking minutes for FUSE filesystems References: <87bmn44ruu.fsf@vostro.rath.org> <87o9qyrbs8.fsf@vostro.rath.org> <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> <29de6425-9f92-3bd8-f446-1c9dded33b15@freebsd.org> <87k21dzdrp.fsf@thinkpad.rath.org> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Sep 2017 18:22:47 -0000 >>>>> On Tue, 05 Sep 2017 11:38:18 +0200, Nikolaus Rath said: > > On Sep 05 2017, Stefan Esser wrote: > > Am 04.09.17 um 23:14 schrieb Ben RUBSON: > >> I managed to reproduce the issue. > >> unmount takes exactly 60 seconds, as if a timeout was running. > >> > >> # procstat -kk $! > >> COMM TDNAME KSTACK > >> printcap - mi_switch+0xd2 sleepq_catch_signals+0xb7 > >> sleepq_timedwait_sig+0x10 _sleep+0x26f fdisp_wait_answ+0x171 > >> fuse_vfsop_unmount+0xf5 dounmount+0x9b6 sys_unmount+0x41b > >> amd64_syscall+0x4ce Xfast_syscall+0xfb > >> > >> # uname -sr > >> FreeBSD 11.0-RELEASE-p9 > > > > I have given the exact position of this 60 second msleep() in multiple > > mails before. It is in fuse_ipc.c, the particular msleep with "fu_ans" > > (line 333 in -CURRENT). > > > > I did not try to diagnose, why this particular umount() takes so long, > > while others are fast, but it is obvious that the kernel module does > > wait for a signal at the end of some IPC and the signal is either lost > > or never sent. There is a check for a dead connection, just before the > > msleep() and the connection is considered alive at that point (and > > should be, to support the umount() result being reported). > > > > I did not have time to look into this during the previous week and > > won't during this week, but it should not be too hard to see, what's > > going on. A starting point could be to compare this test with those > > that perform the unmount without delay. > > Probably the crucial difference is that the test that takes long exits > its main loop on its own and then informs the FUSE kernel module about > that, while the other tests terminate the main loop because the kernel > module tells them to do so. What does "informs the FUSE kernel module about that" do to inform it? __Martin From owner-freebsd-fs@freebsd.org Tue Sep 5 19:06:30 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 91D04E197B1 for ; Tue, 5 Sep 2017 19:06:30 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F26C36E03C for ; Tue, 5 Sep 2017 19:06:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id v85J6Twt083135 for ; Tue, 5 Sep 2017 19:06:29 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 161438] [zfs] [panic] recursed on non-recursive spa_namespace_lock when creating zpool from zvol Date: Tue, 05 Sep 2017 19:06:29 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: asomers@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: Overcome By Events X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: resolution cc bug_status Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Sep 2017 19:06:30 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D161438 Alan Somers changed: What |Removed |Added ---------------------------------------------------------------------------- Resolution|--- |Overcome By Events CC| |asomers@FreeBSD.org Status|In Progress |Closed --- Comment #2 from Alan Somers --- As of FreeBSD 12 revision 322823 this is no longer reproducible. However, = its still possible to hit deadlocks when building zpools atop zvols (see https://svnweb.freebsd.org/base?view=3Drevision&revision=3D294329). So that feature is gated by the off-by-default sysctl vfs.zfs.vol.recursive. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed Sep 6 15:36:40 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B97CDE0860A for ; Wed, 6 Sep 2017 15:36:40 +0000 (UTC) (envelope-from paulz@vanderzwan.org) Received: from mail.paztec.nl (mail.paztec.nl [81.171.19.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.paztec.nl", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 31EE17C516 for ; Wed, 6 Sep 2017 15:36:39 +0000 (UTC) (envelope-from paulz@vanderzwan.org) Received: from gaspode.vanderzwan.org (static-145.133.145.71.ip.telfort.nl [145.133.145.71]) (authenticated bits=0) by mail.paztec.nl (8.15.2/8.15.2) with ESMTPSA id v86FZ2Lr040621 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Wed, 6 Sep 2017 17:35:02 +0200 (CEST) (envelope-from paulz@vanderzwan.org) X-Authentication-Warning: vps3.vanderzwan.org: Host static-145.133.145.71.ip.telfort.nl [145.133.145.71] claimed to be gaspode.vanderzwan.org From: Paul van der Zwan Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Arc limits ignored on 11 ? Message-Id: <9BBD08CB-78DC-491B-9AD6-3260655F2CD1@vanderzwan.org> Date: Wed, 6 Sep 2017 17:35:01 +0200 To: FreeBSD FS X-Mailer: Apple Mail (2.3273) X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.6.2 (mail.paztec.nl [81.171.19.13]); Wed, 06 Sep 2017 17:35:02 +0200 (CEST) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Sep 2017 15:36:40 -0000 Hi,=20 I am running 11 (FreeBSD 11.1-STABLE #0 r323186: Tue Sep 5 23:58:23 = CEST 2017) on a 1GB VPS and want to limit ZFS ARC size to leave some room for mysql = etc. It seems that on 11 the ARC max is sysctl is not a hard limit. I just ran svn update and built world and kernel. After rebooting the new kernel and running make buildworld and = mergemaster the ARC is more than 6 time the size I set as max: $ zfs-stats -A ------------------------------------------------------------------------ ZFS Subsystem Report Wed Sep 6 17:28:05 2017 ------------------------------------------------------------------------ ARC Summary: (HEALTHY) Memory Throttle Count: 0 ARC Misc: Deleted: 20.44k Recycle Misses: 0 Mutex Misses: 19.53k Evict Skips: 15.90m ARC Size: 684.81% 273.93 MiB Target Size: (Adaptive) 100.00% 40.00 MiB Min Size (Hard Limit): 75.21% 30.08 MiB Max Size (High Water): 1:1 40.00 MiB ARC Size Breakdown: Recently Used Cache Size: 11.51% 31.53 MiB Frequently Used Cache Size: 88.49% 242.40 MiB ARC Hash Breakdown: Elements Max: 6.64k Elements Current: 89.33% 5.94k Collisions: 8.44k Chain Max: 2 Chains: 99 = =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80= =94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94= =E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2=80=94=E2= =80=94=E2=80=94=E2=80=94 Values from /boot/loader.conf seem to be read but ignored: $ cat /boot/loader.conf aesni_load=3D"YES" geom_eli_load=3D"YES" vfs.root.mountfrom=3D"zfs:zroot/ROOT/default" geli_vtbd0p4_keyfile0_load=3D"YES" geli_vtbd0p4_keyfile0_type=3D"vtbd0p4:geli_keyfile0" geli_vtbd0p4_keyfile0_name=3D"/boot/encryption.key" zfs_load=3D"YES" kern.geom.label.gptid.enable=3D"0" zpool_cache_load=3D"YES" zpool_cache_type=3D"/boot/zfs/zpool.cache" vfs.zfs.arc_max=3D"40M" vfs.zfs.compressed_arc_enabled=3D0 I have set primarycache=3Dmetadata on the zroot to limit cache use but = that does not seem to help: $ zfs get -r primarycache zroot |grep -v @ NAME PROPERTY VALUE SOURCE zroot primarycache metadata local zroot/ROOT primarycache metadata inherited = from zroot zroot/ROOT/default primarycache metadata inherited = from zroot zroot/tmp primarycache metadata inherited = from zroot zroot/usr primarycache metadata inherited = from zroot zroot/usr/home primarycache metadata inherited = from zroot zroot/usr/obj primarycache metadata inherited = from zroot zroot/usr/ports primarycache metadata inherited = from zroot zroot/usr/src primarycache metadata inherited = from zroot zroot/var primarycache metadata inherited = from zroot zroot/var/crash primarycache metadata inherited = from zroot zroot/var/log primarycache metadata inherited = from zroot zroot/var/mail primarycache metadata inherited = from zroot zroot/var/tmp primarycache metadata inherited = from zroot Any idea how I can really restrict ARC size ?=20 TIA Paul From owner-freebsd-fs@freebsd.org Thu Sep 7 19:14:38 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3C856E099FD for ; Thu, 7 Sep 2017 19:14:38 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D8FD57D628 for ; Thu, 7 Sep 2017 19:14:37 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id AFBE620D5D for ; Thu, 7 Sep 2017 15:14:35 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Thu, 07 Sep 2017 15:14:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=WzxXWu86TcDIooD6UvprJkgR9/SsGXqyiIeYNoUlv3I=; b=r1zH6JTe AOa3Z+PqDHpzn7K/3jzYNDb93sNN/iIMxbeEVCGCYYuW51wxL/nI2Yr9TZnR+nY+ xLwqk87nJ3pdOeEVbPJ0n7jZ4tzXKD6hR27qeofbVtKeLbE/41OgtvibnbkUGFjx Scvu4ZFOVyJP+PvPBPMCs8kZmBhVDefRKbYiHUmzQ9TqBhopb57E8MH0ASlV8OjS cYJoWA5mwq//TcjHQcDjPIZI4h53VdV4XaDMlMlHwIKDVPrrRk56kKvTJnDvQHaz cnW48ZCytHw6h/Ps1ony4kOv4b4GMSoaM7VzMbKw7za1hoAD7aF3Dv/H87o75NuW SWRYk5ncuaBz/g== X-ME-Sender: X-Sasl-enc: NyK5mJReodyFPwLz68e3DIKy+p0oxAeerhFEtJBvwqFL 1504811675 Received: from ebox.rath.org (ebox.rath.org [45.79.69.51]) by mail.messagingengine.com (Postfix) with ESMTPA id 757C37F982 for ; Thu, 7 Sep 2017 15:14:35 -0400 (EDT) Received: from thinkpad.rath.org (thinkpad [192.168.12.2]) by ebox.rath.org (Postfix) with ESMTPS id 6E8A2115 for ; Thu, 7 Sep 2017 19:14:34 +0000 (UTC) Received: by thinkpad.rath.org (Postfix, from userid 1000) id 44B56BFEE0; Thu, 7 Sep 2017 21:14:32 +0200 (CEST) From: Nikolaus Rath To: freebsd-fs@freebsd.org Subject: Re: umount() taking minutes for FUSE filesystems References: <87bmn44ruu.fsf@vostro.rath.org> <87o9qyrbs8.fsf@vostro.rath.org> <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> <29de6425-9f92-3bd8-f446-1c9dded33b15@freebsd.org> <87k21dzdrp.fsf@thinkpad.rath.org> <201709051811.v85IBmbO005440@higson.cam.lispworks.com> Mail-Copies-To: never Mail-Followup-To: freebsd-fs@freebsd.org Date: Thu, 07 Sep 2017 21:14:32 +0200 In-Reply-To: <201709051811.v85IBmbO005440@higson.cam.lispworks.com> (Martin Simmons's message of "Tue, 5 Sep 2017 19:11:48 +0100") Message-ID: <87ingugw2v.fsf@thinkpad.rath.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Sep 2017 19:14:38 -0000 On Sep 05 2017, Martin Simmons wrote: >> Probably the crucial difference is that the test that takes long exits >> its main loop on its own and then informs the FUSE kernel module about >> that, while the other tests terminate the main loop because the kernel >> module tells them to do so. > > What does "informs the FUSE kernel module about that" do to inform it? It calls unmount. https://github.com/libfuse/libfuse/blob/master/lib/mount_bsd.c#L127 Best, Nikolaus --=20 GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2= =AB From owner-freebsd-fs@freebsd.org Fri Sep 8 11:43:25 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46CC1E130C8 for ; Fri, 8 Sep 2017 11:43:25 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [46.17.166.21]) by mx1.freebsd.org (Postfix) with ESMTP id A74026AFAA for ; Fri, 8 Sep 2017 11:43:23 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (higson.cam.lispworks.com [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.15.2/8.15.2) with ESMTP id v88BhEZb076758; Fri, 8 Sep 2017 12:43:14 +0100 (BST) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (localhost.localdomain [127.0.0.1]) by higson.cam.lispworks.com (8.14.4) id v88BhEL5001630; Fri, 8 Sep 2017 12:43:14 +0100 Received: (from martin@localhost) by higson.cam.lispworks.com (8.14.4/8.14.4/Submit) id v88BhEOn001626; Fri, 8 Sep 2017 12:43:14 +0100 Date: Fri, 8 Sep 2017 12:43:14 +0100 Message-Id: <201709081143.v88BhEOn001626@higson.cam.lispworks.com> From: Martin Simmons To: freebsd-fs@freebsd.org In-reply-to: <87ingugw2v.fsf@thinkpad.rath.org> (message from Nikolaus Rath on Thu, 07 Sep 2017 21:14:32 +0200) Subject: Re: umount() taking minutes for FUSE filesystems References: <87bmn44ruu.fsf@vostro.rath.org> <87o9qyrbs8.fsf@vostro.rath.org> <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> <29de6425-9f92-3bd8-f446-1c9dded33b15@freebsd.org> <87k21dzdrp.fsf@thinkpad.rath.org> <201709051811.v85IBmbO005440@higson.cam.lispworks.com> <87ingugw2v.fsf@thinkpad.rath.org> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Sep 2017 11:43:25 -0000 >>>>> On Thu, 07 Sep 2017 21:14:32 +0200, Nikolaus Rath said: > > On Sep 05 2017, Martin Simmons wrote: > >> Probably the crucial difference is that the test that takes long exits > >> its main loop on its own and then informs the FUSE kernel module about > >> that, while the other tests terminate the main loop because the kernel > >> module tells them to do so. > > > > What does "informs the FUSE kernel module about that" do to inform it? > > It calls unmount. > https://github.com/libfuse/libfuse/blob/master/lib/mount_bsd.c#L127 It seems to me that the following occurs: 1. The user program exits the main loop. 2. The user program sends unmount to the kernel. 3. The kernel sends FUSE_DESTROY to the user program. 4. The kernel waits for the user program to respond to FUSE_DESTROY. 5. The user program does not respond because it has exited the main loop. 6. The kernel wait times out after 60 seconds. Perhaps the solution is to close the fd before calling unmount(), like the Linux code in https://github.com/libfuse/libfuse/blob/master/lib/mount.c#L275 does? That will prevent the kernel from sending FUSE_DESTROY, but what is the purpose of FUSE_DESTROY if the user program can't respond to it? __Martin From owner-freebsd-fs@freebsd.org Fri Sep 8 13:42:25 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B1F3E18A85 for ; Fri, 8 Sep 2017 13:42:25 +0000 (UTC) (envelope-from longwitz@incore.de) Received: from dss.incore.de (dss.incore.de [195.145.1.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E280D7FD44 for ; Fri, 8 Sep 2017 13:42:24 +0000 (UTC) (envelope-from longwitz@incore.de) Received: from inetmail.dmz (inetmail.dmz [10.3.0.3]) by dss.incore.de (Postfix) with ESMTP id E7718679D8 for ; Fri, 8 Sep 2017 15:42:15 +0200 (CEST) X-Virus-Scanned: amavisd-new at incore.de Received: from dss.incore.de ([10.3.0.3]) by inetmail.dmz (inetmail.dmz [10.3.0.3]) (amavisd-new, port 10024) with LMTP id O42wkulTjw4a for ; Fri, 8 Sep 2017 15:42:14 +0200 (CEST) Received: from mail.local.incore (fwintern.dmz [10.0.0.253]) by dss.incore.de (Postfix) with ESMTP id 8BD4A6798C for ; Fri, 8 Sep 2017 15:42:13 +0200 (CEST) Received: from bsdlo.incore (bsdlo.incore [192.168.0.84]) by mail.local.incore (Postfix) with ESMTP id 73D05508A9 for ; Fri, 8 Sep 2017 15:42:13 +0200 (CEST) Message-ID: <59B29E35.9000506@incore.de> Date: Fri, 08 Sep 2017 15:42:13 +0200 From: Andreas Longwitz User-Agent: Thunderbird 2.0.0.19 (X11/20090113) MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: fsync: giving up on dirty on ufs partitions running vfs_write_suspend() Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Sep 2017 13:42:25 -0000 I try to describe the cause for the "fsync: given up on dirty" problem described in https://lists.freebsd.org/pipermail/freebsd-fs/2012-February/013804.html or https://lists.freebsd.org/pipermail/freebsd-fs/2013-August/018163.html Now I run FreeBSD 10.3 Stable r317936 and sometimes I see messages like dssbkp4 kernel: fsync: giving up on dirty dssbkp4 kernel: 0xfffff80040d6c938: tag devfs, type VCHR dssbkp4 kernel: usecount 1, writecount 0, refcount 47 mountedhere 0xfffff8004083a200 dssbkp4 kernel: flags (VI_ACTIVE) dssbkp4 kernel: v_object 0xfffff800409b3500 ref 0 pages 1138 cleanbuf 42 dirtybuf 4 dssbkp4 kernel: lock type devfs: EXCL by thread 0xfffff800403a8a00 (pid 26, g_journal switcher, tid 100181) dssbkp4 kernel: dev mirror/gmbkp4p5.journal dssbkp4 kernel: GEOM_JOURNAL: Cannot suspend file system /home (error=35). on all of my servers running gjournal. Similar messages can be seen when a snapshot is taken (e.g. dump -L) on a arbitrary ufs partition. In all these cases the function vfs_write_suspend() was called which returned EAGAIN. This error code is set in vop_stdfsync(), when the above messages are created. First I was confused about the "mountedhere" address, because the given address does not point to a "struct mount" but (as type = VCHR indicates) to a "struct cdev". Threfore I suggest the following patch to improve the output of vn_printf() using the textstrings from defines in /sys/sys/vnode.h: --- vfs_subr.c.orig 2017-05-08 14:17:38.000000000 +0200 +++ vfs_subr.c 2017-08-30 10:45:47.549740000 +0200 @@ -3003,6 +3003,8 @@ static char *typename[] = {"VNON", "VREG", "VDIR", "VBLK", "VCHR", "VLNK", "VSOCK", "VFIFO", "VBAD", "VMARKER"}; +static char *typetext[] = +{"", "", "mountedhere", "", "rdev", "", "socket", "fifoinfo", "", ""}; void vn_printf(struct vnode *vp, const char *fmt, ...) @@ -3016,8 +3018,9 @@ va_end(ap); printf("%p: ", (void *)vp); printf("tag %s, type %s\n", vp->v_tag, typename[vp->v_type]); - printf(" usecount %d, writecount %d, refcount %d mountedhere %p\n", - vp->v_usecount, vp->v_writecount, vp->v_holdcnt, vp->v_mountedhere); + printf(" usecount %d, writecount %d, refcount %d %s %p\n", + vp->v_usecount, vp->v_writecount, vp->v_holdcnt, typetext[vp->v_type], + vp->v_mountedhere); buf[0] = '\0'; buf[1] = '\0'; if (vp->v_vflag & VV_ROOT) Second I found, that the "dirty" situation during vfs_write_suspend() only occurs when a big file (more than 10G on a partition of 116G) is removed. If vfs_write_suspend() is called immediately after "rm bigfile", then in vop_stdfsync() 1000 tries (maxretry) are done to wait for the "rm bigfile" to complete. Because a lot of bitmap writes must be done, the value 1000 is not sufficient on my servers. I have increased maxretry and in the worst case I saw 8650 tries to complete without "dirty". In this case the time spent in vop_stdfsync() was about 0,5 seconds. The following patch solves the "dirty problem" for me: --- vfs_default.c.orig 2016-10-24 12:26:57.000000000 +0200 +++ vfs_default.c 2017-09-08 12:49:18.059970000 +0200 @@ -644,7 +644,7 @@ struct bufobj *bo; struct buf *nbp; int error = 0; - int maxretry = 1000; /* large, arbitrarily chosen */ + int maxretry = 100000; /* large, arbitrarily chosen */ bo = &vp->v_bufobj; BO_LOCK(bo); --- Andreas Longwitz From owner-freebsd-fs@freebsd.org Fri Sep 8 14:22:24 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9A5D8E1AA2C for ; Fri, 8 Sep 2017 14:22:24 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 690D181270 for ; Fri, 8 Sep 2017 14:22:23 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 94AD821006 for ; Fri, 8 Sep 2017 10:22:22 -0400 (EDT) Received: from frontend2 ([10.202.2.161]) by compute1.internal (MEProxy); Fri, 08 Sep 2017 10:22:22 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=5OSY6kzqL0Zkm7MSNCvXHonGIjkCdZZ5/NvfdTQczn4=; b=Tkn+SY5Q pCtA8IQ1tdVJePdFONwwacSj9kBoG2Scx6XqdP16Gfto/ltJsy2udaqo7iZ89Qhx 6LWbpGfz+dvP5ctlFY/iD1SfuM26CDYUlaNWvEdOc+dlY7Hfg+VUFaQebFgM7B7y bc+lL/WNN48CccUAfzkbzEHcmEUZPHygIOuHY0v2CEEBJRQk3JBaSxcAIV4UB/4I Z9RjuBrub2X5o7xLJ4L3YcJ/xI2br/pzUwGOBCKADCzLJ85qMhZrD+BFrkawv1yU xC8pSyrD5MoZUJxzrAmBumjh79xugToX8Ar1odqpUuwb6aJvm6CjYnwJuxP7E8Ie v32wiSgrVorofw== X-ME-Sender: X-Sasl-enc: Pwr0zfPR2djfkEcKhKGTK/QnHD7It47nsUdJe/ng62c9 1504880542 Received: from ebox.rath.org (ebox.rath.org [45.79.69.51]) by mail.messagingengine.com (Postfix) with ESMTPA id 50D7C2494C for ; Fri, 8 Sep 2017 10:22:22 -0400 (EDT) Received: from vostro.rath.org (vostro [192.168.12.4]) by ebox.rath.org (Postfix) with ESMTPS id 6B273195 for ; Fri, 8 Sep 2017 14:22:21 +0000 (UTC) Received: by vostro.rath.org (Postfix, from userid 1000) id 19C62102B9C; Fri, 8 Sep 2017 16:22:20 +0200 (CEST) From: Nikolaus Rath To: freebsd-fs@freebsd.org Subject: Re: umount() taking minutes for FUSE filesystems References: <87bmn44ruu.fsf@vostro.rath.org> <87o9qyrbs8.fsf@vostro.rath.org> <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> <29de6425-9f92-3bd8-f446-1c9dded33b15@freebsd.org> <87k21dzdrp.fsf@thinkpad.rath.org> <201709051811.v85IBmbO005440@higson.cam.lispworks.com> <87ingugw2v.fsf@thinkpad.rath.org> <201709081143.v88BhEOn001626@higson.cam.lispworks.com> Mail-Copies-To: never Mail-Followup-To: freebsd-fs@freebsd.org Date: Fri, 08 Sep 2017 15:22:19 +0100 In-Reply-To: <201709081143.v88BhEOn001626@higson.cam.lispworks.com> (Martin Simmons's message of "Fri, 8 Sep 2017 12:43:14 +0100") Message-ID: <87lglp6zj8.fsf@vostro.rath.org> User-Agent: Gnus/5.130014 (Ma Gnus v0.14) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Sep 2017 14:22:24 -0000 On Sep 08 2017, Martin Simmons wrote: >>>>>> On Thu, 07 Sep 2017 21:14:32 +0200, Nikolaus Rath said: >>=20 >> On Sep 05 2017, Martin Simmons wrote: >> >> Probably the crucial difference is that the test that takes long exits >> >> its main loop on its own and then informs the FUSE kernel module about >> >> that, while the other tests terminate the main loop because the kernel >> >> module tells them to do so. >> > >> > What does "informs the FUSE kernel module about that" do to inform it? >>=20 >> It calls unmount. >> https://github.com/libfuse/libfuse/blob/master/lib/mount_bsd.c#L127 > > It seems to me that the following occurs: > > 1. The user program exits the main loop. > 2. The user program sends unmount to the kernel. > 3. The kernel sends FUSE_DESTROY to the user program. > 4. The kernel waits for the user program to respond to FUSE_DESTROY. > 5. The user program does not respond because it has exited the main loop. > 6. The kernel wait times out after 60 seconds. > > Perhaps the solution is to close the fd before calling unmount(), like the > Linux code in https://github.com/libfuse/libfuse/blob/master/lib/mount.c#= L275 > does? That sounds reasonable, will give it a shot. > That will prevent the kernel from sending FUSE_DESTROY, but what is the > purpose of FUSE_DESTROY if the user program can't respond to it? In most cases, umount() is not called by the filesystem but by the user. In that case, the FUSE_DESTROY request gives the filesystem a chance to clean-up and exit the main loop. That said, the filesystem could also detect the unmount by the kernel closing the fd. So I'm not sure what is gained by the extra request either... Theoretically, at least under Linux the destroy handler could perform some notify_* operations, but I don't see how that would be useful when the filesystem will be unmounted anyway. Best, -Nikolaus --=20 GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2= =AB From owner-freebsd-fs@freebsd.org Fri Sep 8 14:54:25 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E4870E1C304 for ; Fri, 8 Sep 2017 14:54:25 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: from mail-yw0-f181.google.com (mail-yw0-f181.google.com [209.85.161.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B1F8D8252C for ; Fri, 8 Sep 2017 14:54:25 +0000 (UTC) (envelope-from cse.cem@gmail.com) Received: by mail-yw0-f181.google.com with SMTP id v72so5965825ywa.3 for ; Fri, 08 Sep 2017 07:54:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:reply-to:in-reply-to:references :from:date:message-id:subject:to; bh=LaOEVO7M/7fHC3WL1EfJF2nbz4/1LQYJLOr+6oY35ms=; b=rX2x4nWcPcet8Me2jHBvXS6wYmxjcbGsRTZWLZxhEUZTpG4bX9FYtve/art5gTSUYX fWXgrAo6a8LvbFxUsUPDSQ1THsDyxcroz9jv2jm2DrzA2WD1o7bfnAysFJnpf3BrywhI tahZtNJcmdbb19f1vaycWJFPuw1kSeQLXg15UIJciNPcIW/8ack6F8b+RuuuBsYnIGo7 yj11N0puThu6gu4pZIvySP0x2v5DKcNcIoUtr/nkwMamAT+JV7irEuQRinMJrHzbRLFw YaT2dWQ1aiZar7MKDi2eqFCrMyvwgtalTRluig5CnyPQ7DbvOLxbZwWZvZ5COnpMetAs iqfw== X-Gm-Message-State: AHPjjUjMmwf/0sNNS4+izHFHqkdUS6xse7VZvjKIEHJ1JrqWhso6ACPj IzGITHpFGosoNCj/MNo= X-Received: by 10.129.175.103 with SMTP id x39mr2691330ywj.226.1504882458581; Fri, 08 Sep 2017 07:54:18 -0700 (PDT) Received: from mail-it0-f52.google.com (mail-it0-f52.google.com. [209.85.214.52]) by smtp.gmail.com with ESMTPSA id y7sm746592ywg.36.2017.09.08.07.54.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 Sep 2017 07:54:18 -0700 (PDT) Received: by mail-it0-f52.google.com with SMTP id c195so2871084itb.1 for ; Fri, 08 Sep 2017 07:54:18 -0700 (PDT) X-Google-Smtp-Source: AOwi7QD40IyzQC36FaNa3YbNGh+/JLPTT/5KEuYGyEMKuGsQdnyXDhsYGCy/EEacBc3jKXLBgms2aPYwF2yKPJ4SHh4= X-Received: by 10.36.17.85 with SMTP id 82mr1274731itf.78.1504882457856; Fri, 08 Sep 2017 07:54:17 -0700 (PDT) MIME-Version: 1.0 Reply-To: cem@freebsd.org Received: by 10.2.81.131 with HTTP; Fri, 8 Sep 2017 07:54:17 -0700 (PDT) In-Reply-To: <87lglp6zj8.fsf@vostro.rath.org> References: <87bmn44ruu.fsf@vostro.rath.org> <87o9qyrbs8.fsf@vostro.rath.org> <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> <29de6425-9f92-3bd8-f446-1c9dded33b15@freebsd.org> <87k21dzdrp.fsf@thinkpad.rath.org> <201709051811.v85IBmbO005440@higson.cam.lispworks.com> <87ingugw2v.fsf@thinkpad.rath.org> <201709081143.v88BhEOn001626@higson.cam.lispworks.com> <87lglp6zj8.fsf@vostro.rath.org> From: Conrad Meyer Date: Fri, 8 Sep 2017 07:54:17 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: umount() taking minutes for FUSE filesystems To: freebsd-fs@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Sep 2017 14:54:26 -0000 On Fri, Sep 8, 2017 at 7:22 AM, Nikolaus Rath wrote: > In most cases, umount() is not called by the filesystem but by the > user. In that case, the FUSE_DESTROY request gives the filesystem a > chance to clean-up and exit the main loop. > > That said, the filesystem could also detect the unmount by the kernel > closing the fd. So I'm not sure what is gained by the extra request > either... Theoretically, at least under Linux the destroy handler could > perform some notify_* operations, but I don't see how that would be > useful when the filesystem will be unmounted anyway. Hi Nikolaus, It seems like the destroy notification is a good chance for the filesystem to flush any dirty buffers before umount completes. Sure, the filesystem could just do that when it detects the kernel has closed the fd, but then there is no guarantee it happens during umount. It seems that it should happen during umount for logical correctness. Best, Conrad From owner-freebsd-fs@freebsd.org Fri Sep 8 17:28:39 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D2590E23E26 for ; Fri, 8 Sep 2017 17:28:39 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [46.17.166.21]) by mx1.freebsd.org (Postfix) with ESMTP id 6ACED633C9 for ; Fri, 8 Sep 2017 17:28:38 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (higson.cam.lispworks.com [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.15.2/8.15.2) with ESMTP id v88HSXhX093393; Fri, 8 Sep 2017 18:28:33 +0100 (BST) (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com (localhost.localdomain [127.0.0.1]) by higson.cam.lispworks.com (8.14.4) id v88HSXRY010333; Fri, 8 Sep 2017 18:28:33 +0100 Received: (from martin@localhost) by higson.cam.lispworks.com (8.14.4/8.14.4/Submit) id v88HSXP7010329; Fri, 8 Sep 2017 18:28:33 +0100 Date: Fri, 8 Sep 2017 18:28:33 +0100 Message-Id: <201709081728.v88HSXP7010329@higson.cam.lispworks.com> From: Martin Simmons To: freebsd-fs@freebsd.org In-reply-to: <87lglp6zj8.fsf@vostro.rath.org> (message from Nikolaus Rath on Fri, 08 Sep 2017 15:22:19 +0100) Subject: Re: umount() taking minutes for FUSE filesystems References: <87bmn44ruu.fsf@vostro.rath.org> <87o9qyrbs8.fsf@vostro.rath.org> <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> <29de6425-9f92-3bd8-f446-1c9dded33b15@freebsd.org> <87k21dzdrp.fsf@thinkpad.rath.org> <201709051811.v85IBmbO005440@higson.cam.lispworks.com> <87ingugw2v.fsf@thinkpad.rath.org> <201709081143.v88BhEOn001626@higson.cam.lispworks.com> <87lglp6zj8.fsf@vostro.rath.org> X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Sep 2017 17:28:40 -0000 >>>>> On Fri, 08 Sep 2017 15:22:19 +0100, Nikolaus Rath said: > > On Sep 08 2017, Martin Simmons wrote: > > > That will prevent the kernel from sending FUSE_DESTROY, but what is the > > purpose of FUSE_DESTROY if the user program can't respond to it? > > In most cases, umount() is not called by the filesystem but by the > user. In that case, the FUSE_DESTROY request gives the filesystem a > chance to clean-up and exit the main loop. OK, that makes sense. > That said, the filesystem could also detect the unmount by the kernel > closing the fd. So I'm not sure what is gained by the extra request > either... Theoretically, at least under Linux the destroy handler could > perform some notify_* operations, but I don't see how that would be > useful when the filesystem will be unmounted anyway. I don't think the kernel itself ever closes the fd (except implicitly in the case where the filesystem program dies unexpected). __Martin From owner-freebsd-fs@freebsd.org Fri Sep 8 22:08:33 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 66D46E0DC00; Fri, 8 Sep 2017 22:08:33 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from citapm.icyb.net.ua (citapm.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 80E346D05C; Fri, 8 Sep 2017 22:08:32 +0000 (UTC) (envelope-from avg@FreeBSD.org) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citapm.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id BAA02683; Sat, 09 Sep 2017 01:08:29 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1dqRRt-000JVH-CU; Sat, 09 Sep 2017 01:08:29 +0300 To: freebsd-doc@FreeBSD.org, freebsd-fs@FreeBSD.org From: Andriy Gapon Subject: re-synchronize zfs man pages with illumos Message-ID: <19090d61-57d1-e436-f1d1-4e6e49ec291e@FreeBSD.org> Date: Sat, 9 Sep 2017 01:07:33 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Sep 2017 22:08:33 -0000 Originally ZFS manual pages were written using a different formatting language from what we use in FreeBSD. I am not very knowledgeable in this area, but the experts should recognize the language by things like \fB, \fR, etc. Martin Matuska (mm) converted the manuals to the FreeBSD mdoc in r228019 https://svnweb.freebsd.org/base?view=revision&revision=228019 Recently illumos switched to using mdoc format as well. That happened gradually and the changes were usually mixed with other changes. For example, the zpool manual was converted in this change: https://svnweb.freebsd.org/base?view=revision&revision=318936 https://github.com/illumos/illumos-gate/commit/c8323d4323a565676ba44883bfeb289d9ed8813e It is not surprising, of course, that the illumos mdoc pages are quite different from their FreeBSD counterparts given the independent conversions. So, at the moment we have several classes of differences that are not easy to identify via textual differences of the source files alone. 1. Differences in the content due to underlying differences between illumos and FreeBSD, e.g. zones vs jails or different sample disk names, etc. 2. Whitespace differences where exactly the same words are split differently between the lines, etc. 3. Style differences. For example, explicit parentheses vs Pq/Po/Pc; use of Tn in FreeBSD; much more frequent use of Ns macro in illumos; quotes vs a different font (e.g. Qq vs Sy); etc. 4. Mistakes in either FreeBSD or illumos version. For example, the FreeBSD versions often continue a line after a period. 5. Independent / diverging language corrections. My impression is that the illumos pages are cleaner than the FreeBSD ones at the moment. My inclination is to suggest that we "restart" our pages from the current illumos pages and "rebase" FreeBSD-specific changes on top of those. I realize that that would be a lot of work. But the benefits are quite obvious as it would be much easier to merge changes both ways. Comments, suggestions, offers of help are all very welcome! Thanks. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Fri Sep 8 22:34:41 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 725AFE0EF5C; Fri, 8 Sep 2017 22:34:41 +0000 (UTC) (envelope-from russ.haley@gmail.com) Received: from mail-lf0-x22e.google.com (mail-lf0-x22e.google.com [IPv6:2a00:1450:4010:c07::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ED5816DB77; Fri, 8 Sep 2017 22:34:40 +0000 (UTC) (envelope-from russ.haley@gmail.com) Received: by mail-lf0-x22e.google.com with SMTP id q132so8319089lfe.5; Fri, 08 Sep 2017 15:34:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=gFpsnnIFqYXLYl7f+rZd8tBy00VQ/mK1DtH6UIfbdjc=; b=WUVLf0ZwDz/g1NI1zmDXQ/1ta7KU6NrfMnnAKdFL5aS4AEKW62dtEEEhVYITUEtvgI lMb8qm0tQnWMwWx54xjJSEefXzLQB0aJc1ZDoBnBOFjha4RzghT5P2Qm1Rt5O3QZr78w Ugf/Y6AqZwju0Fwcc+ktYpo9wrhARy89pgWP9Xdks7DnuX8wXI6Pudu2joOErXHv6d2k TY9oUCYntpHB2ZdZCSgDxhvjbGMRNnyoJPwAFYBvefi+9aSpMySPEns2YMUi+HsKk8Xp W+CzIzHAeZ8GG0NuYfGeIvKzS7WYXUCAyc17DfagJdv4zKCtXiqPgu1+9h+nu8zVTLwe fCLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=gFpsnnIFqYXLYl7f+rZd8tBy00VQ/mK1DtH6UIfbdjc=; b=kUZvF1ssYWsrT8Q+YxB1wdNjGIczduGZTkVwFNrGbfMarMw4dYyTJ7s9Lq0igPFV+Y 5DOZtZ71WUJk0cTTTWsGzpxu226ZCM7IJbTpwoAk3GJh5BhdzeAMDl2YDQS2DZEAj4iQ MAFLX5fdoHqU7DRuunocI9scn6lpJnanLN0PVYv2ubNdUDgpBDsl0jQsAsU9gZ+3cNPG yEyiDLtIcZyHFDWDWcbo6ZcYQN8lkfNQvTRelpDUMlP4lF0QDo1NXkvycFohZ90vrGlj DRhEap3aXW5IlsisqjDQw6/YZPFgLujvO7NhSIEMGbNXI1/94zVzWzlvOWlJa9zzJ25s l+qw== X-Gm-Message-State: AHPjjUjNOyZUnYMFGxfHz3ZYYKcMnozEDA+CjFLFe0rJfeU2DU+mkTZH bVIPSZUb5vkQEPuzQ95w4wbIhZfZd8wG X-Google-Smtp-Source: AOwi7QDcT2iy1jfKuFPTbg1xN/jDOkS66nFP8h1CNAiZt1bofaAt02x0rIewFx9OrJAT+NzkaGl3mCKVqt85R3HNmow= X-Received: by 10.46.83.23 with SMTP id h23mr1559099ljb.97.1504910078025; Fri, 08 Sep 2017 15:34:38 -0700 (PDT) MIME-Version: 1.0 Received: by 10.46.81.18 with HTTP; Fri, 8 Sep 2017 15:34:37 -0700 (PDT) In-Reply-To: <19090d61-57d1-e436-f1d1-4e6e49ec291e@FreeBSD.org> References: <19090d61-57d1-e436-f1d1-4e6e49ec291e@FreeBSD.org> From: Russell Haley Date: Fri, 8 Sep 2017 15:34:37 -0700 Message-ID: Subject: Re: re-synchronize zfs man pages with illumos To: Andriy Gapon Cc: freebsd-doc@freebsd.org, freebsd-fs@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Sep 2017 22:34:41 -0000 On Fri, Sep 8, 2017 at 3:07 PM, Andriy Gapon wrote: > > Originally ZFS manual pages were written using a different formatting language > from what we use in FreeBSD. I am not very knowledgeable in this area, but the > experts should recognize the language by things like \fB, \fR, etc. > > Martin Matuska (mm) converted the manuals to the FreeBSD mdoc in r228019 > https://svnweb.freebsd.org/base?view=revision&revision=228019 > > Recently illumos switched to using mdoc format as well. > That happened gradually and the changes were usually mixed with other changes. > For example, the zpool manual was converted in this change: > https://svnweb.freebsd.org/base?view=revision&revision=318936 > https://github.com/illumos/illumos-gate/commit/c8323d4323a565676ba44883bfeb289d9ed8813e > > It is not surprising, of course, that the illumos mdoc pages are quite different > from their FreeBSD counterparts given the independent conversions. > > So, at the moment we have several classes of differences that are not easy to > identify via textual differences of the source files alone. > > 1. Differences in the content due to underlying differences between illumos and > FreeBSD, e.g. zones vs jails or different sample disk names, etc. > > 2. Whitespace differences where exactly the same words are split differently > between the lines, etc. > > 3. Style differences. For example, explicit parentheses vs Pq/Po/Pc; use of Tn > in FreeBSD; much more frequent use of Ns macro in illumos; quotes vs a different > font (e.g. Qq vs Sy); etc. > > 4. Mistakes in either FreeBSD or illumos version. For example, the FreeBSD > versions often continue a line after a period. > > 5. Independent / diverging language corrections. > > My impression is that the illumos pages are cleaner than the FreeBSD ones at the > moment. My inclination is to suggest that we "restart" our pages from the > current illumos pages and "rebase" FreeBSD-specific changes on top of those. > > I realize that that would be a lot of work. But the benefits are quite obvious > as it would be much easier to merge changes both ways. > > Comments, suggestions, offers of help are all very welcome! > Thanks. I think anything that can be done to both strengthen the Illumos projects and create tighter coupling between BSD projects is a good thing. Documentation work is also an excellent way to learn a technology. I would happily contribute to this project and it would be a good one to push towards new comers looking to learn/leverage ZFS. It would also create an uptick in activity and could be used as advocacy/advertising in tech publications. ZFS is a big pull from GNU/Linux to FreeBSD. I also think that DTrace documentation on FreeBSD could use some cleanup and renewal. It's such an awesome feature. Russ From owner-freebsd-fs@freebsd.org Sat Sep 9 02:23:49 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B150AE1C795 for ; Sat, 9 Sep 2017 02:23:49 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-lf0-x235.google.com (mail-lf0-x235.google.com [IPv6:2a00:1450:4010:c07::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 367D475682 for ; Sat, 9 Sep 2017 02:23:49 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-lf0-x235.google.com with SMTP id c80so8927287lfh.0 for ; Fri, 08 Sep 2017 19:23:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=4Zo3R73KSx+Zge1Ehyp36DXJhxip0y6nnpPryEsmSCs=; b=EQs3D1MoKHF+gsA6NOb9Y4b7Osts+v+QYrvOGjUujz5x74h2vQhetaT7wMfbJy9YWU zaJST5mS+/0XYTByi7A25+NU/5wMYV6/XJkSaPYyYLw/AVJVenDYTtWKeP7zbhPIkslq hbfzbxj45GPWHJCjM7JKCmJJmwA20HFj64yTE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=4Zo3R73KSx+Zge1Ehyp36DXJhxip0y6nnpPryEsmSCs=; b=phUCSn2i4BY1NymIs5cdGNW6QhgZlpxrQarwahByo0OWoqPLjRRrplZ6rjD7A9clWg FTsybAW48EFnOE8AmkQMpmY/OliT/CrF4gezBfMKEaKQg00cEi7CZtVMNv387VODZtKq 8r0lX4OVuzop4sDhaAr5XwJX2KmU+jnXlW9uo5GP4Gt4/JxNiTDOY5XQy/+4VfhR99vw EmyTgRWv152Jq/ApwkalXg7uYom4xrPuhpgq6kFz20/uS77ml94oW/XtbXeUbbKdwDgu OhcYh5yMZpgGisSFeMlHz3jiRyYR24LfmoQ+viEmCLAsucR/pv7Qzukxe4bLDQ2nVC6+ I11Q== X-Gm-Message-State: AHPjjUhu95uRlNN56m2JXF6F6YhkkW9elD6ZuoLvIyUJ6blEdQLBZbU4 pFcJqVP6GstBYKSecNEnZ4GFXLMySMME X-Google-Smtp-Source: AOwi7QBmui4PRVniJ4p+eXPzu+Jbo9UFofW01atfz28SPDSjRlDzweoN4h6kbyU5qM+G5Vg+Yuyd2Z2eh9RX8Ez8Mhw= X-Received: by 10.25.22.106 with SMTP id m103mr1725229lfi.168.1504923827175; Fri, 08 Sep 2017 19:23:47 -0700 (PDT) MIME-Version: 1.0 Received: by 10.25.83.213 with HTTP; Fri, 8 Sep 2017 19:23:46 -0700 (PDT) In-Reply-To: <19090d61-57d1-e436-f1d1-4e6e49ec291e@FreeBSD.org> References: <19090d61-57d1-e436-f1d1-4e6e49ec291e@FreeBSD.org> From: Matthew Ahrens Date: Fri, 8 Sep 2017 19:23:46 -0700 Message-ID: Subject: Re: re-synchronize zfs man pages with illumos To: Andriy Gapon Cc: freebsd-doc@freebsd.org, freebsd-fs Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Sep 2017 02:23:49 -0000 Andriy, thanks for your continued efforts to improve OpenZFS on both FreeBSD and illumos. Anything we can do to ease porting changes both ways will be great. If you find things that could be improved in the illumos manpages, I'd be happy to help you get those upstream too. I'm not an expert in mandoc, but if you think the FreeBSD style or content is better than what was done in illumos, we should consider making changes in illumos too. --matt On Fri, Sep 8, 2017 at 3:07 PM, Andriy Gapon wrote: > > Originally ZFS manual pages were written using a different formatting > language > from what we use in FreeBSD. I am not very knowledgeable in this area, > but the > experts should recognize the language by things like \fB, \fR, etc. > > Martin Matuska (mm) converted the manuals to the FreeBSD mdoc in r228019 > https://svnweb.freebsd.org/base?view=revision&revision=228019 > > Recently illumos switched to using mdoc format as well. > That happened gradually and the changes were usually mixed with other > changes. > For example, the zpool manual was converted in this change: > https://svnweb.freebsd.org/base?view=revision&revision=318936 > https://github.com/illumos/illumos-gate/commit/ > c8323d4323a565676ba44883bfeb289d9ed8813e > > It is not surprising, of course, that the illumos mdoc pages are quite > different > from their FreeBSD counterparts given the independent conversions. > > So, at the moment we have several classes of differences that are not easy > to > identify via textual differences of the source files alone. > > 1. Differences in the content due to underlying differences between > illumos and > FreeBSD, e.g. zones vs jails or different sample disk names, etc. > > 2. Whitespace differences where exactly the same words are split > differently > between the lines, etc. > > 3. Style differences. For example, explicit parentheses vs Pq/Po/Pc; use > of Tn > in FreeBSD; much more frequent use of Ns macro in illumos; quotes vs a > different > font (e.g. Qq vs Sy); etc. > > 4. Mistakes in either FreeBSD or illumos version. For example, the FreeBSD > versions often continue a line after a period. > > 5. Independent / diverging language corrections. > > My impression is that the illumos pages are cleaner than the FreeBSD ones > at the > moment. My inclination is to suggest that we "restart" our pages from the > current illumos pages and "rebase" FreeBSD-specific changes on top of > those. > > I realize that that would be a lot of work. But the benefits are quite > obvious > as it would be much easier to merge changes both ways. > > Comments, suggestions, offers of help are all very welcome! > Thanks. > -- > Andriy Gapon > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Sat Sep 9 21:53:28 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BAAAE0CA4E for ; Sat, 9 Sep 2017 21:53:28 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5B3437F155 for ; Sat, 9 Sep 2017 21:53:27 +0000 (UTC) (envelope-from Nikolaus@rath.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 2D8AD210AF for ; Sat, 9 Sep 2017 17:53:21 -0400 (EDT) Received: from frontend1 ([10.202.2.160]) by compute1.internal (MEProxy); Sat, 09 Sep 2017 17:53:21 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc:x-sasl-enc; s= fm1; bh=AWgC1uNeU/HTQFc5T/rtTrd27K5zTUpZP/0k0FoSK0o=; b=B7UmXxA/ YhvPpEoiw+rBw0EGHcC9WyN1hD3DieKXEpvL0qf1RPUarrt3MHoPrvKUeFACAWSP ITjzGSgGlsqQ7dYRHyMYWJlxmk3vh0ydiGALLkg4nGnIuOd/wnMxqToz/o8XAzm2 hJrEvwWAdYvR4VXeOk2kBrZT7dtN2eid2AntZV9+19zPg0W5RQe96UufxdhCLPy4 eXajaiyuCVCfYo0pHqHwOhIHfOkCR9YSlVc2FG9KZV8fiHagrHkkDxKEKNX48eVm AAuzLsHykbtcyWgZJQhZPkFig3FRCpjf+d6/8dR35pZ7d8AqDeoOxvDy8UQ6EQhh hHl27bvLzg1cJg== X-ME-Sender: X-Sasl-enc: DLZjO9TiCWP9xM4okN71J4+Ujkjz6bbzYl6YEtlMnBYS 1504994000 Received: from ebox.rath.org (ebox.rath.org [45.79.69.51]) by mail.messagingengine.com (Postfix) with ESMTPA id E0E1B7F9CE for ; Sat, 9 Sep 2017 17:53:20 -0400 (EDT) Received: from vostro.rath.org (vostro [192.168.12.4]) by ebox.rath.org (Postfix) with ESMTPS id 021A1EC for ; Sat, 9 Sep 2017 21:53:20 +0000 (UTC) Received: by vostro.rath.org (Postfix, from userid 1000) id 9B393102BD7; Sat, 9 Sep 2017 22:53:18 +0100 (BST) From: Nikolaus Rath To: freebsd-fs@freebsd.org Subject: Re: umount() taking minutes for FUSE filesystems References: <87bmn44ruu.fsf@vostro.rath.org> <87o9qyrbs8.fsf@vostro.rath.org> <2FAD66DE-031B-4B36-9E85-C7BC6B52B5E6@gmail.com> <29de6425-9f92-3bd8-f446-1c9dded33b15@freebsd.org> <87k21dzdrp.fsf@thinkpad.rath.org> <201709051811.v85IBmbO005440@higson.cam.lispworks.com> <87ingugw2v.fsf@thinkpad.rath.org> <201709081143.v88BhEOn001626@higson.cam.lispworks.com> <87lglp6zj8.fsf@vostro.rath.org> <201709081728.v88HSXP7010329@higson.cam.lispworks.com> Mail-Copies-To: never Mail-Followup-To: freebsd-fs@freebsd.org Date: Sat, 09 Sep 2017 22:53:18 +0100 In-Reply-To: <201709081728.v88HSXP7010329@higson.cam.lispworks.com> (Martin Simmons's message of "Fri, 8 Sep 2017 18:28:33 +0100") Message-ID: <87ingr7d4h.fsf@vostro.rath.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Sep 2017 21:53:28 -0000 On Sep 08 2017, Martin Simmons wrote: >> That said, the filesystem could also detect the unmount by the kernel >> closing the fd. So I'm not sure what is gained by the extra request >> either... Theoretically, at least under Linux the destroy handler could >> perform some notify_* operations, but I don't see how that would be >> useful when the filesystem will be unmounted anyway. > > I don't think the kernel itself ever closes the fd (except implicitly in = the > case where the filesystem program dies unexpected). Sorry, I should have been more precise. The umount is not detected by the kernel closing the fd, but by the kernel returning ENODEV when accessing the fd. Best, -Nikolaus --=20 GPG Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F =C2=BBTime flies like an arrow, fruit flies like a Banana.=C2= =AB