From owner-freebsd-fs@freebsd.org Sun May 13 15:07:08 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF22BFE1614 for ; Sun, 13 May 2018 15:07:07 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8511381C3C for ; Sun, 13 May 2018 15:07:07 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 3D286FE1611; Sun, 13 May 2018 15:07:07 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2AFB4FE160D for ; Sun, 13 May 2018 15:07:07 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-qt0-x22a.google.com (mail-qt0-x22a.google.com [IPv6:2607:f8b0:400d:c0d::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BA39881C1E for ; Sun, 13 May 2018 15:07:06 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-qt0-x22a.google.com with SMTP id q6-v6so13127738qtn.3 for ; Sun, 13 May 2018 08:07:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=T0QADsXRsMzyT57N0cD8AjkkkO1LN5PlsG1Y+YQRqXU=; b=DlGDKK+3I7A+3CTeuZgf7YXt9YbFwhvEf1wdssqPUWskm/eF4ID/3kN9hAIqF2oZMI DfamIwDZ5tQ6TdxxbU+Z/fblP118CYierSyn2slIYrW0PZEGygKMGAjP+mn24UmZJBiN PyKqNtUAwEzxWK40NKZJHxcaJBpxEamZvXm0rpI5YIqo3bJ6Fpqm3X//2hoft+dDu6Jz 4l7s6cDqydA7ZuQDNO9lvVTcYnRSs3dlHf37gb0P4s0+u25GE3SOr9sipRtZ6lu+mXSc dal1AuwshroyN9+aMMYxIiSPcDxHLBcCQuyo3eun6msD2a80VEIJQVyhSDxvVRX1YSqz eWAg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=T0QADsXRsMzyT57N0cD8AjkkkO1LN5PlsG1Y+YQRqXU=; b=prVpYSCDaYsxoM+g+3n7OFDb2agO67EZ5vsJbWjBwzoevWRRd3jan1yPDmRXNAUT4d rAE1ppUl91RIKCPSCnN0RNFVQ9Mlv38WcXT+zAf2mxC7ijtndg/HBCihws0wtaAr47je 6l6fuHc3oTM2MrA2p1gkfXfZ5RYAFSgtA7p28a3tPva3el6ffNEcNpFxoAebbxFIcLLO vW8x6hth7hE1cOj1zEr3Ty6y/lsGx9G5hayjC+cMB1unQlDqcDXL5uU7XlM/TiAzbWvu fiAfSY4WJGe/qaTbLwFMJRkaOEdwPva/gl4px9X/hR9PApTwJv4SUt3iiZ8aDBqaUF6G FFcg== X-Gm-Message-State: ALKqPwdRzJR5ulYHLBejzDwoLflYaAKpJ0zdzw6/bbMLyEcpGxVIFgFQ /3sR1tPOH58vv+kx9Q8P6yefbRF5KCKXqmO4vAroFQ== X-Google-Smtp-Source: AB8JxZqwmMvz6xt3ZVNupLIO+equyy7wI3oJFWWnj9Ffu5heS8/iJXL5KbyD32kNBaxfr7TdYDdY4PCHsztJnUgP4t0= X-Received: by 2002:ac8:90b:: with SMTP id t11-v6mr5555756qth.107.1526224026086; Sun, 13 May 2018 08:07:06 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:ac8:6707:0:0:0:0:0 with HTTP; Sun, 13 May 2018 08:07:05 -0700 (PDT) From: Ryan Stone Date: Sun, 13 May 2018 11:07:05 -0400 Message-ID: Subject: "NFS append race" To: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 May 2018 15:07:08 -0000 I have a 11.1-RELEASE client mounting an NFS directory from a 12-CURRENT server. I'm doing a buildworld right now on it, with the src dir mounted on a local drive on the client and the objdir on the NFS mount. I'm seeing the following printed to the console every few minutes: NFS append race @10000:4040 NFS append race @10000:4040 NFS append race @10000:564 NFS append race @10000:564 The "10000" is never changes, but the number after the : is always different (except that the exact same message will repeat once within a second of the first message) What does this mean? Is it something to be concerned about? From owner-freebsd-fs@freebsd.org Sun May 13 18:41:32 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4410FCB866 for ; Sun, 13 May 2018 18:41:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 5BA4986252 for ; Sun, 13 May 2018 18:41:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id 1B3E8FCB865; Sun, 13 May 2018 18:41:32 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 077E9FCB861 for ; Sun, 13 May 2018 18:41:32 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6E5448622B for ; Sun, 13 May 2018 18:41:31 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w4DIfKfe055110 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Sun, 13 May 2018 21:41:24 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w4DIfKfe055110 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w4DIfKZp055109; Sun, 13 May 2018 21:41:20 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 13 May 2018 21:41:20 +0300 From: Konstantin Belousov To: Ryan Stone Cc: "freebsd-fs@freebsd.org" Subject: Re: "NFS append race" Message-ID: <20180513184120.GY6887@kib.kiev.ua> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.5 (2018-04-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 May 2018 18:41:32 -0000 On Sun, May 13, 2018 at 11:07:05AM -0400, Ryan Stone wrote: > I have a 11.1-RELEASE client mounting an NFS directory from a > 12-CURRENT server. I'm doing a buildworld right now on it, with the > src dir mounted on a local drive on the client and the objdir on the > NFS mount. I'm seeing the following printed to the console every few > minutes: > > > NFS append race @10000:4040 > NFS append race @10000:4040 > NFS append race @10000:564 > NFS append race @10000:564 > > > The "10000" is never changes, but the number after the : is always > different (except that the exact same message will repeat once within > a second of the first message) > > What does this mean? Is it something to be concerned about? I remember that nfsiods threads do not lock the vnode when performing the async io. This seems to be the most feasible explanation for the race. Basically, kernel tracks the dirty regions in the buffers, and apparently the append write covers less region that the currently recorder dirty region. If truncation occured while there is still a scheduled nfsiod write, then the write would re-dirty the truncated range. I suspect that this is mostly innocent. Of course, we should start locking the vnodes in nfsiod. From owner-freebsd-fs@freebsd.org Sun May 13 21:00:29 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AC2D6FD2EA2 for ; Sun, 13 May 2018 21:00:29 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6AA848690C for ; Sun, 13 May 2018 21:00:29 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.ysv.freebsd.org (Postfix) id 29C46FD2E96; Sun, 13 May 2018 21:00:29 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 07A5FFD2E93 for ; Sun, 13 May 2018 21:00:29 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7CFAE868FB for ; Sun, 13 May 2018 21:00:28 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C046A64F for ; Sun, 13 May 2018 21:00:27 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4DL0Ru5018613 for ; Sun, 13 May 2018 21:00:27 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4DL0Rds018609 for fs@FreeBSD.org; Sun, 13 May 2018 21:00:27 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201805132100.w4DL0Rds018609@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: fs@FreeBSD.org Subject: Problem reports for fs@FreeBSD.org that need special attention Date: Sun, 13 May 2018 21:00:27 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.25 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 May 2018 21:00:29 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- New | 203492 | mount_unionfs -o below causes panic New | 217062 | for file systems mounted with -o noexec, exec=off New | 221909 | [ZFS] Add a sysctl to toggle send_corrupt_data Open | 136470 | [nfs] Cannot mount / in read-only, over NFS Open | 139651 | [nfs] mount(8): read-only remount of NFS volume d Open | 140068 | [smbfs] [patch] smbfs does not allow semicolon in Open | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non f Open | 211491 | System hangs after "Uptime" on reboot with ZFS 8 problems total for which you should take action. From owner-freebsd-fs@freebsd.org Mon May 14 05:07:57 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A0FBFCFF21 for ; Mon, 14 May 2018 05:07:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B04EB742E1 for ; Mon, 14 May 2018 05:07:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 712B7FCFF1F; Mon, 14 May 2018 05:07:56 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5D5F6FCFF1D for ; Mon, 14 May 2018 05:07:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E7E6E742C0 for ; Mon, 14 May 2018 05:07:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 3A05C7728 for ; Mon, 14 May 2018 05:07:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4E4v4cx014445 for ; Mon, 14 May 2018 04:57:04 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4E4v44K014444 for fs@FreeBSD.org; Mon, 14 May 2018 04:57:04 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 228174] [dump] dump(8) can read garbage and loop forever Date: Mon, 14 May 2018 04:57:04 +0000 X-Bugzilla-Reason: CC X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: stable@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2018 05:07:57 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D228174 --- Comment #3 from Kirk McKusick --- Taking a snapshot is the same as if you unmounted the filesystem and dumped= it. When you take a snapshot, the following steps are taken: 1) Any new system calls that want to write to the filesystem are suspended. 2) All system calls currently writing to the filesystem are allowed to fini= sh. 3) All dirty blocks are flushed to the filesystem. 4) A snapshot of the fully flushed filesystem is taken. 5) The suspended system calls are awaken, but none of their changes appear = in the snapshot. The dump of the snapshot therefore gets a consistent view of the filesystem. The date of the snapshot is recorded, so that when you later take a level-1 dump (using another snapshot) you get exactlty the changes between the two snapshots. Putting in error checking for every possible inconsistency is a massive undertaking. And the resulting dumps are inconsistent and prone to all sort= of errors when you attempt to restore from them. That is why snapshots were ad= ded as they allow consistent and correct dumps to be made on a live system. --=20 You are receiving this mail because: You are on the CC list for the bug.= From owner-freebsd-fs@freebsd.org Mon May 14 05:14:24 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B3F01FD05D5 for ; Mon, 14 May 2018 05:14:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 496F875744 for ; Mon, 14 May 2018 05:14:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0D4B3FD05D2; Mon, 14 May 2018 05:14:24 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF04FFD05D0 for ; Mon, 14 May 2018 05:14:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 253677573D for ; Mon, 14 May 2018 05:14:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 3EC457892 for ; Mon, 14 May 2018 05:14:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4E5EMuY079999 for ; Mon, 14 May 2018 05:14:22 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4E5EMfi079998 for fs@FreeBSD.org; Mon, 14 May 2018 05:14:22 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 228197] panic: solaris assert: arc_buf_alloc_impl Date: Mon, 14 May 2018 05:14:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 11.1-STABLE X-Bugzilla-Keywords: needs-qa, regression X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2018 05:14:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D228197 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Mon May 14 15:35:24 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7129DDF5BAC for ; Mon, 14 May 2018 15:35:24 +0000 (UTC) (envelope-from paul.esson@redstor.com) Received: from EUR02-AM5-obe.outbound.protection.outlook.com (mail-am5eur02on0603.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe07::603]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BF8FD79013 for ; Mon, 14 May 2018 15:35:23 +0000 (UTC) (envelope-from paul.esson@redstor.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redstorltd.onmicrosoft.com; s=selector1-redstor-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=rRkWcbQnR9Dj+eaygA4UgIxa9DfmnKbrg59f5k18f/Q=; b=bBRGb5noqEjMA5GNTsljTjmboA+6Ie77HjYHpA7sJlBeZRsZtgUpykslZbW2EkbkmmYU1C88+HGR+9r6ow6ThRxMpGO/3bltLpgYHWdRq6P60L7qTLYhJLWgGsPyPmaImgBGYaUfSSmlatCq5v1aHlGVbYtliXyYml8UedUexFI= Received: from AM5PR0102MB2578.eurprd01.prod.exchangelabs.com (10.167.168.139) by AM5PR0102MB2625.eurprd01.prod.exchangelabs.com (10.167.168.150) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.755.16; Mon, 14 May 2018 15:35:21 +0000 Received: from AM5PR0102MB2578.eurprd01.prod.exchangelabs.com ([fe80::ac97:828b:b059:2244]) by AM5PR0102MB2578.eurprd01.prod.exchangelabs.com ([fe80::ac97:828b:b059:2244%13]) with mapi id 15.20.0755.018; Mon, 14 May 2018 15:35:21 +0000 From: Paul Esson To: "freebsd-fs@freebsd.org" Subject: Releasing deleted blocks from a sparse ZVOL Thread-Topic: Releasing deleted blocks from a sparse ZVOL Thread-Index: AdPrmSj1/1GnWlHITtWaBCXmXJBXmQ== Date: Mon, 14 May 2018 15:35:21 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=paul.esson@redstor.com; x-originating-ip: [81.136.184.115] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; AM5PR0102MB2625; 7:qfuJeolji1QR91xJ7UgKzTgcIyECmbM5xQgHltZNvj/NURjbcQS/zvaxZ7d3W566+8Re+//L1mfS/Sx+DHStHsyhtKDiimB04116TQuXTQX37R8QqmzRaP3RuLEL6B174vZdaRLfDddUX+IZZ+/pFw+Z73NFRStN4bXah/i50/xHRXjByau0Mi+l3xXIgGaVyC36CHKt4JP/Ibmw+o8d6bxMbmQ/Sl3H37eHut+kNpb4+9CfDW6AjjuVS0pQ45cJ x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:AM5PR0102MB2625; x-ms-traffictypediagnostic: AM5PR0102MB2625: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(21748063052155); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(3231254)(944501410)(52105095)(10201501046)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(20161123558120)(20161123560045)(6072148)(201708071742011); SRVR:AM5PR0102MB2625; BCL:0; PCL:0; RULEID:; SRVR:AM5PR0102MB2625; x-forefront-prvs: 067270ECAF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39850400004)(376002)(39380400002)(396003)(366004)(346002)(199004)(189003)(42174003)(74316002)(99286004)(316002)(9686003)(236005)(5250100002)(25786009)(7696005)(106356001)(2501003)(102836004)(186003)(26005)(4743002)(97736004)(59450400001)(5630700001)(86362001)(2351001)(9326002)(55016002)(6916009)(68736007)(105586002)(66066001)(486006)(54896002)(44832011)(6306002)(81156014)(3660700001)(790700001)(3846002)(6116002)(5640700003)(53946003)(5660300001)(2900100001)(53936002)(3280700002)(2906002)(7736002)(6436002)(6506007)(33656002)(478600001)(476003)(14454004)(81166006)(8676002)(8936002)(579004); DIR:OUT; SFP:1101; SCL:1; SRVR:AM5PR0102MB2625; H:AM5PR0102MB2578.eurprd01.prod.exchangelabs.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: redstor.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: TjySQtJEVbV/Az/yqm/kkY3MD00TsJYFnK6e53U0o2hRUiuOc30by6kZ1K4H7MAwkefLYBSS9mfBNNzPYgXtXOKvG1arkbM13toeIAzwdb+3UwVPxLVdQhpuwHjA3Dgek+vN4bpL/EdSn9n4mlYjwSf8/1vfFYaDtOmsIzsTLQy1aOtHbhApshpMVhRilyPa spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: a6478e25-850c-41a7-6fa5-08d5b9b04db9 X-OriginatorOrg: redstor.com X-MS-Exchange-CrossTenant-Network-Message-Id: a6478e25-850c-41a7-6fa5-08d5b9b04db9 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 May 2018 15:35:21.0924 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 24ac53ae-15a7-4211-afef-61d8f34e2571 X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM5PR0102MB2625 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2018 15:35:25 -0000 Hi Folks, I'm attempting to use a sparse ZVOL from a FreeBSD 11.1-RELEASE host as a d= ata disk for a bhyve FreeBSD 11.1-RELEASE guest that in turn uses this devi= ce as the source for a ZFS filesystem. When I delete data from the filesys= tem on the guest this is reflected in its used space, but not in the ZVOL o= n the host. I've read suggestions that the use of a block driver that supp= orts TRIM or SCSi unmap is crucial such as virtio-scsi, but from what I can= tell my vm-bhyve implementation is using virtio - certainly the disk1_type= parameter in the configuration file has a value of "virtio-blk". Can anyone confirm this is indeed the issue and whether it is possible to s= witch to the relevant driver? Or am I barking up the wrong tree altogether= ? Host ZFS Filesystem NAME PROPERTY VALUE SOURCE dc1-hn-01 type filesystem - dc1-hn-01 creation Mon Apr 23 14:35 2018 - dc1-hn-01 used 36.7G - dc1-hn-01 available 78.2T - dc1-hn-01 referenced 201K - dc1-hn-01 compressratio 1.00x - dc1-hn-01 mounted yes - dc1-hn-01 quota none default dc1-hn-01 reservation none default dc1-hn-01 recordsize 128K default dc1-hn-01 mountpoint /export/data/dc1-hn-01 local dc1-hn-01 sharenfs off default dc1-hn-01 checksum on default dc1-hn-01 compression off default dc1-hn-01 atime on default dc1-hn-01 devices on default dc1-hn-01 exec on default dc1-hn-01 setuid on default dc1-hn-01 readonly off default dc1-hn-01 jailed off default dc1-hn-01 snapdir hidden default dc1-hn-01 aclmode discard default dc1-hn-01 aclinherit restricted default dc1-hn-01 canmount on default dc1-hn-01 xattr off temporary dc1-hn-01 copies 1 default dc1-hn-01 version 5 - dc1-hn-01 utf8only off - dc1-hn-01 normalization none - dc1-hn-01 casesensitivity sensitive - dc1-hn-01 vscan off default dc1-hn-01 nbmand off default dc1-hn-01 sharesmb off default dc1-hn-01 refquota none default dc1-hn-01 refreservation none default dc1-hn-01 primarycache all default dc1-hn-01 secondarycache all default dc1-hn-01 usedbysnapshots 0 - dc1-hn-01 usedbydataset 201K - dc1-hn-01 usedbychildren 36.7G - dc1-hn-01 usedbyrefreservation 0 - dc1-hn-01 logbias latency default dc1-hn-01 dedup off default dc1-hn-01 mlslabel - dc1-hn-01 sync standard default dc1-hn-01 refcompressratio 1.00x - dc1-hn-01 written 201K - dc1-hn-01 logicalused 4.12G - dc1-hn-01 logicalreferenced 36.5K - dc1-hn-01 volmode default default dc1-hn-01 filesystem_limit none default dc1-hn-01 snapshot_limit none default dc1-hn-01 filesystem_count none default dc1-hn-01 snapshot_count none default dc1-hn-01 redundant_metadata all default Host ZFS volume - used by vm-bhyve guest NAME PROPERTY VALUE = SOURCE dc1-hn-01/vm/dc1-olbp-sn-11/disk1 type volume = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 creation Wed May 2 14:06 2= 018 - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 used 4.70G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 available 78.2T = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 referenced 4.70G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compressratio 1.00x = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 reservation none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volsize 28T = local dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volblocksize 8K = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 checksum on = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compression off = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 readonly off = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 copies 1 = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refreservation none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 primarycache all = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 secondarycache all = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbysnapshots 0 = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbydataset 4.70G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbychildren 0 = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbyrefreservation 0 = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logbias latency = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 dedup off = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 mlslabel = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 sync standard = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refcompressratio 1.00x = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 written 4.70G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalused 2.06G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalreferenced 2.06G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volmode dev = local dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_limit none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_count none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 redundant_metadata all = default Guest ZFS Filesystem NAME PROPERTY VALUE SOURCE dc1-sn-11 type filesystem - dc1-sn-11 creation Wed May 2 15:49 2018 - dc1-sn-11 used 752K - dc1-sn-11 available 26.9T - dc1-sn-11 referenced 176K - dc1-sn-11 compressratio 1.00x - dc1-sn-11 mounted yes - dc1-sn-11 quota none default dc1-sn-11 reservation none default dc1-sn-11 recordsize 128K default dc1-sn-11 mountpoint /export/data/dc1-sn-11 local dc1-sn-11 sharenfs off default dc1-sn-11 checksum on default dc1-sn-11 compression off default dc1-sn-11 atime on default dc1-sn-11 devices on default dc1-sn-11 exec on default dc1-sn-11 setuid on default dc1-sn-11 readonly off default dc1-sn-11 jailed off default dc1-sn-11 snapdir hidden default dc1-sn-11 aclmode discard default dc1-sn-11 aclinherit restricted default dc1-sn-11 canmount on default dc1-sn-11 xattr off temporary dc1-sn-11 copies 1 default dc1-sn-11 version 5 - dc1-sn-11 utf8only off - dc1-sn-11 normalization none - dc1-sn-11 casesensitivity sensitive - dc1-sn-11 vscan off default dc1-sn-11 nbmand off default dc1-sn-11 sharesmb off default dc1-sn-11 refquota none default dc1-sn-11 refreservation none default dc1-sn-11 primarycache all default dc1-sn-11 secondarycache all default dc1-sn-11 usedbysnapshots 0 - dc1-sn-11 usedbydataset 176K - dc1-sn-11 usedbychildren 576K - dc1-sn-11 usedbyrefreservation 0 - dc1-sn-11 logbias latency default dc1-sn-11 dedup off default dc1-sn-11 mlslabel - dc1-sn-11 sync standard default dc1-sn-11 refcompressratio 1.00x - dc1-sn-11 written 176K - dc1-sn-11 logicalused 153K - dc1-sn-11 logicalreferenced 55K - dc1-sn-11 volmode default default dc1-sn-11 filesystem_limit none default dc1-sn-11 snapshot_limit none default dc1-sn-11 filesystem_count none default dc1-sn-11 snapshot_count none default dc1-sn-11 redundant_metadata all default Regards, Paul Esson | Redstor Limited e paul.esson@redstor.com From owner-freebsd-fs@freebsd.org Mon May 14 16:17:52 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4B6CEDF826C for ; Mon, 14 May 2018 16:17:52 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from mail.inparadise.se (h-112-105.A444.priv.bahnhof.se [158.174.112.105]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C2D9F83599 for ; Mon, 14 May 2018 16:17:50 +0000 (UTC) (envelope-from karli@inparadise.se) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 4BD1741B64; Mon, 14 May 2018 18:10:06 +0200 (CEST) Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id bAw6x8dggXkV; Mon, 14 May 2018 18:10:03 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by mail.inparadise.se (Postfix) with ESMTP id 3A46641BC2; Mon, 14 May 2018 18:10:03 +0200 (CEST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.inparadise.se 3A46641BC2 X-Virus-Scanned: amavisd-new at inparadise.se Received: from mail.inparadise.se ([127.0.0.1]) by localhost (mail.inparadise.se [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 6EWP-e4pKz0V; Mon, 14 May 2018 18:10:03 +0200 (CEST) Received: from nexus6p.inparadise.se (nexus6p.inparadise.se [172.16.1.122]) by mail.inparadise.se (Postfix) with ESMTPSA id 0840041B64; Mon, 14 May 2018 18:10:03 +0200 (CEST) Date: Mon, 14 May 2018 18:10:01 +0200 User-Agent: K-9 Mail for Android In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: Releasing deleted blocks from a sparse ZVOL To: freebsd-fs@freebsd.org, Paul Esson , "freebsd-fs@freebsd.org" From: =?ISO-8859-1?Q?Karli_Sj=F6berg?= Message-ID: X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2018 16:17:52 -0000 Paul Esson skrev: (14 maj 2018 17:35:21 CEST) >Hi Folks, > >I'm attempting to use a sparse ZVOL from a FreeBSD 11=2E1-RELEASE host as >a data disk for a bhyve FreeBSD 11=2E1-RELEASE guest that in turn uses >this device as the source for a ZFS filesystem=2E When I delete data >from the filesystem on the guest this is reflected in its used space, >but not in the ZVOL on the host=2E I've read suggestions that the use of >a block driver that supports TRIM or SCSi unmap is crucial such as >virtio-scsi, but from what I can tell my vm-bhyve implementation is >using virtio - certainly the disk1_type parameter in the configuration >file has a value of "virtio-blk"=2E Hey! Can't help you with changing the driver, I'll just state the obvious that = since it's called SCSI unmap, of course you need virtio-scsi in order for i= t to work:) Good luck with it! /K > >Can anyone confirm this is indeed the issue and whether it is possible >to switch to the relevant driver? Or am I barking up the wrong tree >altogether? > >Host ZFS Filesystem > >NAME PROPERTY VALUE SOURCE >dc1-hn-01 type filesystem - >dc1-hn-01 creation Mon Apr 23 14:35 2018 - >dc1-hn-01 used 36=2E7G - >dc1-hn-01 available 78=2E2T - >dc1-hn-01 referenced 201K - >dc1-hn-01 compressratio 1=2E00x - >dc1-hn-01 mounted yes - >dc1-hn-01 quota none default >dc1-hn-01 reservation none default >dc1-hn-01 recordsize 128K default >dc1-hn-01 mountpoint /export/data/dc1-hn-01 local >dc1-hn-01 sharenfs off default >dc1-hn-01 checksum on default >dc1-hn-01 compression off default >dc1-hn-01 atime on default >dc1-hn-01 devices on default >dc1-hn-01 exec on default >dc1-hn-01 setuid on default >dc1-hn-01 readonly off default >dc1-hn-01 jailed off default >dc1-hn-01 snapdir hidden default >dc1-hn-01 aclmode discard default >dc1-hn-01 aclinherit restricted default >dc1-hn-01 canmount on default >dc1-hn-01 xattr off temporary >dc1-hn-01 copies 1 default >dc1-hn-01 version 5 - >dc1-hn-01 utf8only off - >dc1-hn-01 normalization none - >dc1-hn-01 casesensitivity sensitive - >dc1-hn-01 vscan off default >dc1-hn-01 nbmand off default >dc1-hn-01 sharesmb off default >dc1-hn-01 refquota none default >dc1-hn-01 refreservation none default >dc1-hn-01 primarycache all default >dc1-hn-01 secondarycache all default >dc1-hn-01 usedbysnapshots 0 - >dc1-hn-01 usedbydataset 201K - >dc1-hn-01 usedbychildren 36=2E7G - >dc1-hn-01 usedbyrefreservation 0 - >dc1-hn-01 logbias latency default >dc1-hn-01 dedup off default >dc1-hn-01 mlslabel - >dc1-hn-01 sync standard default >dc1-hn-01 refcompressratio 1=2E00x - >dc1-hn-01 written 201K - >dc1-hn-01 logicalused 4=2E12G - >dc1-hn-01 logicalreferenced 36=2E5K - >dc1-hn-01 volmode default default >dc1-hn-01 filesystem_limit none default >dc1-hn-01 snapshot_limit none default >dc1-hn-01 filesystem_count none default >dc1-hn-01 snapshot_count none default >dc1-hn-01 redundant_metadata all default > > >Host ZFS volume - used by vm-bhyve guest > >NAME PROPERTY VALUE =20 > SOURCE >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 type volume =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 creation Wed May 2 >14:06 2018 - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 used 4=2E70G = =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 available 78=2E2T = =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 referenced 4=2E70G = =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compressratio 1=2E00x = =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 reservation none =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volsize 28T =20 > local >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volblocksize 8K =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 checksum on =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compression off =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 readonly off =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 copies 1 =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refreservation none =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 primarycache all =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 secondarycache all =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbysnapshots 0 =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbydataset 4=2E70G = =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbychildren 0 =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbyrefreservation 0 =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logbias latency =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 dedup off =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 mlslabel =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 sync standard =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refcompressratio 1=2E00x = =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 written 4=2E70G = =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalused 2=2E06G = =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalreferenced 2=2E06G = =20 > - >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volmode dev =20 > local >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_limit none =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_count none =20 > default >dc1-hn-01/vm/dc1-olbp-sn-11/disk1 redundant_metadata all =20 > default > >Guest ZFS Filesystem > >NAME PROPERTY VALUE SOURCE >dc1-sn-11 type filesystem - >dc1-sn-11 creation Wed May 2 15:49 2018 - >dc1-sn-11 used 752K - >dc1-sn-11 available 26=2E9T - >dc1-sn-11 referenced 176K - >dc1-sn-11 compressratio 1=2E00x - >dc1-sn-11 mounted yes - >dc1-sn-11 quota none default >dc1-sn-11 reservation none default >dc1-sn-11 recordsize 128K default >dc1-sn-11 mountpoint /export/data/dc1-sn-11 local >dc1-sn-11 sharenfs off default >dc1-sn-11 checksum on default >dc1-sn-11 compression off default >dc1-sn-11 atime on default >dc1-sn-11 devices on default >dc1-sn-11 exec on default >dc1-sn-11 setuid on default >dc1-sn-11 readonly off default >dc1-sn-11 jailed off default >dc1-sn-11 snapdir hidden default >dc1-sn-11 aclmode discard default >dc1-sn-11 aclinherit restricted default >dc1-sn-11 canmount on default >dc1-sn-11 xattr off temporary >dc1-sn-11 copies 1 default >dc1-sn-11 version 5 - >dc1-sn-11 utf8only off - >dc1-sn-11 normalization none - >dc1-sn-11 casesensitivity sensitive - >dc1-sn-11 vscan off default >dc1-sn-11 nbmand off default >dc1-sn-11 sharesmb off default >dc1-sn-11 refquota none default >dc1-sn-11 refreservation none default >dc1-sn-11 primarycache all default >dc1-sn-11 secondarycache all default >dc1-sn-11 usedbysnapshots 0 - >dc1-sn-11 usedbydataset 176K - >dc1-sn-11 usedbychildren 576K - >dc1-sn-11 usedbyrefreservation 0 - >dc1-sn-11 logbias latency default >dc1-sn-11 dedup off default >dc1-sn-11 mlslabel - >dc1-sn-11 sync standard default >dc1-sn-11 refcompressratio 1=2E00x - >dc1-sn-11 written 176K - >dc1-sn-11 logicalused 153K - >dc1-sn-11 logicalreferenced 55K - >dc1-sn-11 volmode default default >dc1-sn-11 filesystem_limit none default >dc1-sn-11 snapshot_limit none default >dc1-sn-11 filesystem_count none default >dc1-sn-11 snapshot_count none default >dc1-sn-11 redundant_metadata all default > > >Regards, > >Paul Esson | Redstor Limited >e paul=2Eesson@redstor=2Ecom > > > >_______________________________________________ >freebsd-fs@freebsd=2Eorg mailing list >https://lists=2Efreebsd=2Eorg/mailman/listinfo/freebsd-fs >To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd=2Eorg" --=20 Skickat fr=C3=A5n min Android-enhet med K-9 Mail=2E Urs=C3=A4kta min f=C3= =A5ordighet=2E From owner-freebsd-fs@freebsd.org Mon May 14 20:37:46 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7A23BE76A36 for ; Mon, 14 May 2018 20:37:46 +0000 (UTC) (envelope-from eborisch@alumni.stanford.edu) Received: from mail-lf0-x232.google.com (mail-lf0-x232.google.com [IPv6:2a00:1450:4010:c07::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id F0C3F82F7B for ; Mon, 14 May 2018 20:37:45 +0000 (UTC) (envelope-from eborisch@alumni.stanford.edu) Received: by mail-lf0-x232.google.com with SMTP id r2-v6so20014420lff.4 for ; Mon, 14 May 2018 13:37:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=lfsAkfca1D1+DH6z17qUZ/RfA27k9fA5XSyJ+dDeTho=; b=MEw5kw2Uxw28bUyCw0HjXug2Eh6CWBJo7O82pBlsX1BSqEcphj/056a4s6wEWSg804 A5Rh3/5TXTueaq9IoepoIrZxfV282o9/zzxGMjsZWw+LfQXk5RykMhqn2ewJd5UZ1Y7/ jW6RWC15QTt2Qt9o3bs83LlflFZHmoYJorBSVcvQcooO8VDJLfIMmBHf8Dl0LnEuEpnY dItw1J8Wb/DOx2gVF/inmWwQLh2SzPTspPXS0Uw1tPooHtjIRptFX6yoWdWrjB/mRujv U51mXkQZtyHqXhPQYQceuKgcf5kvySlgKMfzUDMTXK+gpP6+aJ4+M/Vv7OxMucRAqXfb qVWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=lfsAkfca1D1+DH6z17qUZ/RfA27k9fA5XSyJ+dDeTho=; b=AKsy8owSyEvQrxwW6UV7aQqCrq3xf06c3QE7uREHUrUDmrtpukdg6U9UtcITEAdyIw bY0bFZaXeIeguEgFJRcTl3BcI8CjjvK60fzABnuUmkGpNLwXDOzSEi60YnH4gr0iDYFo bpdOFoCZVcEskSYSK+91Fwnpk9H/MZW5pj+sR62Wa+jRfU3gskmTJSFlCEpt6cAF5ksX RMJpUP2U6OHhS5+BTBhBfnM67agnvF3RyBjAI7C1AcdaBYtDoB/2f7SLUF+A7S2lLXSA SeV5Ua3OksbAOvpj8sT6W7/5OjTTTHE8QfTJMCZ8wcAssKI639Xb4iGz5pHMYX5TFtcY YwkQ== X-Gm-Message-State: ALKqPweaLCMxytjbkk0M3/Sq2WcblpFlzlLTyOqHmPo7c8TsKDrMDzR9 pGXmpU+LPxT+aPI/w7Vl8h9c/FGzNPxKMo3uu+bDgg== X-Google-Smtp-Source: AB8JxZpSL02m1QoJGQH95JMt96ySuJ3mzirx6Sd/p+RgUHiD6x5vSudzOX26MJiuEXDm2/NUgFzlHU5askduEleE30E= X-Received: by 2002:a19:23c8:: with SMTP id j191-v6mr9290420lfj.1.1526330264605; Mon, 14 May 2018 13:37:44 -0700 (PDT) MIME-Version: 1.0 Sender: eborisch@alumni.stanford.edu Received: by 2002:a2e:84cc:0:0:0:0:0 with HTTP; Mon, 14 May 2018 13:37:43 -0700 (PDT) In-Reply-To: References: From: Eric Borisch Date: Mon, 14 May 2018 15:37:43 -0500 X-Google-Sender-Auth: 7KrRhX6Bm6J6007gWiKsfVhA4Bs Message-ID: Subject: Re: Releasing deleted blocks from a sparse ZVOL To: =?UTF-8?Q?Karli_Sj=C3=B6berg?= Cc: Paul Esson , "freebsd-fs@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2018 20:37:46 -0000 I've had success in a similar config by attaching the zvol as 'ahci-hd', which shows up as a 'BHYVE SATA DISK' in a BSD guest. With such a guest (running a zpool on the attached device): kstat.zfs.misc.zio_trim.failed: 0 kstat.zfs.misc.zio_trim.unsupported: 0 kstat.zfs.misc.zio_trim.success: 179883 kstat.zfs.misc.zio_trim.bytes: 13126696960 Also working with a linux guest. I see the expected tracking of the 'logicalreferenced' size in the host with the ALLOC size in the guest (allowing for upward bias due to using 32k sector sizes on my zvols). Note that TRIM and UNMAP are two extremely similar but distinct (used for SATA & SCSI, respectively) commands. Both are commonly referred to as 'trim' by layers above the device drivers. - Eric From owner-freebsd-fs@freebsd.org Mon May 14 20:55:11 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5660E777A7 for ; Mon, 14 May 2018 20:55:10 +0000 (UTC) (envelope-from paul.esson@redstor.com) Received: from EUR01-DB5-obe.outbound.protection.outlook.com (mail-db5eur01on0087.outbound.protection.outlook.com [104.47.2.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2456187C6F for ; Mon, 14 May 2018 20:55:09 +0000 (UTC) (envelope-from paul.esson@redstor.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redstorltd.onmicrosoft.com; s=selector1-redstor-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=Da5Nn3wXJusIdHqAlZuD/fAzrbScDxpJBKXmyfnIAP4=; b=cYW8VVAmqWhUMZM3T0C6Ewaxl4CROiF1sdNgYFiJ65DoL/Rs9oE2OwQnO7OhOQ53POA8NXJJ4iLylrhPE3RNj3QsNs3VTTJh5kyIbdgjKL1yXvf+wAqX4OvJK43R0UV0GR5CZc/6xexjr53vD3277QBej18xAKbzkyEmuHXAXUw= Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com (10.170.251.141) by HE1PR0102MB3177.eurprd01.prod.exchangelabs.com (10.167.124.18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.755.16; Mon, 14 May 2018 20:55:06 +0000 Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::d508:b6bc:e82:73e]) by HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::d508:b6bc:e82:73e%13]) with mapi id 15.20.0755.018; Mon, 14 May 2018 20:55:06 +0000 From: Paul Esson To: Eric Borisch , =?iso-8859-1?Q?Karli_Sj=F6berg?= CC: "freebsd-fs@freebsd.org" Subject: Re: Releasing deleted blocks from a sparse ZVOL Thread-Topic: Releasing deleted blocks from a sparse ZVOL Thread-Index: AdPrmSj1/1GnWlHITtWaBCXmXJBXmQABNnyAAAlZbYAAAJtC6w== Date: Mon, 14 May 2018 20:55:05 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [40.68.209.210] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0102MB3177; 7:1Fml1XCCEdZzU5msybryyzQjTh6QMJnHRvvfoZ/OueOnADx1Ps2Q5FMDbVVfhIU45B+KpsIBwRYS4gHpGGOq1/+BnHvSbRNRLjMX+r0BpLi7BItSsUHHk2A8bJU/PWhBCLfQt0tp4IkPxjKEnR2oaYABSu/H3P7OJR+QoM7O0aofAC4ZsR2h6k64E3lEVu9kt9BZwDbusOtMscH2nGc0z1025158R0aeiqeaFMs6PpWRD2Ppq/Qs2LogZQRiXUGV x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0102MB3177; x-ms-traffictypediagnostic: HE1PR0102MB3177: authentication-results: spf=none (sender IP is ) smtp.mailfrom=paul.esson@redstor.com; x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(85827821059158)(81439100147899); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(3231254)(944501410)(52105095)(10201501046)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123562045)(20161123558120)(20161123560045)(6072148)(201708071742011); SRVR:HE1PR0102MB3177; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0102MB3177; x-forefront-prvs: 067270ECAF x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(366004)(376002)(39380400002)(346002)(39850400004)(396003)(189003)(199004)(81166006)(3280700002)(81156014)(486006)(110136005)(86362001)(7736002)(186003)(76176011)(6506007)(33656002)(53546011)(102836004)(7696005)(316002)(3660700001)(14454004)(2906002)(66066001)(2900100001)(8676002)(44832011)(5250100002)(99286004)(8936002)(6436002)(26005)(476003)(68736007)(5890100001)(446003)(105586002)(229853002)(478600001)(106356001)(6116002)(3846002)(4326008)(25786009)(11346002)(6246003)(6306002)(54896002)(9686003)(5660300001)(74316002)(97736004)(55016002)(53936002)(39060400002); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0102MB3177; H:HE1PR0102MB2588.eurprd01.prod.exchangelabs.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: redstor.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: pfq2vi70UFcBp+rcPzdohisuDL8VRsbf8DgsrIIMQlEBsBZHk7C57h1nhFIlwF/Ow6/TgCcqVzKZmmQmE6e2OaDslA2XoVX8sAAX5x6vH6mgladG8y3r2aZoRQZUuOhni9WVG/AT2Rr8Rw2W9BIUWzvYCv1DETQLD9kAS3GG4pZUQfZswUNZKS7LGBprphaY spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: a5d026e4-3120-4a36-ab37-08d5b9dcf8e8 X-OriginatorOrg: redstor.com X-MS-Exchange-CrossTenant-Network-Message-Id: a5d026e4-3120-4a36-ab37-08d5b9dcf8e8 X-MS-Exchange-CrossTenant-originalarrivaltime: 14 May 2018 20:55:06.0703 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 24ac53ae-15a7-4211-afef-61d8f34e2571 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0102MB3177 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 14 May 2018 20:55:11 -0000 Hi Eric, Thanks for your feedback which sounds encouraging. I'll give that a try. Regards, Paul.E Paul Esson | Redstor Limited t +44 (0)118 951 5235 m +44 (0)776 690 6514 e paul.esson@redstor.com ________________________________ From: eborisch@alumni.stanford.edu on behalf= of Eric Borisch Sent: Monday, May 14, 2018 9:37:43 PM To: Karli Sj=F6berg Cc: Paul Esson; freebsd-fs@freebsd.org Subject: Re: Releasing deleted blocks from a sparse ZVOL I've had success in a similar config by attaching the zvol as 'ahci-hd', wh= ich shows up as a 'BHYVE SATA DISK' in a BSD guest. With such a guest (runn= ing a zpool on the attached device): kstat.zfs.misc.zio_trim.failed: 0 kstat.zfs.misc.zio_trim.unsupported: 0 kstat.zfs.misc.zio_trim.success: 179883 kstat.zfs.misc.zio_trim.bytes: 13126696960 Also working with a linux guest. I see the expected tracking of the 'logicalreferenced' size in the host wit= h the ALLOC size in the guest (allowing for upward bias due to using 32k se= ctor sizes on my zvols). Note that TRIM and UNMAP are two extremely similar but distinct (used for S= ATA & SCSI, respectively) commands. Both are commonly referred to as 'trim'= by layers above the device drivers. - Eric From owner-freebsd-fs@freebsd.org Tue May 15 11:07:14 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31538EADD7C for ; Tue, 15 May 2018 11:07:14 +0000 (UTC) (envelope-from lopez.on.the.lists@yellowspace.net) Received: from mail.yellowspace.net (mail.yellowspace.net [62.146.62.104]) by mx1.freebsd.org (Postfix) with ESMTP id C7A026CA1A for ; Tue, 15 May 2018 11:07:13 +0000 (UTC) (envelope-from lopez.on.the.lists@yellowspace.net) Received: from furia.intranet ([212.114.226.80]) (AUTH: PLAIN lopez.on.the.lists@yellowspace.net, SSL: TLSv1/SSLv3, 128bits, AES128-SHA) by mail.yellowspace.net with esmtp; Tue, 15 May 2018 13:02:06 +0200 id 000147E9.000000005AFABE2E.0000A672 Subject: Re: Native ZFS encryption To: freebsd-fs@freebsd.org References: <3e0b2644-9739-5341-0295-c746275588ec@aon.at> <0bf3fd08-1803-e15b-cadf-755d1c0af325@multiplay.co.uk> From: Lorenzo Perone Message-ID: Date: Tue, 15 May 2018 13:02:05 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <0bf3fd08-1803-e15b-cadf-755d1c0af325@multiplay.co.uk> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 May 2018 11:07:14 -0000 Hi, does anyone know whether there is an update on this, FreeBSD-wise, or if it is planned? It looks like the new pull request for openzfs, https://github.com/openzfs/openzfs/pull/489 has come a few steps further. I've also seen the dataset/zvol encryption functionality "sort of working" on one of the latest OpenZFS on OS X version (1.7.3 Beta). Being able to have encryption at the dataset/zvol level is, imo, a very nice option: It allows to share the space and hardware redundancy between different users, while keeping their data encrypted with separate keys - as opposed to the (very well working) "GELI below" option. On the more general matter, I wondered if anyone has a pointer to any documentation / article / comment (other than the code itself) on how "compatible" the openzfs and FreeBSD-zfs implementations actually are? I wonder for example how FreeBSD specific features like "zfs jail" are kept while/if merging code from openzfs? (as a side note: another FreeBSD specific feature being, of course, longterm-proven reliability..) Best Regards to all active developers, Lorenzo On 04.01.17 18:32, Steven Hartland wrote: > Typically once its merged upstream (openzfs) after then it will get > pulled in. > > Status of that request is currently conflicting: > https://github.com/openzfs/openzfs/pull/124 > > On 04/01/2017 15:15, Martin Birgmeier wrote: >> Tom Caputi is working on Native ZFS encryption for Linux: >> https://github.com/zfsonlinux/zfs/pull/4329 >> >> What are the plans/what is the status for incorporating this into >> FreeBSD? >> >> -- Martin >> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Wed May 16 12:38:45 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1299EDF8E8 for ; Wed, 16 May 2018 12:38:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6380486B4B for ; Wed, 16 May 2018 12:38:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 1A9BAEDF8E7; Wed, 16 May 2018 12:38:44 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E76B3EDF8E6 for ; Wed, 16 May 2018 12:38:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7F02686B45 for ; Wed, 16 May 2018 12:38:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C546F24BF2 for ; Wed, 16 May 2018 12:38:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GCcgc9085283 for ; Wed, 16 May 2018 12:38:42 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GCcgwT085277 for fs@FreeBSD.org; Wed, 16 May 2018 12:38:42 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media Date: Wed, 16 May 2018 12:38:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 12:38:45 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210316 --- Comment #10 from Andriy Gapon --- (In reply to Konstantin Belousov from comment #9) Kostik, I need to refresh my memory of this problem. Meanwhile, I have a counter-question. Is there a bwrite variant where a failure is a final failure with no side effects? I mean, if that bwriteX fails, then the buffer is just discarded. Something with semantics similar to write(2). My understanding of what happens now. bufwrite calls this code: if ((oldflags & B_ASYNC) =3D=3D 0) { int rtval =3D bufwait(bp); brelse(bp); return (rtval); Then in brelse() the following block is executed: if (bp->b_iocmd =3D=3D BIO_WRITE && (bp->b_ioflags & BIO_ERROR) && (bp->b_error !=3D ENXIO || !LIST_EMPTY(&bp->b_dep)) && !(bp->b_flags & B_INVAL)) { /* * Failed write, redirty. All errors except ENXIO (which * means the device is gone) are expected to be potentially * transient - underlying media might work if tried again * after EIO, and memory might be available after an ENOMEM. * * Do this also for buffers that failed with ENXIO, but have * non-empty dependencies - the soft updates code might need * to access the buffer to untangle them. * * Must clear BIO_ERROR to prevent pages from being scrappe= d. */ bp->b_ioflags &=3D ~BIO_ERROR; bdirty(bp); I think that it would be nice to have a way to instruct the buffer code to never retry a specific synchronous write request. If we had such a mechanism we could use it in filesystems's mount code to c= heck whether we can really write to the backing media. Or should a caller of bwrite() explicitly manipulate b_flags (e.g. B_INVAL, B_RELBUF) to get the desired effect? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 16 12:56:47 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AF5FAEE0B7A for ; Wed, 16 May 2018 12:56:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 47B4087795 for ; Wed, 16 May 2018 12:56:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 05113EE0B74; Wed, 16 May 2018 12:56:47 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E7C5DEE0B73 for ; Wed, 16 May 2018 12:56:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8700487791 for ; Wed, 16 May 2018 12:56:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C4B7424EC3 for ; Wed, 16 May 2018 12:56:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GCujGl004331 for ; Wed, 16 May 2018 12:56:45 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GCujHD004330 for fs@FreeBSD.org; Wed, 16 May 2018 12:56:45 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media Date: Wed, 16 May 2018 12:56:44 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 12:56:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210316 --- Comment #11 from Konstantin Belousov --- (In reply to Andriy Gapon from comment #10) There is no flag 'write once and destroy on error'. It is impossible to achieve this by flags. Buffer cache tries hard to not loose user data, it = is the caller duty to destroy no longer needed dirty buffers. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 16 13:37:00 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 284F8EE2D0A for ; Wed, 16 May 2018 13:37:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id AFC3068FF9 for ; Wed, 16 May 2018 13:36:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 6E010EE2D09; Wed, 16 May 2018 13:36:59 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B4A8EE2D08 for ; Wed, 16 May 2018 13:36:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EA48768FF3 for ; Wed, 16 May 2018 13:36:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 22F6F25454 for ; Wed, 16 May 2018 13:36:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GDaw4v013811 for ; Wed, 16 May 2018 13:36:58 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GDaw8Z013810 for fs@FreeBSD.org; Wed, 16 May 2018 13:36:58 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media Date: Wed, 16 May 2018 13:36:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 13:37:00 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210316 --- Comment #12 from Andriy Gapon --- (In reply to Konstantin Belousov from comment #11) Well, it's just very counter-intuitive (for someone who is not deeply famil= iar with buffer cache, like me) that the default (and, in fact, the only) behav= iour for a failed _synchronous_ request is retry. I mean, I've got the error, I might have acted on it, I even might get surprised if the data gets written later. But this is buffer cache, so I will just shut up. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 16 14:30:18 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E5E5AEE5CA7 for ; Wed, 16 May 2018 14:30:17 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mx0.gentlemail.de (mx0.gentlemail.de [IPv6:2a00:e10:2800::a130]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 6AF216B73E for ; Wed, 16 May 2018 14:30:17 +0000 (UTC) (envelope-from freebsd@omnilan.de) Received: from mh0.gentlemail.de (ezra.dcm1.omnilan.net [78.138.80.135]) by mx0.gentlemail.de (8.14.5/8.14.5) with ESMTP id w4GEUGiN010367 for ; Wed, 16 May 2018 16:30:16 +0200 (CEST) (envelope-from freebsd@omnilan.de) Received: from titan.inop.mo1.omnilan.net (s1.omnilan.de [217.91.127.234]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mh0.gentlemail.de (Postfix) with ESMTPSA id 446FE318 for ; Wed, 16 May 2018 16:30:16 +0200 (CEST) From: Harry Schmalzbauer Subject: Panic after 2 LORs autofs/syncer, zfs/devfs: general protection fault while in kernel mode To: "freebsd-fs@freebsd.org" Organization: OmniLAN Message-ID: <3914f2f0-be52-161c-8544-f813ac85f14c@omnilan.de> Date: Wed, 16 May 2018 16:30:15 +0200 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-Greylist: ACL 130 matched, not delayed by milter-greylist-4.2.7 (mx0.gentlemail.de [78.138.80.130]); Wed, 16 May 2018 16:30:16 +0200 (CEST) X-Milter: Spamilter (Reciever: mx0.gentlemail.de; Sender-ip: 78.138.80.135; Sender-helo: mh0.gentlemail.de; ) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 14:30:18 -0000 Hello, I haven't found a newer LOR list than http://sources.zabbadoz.net/freebsd/lor.html, so I'm not sure where/whether to report panics followed by LOR. Here's one I got while shutting down on r333486 (May 11th): Unread portion of the kernel message buffer: <118>Terminated <118>. <118>May 15 22:30:10 titan syslogd: exiting on signal 15 Waiting (max 60 seconds) for system process `vnlru' to stop... done Waiting (max 60 seconds) for system process `syncer' to stop... Syncing disks, vnodes remaining... 0 0 0 0 0 0 0 done Waiting (max 60 seconds) for system thread `bufdaemon' to stop... done Waiting (max 60 seconds) for system thread `bufspacedaemon-0' to stop... done Waiting (max 60 seconds) for system thread `bufspacedaemon-1' to stop... done Waiting (max 60 seconds) for system thread `bufspacedaemon-2' to stop... done Waiting (max 60 seconds) for system thread `bufspacedaemon-3' to stop... done Waiting (max 60 seconds) for system thread `bufspacedaemon-4' to stop... done Waiting (max 60 seconds) for system thread `bufspacedaemon-5' to stop... done Waiting (max 60 seconds) for system thread `bufspacedaemon-6' to stop... done All buffers synced. lock order reversal:  1st 0xfffff8002711a608 autofs (autofs) @ /usr/local/share/deploy-tools/HEAD/src/sys/kern/vfs_mount.c:1343  2nd 0xfffff80019fad248 syncer (syncer) @ /usr/local/share/deploy-tools/HEAD/src/sys/kern/vfs_subr.c:2734 stack backtrace: #0 0xffffffff808336c3 at witness_debugger+0x73 #1 0xffffffff80833544 at witness_checkorder+0xe34 #2 0xffffffff8079f6eb at lockmgr_lock_fast_path+0x18b #3 0xffffffff80bc91f9 at VOP_LOCK1_APV+0xd9 #4 0xffffffff808af5d6 at _vn_lock+0x66 #5 0xffffffff8089ec97 at vputx+0x157 #6 0xffffffff80896aa9 at dounmount+0x4d9 #7 0xffffffff808a048b at vfs_unmountall+0x6b #8 0xffffffff8087b7c5 at bufshutdown+0x2c5 #9 0xffffffff807cabea at kern_reboot+0x21a #10 0xffffffff807ca979 at sys_reboot+0x3a9 #11 0xffffffff80b0ce9c at amd64_syscall+0x28c #12 0xffffffff80ae92dd at fast_syscall_common+0x101 lock order reversal:  1st 0xfffff8000b168d88 zfs (zfs) @ /usr/local/share/deploy-tools/HEAD/src/sys/kern/vfs_mount.c:1343  2nd 0xfffff8000b169248 devfs (devfs) @ /usr/local/share/deploy-tools/HEAD/src/sys/kern/vfs_subr.c:2572 stack backtrace: #0 0xffffffff808336c3 at witness_debugger+0x73 #1 0xffffffff80833544 at witness_checkorder+0xe34 #2 0xffffffff8079f6eb at lockmgr_lock_fast_path+0x18b #3 0xffffffff80bc91f9 at VOP_LOCK1_APV+0xd9 #4 0xffffffff808af5d6 at _vn_lock+0x66 #5 0xffffffff8089e561 at vget+0x81 #6 0xffffffff8068576d at devfs_allocv+0xdd #7 0xffffffff80685263 at devfs_root+0x43 #8 0xffffffff8089f06a at vflush+0x6a #9 0xffffffff80685153 at devfs_unmount+0x43 #10 0xffffffff80896ae9 at dounmount+0x519 #11 0xffffffff808a04e4 at vfs_unmountall+0xc4 #12 0xffffffff8087b7c5 at bufshutdown+0x2c5 #13 0xffffffff807cabea at kern_reboot+0x21a #14 0xffffffff807ca979 at sys_reboot+0x3a9 #15 0xffffffff80b0ce9c at amd64_syscall+0x28c #16 0xffffffff80ae92dd at fast_syscall_common+0x101 Uptime: 1d13h25m53s Fatal trap 9: general protection fault while in kernel mode cpuid = 2; apic id = 02 instruction pointer    = 0x20:0xffffffff81b13984 stack pointer            = 0x28:0xfffffe0000575980 frame pointer            = 0x28:0xfffffe0000575980 code segment        = base 0x0, limit 0xfffff, type 0x1b             = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags    = interrupt enabled, resume, IOPL = 0 current process        = 0 (dbu_evict) #0  doadump (textdump=0) at pcpu.h:231 #1  0xffffffff803e708b in db_dump (dummy=, dummy2=, dummy3=,     dummy4=) at /usr/local/share/deploy-tools/HEAD/src/sys/ddb/db_command.c:574 #2  0xffffffff803e6e59 in db_command (cmd_table=)     at /usr/local/share/deploy-tools/HEAD/src/sys/ddb/db_command.c:481 #3  0xffffffff803e6bd4 in db_command_loop () at /usr/local/share/deploy-tools/HEAD/src/sys/ddb/db_command.c:534 #4  0xffffffff803e9e6f in db_trap (type=, code=)     at /usr/local/share/deploy-tools/HEAD/src/sys/ddb/db_main.c:252 #5  0xffffffff808129b3 in kdb_trap (type=9, code=0, tf=)     at /usr/local/share/deploy-tools/HEAD/src/sys/kern/subr_kdb.c:697 #6  0xffffffff80b0c48f in trap_fatal (frame=0xfffffe00005758c0, eva=0)     at /usr/local/share/deploy-tools/HEAD/src/sys/amd64/amd64/trap.c:872 #7  0xffffffff80b0ba6d in trap (frame=0xfffffe00005758c0) at counter.h:87 #8  0xffffffff80ae89fc in calltrap () at /usr/local/share/deploy-tools/HEAD/src/sys/amd64/amd64/exception.S:230 #9  0xffffffff81b13984 in spa_get_dsl () from /boot/kernel/zfs.ko #10 0xffffffff81b17d1f in txg_verify () from /boot/kernel/zfs.ko #11 0xffffffff81b1829f in txg_list_member () from /boot/kernel/zfs.ko #12 0xffffffff81acb969 in dsl_dir_evict_async () from /boot/kernel/zfs.ko #13 0xffffffff808260cc in taskqueue_run_locked (queue=0xfffff80004058800)     at /usr/local/share/deploy-tools/HEAD/src/sys/kern/subr_taskqueue.c:465 #14 0xffffffff80826e98 in taskqueue_thread_loop (arg=)     at /usr/local/share/deploy-tools/HEAD/src/sys/kern/subr_taskqueue.c:757 #15 0xffffffff8078add4 in fork_exit (callout=0xffffffff80826e10 , arg=0xfffff80003b46e40,     frame=0xfffffe0000575ac0) at /usr/local/share/deploy-tools/HEAD/src/sys/kern/kern_fork.c:1039 #16 0xffffffff80ae99ae in fork_trampoline () at /usr/local/share/deploy-tools/HEAD/src/sys/amd64/amd64/exception.S:971 #17 0x0000000000000000 in ?? () -harry From owner-freebsd-fs@freebsd.org Wed May 16 18:07:44 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 394B4EF0EE7 for ; Wed, 16 May 2018 18:07:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B9B0974402 for ; Wed, 16 May 2018 18:07:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 7BA6AEF0EE3; Wed, 16 May 2018 18:07:43 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67BDBEF0EE0 for ; Wed, 16 May 2018 18:07:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 07850743E7 for ; Wed, 16 May 2018 18:07:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 40549279E6 for ; Wed, 16 May 2018 18:07:42 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GI7g8M001324 for ; Wed, 16 May 2018 18:07:42 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GI7gcr001316 for fs@FreeBSD.org; Wed, 16 May 2018 18:07:42 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Wed, 16 May 2018 18:07:41 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 18:07:44 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 Damjan Jovanovic changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |damjan.jov@gmail.com --- Comment #10 from Damjan Jovanovic --- This can still be reproduced on CURRENT as follows: dd if=3D/dev/zero of=3D64G bs=3D1G count=3D64 newfs_msdos ./64G mdconfig -a -f ./64G mount -t msdosfs /dev/md0 /mnt/test mkdir /mnt/test/ports cp -r /usr/ports/* /mnt/test/ports At least the following end up with corrupted contents: ports/devel ports/games ports/graphics ports/security ports/sysutils ports/textproc ports/www --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 16 18:19:08 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ED6B4EA8ECB for ; Wed, 16 May 2018 18:19:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 8990774D62 for ; Wed, 16 May 2018 18:19:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 470F6EA8ECA; Wed, 16 May 2018 18:19:07 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34263EA8EC9 for ; Wed, 16 May 2018 18:19:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C11CA74D5F for ; Wed, 16 May 2018 18:19:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id BEB9E27B54 for ; Wed, 16 May 2018 18:19:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GIJ5as009224 for ; Wed, 16 May 2018 18:19:05 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GIJ51J009222 for fs@FreeBSD.org; Wed, 16 May 2018 18:19:05 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Wed, 16 May 2018 18:19:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 18:19:08 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #11 from Damjan Jovanovic --- Konstantin, please can you explain how the "previous operation left the pag= es on the vnode page queue behind the directory denode end"? What must code do, between for example bread() and brelse(), to clean up the "buf" properly? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 16 18:35:00 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D4F70EA9C30 for ; Wed, 16 May 2018 18:35:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6BC1E7582A for ; Wed, 16 May 2018 18:35:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 26E55EA9C2F; Wed, 16 May 2018 18:35:00 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 13EBBEA9C2E for ; Wed, 16 May 2018 18:35:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A269F75820 for ; Wed, 16 May 2018 18:34:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id D127127E20 for ; Wed, 16 May 2018 18:34:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GIYwNC050160 for ; Wed, 16 May 2018 18:34:58 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GIYwni050159 for fs@FreeBSD.org; Wed, 16 May 2018 18:34:58 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media Date: Wed, 16 May 2018 18:34:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: cem@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 18:35:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210316 Conrad Meyer changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |cem@freebsd.org --- Comment #13 from Conrad Meyer --- Write once and if it errors destroy user data is a very surprising semantic= for users. Not sure why we should do that. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 16 19:54:24 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BB45BEADFD6 for ; Wed, 16 May 2018 19:54:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 538907905B for ; Wed, 16 May 2018 19:54:24 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 11CCEEADFD5; Wed, 16 May 2018 19:54:24 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F1FE3EADFD3 for ; Wed, 16 May 2018 19:54:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8A55879058 for ; Wed, 16 May 2018 19:54:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A255E8935 for ; Wed, 16 May 2018 19:54:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GJsM48047891 for ; Wed, 16 May 2018 19:54:22 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GJsMDS047890 for fs@FreeBSD.org; Wed, 16 May 2018 19:54:22 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Wed, 16 May 2018 19:54:22 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: kib@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 19:54:24 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #12 from Konstantin Belousov --- (In reply to Damjan Jovanovic from comment #11) r304232 was a fix for very similar issue on UFS. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 16 23:31:00 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DD651EDCEBA for ; Wed, 16 May 2018 23:30:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4FB1C827FB for ; Wed, 16 May 2018 23:30:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0D601EDCEB8; Wed, 16 May 2018 23:30:59 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0262EDCEB7 for ; Wed, 16 May 2018 23:30:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FB59827F5 for ; Wed, 16 May 2018 23:30:58 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id CEC40A676 for ; Wed, 16 May 2018 23:30:57 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GNUv7d088969 for ; Wed, 16 May 2018 23:30:57 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GNUv7I088968 for fs@FreeBSD.org; Wed, 16 May 2018 23:30:57 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Wed, 16 May 2018 23:30:57 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: commit-hook@freebsd.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 23:31:00 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #13 from commit-hook@freebsd.org --- A commit references this bug: Author: mckusick Date: Wed May 16 23:30:04 UTC 2018 New revision: 333693 URL: https://svnweb.freebsd.org/changeset/base/333693 Log: Revert change made in base r171522 (https://svnweb.freebsd.org/base?view=3Drevision&revision=3D304232) converting clrbuf() (which clears the entire buffer) to vfs_bio_clrbuf() (which clears only the new pages that have been added to the buffer). Failure to properly remove pages from the buffer cache can make pages that appear not to need clearing to actually have bad random data in them. See for example base r304232 (https://svnweb.freebsd.org/base?view=3Drevision&revision=3D304232) which noted the need to set B_INVAL and B_NOCACHE as well as clear the B_CACHE flag before calling brelse() to release the buffer. Rather than trying to find all the incomplete brelse() calls, it is simpler, though more slightly expensive, to simply clear the entire buffer when it is newly allocated. PR: 213507 Submitted by: Damjan Jovanovic Reviewed by: kib Changes: head/sys/fs/msdosfs/msdosfs_fat.c --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 16 23:47:17 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46277EDDCC2 for ; Wed, 16 May 2018 23:47:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D3CFD833CF for ; Wed, 16 May 2018 23:47:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 9751CEDDCC1; Wed, 16 May 2018 23:47:16 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 84D5EEDDCBE for ; Wed, 16 May 2018 23:47:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2241B833C9 for ; Wed, 16 May 2018 23:47:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 5C96CA913 for ; Wed, 16 May 2018 23:47:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GNlFuS026183 for ; Wed, 16 May 2018 23:47:15 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GNlFuD026182 for fs@FreeBSD.org; Wed, 16 May 2018 23:47:15 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Wed, 16 May 2018 23:47:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 23:47:17 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |In Progress CC| |mckusick@FreeBSD.org --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 16 23:48:07 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 61306EDDD65 for ; Wed, 16 May 2018 23:48:07 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id E96A68343A for ; Wed, 16 May 2018 23:48:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id AA8D2EDDD64; Wed, 16 May 2018 23:48:06 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9695AEDDD62 for ; Wed, 16 May 2018 23:48:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2F28E83432 for ; Wed, 16 May 2018 23:48:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 5A0CEA920 for ; Wed, 16 May 2018 23:48:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GNm5O1027202 for ; Wed, 16 May 2018 23:48:05 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GNm5mM027201 for fs@FreeBSD.org; Wed, 16 May 2018 23:48:05 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Wed, 16 May 2018 23:48:05 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 23:48:07 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #14 from Kirk McKusick --- (In reply to Damjan Jovanovic from comment #10) Can you verify that my patch fixes this bug for you. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Wed May 16 23:50:27 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C8652EDDFC2 for ; Wed, 16 May 2018 23:50:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 5537183572 for ; Wed, 16 May 2018 23:50:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 0C60EEDDFC1; Wed, 16 May 2018 23:50:27 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EDB7AEDDFBF for ; Wed, 16 May 2018 23:50:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 886898356D for ; Wed, 16 May 2018 23:50:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id BA2D3A934 for ; Wed, 16 May 2018 23:50:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4GNoPH4030308 for ; Wed, 16 May 2018 23:50:25 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4GNoPJs030307 for fs@FreeBSD.org; Wed, 16 May 2018 23:50:25 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Wed, 16 May 2018 23:50:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 May 2018 23:50:28 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #15 from Kirk McKusick --- (In reply to Vladislav Movchan from comment #6) Can you confirm that the fix that I put in (which should be equivalent to t= he patch that you submitted way too long ago) fixes your problem? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 01:19:01 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3891EE3F8A for ; Thu, 17 May 2018 01:19:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6A91E8857D for ; Thu, 17 May 2018 01:19:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 24C10EE3F74; Thu, 17 May 2018 01:19:01 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 122BBEE3F72 for ; Thu, 17 May 2018 01:19:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A0BD488575 for ; Thu, 17 May 2018 01:19:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id BB151B5A2 for ; Thu, 17 May 2018 01:18:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H1IxRR057383 for ; Thu, 17 May 2018 01:18:59 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H1IxXo057382 for fs@FreeBSD.org; Thu, 17 May 2018 01:18:59 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 227784] zfs: Fatal trap 9: general protection fault while in kernel mode on shutdown Date: Thu, 17 May 2018 01:18:59 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: julian@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 01:19:02 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227784 Julian Elischer changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julian@FreeBSD.org --- Comment #1 from Julian Elischer --- I have a similar crash o shutdown .. screenshot attached to the link above ( https://reviews.freebsd.org/P175)=20 FreeBSD build3.local 12.0-CURRENT FreeBSD 12.0-CURRENT #3 r332467: Fri Apr = 13 04:59:35 PDT 2018=20=20=20=20 root@build3.local:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 02:34:22 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1DF7EEB37C for ; Thu, 17 May 2018 02:34:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6E9828C55F for ; Thu, 17 May 2018 02:34:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 32309EEB376; Thu, 17 May 2018 02:34:22 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1F6B3EEB375 for ; Thu, 17 May 2018 02:34:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B04318C559 for ; Thu, 17 May 2018 02:34:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id DC514C233 for ; Thu, 17 May 2018 02:34:20 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H2YKoe045287 for ; Thu, 17 May 2018 02:34:20 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H2YKEM045283 for fs@FreeBSD.org; Thu, 17 May 2018 02:34:20 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Thu, 17 May 2018 02:34:20 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 02:34:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #16 from Damjan Jovanovic --- (In reply to Kirk McKusick from comment #15) It wasn't me that submitted that patch, it was the original reporter, Vladi= slav Movchan. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 04:43:29 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0ADCFEAA309 for ; Thu, 17 May 2018 04:43:29 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 972B86A0AE for ; Thu, 17 May 2018 04:43:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 50E43EAA308; Thu, 17 May 2018 04:43:28 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F888EAA307 for ; Thu, 17 May 2018 04:43:28 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id D3FF86A0AB for ; Thu, 17 May 2018 04:43:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id D79B9D48F for ; Thu, 17 May 2018 04:43:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H4hQPp010209 for ; Thu, 17 May 2018 04:43:26 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H4hQ7l010208 for fs@FreeBSD.org; Thu, 17 May 2018 04:43:26 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Thu, 17 May 2018 04:43:26 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 04:43:29 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #17 from Kirk McKusick --- (In reply to Damjan Jovanovic from comment #16) Well in comment #10 you said you could reproduce it on CURRENT. So I am wondering if you can still reproduce it on CURRENT after this commit (r3336= 93). --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 05:48:23 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8DB72EAE877 for ; Thu, 17 May 2018 05:48:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 274E46C5D8 for ; Thu, 17 May 2018 05:48:23 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id DFBC9EAE874; Thu, 17 May 2018 05:48:22 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CD7E7EAE873 for ; Thu, 17 May 2018 05:48:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 503996C5D6 for ; Thu, 17 May 2018 05:48:22 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 7C84FDCD4 for ; Thu, 17 May 2018 05:48:21 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H5mLxf073425 for ; Thu, 17 May 2018 05:48:21 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H5mLF7073424 for fs@FreeBSD.org; Thu, 17 May 2018 05:48:21 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media Date: Thu, 17 May 2018 05:48:21 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 05:48:23 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210316 --- Comment #14 from Andriy Gapon --- (In reply to Conrad Meyer from comment #13) Indeed, if we talk about the general behaviour. I see that I utterly failed to explain that I was thinking purely in a cont= ext of what msdos does in markvoldirty. Right now that code can leave behind a perpetually dirty buffer and I was thinking how that can be avoided. Maybe markvoldirty should do bp =3D getblk(...) bp->b_flags |=3D B_INVAL | B_RELBUF | B_NOCACHE; bp->b_flags &=3D ~(B_ASYNC | B_CACHE); brelse(bp); after a failed write? Looks clumsy, but should work. Or maybe markvoldirty should not use buffer cache for its write? It could use g_write_data, for example. But that sounds like layering violation. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 06:00:33 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A92C9EAF771 for ; Thu, 17 May 2018 06:00:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 6369F6CE50 for ; Thu, 17 May 2018 06:00:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 1DD2BEAF766; Thu, 17 May 2018 06:00:33 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0BA6FEAF762 for ; Thu, 17 May 2018 06:00:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5B3CA6CE3B for ; Thu, 17 May 2018 06:00:32 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 352D7DE41 for ; Thu, 17 May 2018 06:00:31 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H60VC5008472 for ; Thu, 17 May 2018 06:00:31 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H60Vew008468 for fs@FreeBSD.org; Thu, 17 May 2018 06:00:31 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media Date: Thu, 17 May 2018 06:00:30 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 06:00:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210316 --- Comment #15 from Andriy Gapon --- (In reply to Konstantin Belousov from comment #9) This proposal looks like something that can help with the problem at hand i= n a generic way. I see that at present g_vfs_close() does not even check the return code of bufobj_invalbuf(). That looks dangerous? But not sure if the code can do better. E.g., if looping on an error would be okay. Then, there remains a question of when to use the proposed g_vfs_close(NOSA= VE). There are obvious cases like a failed mount. And that's what we want to address now, so that's good. But there can be a more complex situation. For example, we can mount a filesystem r/w and do some writes to it. But then the hardware suddenly decides to go to read-only mode (e.g., becau= se some internal counter crosses its threshold). So, we will get some failed writes and some dirty buffers that cannot possi= bly be made clean by writing out. Do we need to somehow handle them? And then we decide to unmount the filesystem. How do we handle those buffers? How do we know if we should call g_vfs_close(SAVE) or g_vfs_close(NOSAVE) ? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 06:14:06 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 500CBEB065D for ; Thu, 17 May 2018 06:14:06 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D7D5E6D8FF for ; Thu, 17 May 2018 06:14:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 96E85EB064E; Thu, 17 May 2018 06:14:05 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 84394EB064A for ; Thu, 17 May 2018 06:14:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 195966D8FB for ; Thu, 17 May 2018 06:14:05 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 645CBE0F1 for ; Thu, 17 May 2018 06:14:04 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H6E4Xg010684 for ; Thu, 17 May 2018 06:14:04 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H6E4bm010676 for fs@FreeBSD.org; Thu, 17 May 2018 06:14:04 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 227784] zfs: Fatal trap 9: general protection fault while in kernel mode on shutdown Date: Thu, 17 May 2018 06:14:04 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 06:14:06 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227784 --- Comment #2 from Andriy Gapon --- A similar report by Harry Schmalzbauer on mailing list= s: ... #8 0xffffffff80ae89fc in calltrap () at /usr/local/share/deploy-tools/HEAD/src/sys/amd64/amd64/exception.S:230 #9 0xffffffff81b13984 in spa_get_dsl () from /boot/kernel/zfs.ko #10 0xffffffff81b17d1f in txg_verify () from /boot/kernel/zfs.ko #11 0xffffffff81b1829f in txg_list_member () from /boot/kernel/zfs.ko #12 0xffffffff81acb969 in dsl_dir_evict_async () from /boot/kernel/zfs.ko #13 0xffffffff808260cc in taskqueue_run_locked (queue=3D0xfffff80004058800) at /usr/local/share/deploy-tools/HEAD/src/sys/kern/subr_taskqueue.c:465 #14 0xffffffff80826e98 in taskqueue_thread_loop (arg=3D) at /usr/local/share/deploy-tools/HEAD/src/sys/kern/subr_taskqueue.c:757 #15 0xffffffff8078add4 in fork_exit (callout=3D0xffffffff80826e10 , arg=3D0xfffff80003b46e40, frame=3D0xfffffe0000575ac0) at /usr/local/share/deploy-tools/HEAD/src/sys/kern/kern_fork.c:1039 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 06:15:53 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14815EB0846 for ; Thu, 17 May 2018 06:15:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A25FF6D9DE for ; Thu, 17 May 2018 06:15:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 66A98EB0845; Thu, 17 May 2018 06:15:52 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 55223EB0844 for ; Thu, 17 May 2018 06:15:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E7A966D9DA for ; Thu, 17 May 2018 06:15:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 0E480E0F6 for ; Thu, 17 May 2018 06:15:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H6FoJF064480 for ; Thu, 17 May 2018 06:15:50 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H6FoPq064467 for fs@FreeBSD.org; Thu, 17 May 2018 06:15:50 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 227784] zfs: Fatal trap 9: general protection fault while in kernel mode on shutdown Date: Thu, 17 May 2018 06:15:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: avg@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 06:15:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227784 --- Comment #3 from Andriy Gapon --- (In reply to Julian Elischer from comment #1) Bugzilla supports attachments too. In fact, I wish that at least the stack trace from the original report was = here as well. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 06:35:53 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5468CEB1D08 for ; Thu, 17 May 2018 06:35:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id DFCE16E78A for ; Thu, 17 May 2018 06:35:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 99966EB1D07; Thu, 17 May 2018 06:35:52 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 85A77EB1D05 for ; Thu, 17 May 2018 06:35:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1CF6A6E784 for ; Thu, 17 May 2018 06:35:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 506F8E3C2 for ; Thu, 17 May 2018 06:35:51 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H6ZpuA094704 for ; Thu, 17 May 2018 06:35:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H6ZpMN094703 for fs@FreeBSD.org; Thu, 17 May 2018 06:35:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 227784] zfs: Fatal trap 9: general protection fault while in kernel mode on shutdown Date: Thu, 17 May 2018 06:35:51 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: eadler@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: attachments.created Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 06:35:53 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227784 --- Comment #4 from Eitan Adler --- Created attachment 193474 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=3D193474&action= =3Dedit initial report and stacktrace --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 06:36:16 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6A766EB1D4F for ; Thu, 17 May 2018 06:36:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id EE3576E7DE for ; Thu, 17 May 2018 06:36:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id A85FEEB1D49; Thu, 17 May 2018 06:36:15 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 95D81EB1D48 for ; Thu, 17 May 2018 06:36:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 315DA6E7DB for ; Thu, 17 May 2018 06:36:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 70BD1E3C7 for ; Thu, 17 May 2018 06:36:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H6aEYs095285 for ; Thu, 17 May 2018 06:36:14 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H6aE1v095283 for fs@FreeBSD.org; Thu, 17 May 2018 06:36:14 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 227784] zfs: Fatal trap 9: general protection fault while in kernel mode on shutdown Date: Thu, 17 May 2018 06:36:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: eadler@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 06:36:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D227784 --- Comment #5 from Eitan Adler --- I added a copy of the report. I wish that bugzilla and phab were more close= ly linked :\ --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 07:15:09 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF141ED9AA2 for ; Thu, 17 May 2018 07:15:09 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 7B106701F5 for ; Thu, 17 May 2018 07:15:09 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: by mailman.ysv.freebsd.org (Postfix) id 350BAED9A86; Thu, 17 May 2018 07:15:09 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 22F85ED9A81 for ; Thu, 17 May 2018 07:15:09 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail104.syd.optusnet.com.au (mail104.syd.optusnet.com.au [211.29.132.246]) by mx1.freebsd.org (Postfix) with ESMTP id 591F2701E6 for ; Thu, 17 May 2018 07:15:07 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail104.syd.optusnet.com.au (Postfix) with ESMTPS id 36AB542C0B2 for ; Thu, 17 May 2018 17:14:52 +1000 (AEST) Date: Thu, 17 May 2018 17:14:50 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org cc: fs@freebsd.org Subject: Re: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media In-Reply-To: Message-ID: <20180517163709.F1129@besplex.bde.org> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=I9sVfJog c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=9cW_t1CCXrUA:10 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=marit_-ZtZOUWzEyCiYA:9 a=CjuIK1q_8ugA:10 a=IjZwj45LgO3ly-622nXo:22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 07:15:10 -0000 On Thu, 17 May 2018 a bug that doesn't want replies@freebsd.org wrote: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210316 > > --- Comment #14 from Andriy Gapon --- > (In reply to Conrad Meyer from comment #13) > > Indeed, if we talk about the general behaviour. > > I see that I utterly failed to explain that I was thinking purely in a context > of what msdos does in markvoldirty. > Right now that code can leave behind a perpetually dirty buffer and I was > thinking how that can be avoided. > > Maybe markvoldirty should do > bp = getblk(...) > bp->b_flags |= B_INVAL | B_RELBUF | B_NOCACHE; > bp->b_flags &= ~(B_ASYNC | B_CACHE); > brelse(bp); > after a failed write? > Looks clumsy, but should work. I think this is the only way to clean up the buffer cache. > Or maybe markvoldirty should not use buffer cache for its write? > It could use g_write_data, for example. But that sounds like layering > violation. Not a good way. Markvoldirty() was obtained from apple and fixed a bit by me, but is still very bad, without even this write protection bug. Before it was implemented, you could use removable media with write protection on, and have no writes occur even if you forgot to mount with ro, and nothing bad happened if the media was removed without unmounting provided it was never explicitly written to. Now, markvoldirty() ensures that bad things happen if the media is removed without unmounting, even if the media is writeable initially so that markvoldirty() doesn't fail. I thought that failures were handled better. markvoldirty() returns bwrite(). There is a lot of error handling for this, but this ends up as just markvoldirty() back to clean with the result voided for the final call. For unwriteable media, the buffer remains in the buffer cache forever. One idea for improving this is to delay markvoldirty() until the first explicit write(). Also, don't clobber the disk to write atimes even if the fs is mounted rw and without -noatime (it takes something like FAT32 before atimes even exist in msdosfs). msdosfs has always had an internal flag pm_fmod which was apparently intended for a similar optimization, but it is useless since it is always set on successful rw mounts and not cleared until unmount, and it is write-only except for a check in msdosfs_sync() where it just causes a panic if it is not set. The voldirty flag and any internal dirty flags should also be set to clean if the file system is not written to for some time after a successful complete sync, so that the fs is usually clean if it is not written to often. All versions of Windows that I have tried seem to do this. Bruce From owner-freebsd-fs@freebsd.org Thu May 17 07:41:16 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B03E0EDB634 for ; Thu, 17 May 2018 07:41:16 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4BC4C70FB5 for ; Thu, 17 May 2018 07:41:16 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: by mailman.ysv.freebsd.org (Postfix) id 10143EDB633; Thu, 17 May 2018 07:41:16 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F042EEDB632 for ; Thu, 17 May 2018 07:41:15 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id 6B20C70FB0 for ; Thu, 17 May 2018 07:41:15 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id BD266107C7A for ; Thu, 17 May 2018 17:41:13 +1000 (AEST) Date: Thu, 17 May 2018 17:41:13 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org cc: fs@freebsd.org Subject: Re: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media In-Reply-To: <20180517163709.F1129@besplex.bde.org> Message-ID: <20180517173539.B1308@besplex.bde.org> References: <20180517163709.F1129@besplex.bde.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=cIaQihWN c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=9cW_t1CCXrUA:10 a=kj9zAlcOel0A:10 a=6I5d2MoRAAAA:8 a=MBVNPijdDLtBr63qWgoA:9 a=CjuIK1q_8ugA:10 a=IjZwj45LgO3ly-622nXo:22 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 07:41:16 -0000 On Thu, 17 May 2018, Bruce Evans wrote: > On Thu, 17 May 2018 a bug that doesn't want replies@freebsd.org wrote: > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210316 >> >> --- Comment #14 from Andriy Gapon --- >> (In reply to Conrad Meyer from comment #13) >> >> Indeed, if we talk about the general behaviour. >> >> I see that I utterly failed to explain that I was thinking purely in a >> context >> of what msdos does in markvoldirty. > ... > Markvoldirty() was obtained from apple and fixed a bit by me, but is still > very bad, without even this write protection bug. > ... I forgot to mention its main bug. msdosfs doesn't check the dirty flag, so it allows even rw mounts of dirty file systems (and I think it is too dangerous to allow even ro mounts for fs's with FAT). So maintaining the dirty flag is useless for FreeBSD. It is just a courtesy for OS'es that do check the flag. Bruce From owner-freebsd-fs@freebsd.org Thu May 17 07:42:14 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 19973EDB721 for ; Thu, 17 May 2018 07:42:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id A0A81711F3 for ; Thu, 17 May 2018 07:42:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 60129EDB720; Thu, 17 May 2018 07:42:13 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4DDFEEDB71E for ; Thu, 17 May 2018 07:42:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DBEA8711E7 for ; Thu, 17 May 2018 07:42:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 1A489EC65 for ; Thu, 17 May 2018 07:42:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H7gBW9077580 for ; Thu, 17 May 2018 07:42:11 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H7gBax077579 for fs@FreeBSD.org; Thu, 17 May 2018 07:42:11 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Thu, 17 May 2018 07:42:12 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: vladislav.movchan@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 07:42:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #18 from Vladislav Movchan --- (In reply to Kirk McKusick from comment #15) I confirm I'm no longer able to reproduce this problem after base r333693. Thanks to everyone involved into investigation and resolution of this probl= em. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 08:12:12 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EC7EBEDDF29 for ; Thu, 17 May 2018 08:12:11 +0000 (UTC) (envelope-from se@freebsd.org) Received: from mailout04.t-online.de (mailout04.t-online.de [194.25.134.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 77D9A72926 for ; Thu, 17 May 2018 08:12:11 +0000 (UTC) (envelope-from se@freebsd.org) Received: from fwd17.aul.t-online.de (fwd17.aul.t-online.de [172.20.27.64]) by mailout04.t-online.de (Postfix) with SMTP id 67485419F8C2; Thu, 17 May 2018 10:04:09 +0200 (CEST) Received: from Stefans-MacBook-Pro-10.local (bRAWReZCYhdRPeo0RTSvpqMZpD2xoL1oUrjbN12MmV7KehE3ZSPC5FIor8JKUvjZPS@[84.154.105.176]) by fwd17.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1fJDtK-2Wh7tg0; Thu, 17 May 2018 10:04:02 +0200 Subject: Re: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media To: Bruce Evans , FreeBSD File-Systems References: <20180517163709.F1129@besplex.bde.org> From: Stefan Esser Openpgp: preference=signencrypt Autocrypt: addr=se@freebsd.org; prefer-encrypt=mutual; keydata= xsBNBFVxiRIBCADOLNOZBsqlplHUQ3tG782FNtVT33rQli9EjNt2fhFERHIo4NxHlWBpHLnU b0s4L/eItx7au0i7Gegv01A9LUMwOnAc9EFAm4EW3Wmoa6MYrcP7xDClohg/Y69f7SNpEs3x YATBy+L6NzWZbJjZXD4vqPgZSDuMcLU7BEdJf0f+6h1BJPnGuwHpsSdnnMrZeIM8xQ8PPUVQ L0GZkVojHgNUngJH6e21qDrud0BkdiBcij0M3TCP4GQrJ/YMdurfc8mhueLpwGR2U1W8TYB7 4UY+NLw0McThOCLCxXflIeF/Y7jSB0zxzvb/H3LWkodUTkV57yX9IbUAGA5RKRg9zsUtABEB AAHNKVN0ZWZhbiBFw59lciAoWWFob28hKSA8c3QuZXNzZXJAeWFob28uZGU+wsCWBBMBCgBA AhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AWIQSjceplnAvsyCtxUxNH67XvWv31RAUC WvLvqwUJCyUBEwAKCRBH67XvWv31REySCACc6vqcSFQCRyBRc2CV5ZBjbbnTy7VBoXbUS3/c 4Hn8I0YQ39q7//2z8vYsgLeM1mMXL4PUIU/0f0dBAFBLpxV7bntGzyCJls6SeGS/qcQKhqaI 6I7NcWg8OkIJIhUL6q238cS1ql9pU65fyHe0PP8JS08m81PDpX2/4wTE6h2jgYUy55eXRzoF MEjr1S8SSnidsBem27o7iWu9ltJsUtE86071iZlLzbuHv2nvucrjAV9cK9tHrxYT/YiY8QhT L48iWj2xIjLjg1ebmgIFZ2k881we/KTIoUugqOOR1gDSc4qwM8CA388cN3frjtl98CwhAT5T UV8tIDqri+/Z1AKwzsBNBFVxiRIBCACxI/aglzGVbnI6XHd0MTP05VK/fJub4hHdc+LQpz1M kVnCAhFbY9oecTB/togdKtfiloavjbFrb0nJhJnx57K+3SdSuu+znaQ4SlWiZOtXnkbpRWNU eMm+gtTDMSvloGAfr76RtFHskdDOLgXsHD70bKuMhlBxUCrSwGzHaD00q8iQPhJZ5itb3WPq z3B4IjiDAWTO2obD1wtAvSuHuUj/XJRsiKDKW3x13cfavkad81bZW4cpNwUv8XHLv/vaZPSA ly+hkY7NrDZydMMXVNQ7AJQufWuTJ0q7sImRcEZ5EIa98esJPey4O7C0vY405wjeyxpVZkpq ThDMurqtQFn1ABEBAAHCwHwEGAEKACYCGwwWIQSjceplnAvsyCtxUxNH67XvWv31RAUCWvLv qwUJCyUBGQAKCRBH67XvWv31RLnrB/9gzcRlpx71sDMosoZULWn7wysBJ/8AIEfIByRaHQe3 pn/KwE57pB+zFbbQqB7YzeZb7/UUgR4zU2ZbOcEfwDZcHUbj0B3fGRsS3t0uiLlAd8w0sBZb SxrqzjdpDjIbOZkxssqUmvrsN67UG1AFWH9aD24keBS7YjPBS8hLxPeYV+Xz6vUL8fRZje/Z JgiBMIwyj6g2lH/zkdnxBdC0iG1xxJOLTaghMMeQyCdH6ef8+VMyAlAJsMckbOTvx63tY8z7 DFcrnTJfbe1EziRilVsEaK8tTzJzhcTfos+f3eBYWEilxe5HzIhYKJeC7lmsSUcGwa6+9VRg a0ctmi9Z8OgX Message-ID: <8c1cb4b3-633a-5b14-0713-727b03f44f4e@freebsd.org> Date: Thu, 17 May 2018 10:04:01 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20180517163709.F1129@besplex.bde.org> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 8bit X-ID: bRAWReZCYhdRPeo0RTSvpqMZpD2xoL1oUrjbN12MmV7KehE3ZSPC5FIor8JKUvjZPS X-TOI-MSGID: cf91b1af-dd1f-4ef4-a04e-440fa0da067a X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 08:12:12 -0000 Am 17.05.18 um 09:14 schrieb Bruce Evans: > On Thu, 17 May 2018 a bug that doesn't want replies@freebsd.org wrote: > >> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210316 >> >> --- Comment #14 from Andriy Gapon --- >> (In reply to Conrad Meyer from comment #13) >> >> Indeed, if we talk about the general behaviour. >> >> I see that I utterly failed to explain that I was thinking purely in a context >> of what msdos does in markvoldirty. >> Right now that code can leave behind a perpetually dirty buffer and I was >> thinking how that can be avoided. >> >> Maybe markvoldirty should do >>    bp = getblk(...) >>    bp->b_flags |= B_INVAL | B_RELBUF | B_NOCACHE; >>    bp->b_flags &= ~(B_ASYNC | B_CACHE); >>    brelse(bp); >> after a failed write? >> Looks clumsy, but should work. > > I think this is the only way to clean up the buffer cache. > >> Or maybe markvoldirty should not use buffer cache for its write? >> It could use g_write_data, for example.  But that sounds like layering >> violation. > > Not a good way. > > Markvoldirty() was obtained from apple and fixed a bit by me, but is still > very bad, without even this write protection bug. > > Before it was implemented, you could use removable media with write > protection on, and have no writes occur even if you forgot to mount > with ro, and nothing bad happened if the media was removed without > unmounting provided it was never explicitly written to.  Now, > markvoldirty() ensures that bad things happen if the media is removed > without unmounting, even if the media is writeable initially so that > markvoldirty() doesn't fail. > > I thought that failures were handled better.  markvoldirty() returns > bwrite().  There is a lot of error handling for this, but this ends > up as just markvoldirty() back to clean with the result voided for the > final call.  For unwriteable media, the buffer remains in the buffer > cache forever. > > One idea for improving this is to delay markvoldirty() until the first > explicit write().  Also, don't clobber the disk to write atimes even if > the fs is mounted rw and without -noatime (it takes something like FAT32 > before atimes even exist in msdosfs).  msdosfs has always had an internal > flag pm_fmod which was apparently intended for a similar optimization, but > it is useless since it is always set on successful rw mounts and not cleared > until unmount, and it is write-only except for a check in msdosfs_sync() > where it just causes a panic if it is not set.  The voldirty flag and > any internal dirty flags should also be set to clean if the file system > is not written to for some time after a successful complete sync, so that > the fs is usually clean if it is not written to often.  All versions of > Windows that I have tried seem to do this. Some 20 years ago I had to work with AIX machines, and I found that they offered a nice feature for accesses to removable media (floppy disks, at that time). If such a media was not written to for a few seconds, it could be removed without unmounting. I proposed to implement a timer that was triggered when the number of dirty buffers for a partition drops to zero and that is canceled when the partition is written to (this does not need to be a timer of course, polling for that case every few seconds works as well), at that time. And pre-soft-updates and journaling that feature had also been of advantage for UFS file systems that are rarely written but where the cause of most fsck delay after an unclean shutdown. In case that a media (whether removable or not) was mounted R/W and not written to (had no dirty buffers) for more than a few seconds, the mount could be downgraded to R/O (in the same way as by a "mount -u -o ro"). A flag that recorded the fact, that this partition may be written to could then be checked in the "write to R/O partition" error case, and if the file system was only temporarily set to R/O, it could be treated like a first access to a writable partition (i.e., write a dirty flag into the super-block or whatever action the file system performs when mounted R/W). In short, the suggestion is to down-grade the mount state of any file-system not used for some configurable time to R/O, with an automatic upgrade to R/W on the next write attempt. I did not try to fully implement that feature when floppy disks became less and less relevant, but with USB and SD media being used as writable media, today, the same situation exists as with floppy disks some 20 years ago. The only requirement for such a mechanism is that the number of dirty buffers per partition is known and accessible for a polling every few seconds, that causes the temporary down-grade to R/O to be triggered. Everything else is trivial (i.e., just check a flag in the "write to R/O" error path and clear the R/O flag in such a way that the dirty flag gets written). That requires a (trivial) change in each file system that wants to be able to upgrade to R/W after the temporary downgrade to R/O, though. Regards, STefan From owner-freebsd-fs@freebsd.org Thu May 17 08:29:56 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A0D3EDF00B for ; Thu, 17 May 2018 08:29:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id B08DA7333F for ; Thu, 17 May 2018 08:29:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 6F8BEEDF00A; Thu, 17 May 2018 08:29:55 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5B7BDEDF009 for ; Thu, 17 May 2018 08:29:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E828B73338 for ; Thu, 17 May 2018 08:29:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 18A17F3FA for ; Thu, 17 May 2018 08:29:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H8TrPG097577 for ; Thu, 17 May 2018 08:29:53 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H8Trlb097576 for fs@FreeBSD.org; Thu, 17 May 2018 08:29:53 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Thu, 17 May 2018 08:29:54 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 08:29:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #19 from Damjan Jovanovic --- I am still doing my test. That isn't the only vfs_bio_clrbuf() in msdosfs. IMHO we really should "find all the incomplete brelse() calls". How do "incomplete brelse() calls" work? What is the general rule? Accordin= g to that UFS patch it seems to be: a buf's b_data can only be changed if you subsequently bwrite() it, or brelse() it with b_flags containing B_INVAL | B_RELBUF | B_NOCACHE and not containing B_ASYNC | B_CACHE. If bwrite() fails you must brelse() the buf with those flags instead. Is that right? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 08:46:49 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3452EE0174 for ; Thu, 17 May 2018 08:46:49 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail110.syd.optusnet.com.au (mail110.syd.optusnet.com.au [211.29.132.97]) by mx1.freebsd.org (Postfix) with ESMTP id E672B74080; Thu, 17 May 2018 08:46:48 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from [192.168.0.102] (c110-21-101-228.carlnfd1.nsw.optusnet.com.au [110.21.101.228]) by mail110.syd.optusnet.com.au (Postfix) with ESMTPS id 6B6381055F3; Thu, 17 May 2018 18:46:46 +1000 (AEST) Date: Thu, 17 May 2018 18:46:45 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Stefan Esser cc: FreeBSD File-Systems Subject: Re: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media In-Reply-To: <8c1cb4b3-633a-5b14-0713-727b03f44f4e@freebsd.org> Message-ID: <20180517183323.I1485@besplex.bde.org> References: <20180517163709.F1129@besplex.bde.org> <8c1cb4b3-633a-5b14-0713-727b03f44f4e@freebsd.org> MIME-Version: 1.0 X-Optus-CM-Score: 0 X-Optus-CM-Analysis: v=2.2 cv=cIaQihWN c=1 sm=1 tr=0 a=PalzARQSbocsUSjMRkwAPg==:117 a=PalzARQSbocsUSjMRkwAPg==:17 a=nlC_4_pT8q9DhB4Ho9EA:9 a=DfJEhno4DlxxcEHO3rYA:9 a=-R1uio8HKrReRHQl:21 a=YrIhb_NiRk3hJ3FI:21 a=45ClL6m2LaAA:10 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN; format=flowed Content-Transfer-Encoding: QUOTED-PRINTABLE X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 08:46:50 -0000 On Thu, 17 May 2018, Stefan Esser wrote: > Am 17.05.18 um 09:14 schrieb Bruce Evans: >> >> [.. about markvoldirty() in msdosfs not being too good] >> >> One idea for improving this is to delay markvoldirty() until the first >> explicit write().=A0 Also, don't clobber the disk to write atimes even i= f >> the fs is mounted rw and without -noatime (it takes something like FAT32 >> before atimes even exist in msdosfs).=A0 msdosfs has always had an inter= nal >> flag pm_fmod which was apparently intended for a similar optimization, b= ut >> it is useless since it is always set on successful rw mounts and not cle= ared >> until unmount, and it is write-only except for a check in msdosfs_sync() >> where it just causes a panic if it is not set.=A0 The voldirty flag and >> any internal dirty flags should also be set to clean if the file system >> is not written to for some time after a successful complete sync, so tha= t >> the fs is usually clean if it is not written to often.=A0 All versions o= f >> Windows that I have tried seem to do this. > > Some 20 years ago I had to work with AIX machines, and I found that they > offered a nice feature for accesses to removable media (floppy disks, at > that time). If such a media was not written to for a few seconds, it coul= d > be removed without unmounting. > > I proposed to implement a timer that was triggered when the number of > dirty buffers for a partition drops to zero and that is canceled when > the partition is written to (this does not need to be a timer of course, > polling for that case every few seconds works as well), at that time. And > pre-soft-updates and journaling that feature had also been of advantage > for UFS file systems that are rarely written but where the cause of most > fsck delay after an unclean shutdown. > > In case that a media (whether removable or not) was mounted R/W and not > written to (had no dirty buffers) for more than a few seconds, the mount > could be downgraded to R/O (in the same way as by a "mount -u -o ro"). A > flag that recorded the fact, that this partition may be written to could > then be checked in the "write to R/O partition" error case, and if the > file system was only temporarily set to R/O, it could be treated like a > first access to a writable partition (i.e., write a dirty flag into the > super-block or whatever action the file system performs when mounted R/W)= =2E > > In short, the suggestion is to down-grade the mount state of any file-sys= tem > not used for some configurable time to R/O, with an automatic upgrade to = R/W > on the next write attempt. > ... I want this more for ffs too. Anither idea is to have a per-cylinder-group (cg) dirty flag and only fsck the dirty cg's. Then an enormous number of cg's would be a feature. Most would remain clean unless locality is bad. This is too large a project for me. I only have small data and just use small ffs file systems and mount m= ost of them ro most of the time. Bruce From owner-freebsd-fs@freebsd.org Thu May 17 09:03:27 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3B3D1EE1240 for ; Thu, 17 May 2018 09:03:27 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id CCCB67494C for ; Thu, 17 May 2018 09:03:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 8D736EE123F; Thu, 17 May 2018 09:03:26 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 796ECEE123E for ; Thu, 17 May 2018 09:03:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 100117494B for ; Thu, 17 May 2018 09:03:26 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 4203AF98E for ; Thu, 17 May 2018 09:03:25 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4H93PCr098435 for ; Thu, 17 May 2018 09:03:25 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4H93Pjv098433 for fs@FreeBSD.org; Thu, 17 May 2018 09:03:25 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 191858] [msdosfs] [panic] Disposed page 0xfffff807d41eeb60 is dirty Date: Thu, 17 May 2018 09:03:25 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: vladislav.movchan@gmail.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 09:03:27 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D191858 Vladislav Movchan changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |vladislav.movchan@gmail.com --- Comment #2 from Vladislav Movchan --- I suppose this problem (panic: Freeing unused sector) might be caused by the same reason described in bug 213507. I.e. in directories with the large amount of files (directories which occupy more than one cluster) garbage-like directory entries could appear. Attempt to interpret such dentries (which are basically random data) could = lead to unexpected results. Could you please try to reproduce this on CURRENT after base r333693? To reproduce this problem myself I tried to create several hundreds of directories with 500 to 1000 small files in each and removing them all at o= nce. I repeated this procedure multiple times but was not able to trigger this on r333698. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 11:42:47 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F2400EEB402 for ; Thu, 17 May 2018 11:42:46 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 64ABF7B5E3; Thu, 17 May 2018 11:42:46 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w4HBgZdj048621 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 17 May 2018 14:42:38 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w4HBgZdj048621 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w4HBgZOC048620; Thu, 17 May 2018 14:42:35 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 17 May 2018 14:42:35 +0300 From: Konstantin Belousov To: Stefan Esser Cc: Bruce Evans , FreeBSD File-Systems Subject: Re: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media Message-ID: <20180517114235.GI6887@kib.kiev.ua> References: <20180517163709.F1129@besplex.bde.org> <8c1cb4b3-633a-5b14-0713-727b03f44f4e@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <8c1cb4b3-633a-5b14-0713-727b03f44f4e@freebsd.org> User-Agent: Mutt/1.9.5 (2018-04-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 11:42:47 -0000 On Thu, May 17, 2018 at 10:04:01AM +0200, Stefan Esser wrote: > In case that a media (whether removable or not) was mounted R/W and not > written to (had no dirty buffers) for more than a few seconds, the mount > could be downgraded to R/O (in the same way as by a "mount -u -o ro"). A > flag that recorded the fact, that this partition may be written to could > then be checked in the "write to R/O partition" error case, and if the > file system was only temporarily set to R/O, it could be treated like a > first access to a writable partition (i.e., write a dirty flag into the > super-block or whatever action the file system performs when mounted R/W). > > In short, the suggestion is to down-grade the mount state of any file-system > not used for some configurable time to R/O, with an automatic upgrade to R/W > on the next write attempt. Downgrade to ro revokes all file descriptors opened for write, otherwise it is not a downgrade. This would be a huge mess. From owner-freebsd-fs@freebsd.org Thu May 17 11:43:16 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 921E8EEB46E for ; Thu, 17 May 2018 11:43:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 2EE5B7B637 for ; Thu, 17 May 2018 11:43:16 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id E69C5EEB46A; Thu, 17 May 2018 11:43:15 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D514FEEB469 for ; Thu, 17 May 2018 11:43:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7673E7B631 for ; Thu, 17 May 2018 11:43:15 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id A9B2410FA0 for ; Thu, 17 May 2018 11:43:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4HBhENH091710 for ; Thu, 17 May 2018 11:43:14 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4HBhEuf091709 for fs@FreeBSD.org; Thu, 17 May 2018 11:43:14 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 191858] [msdosfs] [panic] Disposed page 0xfffff807d41eeb60 is dirty Date: Thu, 17 May 2018 11:43:14 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: pho@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 11:43:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D191858 --- Comment #3 from Peter Holm --- I repeated the test on r333704 and did not find any problems. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 13:24:27 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A8029EF0F69 for ; Thu, 17 May 2018 13:24:27 +0000 (UTC) (envelope-from se@freebsd.org) Received: from mailout02.t-online.de (mailout02.t-online.de [194.25.134.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mailout00.t-online.de", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 2CEED7F431 for ; Thu, 17 May 2018 13:24:27 +0000 (UTC) (envelope-from se@freebsd.org) Received: from fwd13.aul.t-online.de (fwd13.aul.t-online.de [172.20.27.62]) by mailout02.t-online.de (Postfix) with SMTP id 08F9441800EA; Thu, 17 May 2018 15:24:25 +0200 (CEST) Received: from Stefans-MacBook-Pro-10.local (TlW4t4ZHwh9jgBvSXZ+7AgAbD4Fwy388CZxn0EQGsrl00d5X7YNIu8u-VQMf2pagOO@[84.154.105.176]) by fwd13.t-online.de with (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384 encrypted) esmtp id 1fJItG-0KvuqG0; Thu, 17 May 2018 15:24:18 +0200 Subject: Re: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media To: Konstantin Belousov Cc: Bruce Evans , FreeBSD File-Systems References: <20180517163709.F1129@besplex.bde.org> <8c1cb4b3-633a-5b14-0713-727b03f44f4e@freebsd.org> <20180517114235.GI6887@kib.kiev.ua> From: Stefan Esser Openpgp: preference=signencrypt Autocrypt: addr=se@freebsd.org; prefer-encrypt=mutual; keydata= xsBNBFVxiRIBCADOLNOZBsqlplHUQ3tG782FNtVT33rQli9EjNt2fhFERHIo4NxHlWBpHLnU b0s4L/eItx7au0i7Gegv01A9LUMwOnAc9EFAm4EW3Wmoa6MYrcP7xDClohg/Y69f7SNpEs3x YATBy+L6NzWZbJjZXD4vqPgZSDuMcLU7BEdJf0f+6h1BJPnGuwHpsSdnnMrZeIM8xQ8PPUVQ L0GZkVojHgNUngJH6e21qDrud0BkdiBcij0M3TCP4GQrJ/YMdurfc8mhueLpwGR2U1W8TYB7 4UY+NLw0McThOCLCxXflIeF/Y7jSB0zxzvb/H3LWkodUTkV57yX9IbUAGA5RKRg9zsUtABEB AAHNKVN0ZWZhbiBFw59lciAoWWFob28hKSA8c3QuZXNzZXJAeWFob28uZGU+wsCWBBMBCgBA AhsDBwsJCAcDAgEGFQgCCQoLBBYCAwECHgECF4AWIQSjceplnAvsyCtxUxNH67XvWv31RAUC WvLvqwUJCyUBEwAKCRBH67XvWv31REySCACc6vqcSFQCRyBRc2CV5ZBjbbnTy7VBoXbUS3/c 4Hn8I0YQ39q7//2z8vYsgLeM1mMXL4PUIU/0f0dBAFBLpxV7bntGzyCJls6SeGS/qcQKhqaI 6I7NcWg8OkIJIhUL6q238cS1ql9pU65fyHe0PP8JS08m81PDpX2/4wTE6h2jgYUy55eXRzoF MEjr1S8SSnidsBem27o7iWu9ltJsUtE86071iZlLzbuHv2nvucrjAV9cK9tHrxYT/YiY8QhT L48iWj2xIjLjg1ebmgIFZ2k881we/KTIoUugqOOR1gDSc4qwM8CA388cN3frjtl98CwhAT5T UV8tIDqri+/Z1AKwzsBNBFVxiRIBCACxI/aglzGVbnI6XHd0MTP05VK/fJub4hHdc+LQpz1M kVnCAhFbY9oecTB/togdKtfiloavjbFrb0nJhJnx57K+3SdSuu+znaQ4SlWiZOtXnkbpRWNU eMm+gtTDMSvloGAfr76RtFHskdDOLgXsHD70bKuMhlBxUCrSwGzHaD00q8iQPhJZ5itb3WPq z3B4IjiDAWTO2obD1wtAvSuHuUj/XJRsiKDKW3x13cfavkad81bZW4cpNwUv8XHLv/vaZPSA ly+hkY7NrDZydMMXVNQ7AJQufWuTJ0q7sImRcEZ5EIa98esJPey4O7C0vY405wjeyxpVZkpq ThDMurqtQFn1ABEBAAHCwHwEGAEKACYCGwwWIQSjceplnAvsyCtxUxNH67XvWv31RAUCWvLv qwUJCyUBGQAKCRBH67XvWv31RLnrB/9gzcRlpx71sDMosoZULWn7wysBJ/8AIEfIByRaHQe3 pn/KwE57pB+zFbbQqB7YzeZb7/UUgR4zU2ZbOcEfwDZcHUbj0B3fGRsS3t0uiLlAd8w0sBZb SxrqzjdpDjIbOZkxssqUmvrsN67UG1AFWH9aD24keBS7YjPBS8hLxPeYV+Xz6vUL8fRZje/Z JgiBMIwyj6g2lH/zkdnxBdC0iG1xxJOLTaghMMeQyCdH6ef8+VMyAlAJsMckbOTvx63tY8z7 DFcrnTJfbe1EziRilVsEaK8tTzJzhcTfos+f3eBYWEilxe5HzIhYKJeC7lmsSUcGwa6+9VRg a0ctmi9Z8OgX Message-ID: <57342a02-ddf3-0a41-d394-74b8f8a27a83@freebsd.org> Date: Thu, 17 May 2018 15:24:15 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20180517114235.GI6887@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: 7bit X-ID: TlW4t4ZHwh9jgBvSXZ+7AgAbD4Fwy388CZxn0EQGsrl00d5X7YNIu8u-VQMf2pagOO X-TOI-MSGID: 8fe9d50e-b0a9-427d-8dae-c71e614d964b X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 13:24:27 -0000 Am 17.05.18 um 13:42 schrieb Konstantin Belousov: > On Thu, May 17, 2018 at 10:04:01AM +0200, Stefan Esser wrote: >> In case that a media (whether removable or not) was mounted R/W and not >> written to (had no dirty buffers) for more than a few seconds, the mount >> could be downgraded to R/O (in the same way as by a "mount -u -o ro"). A >> flag that recorded the fact, that this partition may be written to could >> then be checked in the "write to R/O partition" error case, and if the >> file system was only temporarily set to R/O, it could be treated like a >> first access to a writable partition (i.e., write a dirty flag into the >> super-block or whatever action the file system performs when mounted R/W). >> >> In short, the suggestion is to down-grade the mount state of any file-system >> not used for some configurable time to R/O, with an automatic upgrade to R/W >> on the next write attempt. > Downgrade to ro revokes all file descriptors opened for write, otherwise > it is not a downgrade. Really? I was not aware of that effect, but I think it can be dealt with in a similar way as an unmount request for a file system with open files. In the case of SD cards or other media used for file transfer, it is unlikely that a file system is in active use for an extended time (except for the duration of the file transfer). This is obviously not true for embedded ARM platforms that use SD cards for normal file systems. But the revocation of file file descriptors open for writing does not appear necessary to me. A clean state should be reached (same as a vetoed unmount request typically creates) and then a "clean" superblock could be written, which only corresponds to the state of the file system, not that individual files had their stdio buffers flushed and the file completely written. The flushing of stdio buffers would cause a remount of the partition as R/W. And this will obviously only ever occur, if the file descriptors have not been revoked. > This would be a huge mess. Yes, definitely. But it has been more than a decade since I looked into this topic and AFAIR, it was sufficient to just wait for no dirty buffers, then set the partition to R/O (and at the same time write a clean super-block) and have a flag that allows restarting writes after marking the super-block dirty again. It is not necessary to actually perform a R/O remount. The only guarantee required is, that the super-block is not marked clean if there are any outstanding writes (to make sure, that data blocks and meta-data are consistent) when a clean super-block is written, and that the next write attempt is intercepted and causes the super-block to be dirtied before the write is allowed to change any on disk data. Regards, STefan From owner-freebsd-fs@freebsd.org Thu May 17 15:45:47 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F8D0EADF71 for ; Thu, 17 May 2018 15:45:47 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 06F50862B9 for ; Thu, 17 May 2018 15:45:47 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id BC606EADF68; Thu, 17 May 2018 15:45:46 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AA2A5EADF67 for ; Thu, 17 May 2018 15:45:46 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-qt0-x22e.google.com (mail-qt0-x22e.google.com [IPv6:2607:f8b0:400d:c0d::22e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4467A862B7 for ; Thu, 17 May 2018 15:45:46 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-qt0-x22e.google.com with SMTP id c2-v6so6354996qtn.9 for ; Thu, 17 May 2018 08:45:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=mSI5ZhfgGvHYQmEgpBsyFeB1vDrZCep4h26A6HNjmRs=; b=l0AMO6/YVeXIEZjApJFK5CvMCiIanScjyLwuHllf6z3IZG/ZMf8QrXLvs61Lmmh8ss 2QycBDPQuw3ADM8UZdQ8cxIR9aE+PmATEm/lAd4CtAhoNU/gRv8e3y7aSvvxIhcUk8JC vsfg953UU0n7GGrb9B1mzE5HOq+s0B89LCHXTl3qmmXBNH+nqFIsfH9GQVX6wOHif0q5 U5Q/jzfpJwBYuB3OQ+VDwY/+y1rQ3kFAB75fgk2Z9SkgIJwUHMx14+eVp3p9KhjKkhu4 tb2OMguiCGKjhqDoJTcHBEuknrNwNZJRFirS0FvW9Qr70rCRm1I+CXxiekfJ+ImMfOb6 LdZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=mSI5ZhfgGvHYQmEgpBsyFeB1vDrZCep4h26A6HNjmRs=; b=DpqxU2S138nt2qk2h0e+zgD+BPaTkUjdI1VWiwntN7bP/W2EfpO8lzEkBhNWvAeBj7 gkoF/oMjuPtXL3ocz7o//+MlbhLuRduwdpWzssB9D/jc5VrRGdV6/dSgB4TskIz+lPCo bIR6dSOFXh5ua5fBd20FbxVCTmtdlyorLsBHC78sGJxL5sSDhkL4O5BQ047lyBLX8mCc bqVFE9TO75xfo9KcF9QM2RUN45wl9UBKY1+OVi4rHUudlxoL6CMJll+OB7gnIN6GEU6G Rm4EWh5S3QtLx9E/UsdQComwVTC0t9gsrrMNpzx/YO9fjWhdGxVtr30hjUNpBOcaYC4b 3joA== X-Gm-Message-State: ALKqPwcZfH5602oLua53T8SBnx7J6pOYlU3318ufZPAi2vquP/UcjowF 1HkrkI/XvUQR8os95dndYLR8Ud+x2VTdBn4kjm0= X-Google-Smtp-Source: AB8JxZps9qt4CvG8XDGp0NvM+pG+gPJMlM14cP2I4Frel4Iw9I1YBijgtCD3sUpRpzuzPuF/JrFJrNOpYrhnTrM9Fnw= X-Received: by 2002:aed:3595:: with SMTP id c21-v6mr5871253qte.203.1526571945578; Thu, 17 May 2018 08:45:45 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:ac8:6707:0:0:0:0:0 with HTTP; Thu, 17 May 2018 08:45:45 -0700 (PDT) In-Reply-To: <20180513184120.GY6887@kib.kiev.ua> References: <20180513184120.GY6887@kib.kiev.ua> From: Ryan Stone Date: Thu, 17 May 2018 11:45:45 -0400 Message-ID: Subject: Re: "NFS append race" To: Konstantin Belousov Cc: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 15:45:47 -0000 Do you have a patch you can try, or some pointers as to where to fix it and how? I'm not very familiar with NFS or VFS but I could give it a shot. From owner-freebsd-fs@freebsd.org Thu May 17 15:46:19 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B068CEAE00F for ; Thu, 17 May 2018 15:46:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4392F86320 for ; Thu, 17 May 2018 15:46:19 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 07A98EAE00D; Thu, 17 May 2018 15:46:19 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E980EEAE00B for ; Thu, 17 May 2018 15:46:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 85CA28631B for ; Thu, 17 May 2018 15:46:18 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id C4B1E13130 for ; Thu, 17 May 2018 15:46:17 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4HFkHPt018389 for ; Thu, 17 May 2018 15:46:17 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4HFkHrG018388 for fs@FreeBSD.org; Thu, 17 May 2018 15:46:17 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Thu, 17 May 2018 15:46:18 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 15:46:19 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #20 from Kirk McKusick --- (In reply to Damjan Jovanovic from comment #19) You are correct in what the code needs to do. The general areas that are li= kely to be problematic are when a write fails and when a file is deleted. For wr= ite failures the UFS strategy is to deallocate and invalidate the block. When a file is deleted, any of its in-memory data needs to be freed so that it does not "reappear" if the file once again is written. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 16:16:12 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2D868EB0080 for ; Thu, 17 May 2018 16:16:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id BC9D268A5A for ; Thu, 17 May 2018 16:16:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 8006FEB007F; Thu, 17 May 2018 16:16:11 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6BE7DEB007E for ; Thu, 17 May 2018 16:16:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 044A168A56 for ; Thu, 17 May 2018 16:16:11 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 52CFE13561 for ; Thu, 17 May 2018 16:16:10 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4HGGALm003346 for ; Thu, 17 May 2018 16:16:10 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4HGGAKd003344 for fs@FreeBSD.org; Thu, 17 May 2018 16:16:10 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 191858] [msdosfs] [panic] Disposed page 0xfffff807d41eeb60 is dirty Date: Thu, 17 May 2018 16:16:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 16:16:12 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D191858 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- Status|New |Closed Resolution|--- |FIXED CC| |mckusick@FreeBSD.org --- Comment #4 from Kirk McKusick --- This problem was resolved with revision: 333693. https://svnweb.freebsd.org/changeset/base/333693 --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 16:29:14 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 52910EB0AE7 for ; Thu, 17 May 2018 16:29:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id D877E69360 for ; Thu, 17 May 2018 16:29:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 92341EB0AE6; Thu, 17 May 2018 16:29:13 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7FBE7EB0AE5 for ; Thu, 17 May 2018 16:29:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1D61C6935F for ; Thu, 17 May 2018 16:29:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 60A94136E0 for ; Thu, 17 May 2018 16:29:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4HGTCAe028114 for ; Thu, 17 May 2018 16:29:12 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4HGTCZE028113 for fs@FreeBSD.org; Thu, 17 May 2018 16:29:12 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media Date: Thu, 17 May 2018 16:29:10 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: imp@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 16:29:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210316 --- Comment #16 from Warner Losh --- (In reply to Andriy Gapon from comment #14) bp =3D getblk(...) bp->b_flags |=3D B_INVAL | B_RELBUF | B_NOCACHE; bp->b_flags &=3D ~(B_ASYNC | B_CACHE); brelse(bp); maybe we need a bfinval(bp) that does that. "Forced" is the 'f' but I'm ope= n to any other name. Basically "Throw away this buffer I know is dirty." We have= a 'flush' operation but no 'invalidate' or 'throw away' operation for these c= ache entries. If we centralize it, we'll have fewer gross hacks like this in the system that "should work" but gradually drift away from working because of = some subtle change in what B_NOCACHE means 5 years from now... --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 16:36:35 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 51538EB142E for ; Thu, 17 May 2018 16:36:35 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id DEB4F69A26 for ; Thu, 17 May 2018 16:36:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 9FF00EB142A; Thu, 17 May 2018 16:36:34 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8BFADEB1429 for ; Thu, 17 May 2018 16:36:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 214C969A1F for ; Thu, 17 May 2018 16:36:34 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 4C49B13838 for ; Thu, 17 May 2018 16:36:33 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4HGaXJt050331 for ; Thu, 17 May 2018 16:36:33 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4HGaXq4050330 for fs@FreeBSD.org; Thu, 17 May 2018 16:36:33 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 210316] panic after trying to r/w mount msdosfs on write protected media Date: Thu, 17 May 2018 16:36:33 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 16:36:35 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D210316 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |mckusick@FreeBSD.org --- Comment #17 from Kirk McKusick --- (In reply to Warner Losh from comment #16) I very much agree with the idea of adding bfinval. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 17:15:46 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 08181ED97FD for ; Thu, 17 May 2018 17:15:46 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 94F326B2B7 for ; Thu, 17 May 2018 17:15:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 58E31ED97E6; Thu, 17 May 2018 17:15:45 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 46378ED97E5 for ; Thu, 17 May 2018 17:15:45 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id CDD966B2B3 for ; Thu, 17 May 2018 17:15:44 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id F1F1713DF3 for ; Thu, 17 May 2018 17:15:43 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4HHFhxa056535 for ; Thu, 17 May 2018 17:15:43 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4HHFhdM056534 for fs@FreeBSD.org; Thu, 17 May 2018 17:15:43 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Thu, 17 May 2018 17:15:43 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: damjan.jov@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 17:15:46 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 --- Comment #21 from Damjan Jovanovic --- (In reply to Kirk McKusick from comment #20) Thank you for the explanation and your patch :). I can no longer reproduce the directory corruption in my test. Let me look through the code and try find the incomplete brelse(). --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 17:20:14 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 988F5ED9C0F for ; Thu, 17 May 2018 17:20:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 314376B46B for ; Thu, 17 May 2018 17:20:14 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id E2F2DED9C09; Thu, 17 May 2018 17:20:13 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D1509ED9C08 for ; Thu, 17 May 2018 17:20:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 6FE7A6B468 for ; Thu, 17 May 2018 17:20:13 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 50F8713E06 for ; Thu, 17 May 2018 17:20:12 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4HHKCrw062401 for ; Thu, 17 May 2018 17:20:12 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4HHKCwi062400 for fs@FreeBSD.org; Thu, 17 May 2018 17:20:12 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 213507] [msdosfs]: Prevent occasional directory corruption while extending it to another cluster Date: Thu, 17 May 2018 17:20:11 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: mckusick@FreeBSD.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 17:20:14 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D213507 Kirk McKusick changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Progress |Closed Resolution|--- |FIXED --- Comment #22 from Kirk McKusick --- This bug has been fixed. A better fix may be worked out in which case this report can be reopened for further discussion. But for now I am closing it. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Thu May 17 17:39:20 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAF20EDAB0B for ; Thu, 17 May 2018 17:39:20 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 455C56C036 for ; Thu, 17 May 2018 17:39:20 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id F3287EDAB07; Thu, 17 May 2018 17:39:19 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E085CEDAB03 for ; Thu, 17 May 2018 17:39:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3FDB46C033 for ; Thu, 17 May 2018 17:39:19 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id w4HHd4kN028145 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Thu, 17 May 2018 20:39:07 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua w4HHd4kN028145 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id w4HHd42p028144; Thu, 17 May 2018 20:39:04 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 17 May 2018 20:39:04 +0300 From: Konstantin Belousov To: Ryan Stone Cc: "freebsd-fs@freebsd.org" Subject: Re: "NFS append race" Message-ID: <20180517173904.GR6887@kib.kiev.ua> References: <20180513184120.GY6887@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.9.5 (2018-04-13) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 May 2018 17:39:20 -0000 On Thu, May 17, 2018 at 11:45:45AM -0400, Ryan Stone wrote: > Do you have a patch you can try, or some pointers as to where to fix > it and how? I'm not very familiar with NFS or VFS but I could give it > a shot. No, I do not have patches. You might take a look at the calls to ncl_asyncio() and the ncl_asyncio() code itself. It does BUF_KERNPROC() to get rid of the lock ownership for the buffer lock, essentially the lock is not a lock but semaphore. Then the io occurs in the context of the nfsiod thread. Vnode lock is not transferred to the nfsiod thread, and since it typically needs to access the nfs node, things are somewhat messy. Note that comparing with the local filesystems, after the buffer logical block number b_lblkno is translated to the device physical block number b_blkno, buffer completion typically does not require interaction with the vnode, so BUF_KERNPROC() is more sane there. From owner-freebsd-fs@freebsd.org Fri May 18 14:41:04 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C2BBEAA3E3 for ; Fri, 18 May 2018 14:41:04 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id F33B478D6E for ; Fri, 18 May 2018 14:41:03 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id B3AC8EAA3E2; Fri, 18 May 2018 14:41:03 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A118AEAA3E1 for ; Fri, 18 May 2018 14:41:03 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: from mail-qt0-x22a.google.com (mail-qt0-x22a.google.com [IPv6:2607:f8b0:400d:c0d::22a]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 31D7978D6B for ; Fri, 18 May 2018 14:41:03 +0000 (UTC) (envelope-from rysto32@gmail.com) Received: by mail-qt0-x22a.google.com with SMTP id m9-v6so10539684qtb.5 for ; Fri, 18 May 2018 07:41:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:from:date:message-id:subject:to; bh=Z8Tt9+Yapf6c4L8XstiwvDlN3Ixeje1wwD8qmYeMnFI=; b=V/LSbHM8ahQW56Gj7LS0kYYB0lX5SSLkNS7Y2mkQ7chYmOoX7eW5Up3HILmlcKb+/n x2FlnRLkYdh4+dEA8WBZXf76HGLG4NSSsNw7IBAF7lmdhSViYXH+sjuHPNRyKRzMBko8 8A2D+rBy7SNVNg4Y3v43MggjwHwn5NnJTTBdH0Re2qOMrGRLQQDPDq48fuf+fHA6Cyvp tUO4wBVMl17If1cLoj58/FuMeUIgqMDWdHy/CiwavxV0xzwoHGD4XaRd5kg11RYM2xtx JeaOrE6Zt+OWqNx6fChdksgjnby7xXEWGK5mFjDBvVLAsFhdAiyZkjdYAs2ezMNOIY+d yd2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:from:date:message-id:subject:to; bh=Z8Tt9+Yapf6c4L8XstiwvDlN3Ixeje1wwD8qmYeMnFI=; b=Ic5L4ucAm7+Um/caWmM1JQsJjDIjCW5YLPRC63UGzxvSq0IE0Tj8WwIsbVFFNeSwOj cAZfrPYFIX89CA0De63lAGay8Enarmpemrcdvh+n7PMhCNmi7xaJqqcm0PsZ/ypiNOD1 Fzkfa7b1iTPWAfkB2UsQ//0dyTcLOfYZ8l2PoWG760GAyyYHuobDMvU737lRWTQWICr2 5DAWeNiETWYsHdUr8/36Skrnlz+J68S+xVeIdUexWx9j2vFiVUAjGf7GiscNdBgyFmwe 5olYrC1En3L+uclhdxLSMT2vtbcggd6s7O95J7pT/AA/PL3Lk5wDm3laFi1nkCU+JYGK 4UOg== X-Gm-Message-State: ALKqPweeVoL8XsdDKeNHuLa2kl+75JAO8BVigrkRlEuEleGjmvenkXXn MMmXRTk5VPnlDur85lAu4mLC4VO2hixuehDI7ROkaA== X-Google-Smtp-Source: AB8JxZowFxqtH7SNNidVHzMTH4v6Tpf9gmZababQjbgE2Akb6wDNHHcWwGA80xZH0/JtCl1RAW06FNKboUqNvhvUXyM= X-Received: by 2002:ac8:3487:: with SMTP id w7-v6mr9486090qtb.220.1526654462217; Fri, 18 May 2018 07:41:02 -0700 (PDT) MIME-Version: 1.0 Received: by 2002:ac8:6707:0:0:0:0:0 with HTTP; Fri, 18 May 2018 07:41:01 -0700 (PDT) From: Ryan Stone Date: Fri, 18 May 2018 10:41:01 -0400 Message-ID: Subject: "nfsd server cache flooded" To: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2018 14:41:04 -0000 Earlier in the week I tried running a buildworld on a 11.1-RELEASE system that had /usr/obj mounted via nfsv4 from a 12-CURRENT server. After some time, the build hung I saw this message start repeating in /var/log/messages: May 15 08:23:03 rstone-server kernel: nfsd server cache flooded, try increasing vfs.nfsd.tcphighwater In 12-CURRENT, sysctl vfs.nfsd.tcphighwater defaults to 0, so I have no idea what a reasonable value to set would be. Any advice? From owner-freebsd-fs@freebsd.org Fri May 18 15:42:09 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34E3CEABFF4 for ; Fri, 18 May 2018 15:42:09 +0000 (UTC) (envelope-from paul.esson@redstor.com) Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0619.outbound.protection.outlook.com [IPv6:2a01:111:f400:fe1e::619]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 45FDB7B73A for ; Fri, 18 May 2018 15:42:08 +0000 (UTC) (envelope-from paul.esson@redstor.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redstorltd.onmicrosoft.com; s=selector1-redstor-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eVQlzuwzBVGBOaBOfSeCS2noGaOCe2jD1IgYZglPtGc=; b=N4kx1/CjbpuDEtgqailSqGu+oZ+lDvLszojsoaFxhKNbj7Q7SIrClX1B2odtp7qkb8zoJTamxjabwUtPG/JiGtX5mk0rcZ3Q5FR/WWR2XgBVQPf8cZ+rwRha55OtLoCvusOJSxRGGtBSLdSVcfR9x1Pmr3z5x0Ttha7ydAElRi4= Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com (10.170.251.141) by HE1PR0102MB3257.eurprd01.prod.exchangelabs.com (10.167.124.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.776.11; Fri, 18 May 2018 15:42:05 +0000 Received: from HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::d508:b6bc:e82:73e]) by HE1PR0102MB2588.eurprd01.prod.exchangelabs.com ([fe80::d508:b6bc:e82:73e%13]) with mapi id 15.20.0755.019; Fri, 18 May 2018 15:42:05 +0000 From: Paul Esson To: "freebsd-fs@freebsd.org" Subject: Unexpected zvol usage Thread-Topic: Unexpected zvol usage Thread-Index: AdPuu8U1d0Yref1cQ2GeqjmmDcH+hQ== Date: Fri, 18 May 2018 15:42:05 +0000 Message-ID: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=paul.esson@redstor.com; x-originating-ip: [81.136.184.115] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR0102MB3257; 7:qSuLhukJl/6vNrS+xH+rI6wU+wjAb4K+UBomSsYyBWC+kHIqLovQrIIsJrWm3Gk8a/UJ8hXslHwog/D5fHy7tPgE5VNcbfA2cIzdr9daG5rZdAVrH/PKUgGLvMKdNkE3B4p7JXJUSmtbAb1qnto4EpLjz1NRDpW80lGC9oAUsoncKYBI2Te+7GX4q2VmbFpRRHFG+g3EGad6HeU7rc/L5v099taFzyBZ67oBBgCRfguAPo2frL3G4ZTK4uQpOLCp x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:HE1PR0102MB3257; x-ms-traffictypediagnostic: HE1PR0102MB3257: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(28532068793085)(155532106045638)(21748063052155); x-ms-exchange-senderadcheck: 1 x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(3002001)(10201501046)(3231254)(944501410)(52105095)(93006095)(93001095)(149027)(150027)(6041310)(20161123560045)(20161123564045)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(6072148)(201708071742011)(7699016); SRVR:HE1PR0102MB3257; BCL:0; PCL:0; RULEID:; SRVR:HE1PR0102MB3257; x-forefront-prvs: 0676F530A9 x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39850400004)(346002)(39380400002)(396003)(376002)(366004)(189003)(199004)(42174003)(5630700001)(316002)(6506007)(53936002)(102836004)(33656002)(53946003)(97736004)(16200700003)(2351001)(106356001)(186003)(9686003)(55016002)(26005)(59450400001)(105586002)(99286004)(6306002)(54896002)(25786009)(68736007)(236005)(9326002)(7696005)(3480700004)(81156014)(81166006)(2906002)(3280700002)(5250100002)(14454004)(8676002)(6916009)(3660700001)(790700001)(8936002)(7116003)(3846002)(86362001)(6116002)(486006)(6436002)(74316002)(66066001)(2501003)(476003)(5660300001)(2900100001)(44832011)(7736002)(478600001)(5640700003)(579004)(569006); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR0102MB3257; H:HE1PR0102MB2588.eurprd01.prod.exchangelabs.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: redstor.com does not designate permitted sender hosts) x-microsoft-antispam-message-info: EgnPDKRQeLeJPt4+IcNlhtg0JiB7WvGkPUkRVJJlpMBl7k70j5y3OjRLkEnRWeHcNDi6YHKx6TknmVVAo72JFkVzB5N5AOEN+WyCe3H6ZTJYGD3j6YHDZSqNpNmhA+5O/Vp+jIYtjQn08PfudrabGRuLDpQYxOKrusCygc5ZhnkKghvEK5UHoXnlPRbkQaJx spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: c0aedfb4-ccfd-454a-8290-08d5bcd5e88a X-OriginatorOrg: redstor.com X-MS-Exchange-CrossTenant-Network-Message-Id: c0aedfb4-ccfd-454a-8290-08d5bcd5e88a X-MS-Exchange-CrossTenant-originalarrivaltime: 18 May 2018 15:42:05.5584 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 24ac53ae-15a7-4211-afef-61d8f34e2571 X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR0102MB3257 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2018 15:42:09 -0000 Hi Folks, I have an 11.1-RELEASE system being used as a host for a bhyve guest. Ther= e is a large zpool on the host created from 12 x 10TB HDDs using raidz2 red= undancy with ashift12. I have created a sparse zvol within the pool using = default settings and presented that to the bhyve vm as an ahci-hd disk type= . The guest has a zpool and filesystem dataset built on this disk. When I= start to write to the filesystem on the guest I am finding that the used/r= eferenced on the host's zvol are more than double those on the guest. The = logicalused/referenced values on the host zvol are more in line with the eq= uivalent guest values, but my problem is that the host zvol is likely to fi= ll before I have written all intended data to the guest. I have included below information from both the host and guest before and a= fter writing. This output shows that the zvol uses a default 8K blocksize = and that the guest zfs is therefore ashift13. I also tried creating the zv= ol with a 4K blocksize and the guest zfs ashift12 so that 4K blocks were co= nsistent across hosts and guest, but still saw the amplification on writes = to the zvol. Any insight greatly appreciated. HOST Zpool RAIDZ2 12 x HDDs, ashift 12 NAME PROPERTY VALUE SOURCE dc1-hn-01 type filesystem - dc1-hn-01 creation Mon Apr 23 14:35 2018 - dc1-hn-01 used 32.0G - dc1-hn-01 available 78.2T - dc1-hn-01 referenced 201K - dc1-hn-01 compressratio 1.00x - dc1-hn-01 mounted yes - dc1-hn-01 quota none default dc1-hn-01 reservation none default dc1-hn-01 recordsize 128K default dc1-hn-01 mountpoint /export/data/dc1-hn-01 local dc1-hn-01 sharenfs off default dc1-hn-01 checksum on default dc1-hn-01 compression off default dc1-hn-01 atime on default dc1-hn-01 devices on default dc1-hn-01 exec on default dc1-hn-01 setuid on default dc1-hn-01 readonly off default dc1-hn-01 jailed off default dc1-hn-01 snapdir hidden default dc1-hn-01 aclmode discard default dc1-hn-01 aclinherit restricted default dc1-hn-01 canmount on default dc1-hn-01 xattr off temporary dc1-hn-01 copies 1 default dc1-hn-01 version 5 - dc1-hn-01 utf8only off - dc1-hn-01 normalization none - dc1-hn-01 casesensitivity sensitive - dc1-hn-01 vscan off default dc1-hn-01 nbmand off default dc1-hn-01 sharesmb off default dc1-hn-01 refquota none default dc1-hn-01 refreservation none default dc1-hn-01 primarycache all default dc1-hn-01 secondarycache all default dc1-hn-01 usedbysnapshots 0 - dc1-hn-01 usedbydataset 201K - dc1-hn-01 usedbychildren 32.0G - dc1-hn-01 usedbyrefreservation 0 - dc1-hn-01 logbias latency default dc1-hn-01 dedup off default dc1-hn-01 mlslabel - dc1-hn-01 sync standard default dc1-hn-01 refcompressratio 1.00x - dc1-hn-01 written 201K - dc1-hn-01 logicalused 2.89G - dc1-hn-01 logicalreferenced 36.5K - dc1-hn-01 volmode default default dc1-hn-01 filesystem_limit none default dc1-hn-01 snapshot_limit none default dc1-hn-01 filesystem_count none default dc1-hn-01 snapshot_count none default dc1-hn-01 redundant_metadata all default NAME AVAIL USED USEDSNAP USEDDS USEDREFR= ESERV USEDCHILD dc1-hn-01 78.2T 32.0G 0 201K = 0 32.0G dc1-hn-01/vm 78.2T 31.9G 0 990M = 0 30.9G dc1-hn-01/vm/dc1-olbp-sn-11 78.2T 30.9G 0 238K = 0 30.9G dc1-hn-01/vm/dc1-olbp-sn-11/disk0 78.2T 30.9G 0 4.35G = 26.6G 0 dc1-hn-01/vm/dc1-olbp-sn-11/disk1 78.2T 4.50M 0 4.50M = 0 0 Sparse ZVOL - baseline NAME PROPERTY VALUE = SOURCE dc1-hn-01/vm/dc1-olbp-sn-11/disk1 type volume = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 creation Fri May 18 15:36 2= 018 - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 used 4.50M = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 available 78.2T = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 referenced 4.50M = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compressratio 1.00x = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 reservation none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volsize 28T = local dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volblocksize 8K = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 checksum on = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compression off = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 readonly off = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 copies 1 = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refreservation none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 primarycache all = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 secondarycache all = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbysnapshots 0 = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbydataset 4.50M = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbychildren 0 = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbyrefreservation 0 = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logbias latency = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 dedup off = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 mlslabel = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 sync standard = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refcompressratio 1.00x = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 written 4.50M = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalused 1.89M = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalreferenced 1.89M = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volmode dev = local dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_limit none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_count none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 redundant_metadata all = default GUEST - baseline 1 x vdisk from host ZVOL ashift 13 NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHIL= D dc1-sn-11 26.9T 632K 0 176K 0 456= K NAME PROPERTY VALUE SOURCE dc1-sn-11 type filesystem - dc1-sn-11 creation Fri May 18 15:40 2018 - dc1-sn-11 used 632K - dc1-sn-11 available 26.9T - dc1-sn-11 referenced 176K - dc1-sn-11 compressratio 1.00x - dc1-sn-11 mounted yes - dc1-sn-11 quota none default dc1-sn-11 reservation none default dc1-sn-11 recordsize 128K default dc1-sn-11 mountpoint /export/data/dc1-sn-11 local dc1-sn-11 sharenfs off default dc1-sn-11 checksum on default dc1-sn-11 compression off default dc1-sn-11 atime on default dc1-sn-11 devices on default dc1-sn-11 exec on default dc1-sn-11 setuid on default dc1-sn-11 readonly off default dc1-sn-11 jailed off default dc1-sn-11 snapdir hidden default dc1-sn-11 aclmode discard default dc1-sn-11 aclinherit restricted default dc1-sn-11 canmount on default dc1-sn-11 xattr off temporary dc1-sn-11 copies 1 default dc1-sn-11 version 5 - dc1-sn-11 utf8only off - dc1-sn-11 normalization none - dc1-sn-11 casesensitivity sensitive - dc1-sn-11 vscan off default dc1-sn-11 nbmand off default dc1-sn-11 sharesmb off default dc1-sn-11 refquota none default dc1-sn-11 refreservation none default dc1-sn-11 primarycache all default dc1-sn-11 secondarycache all default dc1-sn-11 usedbysnapshots 0 - dc1-sn-11 usedbydataset 176K - dc1-sn-11 usedbychildren 456K - dc1-sn-11 usedbyrefreservation 0 - dc1-sn-11 logbias latency default dc1-sn-11 dedup off default dc1-sn-11 mlslabel - dc1-sn-11 sync standard default dc1-sn-11 refcompressratio 1.00x - dc1-sn-11 written 176K - dc1-sn-11 logicalused 49K - dc1-sn-11 logicalreferenced 11.5K - dc1-sn-11 volmode default default dc1-sn-11 filesystem_limit none default dc1-sn-11 snapshot_limit none default dc1-sn-11 filesystem_count none default dc1-sn-11 snapshot_count none default dc1-sn-11 redundant_metadata all default After writing some data to the guest HOST ZVOL NAME PROPERTY VALUE = SOURCE dc1-hn-01/vm/dc1-olbp-sn-11/disk1 type volume = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 creation Fri May 18 15:36 2= 018 - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 used 99.7G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 available 78.1T = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 referenced 99.7G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compressratio 1.00x = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 reservation none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volsize 28T = local dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volblocksize 8K = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 checksum on = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compression off = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 readonly off = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 copies 1 = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refreservation none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 primarycache all = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 secondarycache all = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbysnapshots 0 = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbydataset 99.7G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbychildren 0 = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbyrefreservation 0 = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logbias latency = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 dedup off = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 mlslabel = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 sync standard = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refcompressratio 1.00x = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 written 99.7G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalused 43.6G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalreferenced 43.6G = - dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volmode dev = local dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_limit none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_count none = default dc1-hn-01/vm/dc1-olbp-sn-11/disk1 redundant_metadata all = default GUEST ZFS NAME PROPERTY VALUE SOURCE dc1-sn-11 type filesystem - dc1-sn-11 creation Fri May 18 15:40 2018 - dc1-sn-11 used 44.3G - dc1-sn-11 available 26.8T - dc1-sn-11 referenced 176K - dc1-sn-11 compressratio 1.00x - dc1-sn-11 mounted no - dc1-sn-11 quota none default dc1-sn-11 reservation none default dc1-sn-11 recordsize 128K default dc1-sn-11 mountpoint /export/data/dc1-sn-11 local dc1-sn-11 sharenfs off default dc1-sn-11 checksum on default dc1-sn-11 compression off default dc1-sn-11 atime on default dc1-sn-11 devices on default dc1-sn-11 exec on default dc1-sn-11 setuid on default dc1-sn-11 readonly off default dc1-sn-11 jailed off default dc1-sn-11 snapdir hidden default dc1-sn-11 aclmode discard default dc1-sn-11 aclinherit restricted default dc1-sn-11 canmount on default dc1-sn-11 xattr on default dc1-sn-11 copies 1 default dc1-sn-11 version 5 - dc1-sn-11 utf8only off - dc1-sn-11 normalization none - dc1-sn-11 casesensitivity sensitive - dc1-sn-11 vscan off default dc1-sn-11 nbmand off default dc1-sn-11 sharesmb off default dc1-sn-11 refquota none default dc1-sn-11 refreservation none default dc1-sn-11 primarycache all default dc1-sn-11 secondarycache all default dc1-sn-11 usedbysnapshots 0 - dc1-sn-11 usedbydataset 176K - dc1-sn-11 usedbychildren 44.3G - dc1-sn-11 usedbyrefreservation 0 - dc1-sn-11 logbias latency default dc1-sn-11 dedup off default dc1-sn-11 mlslabel - dc1-sn-11 sync standard default dc1-sn-11 refcompressratio 1.00x - dc1-sn-11 written 176K - dc1-sn-11 logicalused 44.2G - dc1-sn-11 logicalreferenced 11.5K - dc1-sn-11 volmode default default dc1-sn-11 filesystem_limit none default dc1-sn-11 snapshot_limit none default dc1-sn-11 filesystem_count none default dc1-sn-11 snapshot_count none default dc1-sn-11 redundant_metadata all default Regards, Paul Esson t +44 (0)118 951 5235 | m +44 (0)776 690 6514 e paul.esson@redstor.com From owner-freebsd-fs@freebsd.org Fri May 18 16:00:46 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F390CEACCBA for ; Fri, 18 May 2018 16:00:45 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from mail-io0-x235.google.com (mail-io0-x235.google.com [IPv6:2607:f8b0:4001:c06::235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B1577C6C5 for ; Fri, 18 May 2018 16:00:45 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: by mail-io0-x235.google.com with SMTP id e12-v6so6758277iob.8 for ; Fri, 18 May 2018 09:00:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dragondata.com; s=google; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=fEv525FxNUAP/OJFXFE3seA5+gYwjnXxMEv9e9572w8=; b=U7mYlC9M9YZFcSK7sXCVLlAckH0swxmoY2O+AntFUY0oLv1QN6AA0QmAse6fHh1D1B Syj5VR/S6deZBhj+dRvCAY2CDTqHQTbkbf1saKxd5H1o5yOhYhtMakyAYKYQsDZ9xknu OTHD2fU30Bsn1paQcdG3b2h8Lu27SMTCejgYU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=fEv525FxNUAP/OJFXFE3seA5+gYwjnXxMEv9e9572w8=; b=ROad3pNpijchK5uNKkaRsrriEjHz6UGjk6+6YIAT+rCuinoe3T3EMT6hWCCUW1Oikx i0U3DPbDkaTEI2VluU0YpPC7R5UmJIKdlvl8Q32hPxaQqO5mObBkOt7WfXJjpDySo3LD 7oOewu8zYiiqEzMUKdod7tYns5XCVw/16K9xKlNKv9Eu30Wc82sZaTW2c5hNuDEjzG0g FVTL7/2Kn9QR7/lH9E6sU3wj59nS1nkhm3Iu1lGe51KI1ldeiH5iFPcRnd/Eketl52+I 22lOpKvAAVcLcPBQ5DwC0Zw/qNuV9XxdawL+IoB9jpXAL3/d9O9BktRwmn7ZTKLaXuDQ Uvdw== X-Gm-Message-State: ALKqPwfS18PJRWYvRYGW6HXv+2oSG2EZsIRkfEwT+rIUqG5z6JO5A2gO M3ZQhIRHsip8ld7iUzcqcCYUWj92EVs= X-Google-Smtp-Source: AB8JxZqYCE2dQajwMp9zhf134B2FyyZt+CdAZch44583a5+IFAdmdEoEVyeoaEMrJOqssHz1BdkCrw== X-Received: by 2002:a6b:3e8b:: with SMTP id l133-v6mr10792158ioa.65.1526659244437; Fri, 18 May 2018 09:00:44 -0700 (PDT) Received: from unassigned.v6.your.org ([2001:4978:1:45:c57b:919e:3353:907]) by smtp.gmail.com with ESMTPSA id d130-v6sm4976374iog.46.2018.05.18.09.00.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 18 May 2018 09:00:43 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Subject: Re: Unexpected zvol usage From: Kevin Day In-Reply-To: Date: Fri, 18 May 2018 11:00:41 -0500 Cc: "freebsd-fs@freebsd.org" Content-Transfer-Encoding: quoted-printable Message-Id: <2EC9E6E6-F49A-4DBA-B622-61D250C60B6C@dragondata.com> References: To: Paul Esson X-Mailer: Apple Mail (2.3445.6.18) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2018 16:00:46 -0000 Try: zfs create -b 128k -V > On May 18, 2018, at 10:42 AM, Paul Esson = wrote: >=20 > Hi Folks, >=20 > I have an 11.1-RELEASE system being used as a host for a bhyve guest. = There is a large zpool on the host created from 12 x 10TB HDDs using = raidz2 redundancy with ashift12. I have created a sparse zvol within = the pool using default settings and presented that to the bhyve vm as an = ahci-hd disk type. The guest has a zpool and filesystem dataset built = on this disk. When I start to write to the filesystem on the guest I am = finding that the used/referenced on the host's zvol are more than double = those on the guest. The logicalused/referenced values on the host zvol = are more in line with the equivalent guest values, but my problem is = that the host zvol is likely to fill before I have written all intended = data to the guest. >=20 >=20 > I have included below information from both the host and guest before = and after writing. This output shows that the zvol uses a default 8K = blocksize and that the guest zfs is therefore ashift13. I also tried = creating the zvol with a 4K blocksize and the guest zfs ashift12 so that = 4K blocks were consistent across hosts and guest, but still saw the = amplification on writes to the zvol. >=20 > Any insight greatly appreciated. >=20 >=20 >=20 > HOST >=20 > Zpool > RAIDZ2 12 x HDDs, ashift 12 >=20 > NAME PROPERTY VALUE SOURCE > dc1-hn-01 type filesystem - > dc1-hn-01 creation Mon Apr 23 14:35 2018 - > dc1-hn-01 used 32.0G - > dc1-hn-01 available 78.2T - > dc1-hn-01 referenced 201K - > dc1-hn-01 compressratio 1.00x - > dc1-hn-01 mounted yes - > dc1-hn-01 quota none default > dc1-hn-01 reservation none default > dc1-hn-01 recordsize 128K default > dc1-hn-01 mountpoint /export/data/dc1-hn-01 local > dc1-hn-01 sharenfs off default > dc1-hn-01 checksum on default > dc1-hn-01 compression off default > dc1-hn-01 atime on default > dc1-hn-01 devices on default > dc1-hn-01 exec on default > dc1-hn-01 setuid on default > dc1-hn-01 readonly off default > dc1-hn-01 jailed off default > dc1-hn-01 snapdir hidden default > dc1-hn-01 aclmode discard default > dc1-hn-01 aclinherit restricted default > dc1-hn-01 canmount on default > dc1-hn-01 xattr off temporary > dc1-hn-01 copies 1 default > dc1-hn-01 version 5 - > dc1-hn-01 utf8only off - > dc1-hn-01 normalization none - > dc1-hn-01 casesensitivity sensitive - > dc1-hn-01 vscan off default > dc1-hn-01 nbmand off default > dc1-hn-01 sharesmb off default > dc1-hn-01 refquota none default > dc1-hn-01 refreservation none default > dc1-hn-01 primarycache all default > dc1-hn-01 secondarycache all default > dc1-hn-01 usedbysnapshots 0 - > dc1-hn-01 usedbydataset 201K - > dc1-hn-01 usedbychildren 32.0G - > dc1-hn-01 usedbyrefreservation 0 - > dc1-hn-01 logbias latency default > dc1-hn-01 dedup off default > dc1-hn-01 mlslabel - > dc1-hn-01 sync standard default > dc1-hn-01 refcompressratio 1.00x - > dc1-hn-01 written 201K - > dc1-hn-01 logicalused 2.89G - > dc1-hn-01 logicalreferenced 36.5K - > dc1-hn-01 volmode default default > dc1-hn-01 filesystem_limit none default > dc1-hn-01 snapshot_limit none default > dc1-hn-01 filesystem_count none default > dc1-hn-01 snapshot_count none default > dc1-hn-01 redundant_metadata all default >=20 > NAME AVAIL USED USEDSNAP USEDDS = USEDREFRESERV USEDCHILD > dc1-hn-01 78.2T 32.0G 0 201K = 0 32.0G > dc1-hn-01/vm 78.2T 31.9G 0 990M = 0 30.9G > dc1-hn-01/vm/dc1-olbp-sn-11 78.2T 30.9G 0 238K = 0 30.9G > dc1-hn-01/vm/dc1-olbp-sn-11/disk0 78.2T 30.9G 0 4.35G = 26.6G 0 > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 78.2T 4.50M 0 4.50M = 0 0 >=20 > Sparse ZVOL - baseline >=20 > NAME PROPERTY VALUE = SOURCE > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 type volume = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 creation Fri May 18 = 15:36 2018 - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 used 4.50M = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 available 78.2T = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 referenced 4.50M = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compressratio 1.00x = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 reservation none = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volsize 28T = local > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volblocksize 8K = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 checksum on = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compression off = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 readonly off = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 copies 1 = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refreservation none = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 primarycache all = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 secondarycache all = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbysnapshots 0 = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbydataset 4.50M = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbychildren 0 = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbyrefreservation 0 = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logbias latency = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 dedup off = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 mlslabel = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 sync standard = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refcompressratio 1.00x = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 written 4.50M = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalused 1.89M = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalreferenced 1.89M = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volmode dev = local > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_limit none = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_count none = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 redundant_metadata all = default >=20 >=20 > GUEST - baseline >=20 > 1 x vdisk from host ZVOL ashift 13 >=20 > NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV = USEDCHILD > dc1-sn-11 26.9T 632K 0 176K 0 = 456K >=20 > NAME PROPERTY VALUE SOURCE > dc1-sn-11 type filesystem - > dc1-sn-11 creation Fri May 18 15:40 2018 - > dc1-sn-11 used 632K - > dc1-sn-11 available 26.9T - > dc1-sn-11 referenced 176K - > dc1-sn-11 compressratio 1.00x - > dc1-sn-11 mounted yes - > dc1-sn-11 quota none default > dc1-sn-11 reservation none default > dc1-sn-11 recordsize 128K default > dc1-sn-11 mountpoint /export/data/dc1-sn-11 local > dc1-sn-11 sharenfs off default > dc1-sn-11 checksum on default > dc1-sn-11 compression off default > dc1-sn-11 atime on default > dc1-sn-11 devices on default > dc1-sn-11 exec on default > dc1-sn-11 setuid on default > dc1-sn-11 readonly off default > dc1-sn-11 jailed off default > dc1-sn-11 snapdir hidden default > dc1-sn-11 aclmode discard default > dc1-sn-11 aclinherit restricted default > dc1-sn-11 canmount on default > dc1-sn-11 xattr off temporary > dc1-sn-11 copies 1 default > dc1-sn-11 version 5 - > dc1-sn-11 utf8only off - > dc1-sn-11 normalization none - > dc1-sn-11 casesensitivity sensitive - > dc1-sn-11 vscan off default > dc1-sn-11 nbmand off default > dc1-sn-11 sharesmb off default > dc1-sn-11 refquota none default > dc1-sn-11 refreservation none default > dc1-sn-11 primarycache all default > dc1-sn-11 secondarycache all default > dc1-sn-11 usedbysnapshots 0 - > dc1-sn-11 usedbydataset 176K - > dc1-sn-11 usedbychildren 456K - > dc1-sn-11 usedbyrefreservation 0 - > dc1-sn-11 logbias latency default > dc1-sn-11 dedup off default > dc1-sn-11 mlslabel - > dc1-sn-11 sync standard default > dc1-sn-11 refcompressratio 1.00x - > dc1-sn-11 written 176K - > dc1-sn-11 logicalused 49K - > dc1-sn-11 logicalreferenced 11.5K - > dc1-sn-11 volmode default default > dc1-sn-11 filesystem_limit none default > dc1-sn-11 snapshot_limit none default > dc1-sn-11 filesystem_count none default > dc1-sn-11 snapshot_count none default > dc1-sn-11 redundant_metadata all default >=20 > After writing some data to the guest >=20 > HOST ZVOL >=20 > NAME PROPERTY VALUE = SOURCE > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 type volume = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 creation Fri May 18 = 15:36 2018 - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 used 99.7G = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 available 78.1T = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 referenced 99.7G = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compressratio 1.00x = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 reservation none = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volsize 28T = local > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volblocksize 8K = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 checksum on = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compression off = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 readonly off = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 copies 1 = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refreservation none = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 primarycache all = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 secondarycache all = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbysnapshots 0 = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbydataset 99.7G = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbychildren 0 = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbyrefreservation 0 = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logbias latency = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 dedup off = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 mlslabel = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 sync standard = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refcompressratio 1.00x = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 written 99.7G = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalused 43.6G = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalreferenced 43.6G = - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volmode dev = local > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_limit none = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_count none = default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 redundant_metadata all = default >=20 > GUEST ZFS >=20 > NAME PROPERTY VALUE SOURCE > dc1-sn-11 type filesystem - > dc1-sn-11 creation Fri May 18 15:40 2018 - > dc1-sn-11 used 44.3G - > dc1-sn-11 available 26.8T - > dc1-sn-11 referenced 176K - > dc1-sn-11 compressratio 1.00x - > dc1-sn-11 mounted no - > dc1-sn-11 quota none default > dc1-sn-11 reservation none default > dc1-sn-11 recordsize 128K default > dc1-sn-11 mountpoint /export/data/dc1-sn-11 local > dc1-sn-11 sharenfs off default > dc1-sn-11 checksum on default > dc1-sn-11 compression off default > dc1-sn-11 atime on default > dc1-sn-11 devices on default > dc1-sn-11 exec on default > dc1-sn-11 setuid on default > dc1-sn-11 readonly off default > dc1-sn-11 jailed off default > dc1-sn-11 snapdir hidden default > dc1-sn-11 aclmode discard default > dc1-sn-11 aclinherit restricted default > dc1-sn-11 canmount on default > dc1-sn-11 xattr on default > dc1-sn-11 copies 1 default > dc1-sn-11 version 5 - > dc1-sn-11 utf8only off - > dc1-sn-11 normalization none - > dc1-sn-11 casesensitivity sensitive - > dc1-sn-11 vscan off default > dc1-sn-11 nbmand off default > dc1-sn-11 sharesmb off default > dc1-sn-11 refquota none default > dc1-sn-11 refreservation none default > dc1-sn-11 primarycache all default > dc1-sn-11 secondarycache all default > dc1-sn-11 usedbysnapshots 0 - > dc1-sn-11 usedbydataset 176K - > dc1-sn-11 usedbychildren 44.3G - > dc1-sn-11 usedbyrefreservation 0 - > dc1-sn-11 logbias latency default > dc1-sn-11 dedup off default > dc1-sn-11 mlslabel - > dc1-sn-11 sync standard default > dc1-sn-11 refcompressratio 1.00x - > dc1-sn-11 written 176K - > dc1-sn-11 logicalused 44.2G - > dc1-sn-11 logicalreferenced 11.5K - > dc1-sn-11 volmode default default > dc1-sn-11 filesystem_limit none default > dc1-sn-11 snapshot_limit none default > dc1-sn-11 filesystem_count none default > dc1-sn-11 snapshot_count none default > dc1-sn-11 redundant_metadata all default >=20 >=20 > Regards, >=20 >=20 > Paul Esson > t +44 (0)118 951 5235 | m +44 (0)776 690 6514 > e paul.esson@redstor.com >=20 >=20 > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Fri May 18 20:29:47 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C21FFEDAFF4 for ; Fri, 18 May 2018 20:29:47 +0000 (UTC) (envelope-from aminoff@nber.org) Received: from mail2.nber.org (mail2.nber.org [198.71.6.79]) by mx1.freebsd.org (Postfix) with ESMTP id 5BA4268A1E for ; Fri, 18 May 2018 20:29:46 +0000 (UTC) (envelope-from aminoff@nber.org) Received: from [198.71.6.100] (perlw2.nber.org [198.71.6.100]) (authenticated bits=0) by mail2.nber.org (8.15.2/8.15.2) with ESMTPSA id w4IKHXGr038182 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Fri, 18 May 2018 16:17:33 -0400 (EDT) (envelope-from aminoff@nber.org) X-Authentication-Warning: mail2.nber.org: Host perlw2.nber.org [198.71.6.100] claimed to be [198.71.6.100] To: freebsd-fs@freebsd.org From: Alex Aminoff Subject: Can one remove a specific file from all snapshots? Message-ID: <38792b3b-04db-0c7f-c9a4-3f55b908a427@nber.org> Date: Fri, 18 May 2018 16:17:33 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-BS X-KLMS-Rule-ID: 1 X-KLMS-Message-Action: clean X-KLMS-AntiSpam-Status: not scanned, disabled by settings X-KLMS-AntiSpam-Interceptor-Info: not scanned X-KLMS-AntiPhishing: Clean, 2018/05/17 14:45:39 X-KLMS-AntiVirus: Kaspersky Security 8.0 for Linux Mail Server, version 8.0.1.721, bases: 2018/05/18 13:56:00 #8433146 X-KLMS-AntiVirus-Status: Clean, skipped X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2018 20:29:47 -0000 The use case here is that we have data sets of sensitive data that we are contractually obligated to delete with no backups at a certain point in time. The datasets are stored on a ZFS filesystem (on a truenas from IxSystems as it happens). The snapshots are kept for 15 days, so at the moment we have to rm the files 15 days before the deadline to be compliant. Is there a way to delete a specific file from the live filesystem as well as all the snapshots? I did find a post to this list from 2014: https://lists.freebsd.org/pipermail/freebsd-fs/2014-October/020307.html but I don't think it is relevant, and in any case it does not appear that the feature that post describes was implemented. Thanks,  - Alex Aminoff, NBER From owner-freebsd-fs@freebsd.org Fri May 18 21:20:26 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 53E92EDC755 for ; Fri, 18 May 2018 21:20:26 +0000 (UTC) (envelope-from eborisch@alumni.stanford.edu) Received: from mail-lf0-x22f.google.com (mail-lf0-x22f.google.com [IPv6:2a00:1450:4010:c07::22f]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 952416A6B1 for ; Fri, 18 May 2018 21:20:25 +0000 (UTC) (envelope-from eborisch@alumni.stanford.edu) Received: by mail-lf0-x22f.google.com with SMTP id m17-v6so15907625lfj.8 for ; Fri, 18 May 2018 14:20:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=d9SI/l9Y4/oulsbY8wxV+mmv/Wvj3R6nr2U3yCUoKRg=; b=hvnDMuRqhpFBYQFBoeKozRN1JVpBoeFHqwzwF1Ik6McPbQXfeDVVSTovKvJjTRZdz9 1MfMRkptnHtrn0lyQEVfmPYDWcSoScyP/W6biEjR4r/eoPJObHE1qWMrcCbD0NJA7piO BKIu6tb2OPr6c9Y2lbHW/TYqjU3hyMsynv8wBEiZjeM4DIxUfGHWngxAtASyJVZLbn8E SO5IXm+sz87xp6iOhZq2uyyRTywNy9X5iaiiaM6ulC6hbVeeiflsiRxLUlZGzhhmZdlj KZvXIwgBmC2QJ0Fq3GulYxcuCsMqHWCq+psBYZFc0r56weC9jbCfjhy9lmobDNDqgR/v ouWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=d9SI/l9Y4/oulsbY8wxV+mmv/Wvj3R6nr2U3yCUoKRg=; b=UFd0n6O2Bpw4BTM6o9r715+TZ6tzirB34Tdw6uxmx2g6FrBRElCxbNqLPab9TZJoM6 ppzqNrBIGeSe7+7FFL0zu4TEEI4O7ieioo/trr45BxppM8kCuv5bqzT2Tm4DOgVDdDTp bW+zp92sYvGR+2FyGWQFpPJgXIoSCsbY6Cylos6ScBEQm3yB32dZi+3M4LStSuJillxH I0Kna5HZ4wVOdDFbR39liXkVCixmh/qfkBDXFjS7bCfY/VKUAZgN7qmGDKJ44K+gfAUe q8Dd/wdN+J3rvHdj7m48rQ1fO2OsEBvn7J1/PRfnUsWbI35r94ftLBrSw7bs6HzsLj10 Q8Kg== X-Gm-Message-State: ALKqPwcsEYrpAU0y74OsA6qAfa+eo+OvaqXYH77QDFNrgGZ7dnSrCEAV 26M560pgCapnb9W5zw6e/1KhkMR689+YwAm40vDV+w== X-Google-Smtp-Source: AB8JxZqZUa7rM0ubnozSNAz/KlKparAFB2GcFv4AEceK2N3fZQpZe5CZ3JHrXucsl8heb+ipAbWzWfdvdVD/AxtiBcI= X-Received: by 2002:a19:1186:: with SMTP id 6-v6mr22944833lfr.134.1526678423695; Fri, 18 May 2018 14:20:23 -0700 (PDT) MIME-Version: 1.0 Sender: eborisch@alumni.stanford.edu Received: by 2002:a2e:84cc:0:0:0:0:0 with HTTP; Fri, 18 May 2018 14:20:23 -0700 (PDT) In-Reply-To: References: From: Eric Borisch Date: Fri, 18 May 2018 16:20:23 -0500 X-Google-Sender-Auth: SspyM3Q4m239ICaTU9RaMFQkQ6A Message-ID: Subject: Re: Unexpected zvol usage To: Paul Esson Cc: "freebsd-fs@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2018 21:20:26 -0000 You're hitting the raidz-N layout rules: individual allocations must be a multiple of (N+1), or 3 for raidz-2, of the underlying block (ashift=12 -> 4k blocks). This is because each individual allocation carries its own parity, and also to avoid leaving holes in the drive when the allocation is removed: https://www.delphix.com/blog/delphix-engineering/ zfs-raidz-stripe-width-or-how-i-learned-stop-worrying-and-love-raidz So for an 8K block, raidz-2, with D being data, P being parity, and X being padding, you're at: DDP,PXX (',' marking allowable allocation multiples of 3), which means 2/3 (66.7%) of your storage is used for metadata + padding; compared with what you likely expected for parity-2 with 12 drives = 1/6 (16.7%), or an "extra" 50% overhead. The "extra" overhead for various volblocksizes in this layout (2/12) are: 4k: 50.0% 8k: 50.0% 16k:16.7% 32k:16.7% 64k: 7.1% 128k: 7.1% That's why Kevin is suggesting 128k. For this particular layout, you would have very similar space efficiency [2] with 64k, and improved latency. Or 16k to significantly reduce the "extra" overhead with the lowest latency impact. I calculate these "extra" overheads here [1], if you're interested. The compression potential with larger block sizes also helps to counteract this overhead on compressible workloads, so it is better on two fronts, if you enable something like lz4 with very low overhead. You really need to test with your workload to find your "optimal" choice, however. I ran into this myself when spinning up some VMs, and put together the linked sheet (Based on one from Matt Ahrens) to help myself and others when selecting array and zvol layouts / settings. - Eric [1] https://docs.google.com/spreadsheets/d/1kQJJpUtbWB_ Poyc7jcO3mNFrFqeHSWuQ8U8Y5UC3dHY/edit?usp=sharing [2] Similar, but I'm guessing not exact; there must be more overhead in the tracking of twice as many blocks, but it's fairly hidden from userland. On Fri, May 18, 2018 at 10:42 AM, Paul Esson wrote: > Hi Folks, > > I have an 11.1-RELEASE system being used as a host for a bhyve guest. > There is a large zpool on the host created from 12 x 10TB HDDs using raidz2 > redundancy with ashift12. I have created a sparse zvol within the pool > using default settings and presented that to the bhyve vm as an ahci-hd > disk type. The guest has a zpool and filesystem dataset built on this > disk. When I start to write to the filesystem on the guest I am finding > that the used/referenced on the host's zvol are more than double those on > the guest. The logicalused/referenced values on the host zvol are more in > line with the equivalent guest values, but my problem is that the host zvol > is likely to fill before I have written all intended data to the guest. > > > I have included below information from both the host and guest before and > after writing. This output shows that the zvol uses a default 8K blocksize > and that the guest zfs is therefore ashift13. I also tried creating the > zvol with a 4K blocksize and the guest zfs ashift12 so that 4K blocks were > consistent across hosts and guest, but still saw the amplification on > writes to the zvol. > > Any insight greatly appreciated. > > > > HOST > > Zpool > RAIDZ2 12 x HDDs, ashift 12 > > NAME PROPERTY VALUE SOURCE > dc1-hn-01 type filesystem - > dc1-hn-01 creation Mon Apr 23 14:35 2018 - > dc1-hn-01 used 32.0G - > dc1-hn-01 available 78.2T - > dc1-hn-01 referenced 201K - > dc1-hn-01 compressratio 1.00x - > dc1-hn-01 mounted yes - > dc1-hn-01 quota none default > dc1-hn-01 reservation none default > dc1-hn-01 recordsize 128K default > dc1-hn-01 mountpoint /export/data/dc1-hn-01 local > dc1-hn-01 sharenfs off default > dc1-hn-01 checksum on default > dc1-hn-01 compression off default > dc1-hn-01 atime on default > dc1-hn-01 devices on default > dc1-hn-01 exec on default > dc1-hn-01 setuid on default > dc1-hn-01 readonly off default > dc1-hn-01 jailed off default > dc1-hn-01 snapdir hidden default > dc1-hn-01 aclmode discard default > dc1-hn-01 aclinherit restricted default > dc1-hn-01 canmount on default > dc1-hn-01 xattr off temporary > dc1-hn-01 copies 1 default > dc1-hn-01 version 5 - > dc1-hn-01 utf8only off - > dc1-hn-01 normalization none - > dc1-hn-01 casesensitivity sensitive - > dc1-hn-01 vscan off default > dc1-hn-01 nbmand off default > dc1-hn-01 sharesmb off default > dc1-hn-01 refquota none default > dc1-hn-01 refreservation none default > dc1-hn-01 primarycache all default > dc1-hn-01 secondarycache all default > dc1-hn-01 usedbysnapshots 0 - > dc1-hn-01 usedbydataset 201K - > dc1-hn-01 usedbychildren 32.0G - > dc1-hn-01 usedbyrefreservation 0 - > dc1-hn-01 logbias latency default > dc1-hn-01 dedup off default > dc1-hn-01 mlslabel - > dc1-hn-01 sync standard default > dc1-hn-01 refcompressratio 1.00x - > dc1-hn-01 written 201K - > dc1-hn-01 logicalused 2.89G - > dc1-hn-01 logicalreferenced 36.5K - > dc1-hn-01 volmode default default > dc1-hn-01 filesystem_limit none default > dc1-hn-01 snapshot_limit none default > dc1-hn-01 filesystem_count none default > dc1-hn-01 snapshot_count none default > dc1-hn-01 redundant_metadata all default > > NAME AVAIL USED USEDSNAP USEDDS > USEDREFRESERV USEDCHILD > dc1-hn-01 78.2T 32.0G 0 201K > 0 32.0G > dc1-hn-01/vm 78.2T 31.9G 0 990M > 0 30.9G > dc1-hn-01/vm/dc1-olbp-sn-11 78.2T 30.9G 0 238K > 0 30.9G > dc1-hn-01/vm/dc1-olbp-sn-11/disk0 78.2T 30.9G 0 4.35G > 26.6G 0 > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 78.2T 4.50M 0 4.50M > 0 0 > > Sparse ZVOL - baseline > > NAME PROPERTY VALUE > SOURCE > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 type volume > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 creation Fri May 18 15:36 > 2018 - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 used 4.50M > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 available 78.2T > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 referenced 4.50M > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compressratio 1.00x > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 reservation none > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volsize 28T > local > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volblocksize 8K > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 checksum on > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compression off > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 readonly off > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 copies 1 > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refreservation none > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 primarycache all > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 secondarycache all > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbysnapshots 0 > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbydataset 4.50M > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbychildren 0 > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbyrefreservation 0 > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logbias latency > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 dedup off > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 mlslabel > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 sync standard > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refcompressratio 1.00x > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 written 4.50M > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalused 1.89M > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalreferenced 1.89M > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volmode dev > local > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_limit none > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_count none > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 redundant_metadata all > default > > > GUEST - baseline > > 1 x vdisk from host ZVOL ashift 13 > > NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV > USEDCHILD > dc1-sn-11 26.9T 632K 0 176K 0 > 456K > > NAME PROPERTY VALUE SOURCE > dc1-sn-11 type filesystem - > dc1-sn-11 creation Fri May 18 15:40 2018 - > dc1-sn-11 used 632K - > dc1-sn-11 available 26.9T - > dc1-sn-11 referenced 176K - > dc1-sn-11 compressratio 1.00x - > dc1-sn-11 mounted yes - > dc1-sn-11 quota none default > dc1-sn-11 reservation none default > dc1-sn-11 recordsize 128K default > dc1-sn-11 mountpoint /export/data/dc1-sn-11 local > dc1-sn-11 sharenfs off default > dc1-sn-11 checksum on default > dc1-sn-11 compression off default > dc1-sn-11 atime on default > dc1-sn-11 devices on default > dc1-sn-11 exec on default > dc1-sn-11 setuid on default > dc1-sn-11 readonly off default > dc1-sn-11 jailed off default > dc1-sn-11 snapdir hidden default > dc1-sn-11 aclmode discard default > dc1-sn-11 aclinherit restricted default > dc1-sn-11 canmount on default > dc1-sn-11 xattr off temporary > dc1-sn-11 copies 1 default > dc1-sn-11 version 5 - > dc1-sn-11 utf8only off - > dc1-sn-11 normalization none - > dc1-sn-11 casesensitivity sensitive - > dc1-sn-11 vscan off default > dc1-sn-11 nbmand off default > dc1-sn-11 sharesmb off default > dc1-sn-11 refquota none default > dc1-sn-11 refreservation none default > dc1-sn-11 primarycache all default > dc1-sn-11 secondarycache all default > dc1-sn-11 usedbysnapshots 0 - > dc1-sn-11 usedbydataset 176K - > dc1-sn-11 usedbychildren 456K - > dc1-sn-11 usedbyrefreservation 0 - > dc1-sn-11 logbias latency default > dc1-sn-11 dedup off default > dc1-sn-11 mlslabel - > dc1-sn-11 sync standard default > dc1-sn-11 refcompressratio 1.00x - > dc1-sn-11 written 176K - > dc1-sn-11 logicalused 49K - > dc1-sn-11 logicalreferenced 11.5K - > dc1-sn-11 volmode default default > dc1-sn-11 filesystem_limit none default > dc1-sn-11 snapshot_limit none default > dc1-sn-11 filesystem_count none default > dc1-sn-11 snapshot_count none default > dc1-sn-11 redundant_metadata all default > > After writing some data to the guest > > HOST ZVOL > > NAME PROPERTY VALUE > SOURCE > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 type volume > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 creation Fri May 18 15:36 > 2018 - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 used 99.7G > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 available 78.1T > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 referenced 99.7G > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compressratio 1.00x > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 reservation none > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volsize 28T > local > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volblocksize 8K > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 checksum on > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 compression off > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 readonly off > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 copies 1 > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refreservation none > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 primarycache all > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 secondarycache all > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbysnapshots 0 > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbydataset 99.7G > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbychildren 0 > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 usedbyrefreservation 0 > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logbias latency > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 dedup off > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 mlslabel > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 sync standard > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 refcompressratio 1.00x > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 written 99.7G > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalused 43.6G > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 logicalreferenced 43.6G > - > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 volmode dev > local > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_limit none > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 snapshot_count none > default > dc1-hn-01/vm/dc1-olbp-sn-11/disk1 redundant_metadata all > default > > GUEST ZFS > > NAME PROPERTY VALUE SOURCE > dc1-sn-11 type filesystem - > dc1-sn-11 creation Fri May 18 15:40 2018 - > dc1-sn-11 used 44.3G - > dc1-sn-11 available 26.8T - > dc1-sn-11 referenced 176K - > dc1-sn-11 compressratio 1.00x - > dc1-sn-11 mounted no - > dc1-sn-11 quota none default > dc1-sn-11 reservation none default > dc1-sn-11 recordsize 128K default > dc1-sn-11 mountpoint /export/data/dc1-sn-11 local > dc1-sn-11 sharenfs off default > dc1-sn-11 checksum on default > dc1-sn-11 compression off default > dc1-sn-11 atime on default > dc1-sn-11 devices on default > dc1-sn-11 exec on default > dc1-sn-11 setuid on default > dc1-sn-11 readonly off default > dc1-sn-11 jailed off default > dc1-sn-11 snapdir hidden default > dc1-sn-11 aclmode discard default > dc1-sn-11 aclinherit restricted default > dc1-sn-11 canmount on default > dc1-sn-11 xattr on default > dc1-sn-11 copies 1 default > dc1-sn-11 version 5 - > dc1-sn-11 utf8only off - > dc1-sn-11 normalization none - > dc1-sn-11 casesensitivity sensitive - > dc1-sn-11 vscan off default > dc1-sn-11 nbmand off default > dc1-sn-11 sharesmb off default > dc1-sn-11 refquota none default > dc1-sn-11 refreservation none default > dc1-sn-11 primarycache all default > dc1-sn-11 secondarycache all default > dc1-sn-11 usedbysnapshots 0 - > dc1-sn-11 usedbydataset 176K - > dc1-sn-11 usedbychildren 44.3G - > dc1-sn-11 usedbyrefreservation 0 - > dc1-sn-11 logbias latency default > dc1-sn-11 dedup off default > dc1-sn-11 mlslabel - > dc1-sn-11 sync standard default > dc1-sn-11 refcompressratio 1.00x - > dc1-sn-11 written 176K - > dc1-sn-11 logicalused 44.2G - > dc1-sn-11 logicalreferenced 11.5K - > dc1-sn-11 volmode default default > dc1-sn-11 filesystem_limit none default > dc1-sn-11 snapshot_limit none default > dc1-sn-11 filesystem_count none default > dc1-sn-11 snapshot_count none default > dc1-sn-11 redundant_metadata all default > > > Regards, > > > Paul Esson > t +44 (0)118 951 5235 | m +44 (0)776 690 6514 > e paul.esson@redstor.com > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@freebsd.org Fri May 18 21:23:54 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 27CA1EDC9BB for ; Fri, 18 May 2018 21:23:54 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: from connect.ultra-secure.de (connect.ultra-secure.de [88.198.71.201]) by mx1.freebsd.org (Postfix) with ESMTP id 776836A9DA for ; Fri, 18 May 2018 21:23:53 +0000 (UTC) (envelope-from rainer@ultra-secure.de) Received: (Haraka outbound); Fri, 18 May 2018 23:21:53 +0200 Authentication-Results: connect.ultra-secure.de; iprev=pass; auth=pass (plain); spf=none smtp.mailfrom=ultra-secure.de Received-SPF: None (connect.ultra-secure.de: domain of ultra-secure.de does not designate 217.71.83.52 as permitted sender) receiver=connect.ultra-secure.de; identity=mailfrom; client-ip=217.71.83.52; helo=[192.168.1.200]; envelope-from= Received: from [192.168.1.200] (217-071-083-052.ip-tech.ch [217.71.83.52]) by connect.ultra-secure.de (Haraka/2.6.2-toaster) with ESMTPSA id 9DF227E6-573B-4B7A-BF51-0AEC2FD27888.1 envelope-from (authenticated bits=0); Fri, 18 May 2018 23:21:46 +0200 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Subject: Re: Can one remove a specific file from all snapshots? From: Rainer Duffner In-Reply-To: <38792b3b-04db-0c7f-c9a4-3f55b908a427@nber.org> Date: Fri, 18 May 2018 23:22:26 +0200 Cc: freebsd-fs@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <62BA102D-E58F-474D-A4DF-65D2020BEB80@ultra-secure.de> References: <38792b3b-04db-0c7f-c9a4-3f55b908a427@nber.org> To: Alex Aminoff X-Mailer: Apple Mail (2.3445.6.18) X-Haraka-GeoIP: EU, CH, 451km X-Haraka-ASN: 24951 X-Haraka-GeoIP-Received: X-Haraka-ASN: 24951 217.71.80.0/20 X-Haraka-ASN-CYMRU: asn=24951 net=217.71.80.0/20 country=CH assignor=ripencc date=2003-08-07 X-Haraka-FCrDNS: 217-071-083-052.ip-tech.ch X-Haraka-p0f: os="Mac OS X " link_type="DSL" distance=12 total_conn=1 shared_ip=N X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on spamassassin X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,BAYES_00, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.1 X-Haraka-Karma: score: 6, good: 5682, bad: 3, connections: 6226, history: 5679, asn_score: 656, asn_connections: 721, asn_good: 657, asn_bad: 1, pass:asn, relaying X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2018 21:23:54 -0000 > Am 18.05.2018 um 22:17 schrieb Alex Aminoff : >=20 >=20 > The use case here is that we have data sets of sensitive data that we = are contractually obligated to delete with no backups at a certain point = in time. The datasets are stored on a ZFS filesystem (on a truenas from = IxSystems as it happens). The snapshots are kept for 15 days, so at the = moment we have to rm the files 15 days before the deadline to be = compliant. Is there a way to delete a specific file from the live = filesystem as well as all the snapshots? >=20 > I did find a post to this list from 2014: >=20 > = https://lists.freebsd.org/pipermail/freebsd-fs/2014-October/020307.html >=20 > but I don't think it is relevant, and in any case it does not appear = that the feature that post describes was implemented. >=20 > Thanks, >=20 AFAIK, snapshots are still read-only. BTRFS seems to have writable snapshots - but I=E2=80=99ve no idea how it = actually works. I would suggest to change your contracts - that will be much easier than = changing ZFS. From owner-freebsd-fs@freebsd.org Fri May 18 21:38:16 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 040A7EDCEB3 for ; Fri, 18 May 2018 21:38:16 +0000 (UTC) (envelope-from rcv-cmlja0BoYXZva21vbi5jb20AQDE1MjY2NzkwNzk=@vfemail.net) Received: from smtp102-6.vfemail.net (onethreesix.vfemail.net [199.16.11.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 996776AFBE for ; Fri, 18 May 2018 21:38:15 +0000 (UTC) (envelope-from rcv-cmlja0BoYXZva21vbi5jb20AQDE1MjY2NzkwNzk=@vfemail.net) DKIM-Signature: v=1; a=rsa-sha256; c=simple; d=havokmon.com; h=date :message-id:from:to:subject:references:in-reply-to:content-type :mime-version:content-transfer-encoding; s=default; bh=Bk4ApJZmF AZko4gKHcZagIOTuCktlMQ16OQkpqUU2UI=; b=I+k3fZ0KMIa9iTH1WeJoMbVRV xcgSS18BtQeZ+Ok2fbxu/Lo7h5RvV5cJKYRFwRR64bGxtnuTUfYtBzSElYtjBzqZ F+tlpJizOrjHprNJDAPLmagv0Qj88K8zPSb2IsFU23qc090nhtuh43h/GN3NvyiS dBhZFuiTJIufgd9TDM= Received: (qmail 20770 invoked by uid 89); 18 May 2018 21:31:19 -0000 Received: by simscan 1.4.0 ppid: 20764, pid: 20767, t: 0.0506s scanners:none Received: from unknown (HELO bG9jYWxob3N0QDE1MjY2NzkwNzk=) (cmlja0BoYXZva21vbi5jb21AMTUyNjY3OTA3OQ==@MTcyLjE2LjEwMC45MkAxNTI2Njc5MDc5) by 172.16.100.62 with ESMTPSA (DHE-RSA-AES128-GCM-SHA256 encrypted, authenticated); 18 May 2018 21:31:19 -0000 Date: Fri, 18 May 2018 21:31:28 +0000 Message-ID: <20180518213128.Horde.1N5mXCiMFdOfph_qVwqZNS7@www.vfemail.net> From: Rick Romero To: freebsd-fs@freebsd.org Subject: Re: Can one remove a specific file from all snapshots? References: <38792b3b-04db-0c7f-c9a4-3f55b908a427@nber.org> <62BA102D-E58F-474D-A4DF-65D2020BEB80@ultra-secure.de> In-Reply-To: <62BA102D-E58F-474D-A4DF-65D2020BEB80@ultra-secure.de> User-Agent: Horde Application Framework 5 Accept-Language: en MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes Content-Transfer-Encoding: 8bit Content-Disposition: inline Content-Description: Plaintext Message X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 May 2018 21:38:16 -0000 Quoting Rainer Duffner : >> Am 18.05.2018 um 22:17 schrieb Alex Aminoff : >> >> The use case here is that we have data sets of sensitive data that >> we are contractually obligated to delete with no backups at a >> certain point in time. The datasets are stored on a ZFS filesystem >> (on a truenas from IxSystems as it happens). The snapshots are kept >> for 15 days, so at the moment we have to rm the files 15 days >> before the deadline to be compliant. Is there a way to delete a >> specific file from the live filesystem as well as all the snapshots? >> >> I did find a post to this list from 2014: >> >> https://lists.freebsd.org/pipermail/freebsd-fs/2014-October/020307.html >> >> but I don't think it is relevant, and in any case it does not >> appear that the feature that post describes was implemented. >> >> Thanks, > > AFAIK, snapshots are still read-only. > > BTRFS seems to have writable snapshots - but I’ve no idea how it > actually works. > > I would suggest to change your contracts - that will be much easier > than changing ZFS. Create a dataset and mount it below the current file location, and put that file there.  Assuming SMB, it should still be shared out, just one folder below.  Then don't include that dataset in the snapshots (or do a different rotation). Rick From owner-freebsd-fs@freebsd.org Sat May 19 02:06:04 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3F51BEE5DB6 for ; Sat, 19 May 2018 02:06:04 +0000 (UTC) (envelope-from large.hadron.collider@gmx.com) Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "mout.gmx.net", Issuer "TeleSec ServerPass DE-2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9698974E0A for ; Sat, 19 May 2018 02:06:03 +0000 (UTC) (envelope-from large.hadron.collider@gmx.com) Received: from Regeneratus.BC.CA.Umbrellix.NET ([50.69.217.36]) by mail.gmx.com (mrgmx101 [212.227.17.174]) with ESMTPSA (Nemesis) id 0LlYrb-1eialT06h5-00bJa8 for ; Sat, 19 May 2018 04:06:00 +0200 Subject: Re: Can one remove a specific file from all snapshots? (accidentally priv'd) To: freebsd-fs@freebsd.org References: <38792b3b-04db-0c7f-c9a4-3f55b908a427@nber.org> From: Large Hadron Collider Followup-To: Alex Aminoff Message-ID: <40e26a54-8a7b-c41b-03b0-89ee88795dd0@gmx.com> Date: Sat, 19 May 2018 02:05:57 +0000 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.6.0 MIME-Version: 1.0 In-Reply-To: <38792b3b-04db-0c7f-c9a4-3f55b908a427@nber.org> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Provags-ID: V03:K1:Gq/iW4ixG0cDv7e51hK3ZuIfYf2GKI5gpgwTxZWfrXc2w30MMOE lmxnaznvK8Ys2ujhotul1t0w+7GOdjLeJG+E0XZ7IM28DrOTz00IjzQjnisqP7M9ibPILx0 9tJfTAFJn6pVB3r9NsquG26LElB6Kxhq6Qyd9qCpMr42rdBTrlK8ATKeLxoDl+mXcMGfmDh SniDmogFDAEjGE5HVlFhw== X-UI-Out-Filterresults: notjunk:1;V01:K0:vjFfOm6qseQ=:Y7wALL/tvwLNzeOvWZSKnh VWyoTN38xeRWLuw368UdeFL5ibOoxvwL/pJNrnRRmigPpmKC0nZAngiN7nrc466h5PIUKM8rD KluawPoTRTHdqUlkW0nCH7tnpkOJDAX9JaZuCFBKqwaHPfCdQV6KeTU2Fc/MrqegDePI6Gide +zfC1Z593pyUqzxpUrUpkCVZKYhsBgRZsrQZgUFsApuy1UC05US4AzHvgYna8gh65VefDgkyT UPjMbQDWocTmw8HpDJ3swtSPQJL4T7mSJ4ZPXtsi/ooxphIYrtnkKVDVBfonV0pWxdx5bm2/r ae7w0qEe++x6NTg1hkLBjwwoGyNoCAaZGi4xN3Qqrpt75G0Uj3paKNjrbCuTjTxbgM8Oy0Bqt ulPblgndW3kCOZQgkN/edRFgZqz/2yWjwCfpdAfCAqCsBHGJlHUTUxU3Cj7hTD1JBGs6x3lja Y255LBw60opZACfZDMsG5O9oVBikQfNiVS5FMKb708DMQOsmJKI4rm1Wqa3cRYRLlyeUP8xqf wxlSvSNMt4cCHpt/wzQvSHOuIJcE73ckzh6YSGyWmaVzC+Py5eCpJjb7f7tvzvjbVELUiaQrM gXspJ/YiW5Jo19GROCDdlj9kbT8DvfqWBcwFr7D17zv5rKymMwR2wmyGhyaxdzQiaZ/sLk6R4 kBMNJrlLCMMRJwlKE+S88Xnc3e0O8snLsrFuujXLZOD8UgC29m20xV9THhKzgpW0cFXWdsYxT VUL0pJ4Jit69S9uERVerb6h0rAVmDqQfPGkiWxlFg0fCAV8fbAeUtkgE4jv32Ebew2GXwKJPu mjtjKhy X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2018 02:06:04 -0000 Expire the snapshots faster (5 days instead of 15) so you have 10 days to dawdle. On 05/18/2018 20:17, Alex Aminoff wrote: > > The use case here is that we have data sets of sensitive data that we > are contractually obligated to delete with no backups at a certain > point in time. The datasets are stored on a ZFS filesystem (on a > truenas from IxSystems as it happens). The snapshots are kept for 15 > days, so at the moment we have to rm the files 15 days before the > deadline to be compliant. Is there a way to delete a specific file > from the live filesystem as well as all the snapshots? > > I did find a post to this list from 2014: > > https://lists.freebsd.org/pipermail/freebsd-fs/2014-October/020307.html > > but I don't think it is relevant, and in any case it does not appear > that the feature that post describes was implemented. > > Thanks, > >  - Alex Aminoff, NBER > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@freebsd.org Sat May 19 04:48:54 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E9D0BEEE05C for ; Sat, 19 May 2018 04:48:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 83ADE7D6DE for ; Sat, 19 May 2018 04:48:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id 446B1EEE059; Sat, 19 May 2018 04:48:53 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 31C2CEEE058 for ; Sat, 19 May 2018 04:48:53 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C310C7D6D9 for ; Sat, 19 May 2018 04:48:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 12E5C266BC for ; Sat, 19 May 2018 04:48:52 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4J4mpu2034367 for ; Sat, 19 May 2018 04:48:51 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4J4mpvI034366 for fs@FreeBSD.org; Sat, 19 May 2018 04:48:51 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 228354] mount_smbfs - stack overflow detected - dumps core Date: Sat, 19 May 2018 04:48:52 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: linimon@FreeBSD.org X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to keywords Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2018 04:48:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D228354 Mark Linimon changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|bugs@FreeBSD.org |fs@FreeBSD.org Keywords| |regression --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat May 19 17:59:56 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 70A87EDC21B for ; Sat, 19 May 2018 17:59:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 0B7F77845C for ; Sat, 19 May 2018 17:59:56 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id BE94FEDC20F; Sat, 19 May 2018 17:59:55 +0000 (UTC) Delivered-To: fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id ABD1CEDC20C for ; Sat, 19 May 2018 17:59:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 42E9D7845A for ; Sat, 19 May 2018 17:59:55 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 8E3A6D58F for ; Sat, 19 May 2018 17:59:54 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id w4JHxsrE023968 for ; Sat, 19 May 2018 17:59:54 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id w4JHxsZc023967 for fs@FreeBSD.org; Sat, 19 May 2018 17:59:54 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: fs@FreeBSD.org Subject: [Bug 228354] mount_smbfs - stack overflow detected - dumps core Date: Sat, 19 May 2018 17:59:54 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bin X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: regression X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: donald.buchholz@intel.com X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: fs@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2018 17:59:56 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D228354 --- Comment #1 from Don Buchholz --- The 12-CURRENT pre-release was installed from "-disc1" images from https://download.freebsd.org/ftp/snapshots/amd64/amd64/ISO-IMAGES/12.0/ Both 20180426 and 20180514 releases were used and found to have this proble= m. --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@freebsd.org Sat May 19 22:09:58 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B9042EE60AD for ; Sat, 19 May 2018 22:09:58 +0000 (UTC) (envelope-from lopez.on.the.lists@yellowspace.net) Received: from mail.yellowspace.net (mail.yellowspace.net [62.146.62.104]) by mx1.freebsd.org (Postfix) with ESMTP id 26C3082391 for ; Sat, 19 May 2018 22:09:57 +0000 (UTC) (envelope-from lopez.on.the.lists@yellowspace.net) Received: from zyx-2.local ([88.217.180.123]) (AUTH: PLAIN lopez.on.the.lists@yellowspace.net, SSL: TLSv1/SSLv3, 128bits, AES128-SHA) by mail.yellowspace.net with esmtp; Sun, 20 May 2018 00:09:50 +0200 id 000116EA.000000005B00A0AE.00014331 Subject: Re: Native ZFS encryption To: freebsd-fs@freebsd.org References: <3e0b2644-9739-5341-0295-c746275588ec@aon.at> <0bf3fd08-1803-e15b-cadf-755d1c0af325@multiplay.co.uk> From: Lorenzo Perone Openpgp: preference=signencrypt Autocrypt: addr=lopez.on.the.lists@yellowspace.net; prefer-encrypt=mutual; keydata= xsFNBFZPDLEBEAC8ucr22czPsPRVfF/aFLN07C9uIi00gQWNFmawBbLDx0z8h159t0c0TVhg iw79mvDGWEjO58+iUuQQop6BIr6UBy7OBW4Y1+N2449LIePkwn1wvB56HIucZzBpbnww7tZA dF8o2MRsL5E8mW3gBA01IiaqyWxo9uwUeI9jLk/EkJpP6QEzhydwWB474dog5/+E7HVnhBHy PKoEF7w8FbvzaaxCGNQSiOPZbETzuZCjuh7YXAAzrr5cwuqrfTs0XXHgBSjBDO7oe1fVtT0q 7KRcLOmhZUmg11lHdXzEOkYiV5psX/kiCLBlBCG295hEPMcRuz0F4L9T95O4/RbUAuwqZt29 5n2p4221jsn8cVTOxxwUEzclP0xOAJONX+hvBCgRL9EmwH9nE2IcS6RriuqI7exSozL6ROZp bCPEWJAhqMEUA+70P0QFk4PL3PbvaQa31ZR+g4zf7/W0NFdrZakkR5VZ+uHyA3NMX6+mKbdM HQAP/uQ/Kk6FMKqjfOB/gSEkto3JQOjC0W1Qgk7wjmMNjLrC8769nqbwmLG/y1Vz82ASojek +k6g4zGFLJfCknwfUEEb8xCjhmMqA/Ks6fBEkXZqE1m/NadYpQuOUmwTMIM5yZ86AJwn6+wr qgW2Z1Yu9yNC0OEXiM7XS1TI2hB39LRC3poIPZH+Vy93WEofywARAQABzS9Mb3JlbnpvIFBl cm9uZSA8bG9yZW56by5wZXJvbmVAYnl0ZXNhdHdvcmsuY29tPsLBfgQTAQIAKAUCVk8NkAIb AwUJCWYBgAYLCQgHAwIGFQgCCQoLBBYCAwECHgECF4AACgkQo9wgCxr6qpMRVQ/9HYuHJ36M SdzKYF8D3fxmAdumUZDdVxoJ4CZSJ7NyOlPjbi5KWtxaQ0ZFwSGXvGIg49KDuGp5C9USwCbv +nTmEkqMbTOQX77CQkdFHfkieEXI1DxAwYVpKVMZKOZK7TRE5YWqLklfFBZdwFfB/SWdxyzX MCEV+Y/2g+mPY6uAOD4FS+/iIDWqEUW1H/jxttLm3whX3VQw+Odu83WzjNCrcHaDFqtqtd4M yy5JMAO+nB9hLZV/qfNgUA//ZGV5TbXWN5kLrHWSN5/u7IzNHieAGBYup3g4BTFl5J3sM00/ 9YaAhtpEPRWW98nyPIekC8T1lS/COe1zh3HTyWTQ5NxIE85rkhwg8nGSvocw/4tLRT51ownq IfXolTMbrmsxqyw/jOZ0rqQLkl7cLH82B7n9gmEhiscR5YYw3Zf8fivXIlg+Cy358q6VXAJ8 veexMgQ2Rykih8hj2VVliqhHQst3Zzca/iwKF1biQe5IWjpZ3IgPiMoN7qvEq0Ql8QsRjUAM N+bhKP3+b8GaHdchY8bo6053X0P381fNdbujZipYJe09XuiXWHZ13BOiAKeVyrPz3CftDwTw RS1Qoe6GQ65jkA7MbiK8RpdrF6VuuVdX3ZDYhuFF3TxYBP1f8yIpaAwP6eBGs8Su0q9R4aCQ qJxCdw/QYG9Ne66cVpU3CWIWynfOwU0EVk8MsQEQAK8ARKV0eXw4yV+cSRKz3nkAuJg+yymo bnc3OTToSEZpc4luFN7ijcKhbOA+/kds22Zm185ncqb/Rtq9z18tQAE5W714PNbP/z7ZJhaP Zg/8F5iaB1IOw8tt0xDs5YtgYLYG0diLfyJMTCjEbptn/7LyQtUbsijU5o3tkAceHekvJzMQ QdPV+Hj87AwKYMGGXJUP4FilRMIHg/8zbnFQen/6LVZVxxqBLH330MRrEJZNPeU6mazIeGlC 28ei9Gy2jq9WiRWpTFUUvpf7egKy6e8ObGNZtJnRoTEqZ/iUDCs1jMIM5nKnqCUVQhvZvAK8 c6VJ3pbCmeVlkQAHQ8ZTYIZVRBN+MRBTHkaDCv6UyMZBLzrT/HUy7xhlLQMdVVpLOrTMyC8z 1200bvU94mVHtzP5Nb1QL3ouW+R660LWls8Z/HEGtwCR5gORwa0/m7nBM2pSuh8HSOXPU+Cl cmDTul4mMdONHvWAyQ691C0F4GG9YJ7CQ37dmFPXaVOewErg0k7laZX0eaV5t0d5LJUchZec ywNBilOfBi0BkfRpj1R1qhbo/Y06WmNdQ6FgchtzP1kGJNaCbcviGC6dqoFJHBSSIm08YRVU 48cWMHCi0UeIVQdFaRe7K/9VP55++K0gI+9N5RDzBqz455XYgYG+uYYnvj+Q+AudAk1wYRsn OPBBABEBAAHCwWUEGAECAA8FAlZPDLECGwwFCQlmAYAACgkQo9wgCxr6qpO30A/+M4ZqqsU3 YET4S6uu7teq+HNDe9HKjEC6MfgToDj/LXICCH7g9b70vQupeCX/8TpYdCBh3m4hezUOcZX2 BZMGfxRfmUreV828M/4pbGQVjsZ14Y0AykmbXuTj1/p4ZeKqxeua09w3KA7FBsG5z1RSOksW uSWxzUPvJswpY1Oofw2kCMIo4B0iF8IEz/m/IEPDG1D1UgAQ+l6mPExOfM8sANAbDGaeEHNh 4OFm1ARoUJTSKCvl7Nf/ZY9PIg8kzZ3E/pxEjISAaX8Zmg7wvublDL+KwokNyH+u/X7rhg41 yAB1c7/lfl5l3kaLpvM8QLQ7c0/r57Qs8YVVeFRWr4c+ri9GJwxGzp8P8nqeZiUIV9l0kjnb C9ROfPLy02xfzuRScmS1pKig+ZUD/OEruvB2IM4vYNjO9yX2vOvePrfNx9EyEP20CXAO7U6Z ZRAtxr0e+46KEutLYmUhaAzBqI2vZk+r/ilrfgVt3K3iWZtAYVLxIkQMUB4QAd5XH5cA8GdG zE/WdA0x7cKt4EZBQCPA5uI/EIryd8pS+15REXzZkxdEbej5HVxscyknFxHNKO/FgxYPmirp sAOKU9byWEHR7Y0MrzWlTR8CYZjRxfDWaAsmNtx7vG7gQHOO31fptBMfyXkRDbgN9DDVd6Zk Pu9D8xDramS6y4qVQ8aVGKMdTEg= Message-ID: <311e19da-6a02-ec4c-47c3-9bd99c6e85f5@yellowspace.net> Date: Sun, 20 May 2018 00:09:47 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit In-Reply-To: Content-Language: en-US X-Mime-Autoconverted: from 8bit to 7bit by courier 0.63 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 19 May 2018 22:09:59 -0000 On 15.05.18 13:02, Lorenzo Perone wrote: > On the more general matter, I wondered if anyone has a pointer to any > documentation / article / comment (other than the code itself) on how > "compatible" the openzfs and FreeBSD-zfs implementations actually are? I'll answer to myself, as that might be interesting to other folks here, too. Although it doesn't directly answer all questions of this thread, I can recommend the latest FreeBSD journal. The article "ZFS 2018 and Onward" by Allan Jude is a nice read. I won't spoil - the journal is worth all our support... Reference: https://www.freebsdfoundation.org/journal/ Best, Lorenzo