From owner-freebsd-fs@FreeBSD.ORG Sun Dec 5 00:27:54 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AA0B1106566B for ; Sun, 5 Dec 2010 00:27:54 +0000 (UTC) (envelope-from sdrhodus@gmail.com) Received: from mail-wy0-f182.google.com (mail-wy0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 11CDA8FC0C for ; Sun, 5 Dec 2010 00:27:53 +0000 (UTC) Received: by wyf19 with SMTP id 19so10931651wyf.13 for ; Sat, 04 Dec 2010 16:27:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:cc:content-type :content-transfer-encoding; bh=ainMtIlxMqXTnO0hKtaDd5SJoumy9Z6h9jMR7rWvbcQ=; b=wb6tGxKMZHu0cgZS1lO9NPXYfS1dJ4lgLnlkw/kjGKndRbuSgZ+jgwzyQj+b7q6iHK nWP+6TgcsiPeDqcRRRnhi+Ts+jNxfCT+30WpNDmxta8PZ3tf5TaUMjteYLEokdrjHEZx 6oGTUiGY2VSU31WvSlH8sGIjyz8rvTuo1cRYs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type:content-transfer-encoding; b=pVHEtHTh2Z6wB1Z8HTpmR3wuXYWDbuRAX+uQSrLyqhs/0moc+U5yD6MxZUAq7K0eij nZyiJ6mfEN7myfD2BReWovq0LLQDK7CfVYn9mLnPoJ171alaRGFdTrAzFKuHoZ7IDIKK Y6ETn5NS9HVad5so4L4r7FpkwhaGlNV9e4Ewo= Received: by 10.216.50.72 with SMTP id y50mr3344336web.34.1291507208788; Sat, 04 Dec 2010 16:00:08 -0800 (PST) MIME-Version: 1.0 Received: by 10.217.4.10 with HTTP; Sat, 4 Dec 2010 15:59:48 -0800 (PST) In-Reply-To: <201012021846.oB2IknXX006899@chez.mckusick.com> References: <201012021846.oB2IknXX006899@chez.mckusick.com> From: David Rhodus Date: Sat, 4 Dec 2010 18:59:48 -0500 Message-ID: To: Kirk McKusick Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Jeff Roberson , Michael Butler , freebsd-fs@freebsd.org Subject: Re: PR kern/152605 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Dec 2010 00:27:54 -0000 On Thu, Dec 2, 2010 at 1:46 PM, Kirk McKusick wrote= : >> From: David Rhodus >> Date: Thu, 2 Dec 2010 12:33:35 -0500 >> Subject: Re: How a full fsck screwed up my SU+J filesystem >> To: Michael Butler >> Cc: Kirk McKusick , >> =A0 =A0 =A0 =A0 Kostik Belousov , current@freebsd.o= rg >> X-ASK-Info: Message Queued (2010/12/02 09:34:03) >> X-ASK-Info: Confirmed by User (2010/12/02 09:35:13) >> >> Hello, =A0what do you make of this PR report ? >> >> http://www.freebsd.org/cgi/query-pr.cgi?pr=3Dkern/152605 >> >> This panic happens during bg-fsck every time. I have to boot into >> single user-mode and do a fsck to correct. > > I have not seen this bug report yet. It would be helpful if you > could identify which file is associated with inode 18, e.g., > > =A0 =A0 =A0 =A0cd > =A0 =A0 =A0 =A0find . -x -inum 18 -ls the file is /usr/ports/mail/popper/files > > Also, a transcript of the fsck that you run to clean up the mess > so that I can see what is broken in the filesystem. > > =A0 =A0 =A0 =A0Kirk McKusick > Below is the output from fsck -y ** /dev/ad0s1a (NO WRITE) USE JOURNAL? no ** Skipping journal, falling through to full fsck SETTING DIRTY FLAG IN READ_ONLY MODE UNEXPECTED SOFT UPDATE INCONSISTENCY ** Last Mounted on / ** Root file system ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 3245 files, 179539 used, 833476 free (1052 frags, 104053 blocks, 0.1% fragmentation) ** /dev/ad0s1e ** Last Mounted on /tmp ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes BLK(S) MISSING IN BIT MAPS SALVAGE? yes 27 files, 12 used, 2029019 free (51 frags, 253621 blocks, 0.0% fragmentatio= n) ***** FILE SYSTEM MARKED CLEAN ***** ***** FILE SYSTEM WAS MODIFIED ***** ** /dev/ad0s1f ** Last Mounted on /usr ** Phase 1 - Check Blocks and Sizes INCORRECT BLOCK COUNT I=3D1907725 (12 should be 8) CORRECT? yes INCORRECT BLOCK COUNT I=3D1921779 (16 should be 0) CORRECT? yes INCORRECT BLOCK COUNT I=3D2237503 (20 should be 16) CORRECT? yes INCORRECT BLOCK COUNT I=3D2261101 (8 should be 4) CORRECT? yes INCORRECT BLOCK COUNT I=3D2263653 (56 should be 0) CORRECT? yes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts UNREF FILE I=3D1921771 OWNER=3Droot MODE=3D100644 SIZE=3D2040 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D1921772 OWNER=3Droot MODE=3D100644 SIZE=3D1000 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D1921773 OWNER=3Droot MODE=3D100644 SIZE=3D3428 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D1921774 OWNER=3Droot MODE=3D100644 SIZE=3D3668 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D1921775 OWNER=3Droot MODE=3D100644 SIZE=3D1776 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D1921776 OWNER=3Droot MODE=3D100644 SIZE=3D3092 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D1921777 OWNER=3Droot MODE=3D100644 SIZE=3D8012 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D1921778 OWNER=3Droot MODE=3D100644 SIZE=3D3524 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D1921779 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245824 OWNER=3Droot MODE=3D100644 SIZE=3D3654 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245827 OWNER=3Droot MODE=3D100644 SIZE=3D1485 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245828 OWNER=3Droot MODE=3D100644 SIZE=3D2946 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245829 OWNER=3Droot MODE=3D100644 SIZE=3D1734 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245831 OWNER=3Droot MODE=3D100644 SIZE=3D2439 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245832 OWNER=3Droot MODE=3D100644 SIZE=3D1466 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245833 OWNER=3Droot MODE=3D100644 SIZE=3D1463 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245834 OWNER=3Droot MODE=3D100644 SIZE=3D3311 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245835 OWNER=3Droot MODE=3D100644 SIZE=3D2266 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245836 OWNER=3Droot MODE=3D100644 SIZE=3D3203 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245838 OWNER=3Droot MODE=3D100644 SIZE=3D1488 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245839 OWNER=3Droot MODE=3D100644 SIZE=3D1649 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245840 OWNER=3Droot MODE=3D100644 SIZE=3D2216 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245841 OWNER=3Droot MODE=3D100644 SIZE=3D1254 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245842 OWNER=3Droot MODE=3D100644 SIZE=3D2256 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245843 OWNER=3Droot MODE=3D100644 SIZE=3D1453 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245844 OWNER=3Droot MODE=3D100644 SIZE=3D1353 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245846 OWNER=3Droot MODE=3D100644 SIZE=3D1780 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245848 OWNER=3Droot MODE=3D100644 SIZE=3D1806 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245856 OWNER=3Droot MODE=3D100644 SIZE=3D1243 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245857 OWNER=3Droot MODE=3D100644 SIZE=3D2307 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245858 OWNER=3Droot MODE=3D100644 SIZE=3D1362 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245859 OWNER=3Droot MODE=3D100644 SIZE=3D1336 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245860 OWNER=3Droot MODE=3D100644 SIZE=3D1172 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245861 OWNER=3Droot MODE=3D100644 SIZE=3D1451 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245862 OWNER=3Droot MODE=3D100644 SIZE=3D2215 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245863 OWNER=3Droot MODE=3D100644 SIZE=3D2079 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245864 OWNER=3Droot MODE=3D100644 SIZE=3D1337 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245865 OWNER=3Droot MODE=3D100644 SIZE=3D2534 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245866 OWNER=3Droot MODE=3D100644 SIZE=3D3293 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245867 OWNER=3Droot MODE=3D100644 SIZE=3D1568 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245868 OWNER=3Droot MODE=3D100644 SIZE=3D1265 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245869 OWNER=3Droot MODE=3D100644 SIZE=3D2397 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245870 OWNER=3Droot MODE=3D100644 SIZE=3D1239 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245871 OWNER=3Droot MODE=3D100644 SIZE=3D1990 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245877 OWNER=3Droot MODE=3D100644 SIZE=3D3839 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245878 OWNER=3Droot MODE=3D100644 SIZE=3D2359 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245879 OWNER=3Droot MODE=3D100644 SIZE=3D1715 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245880 OWNER=3Droot MODE=3D100644 SIZE=3D2127 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245881 OWNER=3Droot MODE=3D100644 SIZE=3D1371 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245882 OWNER=3Droot MODE=3D100644 SIZE=3D3222 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245883 OWNER=3Droot MODE=3D100644 SIZE=3D1358 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245884 OWNER=3Droot MODE=3D100644 SIZE=3D3743 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245885 OWNER=3Droot MODE=3D100644 SIZE=3D1539 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245886 OWNER=3Droot MODE=3D100644 SIZE=3D1537 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245895 OWNER=3Droot MODE=3D100644 SIZE=3D1398 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245896 OWNER=3Droot MODE=3D100644 SIZE=3D1791 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245897 OWNER=3Droot MODE=3D100644 SIZE=3D1695 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245898 OWNER=3Droot MODE=3D100644 SIZE=3D3247 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245899 OWNER=3Droot MODE=3D100644 SIZE=3D1611 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245900 OWNER=3Droot MODE=3D100644 SIZE=3D1574 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245901 OWNER=3Droot MODE=3D100644 SIZE=3D1547 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245902 OWNER=3Droot MODE=3D100644 SIZE=3D1290 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245903 OWNER=3Droot MODE=3D100644 SIZE=3D1663 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245904 OWNER=3Droot MODE=3D100644 SIZE=3D2523 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245905 OWNER=3Droot MODE=3D100644 SIZE=3D2422 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245906 OWNER=3Droot MODE=3D100644 SIZE=3D1838 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245907 OWNER=3Droot MODE=3D100644 SIZE=3D6196 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245908 OWNER=3Droot MODE=3D100644 SIZE=3D2142 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245909 OWNER=3Droot MODE=3D100644 SIZE=3D6504 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245910 OWNER=3Droot MODE=3D100644 SIZE=3D1496 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245911 OWNER=3Droot MODE=3D100644 SIZE=3D12392 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245912 OWNER=3Droot MODE=3D100644 SIZE=3D4488 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245913 OWNER=3Droot MODE=3D100644 SIZE=3D9072 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245914 OWNER=3Droot MODE=3D100644 SIZE=3D3084 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245915 OWNER=3Droot MODE=3D100644 SIZE=3D724 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245916 OWNER=3Droot MODE=3D100644 SIZE=3D15384 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245917 OWNER=3Droot MODE=3D100644 SIZE=3D1480 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245918 OWNER=3Droot MODE=3D100644 SIZE=3D2356 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245919 OWNER=3Droot MODE=3D100644 SIZE=3D1004 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245920 OWNER=3Droot MODE=3D100644 SIZE=3D6208 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245921 OWNER=3Droot MODE=3D100644 SIZE=3D1564 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245924 OWNER=3Droot MODE=3D100644 SIZE=3D3328 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245928 OWNER=3Droot MODE=3D100644 SIZE=3D9736 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245932 OWNER=3Droot MODE=3D100644 SIZE=3D10000 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245934 OWNER=3Droot MODE=3D100644 SIZE=3D2916 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245935 OWNER=3Droot MODE=3D100644 SIZE=3D8700 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245939 OWNER=3Droot MODE=3D100644 SIZE=3D583 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245942 OWNER=3Droot MODE=3D100644 SIZE=3D924 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245943 OWNER=3Droot MODE=3D100644 SIZE=3D11516 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245944 OWNER=3Droot MODE=3D100644 SIZE=3D2740 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245945 OWNER=3Droot MODE=3D100644 SIZE=3D1216 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245946 OWNER=3Droot MODE=3D100644 SIZE=3D976 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245947 OWNER=3Droot MODE=3D100644 SIZE=3D3224 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2245948 OWNER=3Droot MODE=3D100644 SIZE=3D43508 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2263643 OWNER=3Droot MODE=3D100644 SIZE=3D963 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2263644 OWNER=3Droot MODE=3D100644 SIZE=3D5622 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2263645 OWNER=3Droot MODE=3D100644 SIZE=3D175559 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2263646 OWNER=3Droot MODE=3D100644 SIZE=3D179163 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2263647 OWNER=3Droot MODE=3D100644 SIZE=3D5626 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2263648 OWNER=3Droot MODE=3D100644 SIZE=3D11652 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2263650 OWNER=3Droot MODE=3D100644 SIZE=3D3240 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2263651 OWNER=3Droot MODE=3D100644 SIZE=3D972 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2263652 OWNER=3Droot MODE=3D100644 SIZE=3D1656 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D2263653 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DNov 26 23:48 2010 RECONNECT? yes UNREF FILE I=3D4688357 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D4688359 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D4971615 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5136763 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181741 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181749 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181753 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181754 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181755 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181756 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181760 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181765 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181834 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181838 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181839 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181840 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181841 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181859 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181872 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181876 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181877 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181878 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181879 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181893 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181904 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181908 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181909 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181910 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181911 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181929 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181933 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181937 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181938 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181939 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181940 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181945 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181949 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181950 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181951 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181952 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181956 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181963 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181967 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181968 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181969 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181970 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181974 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181981 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181985 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181986 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181987 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181988 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181992 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5181999 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182003 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182004 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182005 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182006 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182010 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182017 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182021 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182022 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182023 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182024 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182028 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182035 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182039 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182040 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182041 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182042 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182046 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182053 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182057 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182058 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182059 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182060 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182064 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182068 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182086 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182090 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182091 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182092 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182093 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182095 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182096 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182097 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182098 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182100 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182103 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:29 2010 RECONNECT? yes UNREF FILE I=3D5182107 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182108 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182109 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182110 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182120 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182129 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182133 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182134 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182135 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182136 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182152 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182164 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182168 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182169 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182170 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182171 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182189 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182202 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182206 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182207 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182208 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182209 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182225 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes UNREF FILE I=3D5182237 OWNER=3Droot MODE=3D100644 SIZE=3D0 MTIME=3DDec 4 17:49 2010 RECONNECT? yes ** Phase 5 - Check Cyl groups FREE BLK COUNT(S) WRONG IN SUPERBLK SALVAGE? yes SUMMARY INFORMATION BAD SALVAGE? yes BLK(S) MISSING IN BIT MAPS SALVAGE? yes 1545806 files, 7224118 used, 48420593 free (236321 frags, 6023034 blocks, 0.4% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** ***** FILE SYSTEM WAS MODIFIED ***** ** /dev/ad0s1d ** Last Mounted on /var ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 22935 files, 194816 used, 1834215 free (303 frags, 229239 blocks, 0.0% fragmentation) ***** FILE SYSTEM MARKED CLEAN ***** From owner-freebsd-fs@FreeBSD.ORG Sun Dec 5 05:42:27 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB192106564A for ; Sun, 5 Dec 2010 05:42:27 +0000 (UTC) (envelope-from joe@netmusician.org) Received: from mail.netmusician.org (dorian.netmusician.org [66.244.95.101]) by mx1.freebsd.org (Postfix) with ESMTP id 925128FC0C for ; Sun, 5 Dec 2010 05:42:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netmusician.org (Postfix) with ESMTP id BEAAFB990 for ; Sun, 5 Dec 2010 00:24:15 -0500 (EST) X-Virus-Scanned: amavisd-new at netmusician.org Received: from mail.netmusician.org ([127.0.0.1]) by localhost (dorian.netmusician.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id InA2SE1wHAHz for ; Sun, 5 Dec 2010 00:24:15 -0500 (EST) Received: from Shakti.local (c-71-201-100-167.hsd1.in.comcast.net [71.201.100.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.netmusician.org (Postfix) with ESMTPSA id 3B286B98F for ; Sun, 5 Dec 2010 00:24:15 -0500 (EST) Message-ID: <4CFB21FC.9080905@netmusician.org> Date: Sun, 05 Dec 2010 00:24:12 -0500 From: Joe Auty User-Agent: Postbox 2.0.2 (Macintosh/20101025) MIME-Version: 1.0 To: freebsd-fs@freebsd.org X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Dec 2010 05:42:28 -0000 Hello, This is possibly a more fundamental non-FreeBSD specific set of questions, but ultimately this is relevant to usage on FreeBSD, so... I'm fairly certain that NFSv4 is supported under Solaris 10/ZFS and FreeBSD/ZFS via the standard "share" binary or the sharenfs ZFS property, right? In mounting an NFS share on my FreeBSD test machine via the following: mount -t nfs -o rw,nfsv4 ipaddress:/share /path/to/share/directory I'm unable to change the permissions of any of these files via a standard chmod on the client (FreeBSD) side. What are NFSv4 ACLs, and is this in any way relevant to my problem here? Do ACLs need to be set in order to use a volume like I can an NFSv3 volume, which works just fine for me? Or, if you wish to address this question in a different way: what do I have to do to get a volume I can mount as an NFSv3 volume and work on without any problems to work as an NFSv4 volume? I'm using FBSD 8.1 on this test box. -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe@netmusician.org From owner-freebsd-fs@FreeBSD.ORG Sun Dec 5 14:17:45 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 175C0106564A for ; Sun, 5 Dec 2010 14:17:45 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id C39C08FC08 for ; Sun, 5 Dec 2010 14:17:44 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAG4t+0yDaFvO/2dsb2JhbACDT6BTrR6PZYEhgzVzBIRfhg8 X-IronPort-AV: E=Sophos;i="4.59,302,1288584000"; d="scan'208";a="103115640" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 05 Dec 2010 09:17:43 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id A3EB1B3E95; Sun, 5 Dec 2010 09:17:43 -0500 (EST) Date: Sun, 5 Dec 2010 09:17:43 -0500 (EST) From: Rick Macklem To: =?utf-8?Q?Tom=C3=A1=C5=A1_Drbohlav?= Message-ID: <466626878.1184716.1291558663551.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4CF8CC96.3040901@karlov.mff.cuni.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [174.114.46.215] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - SAF3 (Mac)/6.0.7_GA_2473.RHEL4_64) Cc: freebsd-fs@freebsd.org Subject: Re: Linux (client) - FreeBSD (server) NFS4 interoperability problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Dec 2010 14:17:45 -0000 > >> > >> Although I am not an nfs expert, when I opened the dump in > >> WireShark > >> it > >> seems to me, that server does not respond to packet #83, Compound > >> Call > >> containing OPEN opcode. That is probably the thing which disturbes > >> Linux > >> client. But why? It is beyond my capabilities. > >> > >> What can I do to make i work? Is it me or server, who si broken? > >> > > I'll take a look at the packet dump when I get home in a couple of > > days. > > Thank, we appreciate any help! > > > The main thing I'd suggest is trying a non-Kerberized NFSv4 > > mount from Linux->FreeBSD and see how that does? > > Looks almost the same to me. Attaching dump/ Ok, I took a look at the dump and it's kinda interesting. Here's what happens: - Linux client does a SetclientID (#51) specifying a callback via TCP at port 37372. - When client does an Open at #53, the server tries to establish a TCP connection for the callback. (See SYN packets #56, 58-64, 72) - The Linux client does not respond to these SYNs. - At #70, client gives up waiting for the Open reply (which hasn't happened yet because the server hasn't timed out on the TCP connect for the callback) and disconnects the main TCP NFS connection. So, I can think of a couple of ways this can be worked around. 1 - Figure out why the client isn't responding too the SYNs for the callback path and fix it so it connects. (Is port 37372 somehow blocked for the client?) 2 - Disable the server from attempting to do a callback connection even when the client advertises one. (At the moment this can only be done for Kerberized mounts by setting nfsrv_nogsscallback == 1 in sys/fs/nfsserver/nfs_nfsdstate.c. (Maybe I should add a sysctl to disable callback attempts?) 3 - Reconfiguring the client so that it doesn't advertise a callback path. (I have no idea how to do this for Linux, but there is a nfsv4@linux-nfs.org email list where someone might be able to help with doing this?) Basically, it's hard to say if the bug is the server trying to establish the callback TCP connection for too long or the client not waiting long enough for the Open Reply. (The server will eventually give up on trying to establish the callback TCP connection and will then reply to the client. It could also be argued that the server should reply NFS4ERR_DELAY to the Open request while it is trying to establish the TCP connection, if it doesn't happen quickly.) I'll look at the timeout for the callback TCP connect and see if I can make it smaller easily. Not having a callback path doesn't break NFSv4, it just disables issuing of delegations, which aren't currently enabled by default in the FreeBSD server at this time anyhow. If I can come up with a simple patch to reduce the timeout on the TCP connect for the callback, I'll email you that for testing. rick From owner-freebsd-fs@FreeBSD.ORG Sun Dec 5 18:20:55 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B0917106566B for ; Sun, 5 Dec 2010 18:20:55 +0000 (UTC) (envelope-from universite@ukr.net) Received: from otrada.od.ua (universite-1-pt.tunnel.tserv24.sto1.ipv6.he.net [IPv6:2001:470:27:140::2]) by mx1.freebsd.org (Postfix) with ESMTP id 22CE08FC21 for ; Sun, 5 Dec 2010 18:20:54 +0000 (UTC) Received: from [10.0.0.10] (phenom.otrada.od.ua [10.0.0.10]) (authenticated bits=0) by otrada.od.ua (8.14.3/8.14.4) with ESMTP id oB5IKoEo098399 for ; Sun, 5 Dec 2010 20:20:50 +0200 (EET) (envelope-from universite@ukr.net) X-Authentication-Warning: otrada.od.ua: Host phenom.otrada.od.ua [10.0.0.10] claimed to be [10.0.0.10] Message-ID: <4CFBD7F9.3000307@ukr.net> Date: Sun, 05 Dec 2010 20:20:41 +0200 From: "Vladislav V. Prodan" User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; ru; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,AWL autolearn=failed version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on mary-teresa.otrada.od.ua X-Virus-Scanned: clamav-milter 0.95.3 at mary-teresa.otrada.od.ua X-Virus-Status: Clean Subject: How to choose the size of swap? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Dec 2010 18:20:55 -0000 We have several servers with 8|16 GB of memory, 2|4 harddrive (RAID1|RAID10 using ZFS) These servers will work with Mysql, the size of databases that are several times larger than the size of memory. What would recommend to expose the size of swap? On a separate partition or a file? At hand is not loaded with servers that have the swap was over 20 MB ... In an extreme case, let the output "top v" to this maillist -- Vladislav V. Prodan VVP24-UANIC +38[067]4584408 +38[099]4060508 vlad11@jabber.ru From owner-freebsd-fs@FreeBSD.ORG Sun Dec 5 19:09:34 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5E6E71065670 for ; Sun, 5 Dec 2010 19:09:34 +0000 (UTC) (envelope-from ronald-freebsd8@klop.yi.org) Received: from fep21.mx.upcmail.net (fep21.mx.upcmail.net [62.179.121.41]) by mx1.freebsd.org (Postfix) with ESMTP id 93A208FC0C for ; Sun, 5 Dec 2010 19:09:33 +0000 (UTC) Received: from edge05.upcmail.net ([192.168.13.212]) by viefep15-int.chello.at (InterMail vM.8.01.02.02 201-2260-120-106-20100312) with ESMTP id <20101205185305.QNKX1633.viefep15-int.chello.at@edge05.upcmail.net> for ; Sun, 5 Dec 2010 19:53:05 +0100 Received: from pinky ([213.46.23.80]) by edge05.upcmail.net with edge id fWt41f01F1jgp3H05Wt5jF; Sun, 05 Dec 2010 19:53:05 +0100 X-SourceIP: 213.46.23.80 Content-Type: text/plain; charset=us-ascii; format=flowed; delsp=yes To: freebsd-fs@freebsd.org References: <4CFBD7F9.3000307@ukr.net> Date: Sun, 05 Dec 2010 19:53:04 +0100 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit From: "Ronald Klop" Message-ID: In-Reply-To: <4CFBD7F9.3000307@ukr.net> User-Agent: Opera Mail/10.63 (Win32) X-Cloudmark-Analysis: v=1.1 cv=JvXQbuMnWGQeb488dJ7w43Du7THgE+O7ieb9U20/rjk= c=1 sm=0 a=egFB3xdmDTgA:10 a=bgpUlknNv7MA:10 a=kj9zAlcOel0A:10 a=Zvg7VmqrAAAA:8 a=qNW38mDyJ86A-iIDrhoA:9 a=DCCK55qrI-VIbiKVmWqZvf_GnosA:4 a=CjuIK1q_8ugA:10 a=SWIUPOSDCbIA:10 a=HpAAvcLHHh0Zw7uRqdWCyQ==:117 Subject: Re: How to choose the size of swap? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Dec 2010 19:09:34 -0000 On Sun, 05 Dec 2010 19:20:41 +0100, Vladislav V. Prodan wrote: > We have several servers with 8|16 GB of memory, 2|4 harddrive > (RAID1|RAID10 using ZFS) > These servers will work with Mysql, the size of databases that are > several times larger than the size of memory. > > What would recommend to expose the size of swap? > On a separate partition or a file? > At hand is not loaded with servers that have the swap was over 20 MB ... > In an extreme case, let the output "top v" to this maillist You might not need swap to run the database. But the swap space is also used for making a kernel dump in case of a problem. In that case swap space must be as large as the amount of memory in the machine. (As far as I know.) Ronald. From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 03:08:44 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC6BF1065672 for ; Mon, 6 Dec 2010 03:08:44 +0000 (UTC) (envelope-from james-freebsd-fs2@jrv.org) Received: from mail.jrv.org (adsl-70-243-84-13.dsl.austtx.swbell.net [70.243.84.13]) by mx1.freebsd.org (Postfix) with ESMTP id 8963C8FC0A for ; Mon, 6 Dec 2010 03:08:44 +0000 (UTC) Received: from kremvax.housenet.jrv (kremvax.housenet.jrv [192.168.3.124]) by mail.jrv.org (8.14.3/8.14.3) with ESMTP id oB62e8i4069697 for ; Sun, 5 Dec 2010 20:40:08 -0600 (CST) (envelope-from james-freebsd-fs2@jrv.org) Authentication-Results: mail.jrv.org; domainkeys=pass (testing) header.from=james-freebsd-fs2@jrv.org DomainKey-Signature: a=rsa-sha1; s=enigma; d=jrv.org; c=nofws; q=dns; h=message-id:date:from:user-agent:mime-version:to:subject: content-type:content-transfer-encoding; b=CZiOMJhN6Nlx9anc7KIyiA2LDtppYxYKjEk0D7UxNHyXB1OaIZ977dZ7sP5hwLHMz Yto+2Mg5ZPCZNvgGIcOMCQ5O7fkKrqodTTkmc+71NSNzaegGadsGrCbntX9VAw1KO5Q zpQBgpyGuifa9bwnrN9Ix3b5JO4qO1grZgs3zM0= Message-ID: <4CFC4D08.5060004@jrv.org> Date: Sun, 05 Dec 2010 20:40:08 -0600 From: "James R. Van Artsdalen" User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: freebsd-fs Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: zpool suggests "installgrub"? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 03:08:45 -0000 FreeBSD H55ITX.housenet.jrv 9.0-CURRENT FreeBSD 9.0-CURRENT #2 r216088: Thu Dec 2 20:21:21 UTC 2010 root@H55ITX.housenet.jrv:/usr/obj/usr/src/sys/GENERIC amd64 H55ITX:/root# zpool attach H55ITX ada4p3 ada5p3 Please be sure to invoke installgrub(1M) to make 'ada5p3' bootable. H55ITX:/root# >From cddl/contrib/opensolaris/lib/libzfs/common/libzfs_pool.c #if defined(__i386) || defined(__amd64) #define BOOTCMD "installgrub(1M)" #else #define BOOTCMD "installboot(1M)" #endif ... if (rootpool) { /* * XXX - This should be removed once we can * automatically install the bootblocks on the * newly attached disk. */ (void) fprintf(stderr, dgettext(TEXT_DOMAIN, "Please " "be sure to invoke %s to make '%s' bootable.\n"), BOOTCMD, new_disk); } return (0); From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 10:18:32 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3016B1065670 for ; Mon, 6 Dec 2010 10:18:30 +0000 (UTC) (envelope-from drb@karlov.mff.cuni.cz) Received: from mail.karlov.mff.cuni.cz (mail.karlov.mff.cuni.cz [195.113.27.114]) by mx1.freebsd.org (Postfix) with ESMTP id AAA0F8FC0A for ; Mon, 6 Dec 2010 10:18:30 +0000 (UTC) Received: from nat-r.karlov.mff.cuni.cz ([195.113.27.73] helo=[10.32.82.27]) by mail.karlov.mff.cuni.cz with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.68 (FreeBSD)) (envelope-from ) id 1PPY9M-000L3c-LJ for freebsd-fs@freebsd.org; Mon, 06 Dec 2010 11:18:28 +0100 Message-ID: <4CFCB86F.6090800@karlov.mff.cuni.cz> Date: Mon, 06 Dec 2010 11:18:23 +0100 From: =?UTF-8?B?VG9tw6HFoSBEcmJvaGxhdg==?= User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.12) Gecko/20101027 Lightning/1.0b2 Thunderbird/3.1.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <466626878.1184716.1291558663551.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <466626878.1184716.1291558663551.JavaMail.root@erie.cs.uoguelph.ca> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: Linux (client) - FreeBSD (server) NFS4 interoperability problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 10:18:32 -0000 On 5.12.2010 15:17, Rick Macklem wrote: >>>> >>>> Although I am not an nfs expert, when I opened the dump in >>>> WireShark >>>> it >>>> seems to me, that server does not respond to packet #83, Compound >>>> Call >>>> containing OPEN opcode. That is probably the thing which disturbes >>>> Linux >>>> client. But why? It is beyond my capabilities. >>>> >>>> What can I do to make i work? Is it me or server, who si broken? >>>> >>> I'll take a look at the packet dump when I get home in a couple of >>> days. >> >> Thank, we appreciate any help! >> >>> The main thing I'd suggest is trying a non-Kerberized NFSv4 >>> mount from Linux->FreeBSD and see how that does? >> >> Looks almost the same to me. Attaching dump/ > > Ok, I took a look at the dump and it's kinda interesting. Here's > what happens: > - Linux client does a SetclientID (#51) specifying a callback > via TCP at port 37372. > - When client does an Open at #53, the server tries to establish > a TCP connection for the callback. (See SYN packets #56, 58-64, 72) > - The Linux client does not respond to these SYNs. > - At #70, client gives up waiting for the Open reply (which hasn't > happened yet because the server hasn't timed out on the TCP connect > for the callback) and disconnects the main TCP NFS connection. thanks, that sounds reasonable, the last disconnect can be unmount on client side, which we made at the end... in previous test we let it wait more then few minutes (which should enough for timeout I guess) > So, I can think of a couple of ways this can be worked around. > 1 - Figure out why the client isn't responding too the SYNs for > the callback path and fix it so it connects. (Is port 37372 > somehow blocked for the client?) traffic may be blocked, but we do not want server to make callbacks anyway (we are exporting data which are being modified by Samba or locally so no delegations and server is going to be access through NAT) > 2 - Disable the server from attempting to do a callback connection > even when the client advertises one. (At the moment this can > only be done for Kerberized mounts by setting nfsrv_nogsscallback == 1 > in sys/fs/nfsserver/nfs_nfsdstate.c. (Maybe I should add a sysctl > to disable callback attempts?) I have try it and it solved our other problem (small delay after first file open after connection), probably connected with server trying to establish callback. I will check with Linux client and let you know, but this si what we needed anyway. Sysctl is a good idea, btw man nfsv4 mentions vfs.newnfs.issue_delegations ... what is the purpose of callback other than delegation? > 3 - Reconfiguring the client so that it doesn't advertise a callback path. > (I have no idea how to do this for Linux, but there is a > nfsv4@linux-nfs.org email list where someone might be able to help > with doing this?) will try, but it is not good for us anyway, as we do not control all the clients so we want be resistant against discussed lockup. Thank for now, I will let you know about the tests. Drb > Basically, it's hard to say if the bug is the server trying to establish > the callback TCP connection for too long or the client not waiting long > enough for the Open Reply. (The server will eventually give up on trying > to establish the callback TCP connection and will then reply to the > client. It could also be argued that the server should reply NFS4ERR_DELAY > to the Open request while it is trying to establish the TCP connection, > if it doesn't happen quickly.) I'll look at the timeout for the callback > TCP connect and see if I can make it smaller easily. Not having a > callback path doesn't break NFSv4, it just disables issuing of delegations, > which aren't currently enabled by default in the FreeBSD server at this > time anyhow. > > If I can come up with a simple patch to reduce the timeout on the TCP > connect for the callback, I'll email you that for testing. > > rick > From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 11:06:58 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4642D10656E4 for ; Mon, 6 Dec 2010 11:06:58 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 188DB8FC08 for ; Mon, 6 Dec 2010 11:06:58 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.4/8.14.4) with ESMTP id oB6B6vAd068229 for ; Mon, 6 Dec 2010 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.4/8.14.4/Submit) id oB6B6vbq068227 for freebsd-fs@FreeBSD.org; Mon, 6 Dec 2010 11:06:57 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 6 Dec 2010 11:06:57 GMT Message-Id: <201012061106.oB6B6vbq068227@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-fs@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 11:06:58 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/152488 fs [tmpfs] [patch] mtime of file updated when only inode o kern/152079 fs [msdosfs] [patch] Small cleanups from the other NetBSD o kern/152022 fs [nfs] nfs service hangs with linux client [regression] o kern/151942 fs [zfs] panic during ls(1) zfs snapshot directory o kern/151905 fs [zfs] page fault under load in /sbin/zfs o kern/151845 fs [smbfs] [patch] smbfs should be upgraded to support Un o bin/151713 fs [patch] Bug in growfs(8) with respect to 32-bit overfl o kern/151648 fs [zfs] disk wait bug o kern/151629 fs [fs] [patch] Skip empty directory entries during name o kern/151330 fs [zfs] will unshare all zfs filesystem after execute a o kern/151326 fs [nfs] nfs exports fail if netgroups contain duplicate o kern/151251 fs [ufs] Can not create files on filesystem with heavy us o kern/151226 fs [zfs] can't delete zfs snapshot o kern/151111 fs [zfs] vnodes leakage during zfs unmount o kern/150796 fs [panic] [suj] [ufs] [softupdates] Panic on portbuild o kern/150503 fs [zfs] ZFS disks are UNAVAIL and corrupted after reboot o kern/150501 fs [zfs] ZFS vdev failure vdev.bad_label on amd64 o kern/150390 fs [zfs] zfs deadlock when arcmsr reports drive faulted o kern/150336 fs [nfs] mountd/nfsd became confused; refused to reload n o kern/150207 fs zpool(1): zpool import -d /dev tries to open weird dev o kern/149208 fs mksnap_ffs(8) hang/deadlock o kern/149173 fs [patch] [zfs] make OpenSolaris installa o kern/149022 fs [hang] File system operations hangs with suspfs state o kern/149015 fs [zfs] [patch] misc fixes for ZFS code to build on Glib o kern/149014 fs [zfs] [patch] declarations in ZFS libraries/utilities o kern/149013 fs [zfs] [patch] make ZFS makefiles use the libraries fro o kern/148504 fs [zfs] ZFS' zpool does not allow replacing drives to be o kern/148490 fs [zfs]: zpool attach - resilver bidirectionally, and re o kern/148368 fs [zfs] ZFS hanging forever on 8.1-PRERELEASE o bin/148296 fs [zfs] [loader] [patch] Very slow probe in /usr/src/sys o kern/148204 fs [nfs] UDP NFS causes overload o kern/148138 fs [zfs] zfs raidz pool commands freeze o kern/147903 fs [zfs] [panic] Kernel panics on faulty zfs device o kern/147881 fs [zfs] [patch] ZFS "sharenfs" doesn't allow different " o kern/147790 fs [zfs] zfs set acl(mode|inherit) fails on existing zfs o kern/147560 fs [zfs] [boot] Booting 8.1-PRERELEASE raidz system take o kern/147420 fs [ufs] [panic] ufs_dirbad, nullfs, jail panic (corrupt o kern/146941 fs [zfs] [panic] Kernel Double Fault - Happens constantly o kern/146786 fs [zfs] zpool import hangs with checksum errors o kern/146708 fs [ufs] [panic] Kernel panic in softdep_disk_write_compl o kern/146528 fs [zfs] Severe memory leak in ZFS on i386 o kern/146502 fs [nfs] FreeBSD 8 NFS Client Connection to Server o kern/146375 fs [nfs] [patch] Typos in macro variables names in sys/fs s kern/145712 fs [zfs] cannot offline two drives in a raidz2 configurat o kern/145411 fs [xfs] [panic] Kernel panics shortly after mounting an o bin/145309 fs bsdlabel: Editing disk label invalidates the whole dev o kern/145272 fs [zfs] [panic] Panic during boot when accessing zfs on o kern/145246 fs [ufs] dirhash in 7.3 gratuitously frees hashes when it o kern/145238 fs [zfs] [panic] kernel panic on zpool clear tank o kern/145229 fs [zfs] Vast differences in ZFS ARC behavior between 8.0 o kern/145189 fs [nfs] nfsd performs abysmally under load o kern/144929 fs [ufs] [lor] vfs_bio.c + ufs_dirhash.c o kern/144458 fs [nfs] [patch] nfsd fails as a kld p kern/144447 fs [zfs] sharenfs fsunshare() & fsshare_main() non functi o kern/144416 fs [panic] Kernel panic on online filesystem optimization s kern/144415 fs [zfs] [panic] kernel panics on boot after zfs crash o kern/144234 fs [zfs] Cannot boot machine with recent gptzfsboot code o kern/143825 fs [nfs] [panic] Kernel panic on NFS client o bin/143572 fs [zfs] zpool(1): [patch] The verbose output from iostat o kern/143345 fs [ext2fs] [patch] extfs minor header cleanups to better o kern/143212 fs [nfs] NFSv4 client strange work ... o kern/143184 fs [zfs] [lor] zfs/bufwait LOR o kern/142924 fs [ext2fs] [patch] Small cleanup for the inode struct in o kern/142914 fs [zfs] ZFS performance degradation over time o kern/142878 fs [zfs] [vfs] lock order reversal o kern/142597 fs [ext2fs] ext2fs does not work on filesystems with real o kern/142489 fs [zfs] [lor] allproc/zfs LOR o kern/142466 fs Update 7.2 -> 8.0 on Raid 1 ends with screwed raid [re o kern/142401 fs [ntfs] [patch] Minor updates to NTFS from NetBSD o kern/142306 fs [zfs] [panic] ZFS drive (from OSX Leopard) causes two o kern/142068 fs [ufs] BSD labels are got deleted spontaneously o kern/141897 fs [msdosfs] [panic] Kernel panic. msdofs: file name leng o kern/141463 fs [nfs] [panic] Frequent kernel panics after upgrade fro o kern/141305 fs [zfs] FreeBSD ZFS+sendfile severe performance issues ( o kern/141091 fs [patch] [nullfs] fix panics with DIAGNOSTIC enabled o kern/141086 fs [nfs] [panic] panic("nfs: bioread, not dir") on FreeBS o kern/141010 fs [zfs] "zfs scrub" fails when backed by files in UFS2 o kern/140888 fs [zfs] boot fail from zfs root while the pool resilveri o kern/140661 fs [zfs] [patch] /boot/loader fails to work on a GPT/ZFS- o kern/140640 fs [zfs] snapshot crash o kern/140134 fs [msdosfs] write and fsck destroy filesystem integrity o kern/140068 fs [smbfs] [patch] smbfs does not allow semicolon in file o kern/139725 fs [zfs] zdb(1) dumps core on i386 when examining zpool c o kern/139715 fs [zfs] vfs.numvnodes leak on busy zfs o bin/139651 fs [nfs] mount(8): read-only remount of NFS volume does n o kern/139597 fs [patch] [tmpfs] tmpfs initializes va_gen but doesn't u o kern/139564 fs [zfs] [panic] 8.0-RC1 - Fatal trap 12 at end of shutdo o kern/139407 fs [smbfs] [panic] smb mount causes system crash if remot o kern/138790 fs [zfs] ZFS ceases caching when mem demand is high o kern/138662 fs [panic] ffs_blkfree: freeing free block o kern/138421 fs [ufs] [patch] remove UFS label limitations o kern/138202 fs mount_msdosfs(1) see only 2Gb o kern/136968 fs [ufs] [lor] ufs/bufwait/ufs (open) o kern/136945 fs [ufs] [lor] filedesc structure/ufs (poll) o kern/136944 fs [ffs] [lor] bufwait/snaplk (fsync) o kern/136873 fs [ntfs] Missing directories/files on NTFS volume o kern/136865 fs [nfs] [patch] NFS exports atomic and on-the-fly atomic o kern/136470 fs [nfs] Cannot mount / in read-only, over NFS o kern/135667 fs [lor] LORs causing ufs filesystem corruption on XEN Do o kern/135546 fs [zfs] zfs.ko module doesn't ignore zpool.cache filenam o kern/135469 fs [ufs] [panic] kernel crash on md operation in ufs_dirb o kern/135050 fs [zfs] ZFS clears/hides disk errors on reboot o kern/134491 fs [zfs] Hot spares are rather cold... o kern/133676 fs [smbfs] [panic] umount -f'ing a vnode-based memory dis o kern/133614 fs [panic] panic: ffs_truncate: read-only filesystem o kern/133174 fs [msdosfs] [patch] msdosfs must support utf-encoded int o kern/132960 fs [ufs] [panic] panic:ffs_blkfree: freeing free frag o kern/132397 fs reboot causes filesystem corruption (failure to sync b o kern/132331 fs [ufs] [lor] LOR ufs and syncer o kern/132237 fs [msdosfs] msdosfs has problems to read MSDOS Floppy o kern/132145 fs [panic] File System Hard Crashes o kern/131441 fs [unionfs] [nullfs] unionfs and/or nullfs not combineab o kern/131360 fs [nfs] poor scaling behavior of the NFS server under lo o kern/131342 fs [nfs] mounting/unmounting of disks causes NFS to fail o bin/131341 fs makefs: error "Bad file descriptor" on the mount poin o kern/130920 fs [msdosfs] cp(1) takes 100% CPU time while copying file o kern/130210 fs [nullfs] Error by check nullfs o kern/129760 fs [nfs] after 'umount -f' of a stale NFS share FreeBSD l o kern/129488 fs [smbfs] Kernel "bug" when using smbfs in smbfs_smb.c: o kern/129231 fs [ufs] [patch] New UFS mount (norandom) option - mostly o kern/129152 fs [panic] non-userfriendly panic when trying to mount(8) o kern/127787 fs [lor] [ufs] Three LORs: vfslock/devfs/vfslock, ufs/vfs o bin/127270 fs fsck_msdosfs(8) may crash if BytesPerSec is zero o kern/127029 fs [panic] mount(8): trying to mount a write protected zi o kern/126287 fs [ufs] [panic] Kernel panics while mounting an UFS file o kern/125895 fs [ffs] [panic] kernel: panic: ffs_blkfree: freeing free s kern/125738 fs [zfs] [request] SHA256 acceleration in ZFS p kern/124621 fs [ext3] [patch] Cannot mount ext2fs partition o kern/123939 fs [msdosfs] corrupts new files o kern/122380 fs [ffs] ffs_valloc:dup alloc (Soekris 4801/7.0/USB Flash o bin/122172 fs [fs]: amd(8) automount daemon dies on 6.3-STABLE i386, o bin/121898 fs [nullfs] pwd(1)/getcwd(2) fails with Permission denied o bin/121779 fs [ufs] snapinfo(8) (and related tools?) only work for t o bin/121366 fs [zfs] [patch] Automatic disk scrubbing from periodic(8 o bin/121072 fs [smbfs] mount_smbfs(8) cannot normally convert the cha f kern/120991 fs [panic] [ffs] [snapshot] System crashes when manipulat o kern/120483 fs [ntfs] [patch] NTFS filesystem locking changes o kern/120482 fs [ntfs] [patch] Sync style changes between NetBSD and F o kern/118912 fs [2tb] disk sizing/geometry problem with large array o kern/118713 fs [minidump] [patch] Display media size required for a k o bin/118249 fs [ufs] mv(1): moving a directory changes its mtime o kern/118107 fs [ntfs] [panic] Kernel panic when accessing a file at N o kern/117954 fs [ufs] dirhash on very large directories blocks the mac o bin/117315 fs [smbfs] mount_smbfs(8) and related options can't mount o kern/117314 fs [ntfs] Long-filename only NTFS fs'es cause kernel pani o kern/117158 fs [zfs] zpool scrub causes panic if geli vdevs detach on o bin/116980 fs [msdosfs] [patch] mount_msdosfs(8) resets some flags f o conf/116931 fs lack of fsck_cd9660 prevents mounting iso images with p kern/116608 fs [msdosfs] [patch] msdosfs fails to check mount options o kern/116583 fs [ffs] [hang] System freezes for short time when using o kern/116170 fs [panic] Kernel panic when mounting /tmp o bin/115361 fs [zfs] mount(8) gets into a state where it won't set/un o kern/114955 fs [cd9660] [patch] [request] support for mask,dirmask,ui o kern/114847 fs [ntfs] [patch] [request] dirmask support for NTFS ala o kern/114676 fs [ufs] snapshot creation panics: snapacct_ufs2: bad blo o bin/114468 fs [patch] [request] add -d option to umount(8) to detach o kern/113852 fs [smbfs] smbfs does not properly implement DFS referral o bin/113838 fs [patch] [request] mount(8): add support for relative p o bin/113049 fs [patch] [request] make quot(8) use getopt(3) and show o kern/112658 fs [smbfs] [patch] smbfs and caching problems (resolves b o kern/111843 fs [msdosfs] Long Names of files are incorrectly created o kern/111782 fs [ufs] dump(8) fails horribly for large filesystems s bin/111146 fs [2tb] fsck(8) fails on 6T filesystem o kern/109024 fs [msdosfs] [iconv] mount_msdosfs: msdosfs_iconv: Operat o kern/109010 fs [msdosfs] can't mv directory within fat32 file system o bin/107829 fs [2TB] fdisk(8): invalid boundary checking in fdisk / w o kern/106107 fs [ufs] left-over fsck_snapshot after unfinished backgro o kern/106030 fs [ufs] [panic] panic in ufs from geom when a dead disk o kern/104406 fs [ufs] Processes get stuck in "ufs" state under persist o kern/104133 fs [ext2fs] EXT2FS module corrupts EXT2/3 filesystems o kern/103035 fs [ntfs] Directories in NTFS mounted disc images appear o kern/101324 fs [smbfs] smbfs sometimes not case sensitive when it's s o kern/99290 fs [ntfs] mount_ntfs ignorant of cluster sizes s bin/97498 fs [request] newfs(8) has no option to clear the first 12 o kern/97377 fs [ntfs] [patch] syntax cleanup for ntfs_ihash.c o kern/95222 fs [cd9660] File sections on ISO9660 level 3 CDs ignored o kern/94849 fs [ufs] rename on UFS filesystem is not atomic o bin/94810 fs fsck(8) incorrectly reports 'file system marked clean' o kern/94769 fs [ufs] Multiple file deletions on multi-snapshotted fil o kern/94733 fs [smbfs] smbfs may cause double unlock o bin/94635 fs snapinfo(8)/libufs only works for disk-backed filesyst o kern/93942 fs [vfs] [patch] panic: ufs_dirbad: bad dir (patch from D o kern/92272 fs [ffs] [hang] Filling a filesystem while creating a sna o kern/91134 fs [smbfs] [patch] Preserve access and modification time a kern/90815 fs [smbfs] [patch] SMBFS with character conversions somet o kern/88657 fs [smbfs] windows client hang when browsing a samba shar o kern/88555 fs [panic] ffs_blkfree: freeing free frag on AMD 64 o kern/88266 fs [smbfs] smbfs does not implement UIO_NOCOPY and sendfi o bin/87966 fs [patch] newfs(8): introduce -A flag for newfs to enabl o kern/87859 fs [smbfs] System reboot while umount smbfs. o kern/86587 fs [msdosfs] rm -r /PATH fails with lots of small files o bin/85494 fs fsck_ffs: unchecked use of cg_inosused macro etc. f kern/85326 fs [smbfs] [panic] saving a file via samba to an overquot o kern/80088 fs [smbfs] Incorrect file time setting on NTFS mounted vi o bin/74779 fs Background-fsck checks one filesystem twice and omits o kern/73484 fs [ntfs] Kernel panic when doing `ls` from the client si o bin/73019 fs [ufs] fsck_ufs(8) cannot alloc 607016868 bytes for ino o kern/71774 fs [ntfs] NTFS cannot "see" files on a WinXP filesystem o bin/70600 fs fsck(8) throws files away when it can't grow lost+foun o kern/68978 fs [panic] [ufs] crashes with failing hard disk, loose po o kern/65920 fs [nwfs] Mounted Netware filesystem behaves strange o kern/65901 fs [smbfs] [patch] smbfs fails fsx write/truncate-down/tr o kern/61503 fs [smbfs] mount_smbfs does not work as non-root o kern/55617 fs [smbfs] Accessing an nsmb-mounted drive via a smb expo o kern/51685 fs [hang] Unbounded inode allocation causes kernel to loc o kern/51583 fs [nullfs] [patch] allow to work with devices and socket o kern/36566 fs [smbfs] System reboot with dead smb mount and umount o kern/33464 fs [ufs] soft update inconsistencies after system crash o bin/27687 fs fsck(8) wrapper is not properly passing options to fsc o kern/18874 fs [2TB] 32bit NFS servers export wrong negative values t 210 problems total. From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 15:02:53 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A0726106564A for ; Mon, 6 Dec 2010 15:02:53 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 600218FC1C for ; Mon, 6 Dec 2010 15:02:53 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEALaJ/EyDaFvO/2dsb2JhbACDUKBZrRmQJ4EhgzVzBIRfhg8 X-IronPort-AV: E=Sophos;i="4.59,305,1288584000"; d="scan'208";a="103200705" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 06 Dec 2010 10:02:52 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 7C210B3E95; Mon, 6 Dec 2010 10:02:52 -0500 (EST) Date: Mon, 6 Dec 2010 10:02:52 -0500 (EST) From: Rick Macklem To: =?utf-8?Q?Tom=C3=A1=C5=A1_Drbohlav?= Message-ID: <1246727856.1218036.1291647772450.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4CFCB86F.6090800@karlov.mff.cuni.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [174.114.46.215] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - SAF3 (Mac)/6.0.7_GA_2473.RHEL4_64) Cc: freebsd-fs@freebsd.org Subject: Re: Linux (client) - FreeBSD (server) NFS4 interoperability problem X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 15:02:53 -0000 > > traffic may be blocked, but we do not want server to make callbacks > anyway (we are exporting data which are being modified by Samba or > locally so no delegations and server is going to be access through > NAT) > > > 2 - Disable the server from attempting to do a callback connection > > even when the client advertises one. (At the moment this can > > only be done for Kerberized mounts by setting > > nfsrv_nogsscallback == 1 > > in sys/fs/nfsserver/nfs_nfsdstate.c. (Maybe I should add a > > sysctl > > to disable callback attempts?) > > I have try it and it solved our other problem (small delay after first > file open after connection), probably connected with server trying to > establish callback. I will check with Linux client and let you know, > but > this si what we needed anyway. > > Sysctl is a good idea, btw man nfsv4 mentions > vfs.newnfs.issue_delegations ... what is the purpose of callback other > than delegation? > I think the correct solution is to set a short (about 1sec timeout) on the callback connection attempt when no delegations are issued to that client. Unfortunately that patch isn't trivial, since the timeout needs to happen inside of the krpc and there currently isn't a timeout argument that can be passed into it for this. At the moment, vfs.newnfs.issue_delegations == 0 doesn't disable attempting to make a callback connection when the client advertises one, but I agree that it could disable callbacks since, as you note, callbacks are only useful when delegations are being issued. I'll email you a patch that does this. Btw, the other patch shouldn't be in a production server, since it would time out all connection attempts and not just callback ones. (That was why I set the timeout in it at 30sec instead of 1sec. It was just meant to test to see if the Linux client could Open successfully if the callback connection attempt timed out in less than the 1minute it waits before giving up on waiting for a reply.) rick From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 16:19:36 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA25A1065696 for ; Mon, 6 Dec 2010 16:19:36 +0000 (UTC) (envelope-from patpro@patpro.net) Received: from rack.patpro.net (rack.patpro.net [193.30.227.216]) by mx1.freebsd.org (Postfix) with ESMTP id 174ED8FC20 for ; Mon, 6 Dec 2010 16:19:35 +0000 (UTC) Received: from rack.patpro.net (localhost [127.0.0.1]) by rack.patpro.net (Postfix) with ESMTP id 3C85F1CC036 for ; Mon, 6 Dec 2010 17:00:16 +0100 (CET) X-Virus-Scanned: amavisd-new at patpro.net Received: from amavis-at-patpro.net ([127.0.0.1]) by rack.patpro.net (rack.patpro.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RvCi+YoKOdLx for ; Mon, 6 Dec 2010 17:00:12 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) by rack.patpro.net (Postfix) with ESMTP for ; Mon, 6 Dec 2010 17:00:12 +0100 (CET) From: Patrick Proniewski Content-Type: multipart/signed; boundary=Apple-Mail-4--188600245; protocol="application/pkcs7-signature"; micalg=sha1 Date: Mon, 6 Dec 2010 17:00:12 +0100 Message-Id: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> To: freebsd-fs@freebsd.org Mime-Version: 1.0 (Apple Message framework v1082) X-Mailer: Apple Mail (2.1082) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: auto-mounting ZFS snapshots X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 16:19:37 -0000 --Apple-Mail-4--188600245 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hello, I'm using freebsd-snapshot to create/destroy a bunch of daily/weekly = snapshots of my ZFS datasets. I would like to make the automount work, so I've configured amd accordingly, but strangely it fails: $ uname -v FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC 2010 = root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC $ grep ^amd /etc/rc.conf amd_enable=3D"YES" amd_flags=3D"-a /.am -c 1800 -w 60 -l syslog /snap = /usr/local/etc/amd.map.snap" $ df /snap Filesystem 1K-blocks Used Avail Capacity Mounted on pid82732@rack:/snap 0 0 0 100% /snap $ cat /usr/local/etc/amd.map.snap /defaults type:=3Dprogram * mount:=3D"/usr/local/sbin/snapshot snapshot mount /`echo ${key} | = /usr/bin/sed -e 's;+;/;g'` ${fs}";\ unmount:=3D"/usr/local/sbin/snapshot snapshot umount ${fs}" $ snapshot list /home/patpro Filesystem User User% Snap Snap% Snapshot /home/patpro 748MB 0.1% 1024KB 0.0% daily.5 /home/patpro 748MB 0.1% 1024KB 0.0% weekly.0 /home/patpro 748MB 0.1% 1024KB 0.0% daily.4 /home/patpro 748MB 0.1% 1024KB 0.0% daily.3 /home/patpro 748MB 0.1% 1024KB 0.0% daily.2 /home/patpro 748MB 0.1% 1024KB 0.0% daily.1 /home/patpro 748MB 0.1% 1024KB 0.0% daily.0 few attempts: $ ls -l /snap/home/patpro:daily.0/testawk.sh ls: /snap/home/patpro:daily.0/testawk.sh: Unknown error: 2147483647 $ ls -l /snap/home/patpro:daily.0/testawk.sh ls: /snap/home/patpro:daily.0/testawk.sh: Operation timed out $ ls -l /snap/home+patpro:daily.0/testawk.sh ls: /snap/home+patpro:daily.0/testawk.sh: Unknown error: 2147483647 $ ls -l /snap/home+patpro:daily.0/testawk.sh ls: /snap/home+patpro:daily.0/testawk.sh: Operation timed out Any idea is greatly appreciated. thanks, Patrick --Apple-Mail-4--188600245-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 17:20:56 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A6DB2106566B for ; Mon, 6 Dec 2010 17:20:56 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 596D38FC31 for ; Mon, 6 Dec 2010 17:20:56 +0000 (UTC) Received: by yxh35 with SMTP id 35so6490742yxh.13 for ; Mon, 06 Dec 2010 09:20:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=VDn2XsmQ77EqiSaPEMXy/0fCG6aIiEr+hqCT8yZihQc=; b=Pj+ItcehpgUTCpwNoTV4f09TlOPOVHw5tcyvLXZKKULwp+dQjBDrWjlMNS6/iim4pg 51X28H0gJPa1sEX0SqTb6LfRbYblzpMsN7QfwpYpQKY6HK6WL/gw3FdRCDRW7MMzMbPP SPSDQkGNVQUCyk5NhxGlCRgQO72l0Z3Yvbjys= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=MYH8BzoPw2iIyDFlJp9CLGqk6jGkDrGwLiFvp29vIVM1yDukefkbpjnFCOUfUUu0J3 BgMgFkRHF8IvekYqqENvj1O+b1yXZ2yra5l8pTP01g+e1+sr1Mx5B19WTYiDmZcvq5cU 3UP5/jLXNziGdA5bTa5gzupe6LI8XoKkRx2Us= MIME-Version: 1.0 Received: by 10.90.88.20 with SMTP id l20mr8162297agb.57.1291656055329; Mon, 06 Dec 2010 09:20:55 -0800 (PST) Received: by 10.90.226.2 with HTTP; Mon, 6 Dec 2010 09:20:55 -0800 (PST) In-Reply-To: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> References: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> Date: Mon, 6 Dec 2010 09:20:55 -0800 Message-ID: From: Freddie Cash To: Patrick Proniewski Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org Subject: Re: auto-mounting ZFS snapshots X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 17:20:56 -0000 On Mon, Dec 6, 2010 at 8:00 AM, Patrick Proniewski wrote: > I'm using freebsd-snapshot to create/destroy a bunch of daily/weekly snapshots > of my ZFS datasets. > I would like to make the automount work, so I've configured amd > accordingly, but strangely it fails: FreeBSD automounts the "hidden" .zfs filesystem hierarchy where snapshots are "stored" automatically. No need for amd. # zfs set snapdir=visible poolname/fsname # cd /path/to/fsname/.zfs/snapshot # ls # cd snapshot-name Even a simple "ls /path/to/fsname/.zfs/snapshot/snapshot-name/somedir/" will cause it to be mounted. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 17:27:38 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 333AA106564A for ; Mon, 6 Dec 2010 17:27:38 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-gx0-f172.google.com (mail-gx0-f172.google.com [209.85.161.172]) by mx1.freebsd.org (Postfix) with ESMTP id C7A2B8FC12 for ; Mon, 6 Dec 2010 17:27:37 +0000 (UTC) Received: by gxk28 with SMTP id 28so7225086gxk.17 for ; Mon, 06 Dec 2010 09:27:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:date:from:to:cc :subject:in-reply-to:message-id:references:user-agent :x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; bh=O07euSfnc9Y/TZmgvUu3wrgmJzkg+EJ/+BN9djOJG9w=; b=uTWF5ra2yckzPGdH7dJ657AEQQwdXRLICwqAniVwjsy2l+iy7NhflhJG5HZDXLVaTl QhZqvyX1WaqwXcveJ15ZvDUKBDkwKjYB4Uyi7jDXl3p975Pw0qXQtp0/yXvqbJ8CpDv7 4/bDr0JXi1lvf6+aJq92a9D56cp5P3XRafUZo= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:x-openpgp-key-id:x-openpgp-key-fingerprint:mime-version :content-type; b=aYpfvW6kZMoErTbulpc9YewLemTh+qlQYe7rSC7CuH4pkjmqR15oGHJdnLhLLeGkDY ia9qC/TLGkrudGjX7oKyQnXf6pPzhMoHfNvbLguYCpQofhUFMFyfNLczy3R3jCB+utaB R+Y6IUgikPZdzWDZTaG/2uzJyjVurg5FIsdyI= Received: by 10.42.9.101 with SMTP id l37mr1531860icl.479.1291656456699; Mon, 06 Dec 2010 09:27:36 -0800 (PST) Received: from centel.dataix.local (adsl-99-19-45-101.dsl.klmzmi.sbcglobal.net [99.19.45.101]) by mx.google.com with ESMTPS id d21sm5183067ibg.3.2010.12.06.09.27.34 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 06 Dec 2010 09:27:34 -0800 (PST) Sender: "J. Hellenthal" Date: Mon, 6 Dec 2010 12:27:14 -0500 From: jhell To: Patrick Proniewski In-Reply-To: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> Message-ID: References: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-OpenPGP-Key-Id: 0x89D8547E X-OpenPGP-Key-Fingerprint: 85EF E26B 07BB 3777 76BE B12A 9057 8789 89D8 547E MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: FreeBSD Filesystems Subject: Re: auto-mounting ZFS snapshots X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 17:27:38 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Mon, 6 Dec 2010 11:00, Patrick Proniewski wrote: In Message-Id: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> > Hello, > > I'm using freebsd-snapshot to create/destroy a bunch of daily/weekly snapshots > of my ZFS datasets. > I would like to make the automount work, so I've configured amd > accordingly, but strangely it fails: > Unless I am misunderstanding what you are trying to do here. ZFS snapshots are already mounted automatically under: /path/to/mountpoint/.zfs/snapshot/{snapname} A normal unprivileged user can access that information at any time without the extra use of amd(8). Example: $ zfs snapshot pool/home@today $ ls -l /home/.zfs/snapshot/today/ total 1 drwxr-xr-x 65 jhell jhell 156 Dec 6 12:14 jhell $ Regards, - -- jhell,v -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJM/Rz7AAoJEJBXh4mJ2FR+CtoH/08mm1Ufj0pLvAyF9Sdmx+Pi 0KN0/dr1btg35NcS7gPxbevwsxArUsLgTwuNbPwoNkKCwUYnUa1+bXO1fqn4VuFg P1hL5zyK/yhdDCLWeCxA6i0luoX5JzKKOMa5iCGIxW1bROPPvuiUP8+A9Ypn0QBu SEEvvwu4DkYGdjuIfLS6kAgsVnvXOe1tdaZbRLfO9mt/2LTi+4L5maeFiRSCHj5Q huij1SliSOLapy1CZhEd/0bv5QTo2Lyy4HZjOqWVeveSypKRtfYSaK+z4Jiblt7A 6okA7RDC0gYc6oH2aV8jXyuxEN5VHoG3QldgBHktwO4UDjytgGyi8jqZyLW/Hkk= =gNe4 -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 20:20:52 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 421F4106566B for ; Mon, 6 Dec 2010 20:20:52 +0000 (UTC) (envelope-from patpro@patpro.net) Received: from rack.patpro.net (rack.patpro.net [193.30.227.216]) by mx1.freebsd.org (Postfix) with ESMTP id E8DFF8FC17 for ; Mon, 6 Dec 2010 20:20:51 +0000 (UTC) Received: from rack.patpro.net (localhost [127.0.0.1]) by rack.patpro.net (Postfix) with ESMTP id EFD9A1CC036; Mon, 6 Dec 2010 21:20:50 +0100 (CET) X-Virus-Scanned: amavisd-new at patpro.net Received: from amavis-at-patpro.net ([127.0.0.1]) by rack.patpro.net (rack.patpro.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id ZQqIq37qTYj0; Mon, 6 Dec 2010 21:20:47 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) by rack.patpro.net (Postfix) with ESMTP; Mon, 6 Dec 2010 21:20:47 +0100 (CET) Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: multipart/signed; boundary=Apple-Mail-1--172965941; protocol="application/pkcs7-signature"; micalg=sha1 From: Patrick Proniewski In-Reply-To: Date: Mon, 6 Dec 2010 21:20:46 +0100 Message-Id: <2F1F28F8-DDA3-4EA2-89D2-23EDAA0495F0@patpro.net> References: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> To: jhell , Freddie Cash X-Mailer: Apple Mail (2.1082) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: FreeBSD Filesystems Subject: Re: auto-mounting ZFS snapshots X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 20:20:52 -0000 --Apple-Mail-1--172965941 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Freddie, JHell, you posted the same answer to my question, so I make a = group reply ;) On 6 d=E9c. 2010, at 18:27, jhell wrote: > Unless I am misunderstanding what you are trying to do here. ZFS = snapshots are already mounted automatically under: >=20 > /path/to/mountpoint/.zfs/snapshot/{snapname} >=20 > A normal unprivileged user can access that information at any time = without > the extra use of amd(8). That's great news. I was doing this because I was following the = freebsd-snapshot documentation = (). I can unload amd and get rid of it's config file then. Thank you very much. patpro= --Apple-Mail-1--172965941-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 21:13:21 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 647031065675 for ; Mon, 6 Dec 2010 21:13:21 +0000 (UTC) (envelope-from patpro@patpro.net) Received: from rack.patpro.net (rack.patpro.net [193.30.227.216]) by mx1.freebsd.org (Postfix) with ESMTP id 169CA8FC1A for ; Mon, 6 Dec 2010 21:13:20 +0000 (UTC) Received: from rack.patpro.net (localhost [127.0.0.1]) by rack.patpro.net (Postfix) with ESMTP id 4C8371CC036 for ; Mon, 6 Dec 2010 22:13:20 +0100 (CET) X-Virus-Scanned: amavisd-new at patpro.net Received: from amavis-at-patpro.net ([127.0.0.1]) by rack.patpro.net (rack.patpro.net [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 007kSvSw38Zo for ; Mon, 6 Dec 2010 22:13:17 +0100 (CET) Received: from [127.0.0.1] (localhost [127.0.0.1]) by rack.patpro.net (Postfix) with ESMTP for ; Mon, 6 Dec 2010 22:13:17 +0100 (CET) From: Patrick Proniewski Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: multipart/signed; boundary=Apple-Mail-5--169816133; protocol="application/pkcs7-signature"; micalg=sha1 Date: Mon, 6 Dec 2010 22:13:16 +0100 In-Reply-To: To: FreeBSD Filesystems References: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> Message-Id: X-Mailer: Apple Mail (2.1082) X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: auto-mounting ZFS snapshots X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 21:13:21 -0000 --Apple-Mail-5--169816133 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 On 6 d=E9c. 2010, at 18:20, Freddie Cash wrote: > FreeBSD automounts the "hidden" .zfs filesystem hierarchy where > snapshots are "stored" automatically. No need for amd. The zfs man page reads: File system snapshots can be accessed under the ".zfs/snapshot" = direc- tory in the root of the file system. Snapshots are = automatically mounted on demand and may be unmounted at regular intervals. The = visi- bility of the ".zfs" directory can be controlled by the "snapdir" = prop- erty. "may be unmounted at regular intervals" <-- what about this? Do I have = to create a periodic script that'll unmount snapshots? And if so, how = can I make a script that'll unmount only snapshots mounted for over, = say, 1 hour? patpro= --Apple-Mail-5--169816133-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 21:28:49 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A07131065670 for ; Mon, 6 Dec 2010 21:28:49 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id 2CE128FC1B for ; Mon, 6 Dec 2010 21:28:48 +0000 (UTC) Received: by wwf26 with SMTP id 26so8315182wwf.31 for ; Mon, 06 Dec 2010 13:28:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :organization:user-agent:mime-version:to:cc:subject:references :in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; bh=8tgTSusD+nfaxEd8qo3hZROPOc0K44QxOCFtfb73e9o=; b=aGN9xBWsNALfVc3x5r0D+6K0HoRtD9y4/PvdFAXkWLqzluhJRcW4JHF5VB/HpMAITv a0c5iLDwaVnzwcq7Nnwgvbb+Mv43FYq8SV9IjeDmUXhETSkzBzujLjph9boCNmjgXF7B 7neSbgWyG2++dNrUVE3lyR/9sJH09lYyTVwL0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:organization:user-agent:mime-version:to :cc:subject:references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=OrxlD1rwhD40wiDtKV7VyktHM4fSqdt9uBLyn4dsTfizf3646a8Uia6otfU4HBf4gq m78ijGbe7WBaARk+kzdCjp8jEZBxtS2ZIYqXqwUelYfi5Ln0gMvGIHE9lHCe2urkXjma wlje6hHjS31vedllbrPqZRCnWOnxcnCfI217M= Received: by 10.216.11.205 with SMTP id 55mr485880wex.72.1291670927846; Mon, 06 Dec 2010 13:28:47 -0800 (PST) Received: from centel.dataix.local ([99.19.45.101]) by mx.google.com with ESMTPS id x28sm2591579weq.16.2010.12.06.13.28.45 (version=SSLv3 cipher=RC4-MD5); Mon, 06 Dec 2010 13:28:46 -0800 (PST) Sender: "J. Hellenthal" Message-ID: <4CFD558C.2060202@DataIX.net> Date: Mon, 06 Dec 2010 16:28:44 -0500 From: jhell Organization: http://www.DataIX.net User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.12) Gecko/20101028 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Patrick Proniewski References: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: FreeBSD Filesystems Subject: Re: auto-mounting ZFS snapshots X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 21:28:49 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12/06/2010 16:13, Patrick Proniewski wrote: > On 6 déc. 2010, at 18:20, Freddie Cash wrote: > >> FreeBSD automounts the "hidden" .zfs filesystem hierarchy where >> snapshots are "stored" automatically. No need for amd. > > The zfs man page reads: > > File system snapshots can be accessed under the ".zfs/snapshot" > direc- tory in the root of the file system. Snapshots are > automatically mounted on demand and may be unmounted at regular > intervals. The visi- bility of the ".zfs" directory can be controlled > by the "snapdir" prop- erty. > > "may be unmounted at regular intervals" <-- what about this? Do I > have to create a periodic script that'll unmount snapshots? And if > so, how can I make a script that'll unmount only snapshots mounted > for over, say, 1 hour? No this is a no worry type of thing. This is a transparent process that handles what is available. Just consider them mounted all the time but you will not see them. Play with it a little and see what it looks like. mount(1) will not even show them as mounted so there is no worries here. The correct thing happens to keep you from shooting yourself in the foot. - -- jhell,v -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJM/VWMAAoJEJBXh4mJ2FR+rksH+wRM2iGqZGRxJ+HqgBbdl1PT /ppl3Wvf5S+RQaltEYuL5H4oYgfKW08R3isP3zrHIGcfhcoef0ehi33dH0ZZCUGr aOPB9WspWy46OQhz6i5SRuZCGl1cvauT63zufi4meSwlT4e0el16JcQIU0W1DX0s 8if7oah+ZaldHmJscfU6Rv/kw+yNnv40s7XFMSnVGEOaX93+lTts2vNCLZPRs5pF bhPQW1ueNgrV4g+VSeC/N9RuClXHWQXXvehMxosrfvaeB/W06R+xI9c9u6wzKDl1 ZnPKNGhG2ej6dTdpST15k2vj0bYD2JESat/1U95UQShXyvqhNnn4s2PYEVxxNao= =JK4c -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 21:30:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B814106566B for ; Mon, 6 Dec 2010 21:30:00 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id C8FF28FC12 for ; Mon, 6 Dec 2010 21:29:59 +0000 (UTC) Received: by ywp6 with SMTP id 6so6664542ywp.13 for ; Mon, 06 Dec 2010 13:29:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=i3Khy131XgCHLiqcVJEL6uwSURYcetL8BNTvjrsGXUQ=; b=SukTEykkZxUIkLu9JIrZlgWU21jhr1rxUivZoDanQJmv+m+86owupthK8TqPK31zDT U2Xk41xnUNiegKQGriKZvh2hBTWkhUIu5TALyzxR9mdd16/UBbj0gDpoZk2KWxWkfYoD sr34XJR9J3cDHhYg2FrnEyN74tLHE3909DAzE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=jqN1Y8t4f1/mjq/mqWuWtNjW8ZrjPu9V1IWdG1bcEVy7OZIl9xG1vwKMZkzhXpapoI 8vAozMHnXh6Og094xsID9jJjr68rKy/odOQX4S1whwoMCHLK9mrTXZuIL8Gn2VOZdRr8 p8Ef/cbIk8cSRd7zDYl4DGH/AgItpp4pd55Rg= MIME-Version: 1.0 Received: by 10.90.4.26 with SMTP id 26mr8507553agd.84.1291670998952; Mon, 06 Dec 2010 13:29:58 -0800 (PST) Received: by 10.90.226.2 with HTTP; Mon, 6 Dec 2010 13:29:58 -0800 (PST) In-Reply-To: References: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> Date: Mon, 6 Dec 2010 13:29:58 -0800 Message-ID: From: Freddie Cash To: Patrick Proniewski Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Filesystems Subject: Re: auto-mounting ZFS snapshots X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 21:30:00 -0000 On Mon, Dec 6, 2010 at 1:13 PM, Patrick Proniewski wrot= e: > On 6 d=C3=A9c. 2010, at 18:20, Freddie Cash wrote: > >> FreeBSD automounts the "hidden" .zfs filesystem hierarchy where >> snapshots are "stored" automatically. =C2=A0No need for amd. > > The zfs man page reads: > > =C2=A0 =C2=A0 =C2=A0 File =C2=A0system snapshots can be accessed under th= e ".zfs/snapshot" direc- > =C2=A0 =C2=A0 =C2=A0 tory in the root =C2=A0of =C2=A0the =C2=A0file =C2= =A0system. =C2=A0Snapshots =C2=A0are =C2=A0automatically > =C2=A0 =C2=A0 =C2=A0 mounted =C2=A0on demand and may be unmounted at regu= lar intervals. The visi- > =C2=A0 =C2=A0 =C2=A0 bility of the ".zfs" directory can be controlled by = the "snapdir" prop- > =C2=A0 =C2=A0 =C2=A0 erty. > > "may be unmounted at regular intervals" <-- what about this? Do I have to= create a periodic script that'll unmount snapshots? And if so, how can I m= ake a script that'll unmount only snapshots mounted for over, say, 1 hour? If you don't want the output of "mount" to be cluttered with ZFS snapshots, then you can manually clear it out with something like: mount | grep | awk '{ print $1 }' | xargs sudo umount Replace with the ZFS filesystem name as shown in the output of mount. If needed, you could put that into a periodic script or cronjob. However, it shouldn't hurt things to have all the snapshots mounted. We have 205 snapshots mounted on our backups server without any issues so far. --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 21:31:12 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 15A77106564A for ; Mon, 6 Dec 2010 21:31:12 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (lor.one-eyed-alien.net [69.66.77.232]) by mx1.freebsd.org (Postfix) with ESMTP id C24CE8FC15 for ; Mon, 6 Dec 2010 21:31:11 +0000 (UTC) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.3/8.14.3) with ESMTP id oB6LU2mW088372; Mon, 6 Dec 2010 15:30:02 -0600 (CST) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.3/8.14.3/Submit) id oB6LU1L9088371; Mon, 6 Dec 2010 15:30:02 -0600 (CST) (envelope-from brooks) Date: Mon, 6 Dec 2010 15:30:01 -0600 From: Brooks Davis To: Ronald Klop Message-ID: <20101206213001.GI7429@lor.one-eyed-alien.net> References: <4CFBD7F9.3000307@ukr.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ev7mvGV+3JQuI2Eo" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.17 (2007-11-01) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Mon, 06 Dec 2010 15:30:02 -0600 (CST) Cc: freebsd-fs@freebsd.org Subject: Re: How to choose the size of swap? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 21:31:12 -0000 --ev7mvGV+3JQuI2Eo Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Dec 05, 2010 at 07:53:04PM +0100, Ronald Klop wrote: > On Sun, 05 Dec 2010 19:20:41 +0100, Vladislav V. Prodan=20 > wrote: >=20 >> We have several servers with 8|16 GB of memory, 2|4 harddrive >> (RAID1|RAID10 using ZFS) >> These servers will work with Mysql, the size of databases that are >> several times larger than the size of memory. >>=20 >> What would recommend to expose the size of swap? >> On a separate partition or a file? >> At hand is not loaded with servers that have the swap was over 20 MB ... >> In an extreme case, let the output "top v" to this maillist >=20 > You might not need swap to run the database. But the swap space is also= =20 > used for making a kernel dump in case of a problem. In that case swap spa= ce=20 > must be as large as the amount of memory in the machine. (As far as I=20 > know.) With minidumps (the default) you'll need much less that physical memory in general. In most cases 2GB should be more than enough for dumps. -- Brooks --ev7mvGV+3JQuI2Eo Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iD8DBQFM/VXZXY6L6fI4GtQRAvm6AKC/XFFRfz0y47O2C0CH7JicWMZsPQCgsjyB 1s9h1BGxjMBqYIc5R/oeimo= =3/6H -----END PGP SIGNATURE----- --ev7mvGV+3JQuI2Eo-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 21:31:16 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12B4D10656F0 for ; Mon, 6 Dec 2010 21:31:16 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-gx0-f172.google.com (mail-gx0-f172.google.com [209.85.161.172]) by mx1.freebsd.org (Postfix) with ESMTP id B5A9A8FC19 for ; Mon, 6 Dec 2010 21:31:15 +0000 (UTC) Received: by gxk28 with SMTP id 28so7439173gxk.17 for ; Mon, 06 Dec 2010 13:31:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=bLdrAhWPDDUY2kgNMKcezQrhUW0NTv5cjtIsLfqb/bo=; b=A/lO1Tu7cfYOEQqx507n9mnTTsXjQnK5cwAmMVFPJ3bugjJDaWQvHqxOKwm7ErjAMr 5ztHBajVI8KkysMCTHMEfpLdt4LASWLD0YoM9gSpX4enunsjoo/WfYa0giST7IXLuY7a m7rArFtJMnR2/+OM6Z0YkqUgoGYZTb3c7wFsM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=Ec7PpSWITWkDlqybAfrNJfu6AqfqapTWbwBa2VuSDxWaxHaYhe/n1uV7QKkKCZNldm wXEWMi32ps7p7tuOcKPDkIvd9Cc57IlEva4swfEK40FQFvekZxxjOD6c72sR3dOcfbq4 7ZZlMplYtR7BOn6KnumrKxCa0DzhWBO9PW1V8= MIME-Version: 1.0 Received: by 10.91.203.6 with SMTP id f6mr8569780agq.30.1291671074792; Mon, 06 Dec 2010 13:31:14 -0800 (PST) Received: by 10.90.226.2 with HTTP; Mon, 6 Dec 2010 13:31:14 -0800 (PST) In-Reply-To: <4CFD558C.2060202@DataIX.net> References: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> <4CFD558C.2060202@DataIX.net> Date: Mon, 6 Dec 2010 13:31:14 -0800 Message-ID: From: Freddie Cash To: jhell Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: FreeBSD Filesystems , Patrick Proniewski Subject: Re: auto-mounting ZFS snapshots X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 21:31:16 -0000 On Mon, Dec 6, 2010 at 1:28 PM, jhell wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 12/06/2010 16:13, Patrick Proniewski wrote: >> On 6 d=C3=A9c. 2010, at 18:20, Freddie Cash wrote: >> >>> FreeBSD automounts the "hidden" .zfs filesystem hierarchy where >>> snapshots are "stored" automatically. =C2=A0No need for amd. >> >> The zfs man page reads: >> >> File =C2=A0system snapshots can be accessed under the ".zfs/snapshot" >> direc- tory in the root =C2=A0of =C2=A0the =C2=A0file =C2=A0system. =C2= =A0Snapshots =C2=A0are >> automatically mounted =C2=A0on demand and may be unmounted at regular >> intervals. The visi- bility of the ".zfs" directory can be controlled >> by the "snapdir" prop- erty. >> >> "may be unmounted at regular intervals" <-- what about this? Do I >> have to create a periodic script that'll unmount snapshots? And if >> so, how can I make a script that'll unmount only snapshots mounted >> for over, say, 1 hour? > > No this is a no worry type of thing. This is a transparent process that > handles what is available. Just consider them mounted all the time but > you will not see them. > > Play with it a little and see what it looks like. mount(1) will not even > show them as mounted so there is no worries here. The correct thing > happens to keep you from shooting yourself in the foot. Oh, that's right. Mount was updated in 8.x to not show ZFS snapshots, so you don't even have to worry about it anymore. My previous reply applies to 7.x, though (which is what's running on one of our backups boxes). --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 21:42:26 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B56C31065670 for ; Mon, 6 Dec 2010 21:42:26 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta07.westchester.pa.mail.comcast.net (qmta07.westchester.pa.mail.comcast.net [76.96.62.64]) by mx1.freebsd.org (Postfix) with ESMTP id 636208FC2A for ; Mon, 6 Dec 2010 21:42:25 +0000 (UTC) Received: from omta21.westchester.pa.mail.comcast.net ([76.96.62.72]) by qmta07.westchester.pa.mail.comcast.net with comcast id ft2P1f0081ZXKqc57xiSoG; Mon, 06 Dec 2010 21:42:26 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta21.westchester.pa.mail.comcast.net with comcast id fxiR1f0063LrwQ23hxiRAp; Mon, 06 Dec 2010 21:42:26 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id BC09C9B422; Mon, 6 Dec 2010 13:42:23 -0800 (PST) Date: Mon, 6 Dec 2010 13:42:23 -0800 From: Jeremy Chadwick To: Ronald Klop Message-ID: <20101206214223.GA6375@icarus.home.lan> References: <4CFBD7F9.3000307@ukr.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org Subject: Re: How to choose the size of swap? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 21:42:26 -0000 On Sun, Dec 05, 2010 at 07:53:04PM +0100, Ronald Klop wrote: > On Sun, 05 Dec 2010 19:20:41 +0100, Vladislav V. Prodan > wrote: > > >We have several servers with 8|16 GB of memory, 2|4 harddrive > >(RAID1|RAID10 using ZFS) > >These servers will work with Mysql, the size of databases that are > >several times larger than the size of memory. > > > >What would recommend to expose the size of swap? > >On a separate partition or a file? > >At hand is not loaded with servers that have the swap was over 20 MB ... > >In an extreme case, let the output "top v" to this maillist > > You might not need swap to run the database. But the swap space is > also used for making a kernel dump in case of a problem. In that > case swap space must be as large as the amount of memory in the > machine. (As far as I know.) And don't forget that (by default) you'll need a /var filesystem that has enough physical room to hold the dump. You can change this using the "dumpdir" option in rc.conf(5) if your existing /var isn't large enough. Just make sure to copy /var/crash to /wherever, and make sure the file/dir permissions are correct + minfree file exists. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:02:30 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 51F1F106564A for ; Mon, 6 Dec 2010 22:02:30 +0000 (UTC) (envelope-from joe@netmusician.org) Received: from mail.netmusician.org (dorian.netmusician.org [66.244.95.101]) by mx1.freebsd.org (Postfix) with ESMTP id 12E2D8FC08 for ; Mon, 6 Dec 2010 22:02:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netmusician.org (Postfix) with ESMTP id 4162CB874; Mon, 6 Dec 2010 17:02:29 -0500 (EST) X-Virus-Scanned: amavisd-new at netmusician.org Received: from mail.netmusician.org ([127.0.0.1]) by localhost (dorian.netmusician.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 8undCiseawTX; Mon, 6 Dec 2010 17:02:28 -0500 (EST) Received: from Shakti.local (c-71-201-100-167.hsd1.in.comcast.net [71.201.100.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.netmusician.org (Postfix) with ESMTPSA id 8CC2DB803; Mon, 6 Dec 2010 17:02:28 -0500 (EST) Message-ID: <4CFD5D73.1050601@netmusician.org> Date: Mon, 06 Dec 2010 17:02:27 -0500 From: Joe Auty User-Agent: Postbox 2.0.2 (Macintosh/20101025) MIME-Version: 1.0 To: Rick Macklem , freebsd-fs@freebsd.org References: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:02:30 -0000 Rick Macklem wrote: >> Hello, >> >> This is possibly a more fundamental non-FreeBSD specific set of >> questions, but ultimately this is relevant to usage on FreeBSD, so... >> >> I'm fairly certain that NFSv4 is supported under Solaris 10/ZFS and >> FreeBSD/ZFS via the standard "share" binary or the sharenfs ZFS >> property, right? >> >> In mounting an NFS share on my FreeBSD test machine via the following: >> >> mount -t nfs -o rw,nfsv4 ipaddress:/share /path/to/share/directory >> >> I'm unable to change the permissions of any of these files via a >> standard chmod on the client (FreeBSD) side. What are NFSv4 ACLs, and >> is >> this in any way relevant to my problem here? Do ACLs need to be set in >> order to use a volume like I can an NFSv3 volume, which works just >> fine >> for me? >> > It might be worth capturing packets "tcpdump -s 0 -w xxx host " > while trying a "chmod" and seeing what goes over the wire. You can look > at it via wireshark or email me "xxx" and I can take a look. > > I don't know anything about ZFS, but you could try getfacl/setfacl on the > client and see what happens? > > Edward Napierala (trasz@freebsd.org) did commit a recent change w.r.t. > NFSv4 ACLs and I remember the discussion saying something like "after > this change, chmod no longer does anything once ACLs are enabled, but I > have no idea if it is relevant. > > Also, make sure "ls -l" is not reporting "nobody". If the user/group > name mapping isn't working, most Setattr Ops will fail. > > rick > Thanks Rick, I will look into this, but for the benefit of my own education, are NFSv4 ACLs supposed to be intertwined or separate from standard Unix permissions? I'm confused as to how the ACLs have changed from v3, or if this is even relevant to my problem not really knowing how they work and why they are needed :) Care to provide some basic info here? I'll follow up on this and will provide this dump info shortly... -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe@netmusician.org From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:28:41 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BC7BA1065672 for ; Mon, 6 Dec 2010 22:28:41 +0000 (UTC) (envelope-from etnapierala@googlemail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4764A8FC18 for ; Mon, 6 Dec 2010 22:28:40 +0000 (UTC) Received: by fxm16 with SMTP id 16so9911039fxm.13 for ; Mon, 06 Dec 2010 14:28:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:sender:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=QBlRY0G/xM8rK93ZIgs7jG5wW1ayti1SG0h3b/YC4I0=; b=VEkS9rS2nKE5R/AnEe76ZzvyjWDFh0fLvoon9uMQ5q3BsN0PCSpX/VVLKlblGCTFwg m0kMBCrOtpJ0rx9ZguWkM5s149HlZk888pObTKtGgZUalQwgVJUxC777zVCkCAXq/co7 aRI4Tqut5VOBApYNqwtmX+HNat/Wc2XY2mA6k= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=TfRCb0LWa93eXHHZtJk2MkSlQOFkDEjn59Brc2ZBbGFcWXTs8/iDb2VifRyLX3jWrO 0mzsHp98OgpU45Pl2jJQnRD/ZpyUmEnGiHtMh/bDOxk+t81lbONe9Roz5tGvXJBHb6ON Ytd7zfehqG/G3KgUfnhLfsF/FmzJcGS5EyDWQ= Received: by 10.223.100.8 with SMTP id w8mr6223233fan.55.1291674519911; Mon, 06 Dec 2010 14:28:39 -0800 (PST) Received: from [192.168.1.102] (45.81.datacomsa.pl [195.34.81.45]) by mx.google.com with ESMTPS id z1sm1732211fau.21.2010.12.06.14.28.37 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 06 Dec 2010 14:28:38 -0800 (PST) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=iso-8859-2 From: =?iso-8859-2?Q?Edward_Tomasz_Napiera=B3a?= In-Reply-To: <4CFD5D73.1050601@netmusician.org> Date: Mon, 6 Dec 2010 23:28:36 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> <4CFD5D73.1050601@netmusician.org> To: Joe Auty X-Mailer: Apple Mail (2.1082) Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:28:41 -0000 Wiadomo=B6=E6 napisana przez Joe Auty w dniu 2010-12-06, o godz. 23:02: > Rick Macklem wrote: >>=20 >> I don't know anything about ZFS, but you could try getfacl/setfacl on = the >> client and see what happens? >>=20 >> Edward Napierala (trasz@freebsd.org) did commit a recent change = w.r.t. >> NFSv4 ACLs and I remember the discussion saying something like "after >> this change, chmod no longer does anything once ACLs are enabled, but = I >> have no idea if it is relevant. Erm, no. There is a change in the queue that will change chmod = behaviour wrt. ACLs, but 1. it's not committed yet, and 2. chmod will continue to work. >> Also, make sure "ls -l" is not reporting "nobody". If the user/group >> name mapping isn't working, most Setattr Ops will fail. >>=20 >> rick >>=20 >=20 >=20 > Thanks Rick, >=20 > I will look into this, but for the benefit of my own education, are > NFSv4 ACLs supposed to be intertwined or separate from standard Unix > permissions? I'm confused as to how the ACLs have changed from v3, or = if > this is even relevant to my problem not really knowing how they work = and > why they are needed :) Both POSIX.1e and NFSv4 ACLs are similar in that they both influence the mode, and get influenced by it. In other words, when you change the ACL, the mode gets updated; when you change the mode, the ACL gets updated. Also, for both POSIX.1e and NFSv4 ACLs, file mode continues to work as usual if you ignore the ACL part. Good introduction might be the setfacl(1) manual page. -- If you cut off my head, what would I say? Me and my head, or me and my = body? From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:34:55 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 759061065670; Mon, 6 Dec 2010 22:34:55 +0000 (UTC) (envelope-from joe@netmusician.org) Received: from mail.netmusician.org (dorian.netmusician.org [66.244.95.101]) by mx1.freebsd.org (Postfix) with ESMTP id 43FC38FC18; Mon, 6 Dec 2010 22:34:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netmusician.org (Postfix) with ESMTP id 2BF7EB8C8; Mon, 6 Dec 2010 17:34:54 -0500 (EST) X-Virus-Scanned: amavisd-new at netmusician.org Received: from mail.netmusician.org ([127.0.0.1]) by localhost (dorian.netmusician.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id VGML-MjcKVkI; Mon, 6 Dec 2010 17:34:53 -0500 (EST) Received: from Shakti.local (c-71-201-100-167.hsd1.in.comcast.net [71.201.100.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.netmusician.org (Postfix) with ESMTPSA id 3D72DB8C2; Mon, 6 Dec 2010 17:34:53 -0500 (EST) Message-ID: <4CFD6506.7090901@netmusician.org> Date: Mon, 06 Dec 2010 17:34:46 -0500 From: Joe Auty User-Agent: Postbox 2.0.2 (Macintosh/20101025) MIME-Version: 1.0 To: =?UTF-8?B?RWR3YXJkIFRvbWFzeiBOYXBpZXJhxYJh?= References: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> <4CFD5D73.1050601@netmusician.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:34:55 -0000 Edward Tomasz Napiera=C5=82a wrote: > Wiadomo=C5=9B=C4=87 napisana przez Joe Auty w dniu 2010-12-06, o godz. = 23:02: >> Rick Macklem wrote: >>> I don't know anything about ZFS, but you could try getfacl/setfacl on= the >>> client and see what happens? >>> >>> Edward Napierala (trasz@freebsd.org) did commit a recent change w.r.t= . >>> NFSv4 ACLs and I remember the discussion saying something like "after >>> this change, chmod no longer does anything once ACLs are enabled, but= I >>> have no idea if it is relevant. > > Erm, no. There is a change in the queue that will change chmod behavio= ur > wrt. ACLs, but 1. it's not committed yet, and 2. chmod will continue to > work. > >>> Also, make sure "ls -l" is not reporting "nobody". If the user/group >>> name mapping isn't working, most Setattr Ops will fail. >>> >>> rick >>> >> Thanks Rick, >> >> I will look into this, but for the benefit of my own education, are >> NFSv4 ACLs supposed to be intertwined or separate from standard Unix >> permissions? I'm confused as to how the ACLs have changed from v3, or = if >> this is even relevant to my problem not really knowing how they work a= nd >> why they are needed :) > > Both POSIX.1e and NFSv4 ACLs are similar in that they both influence > the mode, and get influenced by it. In other words, when you change > the ACL, the mode gets updated; when you change the mode, the ACL gets > updated. Also, for both POSIX.1e and NFSv4 ACLs, file mode continues > to work as usual if you ignore the ACL part. > Thanks for this! So, if I want to just ignore the NFSv4 ACLs on account of not needing anything beyond the POSIX ACLs, I'm free to do so without consequence... Correct? > Good introduction might be the setfacl(1) manual page. > > -- > If you cut off my head, what would I say? Me and my head, or me and my= body? > --=20 Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe@netmusician.org From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:35:54 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E38F91065670; Mon, 6 Dec 2010 22:35:54 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 775E58FC1C; Mon, 6 Dec 2010 22:35:54 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAJf0/EyDaFvO/2dsb2JhbACDT6BfrjmQd4EhgzVzBIRfhg8 X-IronPort-AV: E=Sophos;i="4.59,307,1288584000"; d="scan'208";a="101527667" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 06 Dec 2010 17:35:53 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 48F6DB3F39; Mon, 6 Dec 2010 17:35:53 -0500 (EST) Date: Mon, 6 Dec 2010 17:35:53 -0500 (EST) From: Rick Macklem To: =?utf-8?Q?Edward_Tomasz_Napiera=C5=82a?= Message-ID: <265929659.1260579.1291674953285.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [174.114.46.215] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - IE8 (Win)/6.0.7_GA_2473.RHEL4_64) Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:35:55 -0000 > Wiadomo=C5=9B=C4=87 napisana przez Joe Auty w dniu 2010-12-06, o godz. 23= :02: > > Rick Macklem wrote: > >> > >> I don't know anything about ZFS, but you could try getfacl/setfacl > >> on the > >> client and see what happens? > >> > >> Edward Napierala (trasz@freebsd.org) did commit a recent change > >> w.r.t. > >> NFSv4 ACLs and I remember the discussion saying something like > >> "after > >> this change, chmod no longer does anything once ACLs are enabled, > >> but I > >> have no idea if it is relevant. >=20 > Erm, no. There is a change in the queue that will change chmod > behaviour > wrt. ACLs, but 1. it's not committed yet, and 2. chmod will continue > to > work. >=20 Oops, sorry. Thanks for clarifying it, rick From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:41:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 989C71065672 for ; Mon, 6 Dec 2010 22:41:28 +0000 (UTC) (envelope-from joe@netmusician.org) Received: from mail.netmusician.org (dorian.netmusician.org [66.244.95.101]) by mx1.freebsd.org (Postfix) with ESMTP id 66D538FC1F for ; Mon, 6 Dec 2010 22:41:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netmusician.org (Postfix) with ESMTP id B8C13B8AD; Mon, 6 Dec 2010 17:41:27 -0500 (EST) X-Virus-Scanned: amavisd-new at netmusician.org Received: from mail.netmusician.org ([127.0.0.1]) by localhost (dorian.netmusician.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id mx2PWOWCmlom; Mon, 6 Dec 2010 17:41:27 -0500 (EST) Received: from Shakti.local (c-71-201-100-167.hsd1.in.comcast.net [71.201.100.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.netmusician.org (Postfix) with ESMTPSA id 216F8B87A; Mon, 6 Dec 2010 17:41:27 -0500 (EST) Message-ID: <4CFD6693.7080100@netmusician.org> Date: Mon, 06 Dec 2010 17:41:23 -0500 From: Joe Auty User-Agent: Postbox 2.0.2 (Macintosh/20101025) MIME-Version: 1.0 To: Rick Macklem , freebsd-fs@freebsd.org References: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:41:28 -0000 Rick Macklem wrote: >> Hello, >> >> This is possibly a more fundamental non-FreeBSD specific set of >> questions, but ultimately this is relevant to usage on FreeBSD, so... >> >> I'm fairly certain that NFSv4 is supported under Solaris 10/ZFS and >> FreeBSD/ZFS via the standard "share" binary or the sharenfs ZFS >> property, right? >> >> In mounting an NFS share on my FreeBSD test machine via the following: >> >> mount -t nfs -o rw,nfsv4 ipaddress:/share /path/to/share/directory >> >> I'm unable to change the permissions of any of these files via a >> standard chmod on the client (FreeBSD) side. What are NFSv4 ACLs, and >> is >> this in any way relevant to my problem here? Do ACLs need to be set in >> order to use a volume like I can an NFSv3 volume, which works just >> fine >> for me? >> > It might be worth capturing packets "tcpdump -s 0 -w xxx host " > while trying a "chmod" and seeing what goes over the wire. You can look > at it via wireshark or email me "xxx" and I can take a look. > > I don't know anything about ZFS, but you could try getfacl/setfacl on the > client and see what happens? > > Edward Napierala (trasz@freebsd.org) did commit a recent change w.r.t. > NFSv4 ACLs and I remember the discussion saying something like "after > this change, chmod no longer does anything once ACLs are enabled, but I > have no idea if it is relevant. > > Also, make sure "ls -l" is not reporting "nobody". If the user/group > name mapping isn't working, most Setattr Ops will fail. > Okay, Here is my dump command... The NFS host is 192.168.0.20: # tcpdump -s 0 -w dumpfile.txt host 192.168.0.20 tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 65535 bytes In NFS mount: # ls -l total 2 -rw-r--r-- 1 root wheel 0 Dec 4 23:19 blah -rw-r--r-- 1 root wheel 0 Dec 4 23:19 test2 -rw-r--r-- 1 root wheel 0 Dec 4 23:19 test3 # chown joe blah (no response) "joe" is indeed a local user on the NFS client side. This is not generating any tcpdump output though. # ls -l total 2 -rw-r--r-- 1 root wheel 0 Dec 4 23:19 blah -rw-r--r-- 1 root wheel 0 Dec 4 23:19 test2 -rw-r--r-- 1 root wheel 0 Dec 4 23:19 test3 No actual permission change I created these files as root, so that much is being recognized... > rick > -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe@netmusician.org From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:42:29 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FCCD106566B; Mon, 6 Dec 2010 22:42:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id C8F828FC0C; Mon, 6 Dec 2010 22:42:28 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAIb1/EyDaFvO/2dsb2JhbACDT6BfrjiQd4MFgVFzBIRfhg8 X-IronPort-AV: E=Sophos;i="4.59,307,1288584000"; d="scan'208";a="103268682" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 06 Dec 2010 17:42:27 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id CF734B3F33; Mon, 6 Dec 2010 17:42:27 -0500 (EST) Date: Mon, 6 Dec 2010 17:42:27 -0500 (EST) From: Rick Macklem To: Joe Auty Message-ID: <462624062.1260806.1291675347837.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4CFD6506.7090901@netmusician.org> MIME-Version: 1.0 X-Originating-IP: [174.114.46.215] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - IE8 (Win)/6.0.7_GA_2473.RHEL4_64) Content-Type: multipart/related; boundary="----=_Part_1260805_36421406.1291675347835" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, =?utf-8?Q?Edward_Tomasz_Napiera=C5=82a?= Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:42:29 -0000 ------=_Part_1260805_36421406.1291675347835 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit > > So, if I want to just ignore the NFSv4 ACLs on account of not needing > anything beyond the POSIX ACLs, I'm free to do so without > consequence... Correct? > Well, NFSv4 won't be able to manipulate POSIX ACLs (really POSIX.1e draft which was never ratified and, as such, isn't a POSIX standard as I understand it). If you meant "beyond chmod" then I think you will be ok, but I haven't used ZFS, so?? ------=_Part_1260805_36421406.1291675347835-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:45:54 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 48FB61065674; Mon, 6 Dec 2010 22:45:54 +0000 (UTC) (envelope-from joe@netmusician.org) Received: from mail.netmusician.org (dorian.netmusician.org [66.244.95.101]) by mx1.freebsd.org (Postfix) with ESMTP id 1706E8FC17; Mon, 6 Dec 2010 22:45:53 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netmusician.org (Postfix) with ESMTP id 16279B8C8; Mon, 6 Dec 2010 17:45:53 -0500 (EST) X-Virus-Scanned: amavisd-new at netmusician.org Received: from mail.netmusician.org ([127.0.0.1]) by localhost (dorian.netmusician.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id bnXRk2-W0+Po; Mon, 6 Dec 2010 17:45:52 -0500 (EST) Received: from Shakti.local (c-71-201-100-167.hsd1.in.comcast.net [71.201.100.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.netmusician.org (Postfix) with ESMTPSA id 715AEB8C2; Mon, 6 Dec 2010 17:45:52 -0500 (EST) Message-ID: <4CFD679C.7020804@netmusician.org> Date: Mon, 06 Dec 2010 17:45:48 -0500 From: Joe Auty User-Agent: Postbox 2.0.2 (Macintosh/20101025) MIME-Version: 1.0 To: Rick Macklem References: <462624062.1260806.1291675347837.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <462624062.1260806.1291675347837.JavaMail.root@erie.cs.uoguelph.ca> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, =?UTF-8?B?RWR3YXJkIFRvbWFzeiBOYXBpZXJhxYJh?= Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:45:54 -0000 Rick Macklem wrote: >> So, if I want to just ignore the NFSv4 ACLs on account of not needing >> anything beyond the POSIX ACLs, I'm free to do so without >> consequence... Correct? >> > > Well, NFSv4 won't be able to manipulate POSIX ACLs (really POSIX.1e draft > which was never ratified and, as such, isn't a POSIX standard as I understand > it). If you meant "beyond chmod" then I think you will be ok, but I haven't > used ZFS, so?? > Well, chmods and POSIX.1e ACLs work fine in NFSv3 with the same ZFS server and everything else being the same on the FreeBSD site, so I don't think that ZFS is the problem here unless ZFS has some sort of NFSv4 host bug. -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe@netmusician.org From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:46:11 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C61BB1065670 for ; Mon, 6 Dec 2010 22:46:11 +0000 (UTC) (envelope-from etnapierala@googlemail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4F7EC8FC1E for ; Mon, 6 Dec 2010 22:46:10 +0000 (UTC) Received: by fxm16 with SMTP id 16so9926253fxm.13 for ; Mon, 06 Dec 2010 14:46:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:sender:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=NngvNMRqAo+e+CyjwlFzr+itg2zwCmBsSyDutE9Vwzs=; b=D1kEhEvHrGidcIJspHbtHRw7xWsd9VvUJAcdCIqHivZII4LDz+E1cImpI5RXDwiGmR N1h95AG6o2+AVhGazC4MwSf4Dqw1AQYZtep01YGn7uUhQL3+8L9K46k+cutW5xSkzYZn D8n8FBQ47U4KjC+myvbHAeN+9Q5Ndtb1YVcNs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=IxnozRj14JNURV8P3bAwcK5FO+INhZUwhcJJDy1FR2cd5k5c0+pvUvc4o1vj7EPU1Y EiSbf3y950+eGdhTbnyT6pj9PhmJXi0yoHRyk8LKwmgZPsfCprgu3MD6G7micG37dTbB sTnOdJetlQF4D9orBq38o2eunbknGYYXFT9ss= Received: by 10.223.87.67 with SMTP id v3mr6080474fal.130.1291675570267; Mon, 06 Dec 2010 14:46:10 -0800 (PST) Received: from [192.168.1.102] (45.81.datacomsa.pl [195.34.81.45]) by mx.google.com with ESMTPS id e17sm1735953fak.34.2010.12.06.14.46.07 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 06 Dec 2010 14:46:08 -0800 (PST) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=iso-8859-2 From: =?iso-8859-2?Q?Edward_Tomasz_Napiera=B3a?= In-Reply-To: <4CFD6506.7090901@netmusician.org> Date: Mon, 6 Dec 2010 23:46:05 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <408E7ECD-C232-47DD-9D24-389F2CA4E406@FreeBSD.org> References: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> <4CFD5D73.1050601@netmusician.org> <4CFD6506.7090901@netmusician.org> To: Joe Auty X-Mailer: Apple Mail (2.1082) Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:46:11 -0000 Wiadomo=B6=E6 napisana przez Joe Auty w dniu 2010-12-06, o godz. 23:34: > Edward Tomasz Napiera=B3a wrote: >> Wiadomo=B6=E6 napisana przez Joe Auty w dniu 2010-12-06, o godz. = 23:02: >>>> Also, make sure "ls -l" is not reporting "nobody". If the = user/group >>>> name mapping isn't working, most Setattr Ops will fail. >>>>=20 >>>> rick >>>>=20 >>> Thanks Rick, >>>=20 >>> I will look into this, but for the benefit of my own education, are >>> NFSv4 ACLs supposed to be intertwined or separate from standard Unix >>> permissions? I'm confused as to how the ACLs have changed from v3, = or if >>> this is even relevant to my problem not really knowing how they work = and >>> why they are needed :) >>=20 >> Both POSIX.1e and NFSv4 ACLs are similar in that they both influence >> the mode, and get influenced by it. In other words, when you change >> the ACL, the mode gets updated; when you change the mode, the ACL = gets >> updated. Also, for both POSIX.1e and NFSv4 ACLs, file mode continues >> to work as usual if you ignore the ACL part. >>=20 > Thanks for this! >=20 > So, if I want to just ignore the NFSv4 ACLs on account of not needing > anything beyond the POSIX ACLs, I'm free to do so without = consequence... > Correct? If you want to just ignore the ACLs on account of not needing anything beyond the file mode, aka standard UNIX permissions. Filesystems support either POSIX.1e ACLs, or NFSv4 ACLs, not both. I didn't actually test NFSv4, but I guess it uses NFSv4 ACLs, not POSIX.1e. ZFS supports NFSv4 only. UFS supports either POSIX.1e or NFSv4, depending on the mount options. -- If you cut off my head, what would I say? Me and my head, or me and my = body? From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:47:42 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7DC9C1065674; Mon, 6 Dec 2010 22:47:42 +0000 (UTC) (envelope-from joe@netmusician.org) Received: from mail.netmusician.org (dorian.netmusician.org [66.244.95.101]) by mx1.freebsd.org (Postfix) with ESMTP id 32ABF8FC14; Mon, 6 Dec 2010 22:47:42 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netmusician.org (Postfix) with ESMTP id D61E4B8B1; Mon, 6 Dec 2010 17:47:41 -0500 (EST) X-Virus-Scanned: amavisd-new at netmusician.org Received: from mail.netmusician.org ([127.0.0.1]) by localhost (dorian.netmusician.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id D4nTMjUETzmX; Mon, 6 Dec 2010 17:47:41 -0500 (EST) Received: from Shakti.local (c-71-201-100-167.hsd1.in.comcast.net [71.201.100.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.netmusician.org (Postfix) with ESMTPSA id 3AA04B87A; Mon, 6 Dec 2010 17:47:41 -0500 (EST) Message-ID: <4CFD6808.8010004@netmusician.org> Date: Mon, 06 Dec 2010 17:47:36 -0500 From: Joe Auty User-Agent: Postbox 2.0.2 (Macintosh/20101025) MIME-Version: 1.0 To: =?UTF-8?B?RWR3YXJkIFRvbWFzeiBOYXBpZXJhxYJh?= References: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> <4CFD5D73.1050601@netmusician.org> <4CFD6506.7090901@netmusician.org> <408E7ECD-C232-47DD-9D24-389F2CA4E406@FreeBSD.org> In-Reply-To: <408E7ECD-C232-47DD-9D24-389F2CA4E406@FreeBSD.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:47:42 -0000 Edward Tomasz Napiera=C5=82a wrote: > Wiadomo=C5=9B=C4=87 napisana przez Joe Auty w dniu 2010-12-06, o godz. = 23:34: >> Edward Tomasz Napiera=C5=82a wrote: >>> Wiadomo=C5=9B=C4=87 napisana przez Joe Auty w dniu 2010-12-06, o godz= . 23:02: >>>>> Also, make sure "ls -l" is not reporting "nobody". If the user/grou= p >>>>> name mapping isn't working, most Setattr Ops will fail. >>>>> >>>>> rick >>>>> >>>> Thanks Rick, >>>> >>>> I will look into this, but for the benefit of my own education, are >>>> NFSv4 ACLs supposed to be intertwined or separate from standard Unix >>>> permissions? I'm confused as to how the ACLs have changed from v3, o= r if >>>> this is even relevant to my problem not really knowing how they work= and >>>> why they are needed :) >>> Both POSIX.1e and NFSv4 ACLs are similar in that they both influence >>> the mode, and get influenced by it. In other words, when you change >>> the ACL, the mode gets updated; when you change the mode, the ACL get= s >>> updated. Also, for both POSIX.1e and NFSv4 ACLs, file mode continues >>> to work as usual if you ignore the ACL part. >>> >> Thanks for this! >> >> So, if I want to just ignore the NFSv4 ACLs on account of not needing >> anything beyond the POSIX ACLs, I'm free to do so without consequence.= .. >> Correct? > > If you want to just ignore the ACLs on account of not needing anything > beyond the file mode, aka standard UNIX permissions. Filesystems > support either POSIX.1e ACLs, or NFSv4 ACLs, not both. I didn't > actually test NFSv4, but I guess it uses NFSv4 ACLs, not POSIX.1e. > ZFS supports NFSv4 only. UFS supports either POSIX.1e or NFSv4, > depending on the mount options. I might be misunderstanding you, but ZFS definitely supports NFSv3 because I've been mounting and using NFS volumes via this protocol version for quite some time now without incident. > -- > If you cut off my head, what would I say? Me and my head, or me and my= body? > --=20 Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe@netmusician.org From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:54:46 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9806A106566C for ; Mon, 6 Dec 2010 22:54:46 +0000 (UTC) (envelope-from etnapierala@googlemail.com) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 200A28FC15 for ; Mon, 6 Dec 2010 22:54:45 +0000 (UTC) Received: by fxm16 with SMTP id 16so9932669fxm.13 for ; Mon, 06 Dec 2010 14:54:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:sender:subject:mime-version :content-type:from:in-reply-to:date:cc:content-transfer-encoding :message-id:references:to:x-mailer; bh=zdTbltgVRuzGbwiw5RPQzh02xEU6mELkJk9Llh1lmm0=; b=eFSFSCaNZc2xJzbcJ7Q+xfa5cmlhPawerGgwcXPQFkQfhcojUhNBGXLm7lk/WLFBOM sMX2fQHRAYFoBmJD2sJpXItqGjtVC9Q6W9+rMkpDmrux7px7B4K5+al+8G0VwOSmWRPB y9d56ZcZfKjKyOCvqA/mZyLW4B7Rch3hDj0M8= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=sender:subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; b=QvQkXLyT0V2QI/KmDk/xJTc7p5xp1RWkU9kZAFUeQg8RsA6t0fEsyiKzL9hUr5WNiF IkoQFRYsXC/kE72Mn3bBeqeRa08N82Fi9z6lceqQGfd11wCuhIpGqAfowJjGCkqgVawu IcCRm6ZTnsZ4+tzcT9FhtgYLAJyaDBFp7h97I= Received: by 10.223.100.4 with SMTP id w4mr5454326fan.26.1291676085055; Mon, 06 Dec 2010 14:54:45 -0800 (PST) Received: from [192.168.1.102] (45.81.datacomsa.pl [195.34.81.45]) by mx.google.com with ESMTPS id r24sm1737021fax.27.2010.12.06.14.54.43 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 06 Dec 2010 14:54:44 -0800 (PST) Sender: =?UTF-8?Q?Edward_Tomasz_Napiera=C5=82a?= Mime-Version: 1.0 (Apple Message framework v1082) Content-Type: text/plain; charset=iso-8859-2 From: =?iso-8859-2?Q?Edward_Tomasz_Napiera=B3a?= In-Reply-To: <4CFD6808.8010004@netmusician.org> Date: Mon, 6 Dec 2010 23:54:41 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> <4CFD5D73.1050601@netmusician.org> <4CFD6506.7090901@netmusician.org> <408E7ECD-C232-47DD-9D24-389F2CA4E406@FreeBSD.org> <4CFD6808.8010004@netmusician.org> To: Joe Auty X-Mailer: Apple Mail (2.1082) Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:54:46 -0000 Wiadomo=B6=E6 napisana przez Joe Auty w dniu 2010-12-06, o godz. 23:47: > Edward Tomasz Napiera=B3a wrote: >> Wiadomo=B6=E6 napisana przez Joe Auty w dniu 2010-12-06, o godz. = 23:34: >>> Edward Tomasz Napiera=B3a wrote: >>>> Wiadomo=B6=E6 napisana przez Joe Auty w dniu 2010-12-06, o godz. = 23:02: >>>>>> Also, make sure "ls -l" is not reporting "nobody". If the = user/group >>>>>> name mapping isn't working, most Setattr Ops will fail. >>>>>>=20 >>>>>> rick >>>>>>=20 >>>>> Thanks Rick, >>>>>=20 >>>>> I will look into this, but for the benefit of my own education, = are >>>>> NFSv4 ACLs supposed to be intertwined or separate from standard = Unix >>>>> permissions? I'm confused as to how the ACLs have changed from v3, = or if >>>>> this is even relevant to my problem not really knowing how they = work and >>>>> why they are needed :) >>>> Both POSIX.1e and NFSv4 ACLs are similar in that they both = influence >>>> the mode, and get influenced by it. In other words, when you = change >>>> the ACL, the mode gets updated; when you change the mode, the ACL = gets >>>> updated. Also, for both POSIX.1e and NFSv4 ACLs, file mode = continues >>>> to work as usual if you ignore the ACL part. >>>>=20 >>> Thanks for this! >>>=20 >>> So, if I want to just ignore the NFSv4 ACLs on account of not = needing >>> anything beyond the POSIX ACLs, I'm free to do so without = consequence... >>> Correct? >>=20 >> If you want to just ignore the ACLs on account of not needing = anything >> beyond the file mode, aka standard UNIX permissions. Filesystems >> support either POSIX.1e ACLs, or NFSv4 ACLs, not both. I didn't >> actually test NFSv4, but I guess it uses NFSv4 ACLs, not POSIX.1e. >> ZFS supports NFSv4 only. UFS supports either POSIX.1e or NFSv4, >> depending on the mount options. > I might be misunderstanding you, but ZFS definitely supports NFSv3 > because I've been mounting and using NFS volumes via this protocol > version for quite some time now without incident. Let me rephrase: ZFS only supports NFSv4 ACLs, it does not support POSIX.1e ACLs. Since ACLs are not a mandatory element of filesystem, sharing ZFS over NFSv3 works, but the client has no way to manipulate the ACLs or retrieve them. When sharing ZFS over NFSv4, the NFSv4 ACLs should work, I guess. Still, I'm not sure if the problem is actually ACL-related. -- If you cut off my head, what would I say? Me and my head, or me and my = body? From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:58:34 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 711351065670; Mon, 6 Dec 2010 22:58:34 +0000 (UTC) (envelope-from joe@netmusician.org) Received: from mail.netmusician.org (dorian.netmusician.org [66.244.95.101]) by mx1.freebsd.org (Postfix) with ESMTP id 3BA198FC12; Mon, 6 Dec 2010 22:58:33 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netmusician.org (Postfix) with ESMTP id 7AEC9B87C; Mon, 6 Dec 2010 17:58:33 -0500 (EST) X-Virus-Scanned: amavisd-new at netmusician.org Received: from mail.netmusician.org ([127.0.0.1]) by localhost (dorian.netmusician.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id Gfhk-BeZ2332; Mon, 6 Dec 2010 17:58:33 -0500 (EST) Received: from Shakti.local (c-71-201-100-167.hsd1.in.comcast.net [71.201.100.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.netmusician.org (Postfix) with ESMTPSA id BE2E0B87A; Mon, 6 Dec 2010 17:58:32 -0500 (EST) Message-ID: <4CFD6A96.9090502@netmusician.org> Date: Mon, 06 Dec 2010 17:58:30 -0500 From: Joe Auty User-Agent: Postbox 2.0.2 (Macintosh/20101025) MIME-Version: 1.0 To: =?ISO-8859-2?Q?Edward_Tomasz_Napiera=B3a?= References: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> <4CFD5D73.1050601@netmusician.org> <4CFD6506.7090901@netmusician.org> <408E7ECD-C232-47DD-9D24-389F2CA4E406@FreeBSD.org> <4CFD6808.8010004@netmusician.org> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:58:34 -0000 Edward Tomasz Napiera=B3a wrote: > Wiadomo=B6=E6 napisana przez Joe Auty w dniu 2010-12-06, o godz. 23:47: >> Edward Tomasz Napiera=B3a wrote: >>> Wiadomo=B6=E6 napisana przez Joe Auty w dniu 2010-12-06, o godz. 23:3= 4: >>>> Edward Tomasz Napiera=B3a wrote: >>>>> Wiadomo=B6=E6 napisana przez Joe Auty w dniu 2010-12-06, o godz. 23= :02: >>>>>>> Also, make sure "ls -l" is not reporting "nobody". If the user/gr= oup >>>>>>> name mapping isn't working, most Setattr Ops will fail. >>>>>>> >>>>>>> rick >>>>>>> >>>>>> Thanks Rick, >>>>>> >>>>>> I will look into this, but for the benefit of my own education, ar= e >>>>>> NFSv4 ACLs supposed to be intertwined or separate from standard Un= ix >>>>>> permissions? I'm confused as to how the ACLs have changed from v3,= or if >>>>>> this is even relevant to my problem not really knowing how they wo= rk and >>>>>> why they are needed :) >>>>> Both POSIX.1e and NFSv4 ACLs are similar in that they both influenc= e >>>>> the mode, and get influenced by it. In other words, when you chang= e >>>>> the ACL, the mode gets updated; when you change the mode, the ACL g= ets >>>>> updated. Also, for both POSIX.1e and NFSv4 ACLs, file mode continu= es >>>>> to work as usual if you ignore the ACL part. >>>>> >>>> Thanks for this! >>>> >>>> So, if I want to just ignore the NFSv4 ACLs on account of not needin= g >>>> anything beyond the POSIX ACLs, I'm free to do so without consequenc= e... >>>> Correct? >>> If you want to just ignore the ACLs on account of not needing anythin= g >>> beyond the file mode, aka standard UNIX permissions. Filesystems >>> support either POSIX.1e ACLs, or NFSv4 ACLs, not both. I didn't >>> actually test NFSv4, but I guess it uses NFSv4 ACLs, not POSIX.1e. >>> ZFS supports NFSv4 only. UFS supports either POSIX.1e or NFSv4, >>> depending on the mount options. >> I might be misunderstanding you, but ZFS definitely supports NFSv3 >> because I've been mounting and using NFS volumes via this protocol >> version for quite some time now without incident. > > Let me rephrase: ZFS only supports NFSv4 ACLs, it does not support > POSIX.1e ACLs. Since ACLs are not a mandatory element of filesystem, > sharing ZFS over NFSv3 works, but the client has no way to manipulate > the ACLs or retrieve them. When sharing ZFS over NFSv4, the NFSv4 ACLs > should work, I guess. > > Still, I'm not sure if the problem is actually ACL-related. > Well, since I'm not setting any or have any need to use them since I've been just fine with POSIX ACLs, they may be irrelevant. I only brought them up because I'm flying blind with trying to figure out my problem here, and in Googling I came across stuff related to the NFSv4 ACLs, which is one obvious change between v3 and v4. I'm going to try my same test on my CentOS box and will report back. Thanks all of your for your patience with this! --=20 Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe@netmusician.org From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:59:14 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D70B106566B for ; Mon, 6 Dec 2010 22:59:14 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id DE6948FC18 for ; Mon, 6 Dec 2010 22:59:13 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAEf5/EyDaFvO/2dsb2JhbACDT6BarkuQeYRWcwSEX4YP X-IronPort-AV: E=Sophos;i="4.59,307,1288584000"; d="scan'208";a="101529827" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 06 Dec 2010 17:59:13 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 10572B3E95; Mon, 6 Dec 2010 17:59:13 -0500 (EST) Date: Mon, 6 Dec 2010 17:59:13 -0500 (EST) From: Rick Macklem To: Joe Auty Message-ID: <2012614235.1261352.1291676353006.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4CFD6693.7080100@netmusician.org> MIME-Version: 1.0 X-Originating-IP: [174.114.46.215] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - IE8 (Win)/6.0.7_GA_2473.RHEL4_64) Content-Type: multipart/related; boundary="----=_Part_1261351_324535824.1291676353005" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:59:14 -0000 ------=_Part_1261351_324535824.1291676353005 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable > Okay, >=20 > Here is my dump command... The NFS host is 192.168.0.20: >=20 > # tcpdump -s 0 -w dumpfile.txt host 192.168.0.20 > tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size > 65535 bytes >=20 This won't put anything on the screen. It's dumping to dumpfile.txt (which is not a text file, but it doesn't matter what you call it). If you email me dumpfile.txt as an attachment, that was what I was referring to. >=20 > In NFS mount: >=20 > # ls -l > total 2 > -rw-r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 0 Dec=C2=A0 4 23:19 blah > -rw-r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 0 Dec=C2=A0 4 23:19 test2 > -rw-r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 0 Dec=C2=A0 4 23:19 test3 >=20 > # chown joe blah >=20 > (no response) >=20 > "joe" is indeed a local user on the NFS client side. >=20 > This is not generating any tcpdump output though. >=20 Do you mean that dumpfile.txt isn't growing while you do this or just that there isn't anything being printed in the window where tcpdump is running? > # ls -l > total 2 > -rw-r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 0 Dec=C2=A0 4 23:19 blah > -rw-r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 0 Dec=C2=A0 4 23:19 test2 > -rw-r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 0 Dec=C2=A0 4 23:19 test3 >=20 > No actual permission change >=20 You could try "chmod 600 blah" and see if that works? (It shouldn't care about uid<->username mapping.) >=20 > I created these files as root, so that much is being recognized... >=20 Yep, which suggests that the uid<->username mapping is working. rick ------=_Part_1261351_324535824.1291676353005-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 22:59:57 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1F07106564A for ; Mon, 6 Dec 2010 22:59:57 +0000 (UTC) (envelope-from joe@netmusician.org) Received: from mail.netmusician.org (dorian.netmusician.org [66.244.95.101]) by mx1.freebsd.org (Postfix) with ESMTP id A42028FC0A for ; Mon, 6 Dec 2010 22:59:56 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netmusician.org (Postfix) with ESMTP id 366E4B87C for ; Mon, 6 Dec 2010 17:59:56 -0500 (EST) X-Virus-Scanned: amavisd-new at netmusician.org Received: from mail.netmusician.org ([127.0.0.1]) by localhost (dorian.netmusician.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id jCO1lwsTxqnV for ; Mon, 6 Dec 2010 17:59:55 -0500 (EST) Received: from Shakti.local (c-71-201-100-167.hsd1.in.comcast.net [71.201.100.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.netmusician.org (Postfix) with ESMTPSA id AAF42B87A for ; Mon, 6 Dec 2010 17:59:55 -0500 (EST) Message-ID: <4CFD6AEA.1040502@netmusician.org> Date: Mon, 06 Dec 2010 17:59:54 -0500 From: Joe Auty User-Agent: Postbox 2.0.2 (Macintosh/20101025) MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <1124305635.1255931.1291670668724.JavaMail.root@erie.cs.uoguelph.ca> <4CFD6693.7080100@netmusician.org> <4CFD6AD1.6020706@netmusician.org> In-Reply-To: <4CFD6AD1.6020706@netmusician.org> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 22:59:57 -0000 Edward Tomasz Napiera=C5=82a wrote: > Wiadomo=C5=9B=C4=87 napisana przez Joe Auty w dniu 2010-12-06, o godz. = 23:41: >> Rick Macklem wrote: >>>> Hello, >>>> >>>> This is possibly a more fundamental non-FreeBSD specific set of >>>> questions, but ultimately this is relevant to usage on FreeBSD, so..= . >>>> >>>> I'm fairly certain that NFSv4 is supported under Solaris 10/ZFS and >>>> FreeBSD/ZFS via the standard "share" binary or the sharenfs ZFS >>>> property, right? >>>> >>>> In mounting an NFS share on my FreeBSD test machine via the followin= g: >>>> >>>> mount -t nfs -o rw,nfsv4 ipaddress:/share /path/to/share/directory >>>> >>>> I'm unable to change the permissions of any of these files via a >>>> standard chmod on the client (FreeBSD) side. What are NFSv4 ACLs, an= d >>>> is >>>> this in any way relevant to my problem here? Do ACLs need to be set = in >>>> order to use a volume like I can an NFSv3 volume, which works just >>>> fine >>>> for me? >>>> >>> It might be worth capturing packets "tcpdump -s 0 -w xxx host " >>> while trying a "chmod" and seeing what goes over the wire. You can lo= ok >>> at it via wireshark or email me "xxx" and I can take a look. >>> >>> I don't know anything about ZFS, but you could try getfacl/setfacl on= the >>> client and see what happens? >>> >>> Edward Napierala (trasz@freebsd.org) did commit a recent change w.r.t= . >>> NFSv4 ACLs and I remember the discussion saying something like "after >>> this change, chmod no longer does anything once ACLs are enabled, but= I >>> have no idea if it is relevant. >>> >>> Also, make sure "ls -l" is not reporting "nobody". If the user/group >>> name mapping isn't working, most Setattr Ops will fail. >>> >> Okay, >> >> Here is my dump command... The NFS host is 192.168.0.20: >> >> # tcpdump -s 0 -w dumpfile.txt host 192.168.0.20 >> tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size >> 65535 bytes > > Looks like the mailing list ate the attachment. Nope... The dump file is empty so I didn't bother with it. Well, it contains "######", but that's probably not terribly useful :) > -- > If you cut off my head, what would I say? Me and my head, or me and my= body? > --=20 Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe@netmusician.org From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 23:03:07 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E6091065672 for ; Mon, 6 Dec 2010 23:03:07 +0000 (UTC) (envelope-from joe@netmusician.org) Received: from mail.netmusician.org (dorian.netmusician.org [66.244.95.101]) by mx1.freebsd.org (Postfix) with ESMTP id 5E8928FC19 for ; Mon, 6 Dec 2010 23:03:07 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netmusician.org (Postfix) with ESMTP id 979A3B87C; Mon, 6 Dec 2010 18:03:06 -0500 (EST) X-Virus-Scanned: amavisd-new at netmusician.org Received: from mail.netmusician.org ([127.0.0.1]) by localhost (dorian.netmusician.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 7KTU9eRTHCMC; Mon, 6 Dec 2010 18:03:06 -0500 (EST) Received: from Shakti.local (c-71-201-100-167.hsd1.in.comcast.net [71.201.100.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.netmusician.org (Postfix) with ESMTPSA id 09A7BB87A; Mon, 6 Dec 2010 18:03:06 -0500 (EST) Message-ID: <4CFD6BA8.8060902@netmusician.org> Date: Mon, 06 Dec 2010 18:03:04 -0500 From: Joe Auty User-Agent: Postbox 2.0.2 (Macintosh/20101025) MIME-Version: 1.0 To: Rick Macklem References: <2012614235.1261352.1291676353006.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <2012614235.1261352.1291676353006.JavaMail.root@erie.cs.uoguelph.ca> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 23:03:07 -0000 Rick Macklem wrote: >> Okay, >> >> Here is my dump command... The NFS host is 192.168.0.20: >> >> # tcpdump -s 0 -w dumpfile.txt host 192.168.0.20 >> tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size >> 65535 bytes >> > This won't put anything on the screen. It's dumping to dumpfile.txt > (which is not a text file, but it doesn't matter what you call it). > If you email me dumpfile.txt as an attachment, that was what I was > referring to. > See my last email, my dumpfile.txt was empty. Sorry for this confusion! >> # ls -l >> total 2 >> -rw-r--r-- 1 root wheel 0 Dec 4 23:19 blah >> -rw-r--r-- 1 root wheel 0 Dec 4 23:19 test2 >> -rw-r--r-- 1 root wheel 0 Dec 4 23:19 test3 >> >> No actual permission change >> > You could try "chmod 600 blah" and see if that works? (It shouldn't > care about uid<->username mapping.) > This actually works: # chmod 600 blah # ls -l total 2 -rw------- 1 root wheel 0 Dec 4 23:19 blah -rw-r--r-- 1 root wheel 0 Dec 4 23:19 test2 -rw-r--r-- 1 root wheel 0 Dec 4 23:19 test3 -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe@netmusician.org From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 23:06:43 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBDA61065672; Mon, 6 Dec 2010 23:06:43 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 573F68FC19; Mon, 6 Dec 2010 23:06:43 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAKD7/EyDaFvO/2dsb2JhbACDT6BarjWQeYMFgVFzBIRfhg8 X-IronPort-AV: E=Sophos;i="4.59,307,1288584000"; d="scan'208";a="101530598" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 06 Dec 2010 18:06:42 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 533EBB3F21; Mon, 6 Dec 2010 18:06:42 -0500 (EST) Date: Mon, 6 Dec 2010 18:06:42 -0500 (EST) From: Rick Macklem To: Joe Auty Message-ID: <1566415453.1261550.1291676802011.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4CFD679C.7020804@netmusician.org> MIME-Version: 1.0 X-Originating-IP: [174.114.46.215] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - IE8 (Win)/6.0.7_GA_2473.RHEL4_64) Content-Type: multipart/related; boundary="----=_Part_1261549_523196553.1291676802010" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, =?utf-8?Q?Edward_Tomasz_Napiera=C5=82a?= Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 23:06:43 -0000 ------=_Part_1261549_523196553.1291676802010 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit > Rick Macklem wrote: > > > > So, if I want to just ignore the NFSv4 ACLs on account of not needing > anything beyond the POSIX ACLs, I'm free to do so without > consequence... Correct? Well, NFSv4 won't be able to manipulate POSIX > ACLs (really POSIX.1e draft which was never ratified and, as such, > isn't a POSIX standard as I understand it). If you meant "beyond > chmod" then I think you will be ok, but I haven't used ZFS, so?? > Well, chmods and POSIX.1e ACLs work fine in NFSv3 with the same ZFS > server and everything else being the same on the FreeBSD site, so I > don't think that ZFS is the problem here unless ZFS has some sort of > NFSv4 host bug. > Ok, it depends on your definition of "works". I guess you mean that the ACLs define the protection applied to the file and can be manipulated locally on the server (or using chmod, given its limitations). NFSv3 knows nothing about ACLs, although Sun has an unpublished side-band protocol that allows a client that knows this protocol (FreeBSD's client doesn't) to manipulate the ACLs. For NFSv4, all the client does is allow the NFSv4 ACLs (not the POSIX.1e draft ones) to be manipulated via getfacl/setfacl at the client side. (It just translates the NFSv4 ACL between the form used by VOP_xxx() and the form that goes on the wire.) Generally the NFS server (at least a FreeBSD one) will simply expect the underlying VOP_xxx() calls to handle checking of the ACL. VOP_ACCESSX() is the main one for FreeBSD-CURRENT. (That's why I know diddly about ACLs, because NFS doesn't need to know about them:-) rick ------=_Part_1261549_523196553.1291676802010-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 23:17:22 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A90F1065675; Mon, 6 Dec 2010 23:17:22 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id C9D138FC2E; Mon, 6 Dec 2010 23:17:20 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEALv9/EyDaFvO/2dsb2JhbACDT6BarjOQeYRWcwSEX4YP X-IronPort-AV: E=Sophos;i="4.59,307,1288584000"; d="scan'208";a="103271885" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 06 Dec 2010 18:17:20 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 155FEB3F21; Mon, 6 Dec 2010 18:17:20 -0500 (EST) Date: Mon, 6 Dec 2010 18:17:20 -0500 (EST) From: Rick Macklem To: Joe Auty Message-ID: <1515785960.1261915.1291677440081.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4CFD6808.8010004@netmusician.org> MIME-Version: 1.0 X-Originating-IP: [174.114.46.215] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - IE8 (Win)/6.0.7_GA_2473.RHEL4_64) Content-Type: multipart/related; boundary="----=_Part_1261914_2017500749.1291677440080" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, =?utf-8?Q?Edward_Tomasz_Napiera=C5=82a?= Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 23:17:22 -0000 ------=_Part_1261914_2017500749.1291677440080 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit > I might be misunderstanding you, but ZFS definitely supports NFSv3 > because I've been mounting and using NFS volumes via this protocol > version for quite some time now without incident. > Yep, but you couldn't do a getfacl or setfacl in the client to manipulate the ACLs. On an NFSv4 mount, you should be able to do a getfacl or setfacl if the volume on the server supports NFSv4 ACLs. I suspect the failing "chown" doesn't have anything to do with ACLs. (It might be that the server doesn't know "joe" as a user, for example. In NFSv3, it would have sent "joe's" uid to the server, which is just a number it always trusts. For NFSv4, it will have sent "joe@" to the server and the NFS server must then know "joe" so it can turn that into "joe's" uid.) It just hit me that you said "joe" was a local user in the client? (For NFSv4 to work, the user names must be in the server's passwd database as well. Usually all the clients and servers share the same user and group databases via LDAP or NIS, but you can just copy /etc/passwd and /etc/group entries around, if you like. After updating the server's /etc/passwd or /etc/group, I don't know what you need to do to get Solaris's NFSv4 server to see the update. I always just reboot it. For a FreeBSD server, it should find additions. For deletions or changes to an entry, you can either wait for it to time out the cache or kill/restart the nfsuserd.) rick ------=_Part_1261914_2017500749.1291677440080-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 23:21:17 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B8C801065679; Mon, 6 Dec 2010 23:21:17 +0000 (UTC) (envelope-from joe@netmusician.org) Received: from mail.netmusician.org (dorian.netmusician.org [66.244.95.101]) by mx1.freebsd.org (Postfix) with ESMTP id 88C798FC1A; Mon, 6 Dec 2010 23:21:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail.netmusician.org (Postfix) with ESMTP id E0970B87C; Mon, 6 Dec 2010 18:21:16 -0500 (EST) X-Virus-Scanned: amavisd-new at netmusician.org Received: from mail.netmusician.org ([127.0.0.1]) by localhost (dorian.netmusician.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id oiA3KgzDXbaP; Mon, 6 Dec 2010 18:21:16 -0500 (EST) Received: from Shakti.local (c-71-201-100-167.hsd1.in.comcast.net [71.201.100.167]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by mail.netmusician.org (Postfix) with ESMTPSA id 485B9B87A; Mon, 6 Dec 2010 18:21:16 -0500 (EST) Message-ID: <4CFD6FE9.4020406@netmusician.org> Date: Mon, 06 Dec 2010 18:21:13 -0500 From: Joe Auty User-Agent: Postbox 2.0.2 (Macintosh/20101025) MIME-Version: 1.0 To: Rick Macklem References: <1515785960.1261915.1291677440081.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <1515785960.1261915.1291677440081.JavaMail.root@erie.cs.uoguelph.ca> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-2 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, =?ISO-8859-2?Q?Edward_Tomasz_Napiera=B3a?= Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 23:21:17 -0000 Rick Macklem wrote: >> I might be misunderstanding you, but ZFS definitely supports NFSv3 >> because I've been mounting and using NFS volumes via this protocol >> version for quite some time now without incident. >> > Yep, but you couldn't do a getfacl or setfacl in the client to > manipulate the ACLs. On an NFSv4 mount, you should be able to do > a getfacl or setfacl if the volume on the server supports NFSv4 ACLs. > > I suspect the failing "chown" doesn't have anything to do with ACLs. > (It might be that the server doesn't know "joe" as a user, for example. > In NFSv3, it would have sent "joe's" uid to the server, which is just > a number it always trusts. For NFSv4, it will have sent "joe@" > to the server and the NFS server must then know "joe" so it can turn > that into "joe's" uid.) > > It just hit me that you said "joe" was a local user in the client? > (For NFSv4 to work, the user names must be in the server's passwd > database as well. Usually all the clients and servers share the > same user and group databases via LDAP or NIS, but you can just > copy /etc/passwd and /etc/group entries around, if you like. > After updating the server's /etc/passwd or /etc/group, I don't > know what you need to do to get Solaris's NFSv4 server to see the > update. I always just reboot it. For a FreeBSD server, it should > find additions. For deletions or changes to an entry, you can > either wait for it to time out the cache or kill/restart the nfsuserd.) > > rick > Aha! Progress... This requirement is problematic for me right now for a variety of reasons including that I'm not using LDAP or NIS (although I will in the future). Is there anyway to get NFSv4 to behave like v3 in this respect so that these users don't need to exist on the NFS server side? -- Joe Auty, NetMusician NetMusician helps musicians, bands and artists create beautiful, professional, custom designed, career-essential websites that are easy to maintain and to integrate with popular social networks. www.netmusician.org joe@netmusician.org From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 23:22:24 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C8750106564A for ; Mon, 6 Dec 2010 23:22:24 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 85B078FC1A for ; Mon, 6 Dec 2010 23:22:24 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEACP//EyDaFvO/2dsb2JhbACDT6BarjGQeYEhgzVzBIRfhg8 X-IronPort-AV: E=Sophos;i="4.59,307,1288584000"; d="scan'208";a="101532179" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 06 Dec 2010 18:22:23 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id C1675B3E95; Mon, 6 Dec 2010 18:22:23 -0500 (EST) Date: Mon, 6 Dec 2010 18:22:23 -0500 (EST) From: Rick Macklem To: Joe Auty Message-ID: <827023472.1262049.1291677743726.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4CFD6AEA.1040502@netmusician.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [174.114.46.215] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - IE8 (Win)/6.0.7_GA_2473.RHEL4_64) Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 23:22:24 -0000 > > > > Looks like the mailing list ate the attachment. > Nope... The dump file is empty so I didn't bother with it. > > Well, it contains "######", but that's probably not terribly useful :) > It's a binary file. To see what's in it, you can: tcpdump -r dumpfile.txt although wireshark is much better for this, since it knows NFS packets. rick From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 23:26:53 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9A2011065670 for ; Mon, 6 Dec 2010 23:26:53 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 587C08FC18 for ; Mon, 6 Dec 2010 23:26:52 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEABMA/UyDaFvO/2dsb2JhbACDT6Bari+QeIMSgURzBIRfhg8 X-IronPort-AV: E=Sophos;i="4.59,307,1288584000"; d="scan'208";a="103272791" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 06 Dec 2010 18:26:52 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 45DEAB3E95; Mon, 6 Dec 2010 18:26:52 -0500 (EST) Date: Mon, 6 Dec 2010 18:26:52 -0500 (EST) From: Rick Macklem To: Joe Auty Message-ID: <380955649.1262172.1291678012262.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4CFD6BA8.8060902@netmusician.org> MIME-Version: 1.0 X-Originating-IP: [174.114.46.215] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - IE8 (Win)/6.0.7_GA_2473.RHEL4_64) Content-Type: multipart/related; boundary="----=_Part_1262171_304558095.1291678012261" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 23:26:53 -0000 ------=_Part_1262171_304558095.1291678012261 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable >=20 > This actually works: >=20 > # chmod 600 blah >=20 > # ls -l > total 2 > -rw-------=C2=A0 1 root=C2=A0 wheel=C2=A0 0 Dec=C2=A0 4 23:19 blah > -rw-r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 0 Dec=C2=A0 4 23:19 test2 > -rw-r--r--=C2=A0 1 root=C2=A0 wheel=C2=A0 0 Dec=C2=A0 4 23:19 test3 >=20 Ok, so it sounds to me like "joe" has to be added to the server's password database (and keep the uids the same, since you aren't using Kerberos). And I think you can forget about ACLs, although the discussion has been educational for me as well. rick ------=_Part_1262171_304558095.1291678012261-- From owner-freebsd-fs@FreeBSD.ORG Mon Dec 6 23:36:18 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2A949106564A; Mon, 6 Dec 2010 23:36:18 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-annu.mail.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id C93138FC0A; Mon, 6 Dec 2010 23:36:17 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ApwEAKwC/UyDaFvO/2dsb2JhbACDT6BarjyQeYRWcwSEX4YP X-IronPort-AV: E=Sophos;i="4.59,307,1288584000"; d="scan'208";a="101533478" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-annu-pri.mail.uoguelph.ca with ESMTP; 06 Dec 2010 18:36:16 -0500 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id F3CC7B3E95; Mon, 6 Dec 2010 18:36:16 -0500 (EST) Date: Mon, 6 Dec 2010 18:36:16 -0500 (EST) From: Rick Macklem To: Joe Auty Message-ID: <418685426.1262450.1291678576945.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <4CFD6FE9.4020406@netmusician.org> MIME-Version: 1.0 X-Originating-IP: [174.114.46.215] X-Mailer: Zimbra 6.0.7_GA_2476.RHEL4 (ZimbraWebClient - IE8 (Win)/6.0.7_GA_2473.RHEL4_64) Content-Type: multipart/related; boundary="----=_Part_1262449_480579799.1291678576944" X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-fs@freebsd.org, =?utf-8?Q?Edward_Tomasz_Napiera=C5=82a?= Subject: Re: Migrating from NFSv3 to v4 - NFSv4 ACL/permission confusion X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Dec 2010 23:36:18 -0000 ------=_Part_1262449_480579799.1291678576944 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit > > This requirement is problematic for me right now for a variety of > reasons including that I'm not using LDAP or NIS (although I will in > the future). Is there anyway to get NFSv4 to behave like v3 in this > respect so that these users don't need to exist on the NFS server > side? > Nope, sorry. NFSv4 uses names on the wire, so the server needs to know them as well as the client. (Early in NFSv4 development and testing, implementations were "cheating" and using names like "504", but that wasn't covered by the RFC and I don't know which ones will still do that. I've got a hunch that the FreeBSD code still has that in it, but only if there is no translation for the number. You could try doing a "chown NNN blah" locally on the server, where NN is joe's uid, and then see what the "ls -l" on the client shows, but that won't fix your "chown joe blah" case on the client.) So, unless you can put "joe" in the server's /etc/passwd, I think you'll need to stick with NFSv3 until you've adopted NIS or LDAP. rick ------=_Part_1262449_480579799.1291678576944-- From owner-freebsd-fs@FreeBSD.ORG Tue Dec 7 11:15:57 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E373C1065675 for ; Tue, 7 Dec 2010 11:15:57 +0000 (UTC) (envelope-from a.smith@ukgrid.net) Received: from mx0.ukgrid.net (mx0.ukgrid.net [89.21.28.41]) by mx1.freebsd.org (Postfix) with ESMTP id A0DAC8FC1A for ; Tue, 7 Dec 2010 11:15:57 +0000 (UTC) Received: from [89.21.28.38] (port=65209 helo=omicron.ukgrid.net) by mx0.ukgrid.net with esmtp (Exim 4.72; FreeBSD) envelope-from a.smith@ukgrid.net envelope-to freebsd-fs@freebsd.org id 1PPvWV-000GP4-FK; Tue, 07 Dec 2010 11:15:55 +0000 Received: from cpc2-mort2-0-0-cust506.croy.cable.virginmedia.com (cpc2-mort2-0-0-cust506.croy.cable.virginmedia.com [82.43.121.251]) by webmail2.ukgrid.net (Horde Framework) with HTTP; Tue, 07 Dec 2010 11:15:55 +0000 Message-ID: <20101207111555.17494gq2v9f0sias@webmail2.ukgrid.net> Date: Tue, 07 Dec 2010 11:15:55 +0000 From: a.smith@ukgrid.net To: freebsd-fs@freebsd.org References: <20101202114522.16601wx7crv5zhz4@webmail2.ukgrid.net> In-Reply-To: <20101202114522.16601wx7crv5zhz4@webmail2.ukgrid.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) H3 (4.3.9) / FreeBSD-8.1 Subject: Re: spurious ZFS "dataset does not exist" errors X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 11:16:02 -0000 Hi, I have now repeated this several times and continue to receive these weird errors. To confirm my steps to produce this, I am simply doing: zpool create nas1bkp raidz ada4p1 ada5p1 ada6p1 ada7p1 zfs send nas1/iscsi-cctv@another | zfs receive nas1bkp/iscsi-cctv The send takes over 12 hours as the volume size is nearly 3TB. I therefore used the disown command to allow the process to complete while not connected to the server. I tested this also using the nohup command with the same result. I also tested using a small test volume of just 100M and I don't get the same errors. So its only occuring with very large send/receive operations. If no one has any ideas, or any other tests I might do then I guess I'm just gona have to ignore these as the pool seems fine aparte from these... :S thanks Andy. Quoting a.smith@ukgrid.net: > Hi, > > I have been setting up a new ZFS pool to which I am replicate via > ZFS send/receive a (raw) volume from another pool. After creating > the duplicate volume in the new pool and having tested that it works > (it does) I exported the ZFS pool and it gives me the following > errors: > > # zpool export nas1bkp > cannot open 'nas1bkp/iscsi-cctvp3': dataset does not exist > cannot open 'nas1bkp/iscsi-cctvp2': dataset does not exist > cannot open 'nas1bkp/iscsi-cctvp1': dataset does not exist > > Its quite right, those datasets don't exist. But why is it even > trying to touch something that doesn't and has never existed. > Devices that actually exist are shown here: > > # zfs list > NAME USED AVAIL REFER MOUNTPOINT > nas1 3.19T 2.15T 648M /nas1 > nas1/cctv 2.53M 2.15T 2.53M /nas1/cctv > nas1/iscsi-cctv 3.19T 2.65T 2.65T - > nas1/zfssend 28.4K 50.0G 28.4K /nas1/zfssend > nas1bkp 3.19T 2.15T 28.4K /nas1bkp > nas1bkp/iscsi-cctv 3.19T 2.65T 2.65T - > nas1bkp/zfsrecv 26.9K 50.0G 26.9K /nas1bkp/zfsrecv > > > Anyone any idea whats going on? Spurious errors make me nervous! > > thanks Andy. > > > > From owner-freebsd-fs@FreeBSD.ORG Tue Dec 7 12:09:21 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2189910656AA for ; Tue, 7 Dec 2010 12:09:21 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [193.34.186.230]) by mx1.freebsd.org (Postfix) with ESMTP id B50DE8FC13 for ; Tue, 7 Dec 2010 12:09:20 +0000 (UTC) Received: from higson.cam.lispworks.com (IDENT:U2FsdGVkX18g78x/WK4i2RRUarHTsGe+S2Pa8gm5fp8@higson [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.14.3/8.14.3) with ESMTP id oB7C9Ifk085592; Tue, 7 Dec 2010 12:09:18 GMT (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com by higson.cam.lispworks.com (8.13.1) id oB7C9ILr006645; Tue, 7 Dec 2010 12:09:18 GMT Received: (from martin@localhost) by higson.cam.lispworks.com (8.13.1/8.13.1/Submit) id oB7C9In9006642; Tue, 7 Dec 2010 12:09:18 GMT Date: Tue, 7 Dec 2010 12:09:18 GMT Message-Id: <201012071209.oB7C9In9006642@higson.cam.lispworks.com> From: Martin Simmons To: freebsd-fs@freebsd.org In-reply-to: <20101206214223.GA6375@icarus.home.lan> (message from Jeremy Chadwick on Mon, 6 Dec 2010 13:42:23 -0800) References: <4CFBD7F9.3000307@ukr.net> <20101206214223.GA6375@icarus.home.lan> Subject: Re: How to choose the size of swap? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 12:09:21 -0000 >>>>> On Mon, 6 Dec 2010 13:42:23 -0800, Jeremy Chadwick said: > > And don't forget that (by default) you'll need a /var filesystem that > has enough physical room to hold the dump. You can change this using > the "dumpdir" option in rc.conf(5) if your existing /var isn't large > enough. Just make sure to copy /var/crash to /wherever, and make sure > the file/dir permissions are correct + minfree file exists. Is that working from swap-based dumps now? Last time I looked, the boot sequence enables swap before the dump is copied, so it is lost. __Martin From owner-freebsd-fs@FreeBSD.ORG Tue Dec 7 12:44:06 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 742EE1065670 for ; Tue, 7 Dec 2010 12:44:06 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [193.34.186.230]) by mx1.freebsd.org (Postfix) with ESMTP id EB37F8FC17 for ; Tue, 7 Dec 2010 12:44:05 +0000 (UTC) Received: from higson.cam.lispworks.com (IDENT:U2FsdGVkX1/Nee4iA+czCETnGmLI6oqj/zip57GVVLg@higson [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.14.3/8.14.3) with ESMTP id oB7Ci2Fb087765; Tue, 7 Dec 2010 12:44:02 GMT (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com by higson.cam.lispworks.com (8.13.1) id oB7Ci2G5006890; Tue, 7 Dec 2010 12:44:02 GMT Received: (from martin@localhost) by higson.cam.lispworks.com (8.13.1/8.13.1/Submit) id oB7Ci2AD006887; Tue, 7 Dec 2010 12:44:02 GMT Date: Tue, 7 Dec 2010 12:44:02 GMT Message-Id: <201012071244.oB7Ci2AD006887@higson.cam.lispworks.com> From: Martin Simmons To: freebsd-fs@freebsd.org In-reply-to: (message from Freddie Cash on Mon, 6 Dec 2010 13:31:14 -0800) References: <4524EE89-E0A2-4CCF-92F9-80BB709C4BF6@patpro.net> <4CFD558C.2060202@DataIX.net> MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Subject: Re: auto-mounting ZFS snapshots X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 12:44:06 -0000 >>>>> On Mon, 6 Dec 2010 13:31:14 -0800, Freddie Cash said: > > On Mon, Dec 6, 2010 at 1:28 PM, jhell wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > On 12/06/2010 16:13, Patrick Proniewski wrote: > >> On 6 déc. 2010, at 18:20, Freddie Cash wrote: > >> > >>> FreeBSD automounts the "hidden" .zfs filesystem hierarchy where > >>> snapshots are "stored" automatically.  No need for amd. > >> > >> The zfs man page reads: > >> > >> File  system snapshots can be accessed under the ".zfs/snapshot" > >> direc- tory in the root  of  the  file  system.  Snapshots  are > >> automatically mounted  on demand and may be unmounted at regular > >> intervals. The visi- bility of the ".zfs" directory can be controlled > >> by the "snapdir" prop- erty. > >> > >> "may be unmounted at regular intervals" <-- what about this? Do I > >> have to create a periodic script that'll unmount snapshots? And if > >> so, how can I make a script that'll unmount only snapshots mounted > >> for over, say, 1 hour? > > > > No this is a no worry type of thing. This is a transparent process that > > handles what is available. Just consider them mounted all the time but > > you will not see them. > > > > Play with it a little and see what it looks like. mount(1) will not even > > show them as mounted so there is no worries here. The correct thing > > happens to keep you from shooting yourself in the foot. > > Oh, that's right. Mount was updated in 8.x to not show ZFS snapshots, > so you don't even have to worry about it anymore. > > My previous reply applies to 7.x, though (which is what's running on > one of our backups boxes). In 8.0 at least, they are still shown in the output of /etc/periodic/security/200.chkmounts by default. __Martin From owner-freebsd-fs@FreeBSD.ORG Tue Dec 7 13:45:57 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D31A31065672 for ; Tue, 7 Dec 2010 13:45:56 +0000 (UTC) (envelope-from avg@freebsd.org) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 255398FC15 for ; Tue, 7 Dec 2010 13:45:55 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id PAA02366; Tue, 07 Dec 2010 15:45:51 +0200 (EET) (envelope-from avg@freebsd.org) Message-ID: <4CFE3A8F.9030102@freebsd.org> Date: Tue, 07 Dec 2010 15:45:51 +0200 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101029 Lightning/1.0b2 Thunderbird/3.1.6 MIME-Version: 1.0 To: Martin Simmons References: <4CFBD7F9.3000307@ukr.net> <20101206214223.GA6375@icarus.home.lan> <201012071209.oB7C9In9006642@higson.cam.lispworks.com> In-Reply-To: <201012071209.oB7C9In9006642@higson.cam.lispworks.com> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=KOI8-U Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: How to choose the size of swap? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 13:45:57 -0000 on 07/12/2010 14:09 Martin Simmons said the following: >>>>>> On Mon, 6 Dec 2010 13:42:23 -0800, Jeremy Chadwick said: >> >> And don't forget that (by default) you'll need a /var filesystem that >> has enough physical room to hold the dump. You can change this using >> the "dumpdir" option in rc.conf(5) if your existing /var isn't large >> enough. Just make sure to copy /var/crash to /wherever, and make sure >> the file/dir permissions are correct + minfree file exists. > > Is that working from swap-based dumps now? Last time I looked, the boot > sequence enables swap before the dump is copied, so it is lost. The sequence is that, but the dump should not be destroyed unless the swap gets actually used during that period. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Tue Dec 7 15:31:31 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EBF86106564A for ; Tue, 7 Dec 2010 15:31:31 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [IPv6:2a01:170:102f::2]) by mx1.freebsd.org (Postfix) with ESMTP id 71B3A8FC08 for ; Tue, 7 Dec 2010 15:31:31 +0000 (UTC) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.3/8.14.3) with ESMTP id oB7FVEdA047728; Tue, 7 Dec 2010 16:31:29 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.3/8.14.3/Submit) id oB7FVEhb047727; Tue, 7 Dec 2010 16:31:14 +0100 (CET) (envelope-from olli) Date: Tue, 7 Dec 2010 16:31:14 +0100 (CET) Message-Id: <201012071531.oB7FVEhb047727@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (FreeBSD/6.4-PRERELEASE-20080904 (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.3.5 (lurza.secnetix.de [127.0.0.1]); Tue, 07 Dec 2010 16:31:29 +0100 (CET) Cc: Subject: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 15:31:32 -0000 Hi, I've bought an OCZ Vertex2 E (120 GB SSD) and installed FreeBSD i386 stable/8 on it, using UFS (UFS2, to be exact). I've made sure that the partitions are aligned properly, and used newfs with 4k fragsize and 32k blocksize. It works very well so far. Now, the only thing missing would be TRIM support for UFS. I did some research and found the following patch: http://people.freebsd.org/~pjd/patches/ufsdel.patch Apparently that patch is supposed to make UFS use TRIM, among other things. However, the patch seems to be rather old (time stamp says 2007-02-03), and nothing has been comitted to current or stable so far (I've grepped through the past 12 months of SVN logs, after TRIM support was added to ada(4) by mav@). So, my question is, are there plans to add TRIM support to UFS? Is anyone working on it? Or is it already there and I just overlooked it? Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "A language that doesn't have everything is actually easier to program in than some that do." -- Dennis M. Ritchie From owner-freebsd-fs@FreeBSD.ORG Tue Dec 7 15:59:36 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 19A00106564A for ; Tue, 7 Dec 2010 15:59:36 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [IPv6:2a01:170:102f::2]) by mx1.freebsd.org (Postfix) with ESMTP id 7A11B8FC16 for ; Tue, 7 Dec 2010 15:59:35 +0000 (UTC) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.3/8.14.3) with ESMTP id oB7FxJee048701; Tue, 7 Dec 2010 16:59:34 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.3/8.14.3/Submit) id oB7FxJqf048700; Tue, 7 Dec 2010 16:59:19 +0100 (CET) (envelope-from olli) Date: Tue, 7 Dec 2010 16:59:19 +0100 (CET) Message-Id: <201012071559.oB7FxJqf048700@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG In-Reply-To: <201011171705.oAHH5age003849@lurza.secnetix.de> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (FreeBSD/6.4-PRERELEASE-20080904 (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.3.5 (lurza.secnetix.de [127.0.0.1]); Tue, 07 Dec 2010 16:59:34 +0100 (CET) Cc: Subject: Re: NFS hangs (7.3) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 15:59:36 -0000 Oliver Fromme wrote: > I've got a problem on a server farm. Every now and then, > some NFS mounts hang. This happens after a few days or > after a few weeks. All processes trying to access files > from the hanging mount go to state "D" and freeze. The > only way to resolve the problem is to reboot the server. > [...] > The machine is quite busy. The hangs seem to always occur > in the night when lots of cron jobs are running. The machine > has 221 NFS mounts and 26 nullfs mounts, and it has 26 jails, > if that matters. All NFS shares are mounted from a virtual > filer running on a NetApp filer. The mounts use the default > settings, so they should be v3 TCP (this is the default, > right?). The only extra option we use is -L in order to > "fake" locking locally. Shortly after I posted the above, I found out that stable/7 does *not* use TCP by default. tcpdump showed that NFS was using UDP. So I changed the mamangement scripts to force TCP when mounting NFS shares. So far, there were no further hangs. It's still possible that it might occur in the future (sometimes it took a few weeks to produce a hang), but it seems as if the problem is really fixed now. In case it happens again, I will follow Rick's and Kostik's advice (procstat -ka, ps axHl etc.). Thanks! Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "... there are two ways of constructing a software design: One way is to make it so simple that there are _obviously_ no deficiencies and the other way is to make it so complicated that there are no _obvious_ deficiencies." -- C.A.R. Hoare, ACM Turing Award Lecture, 1980 From owner-freebsd-fs@FreeBSD.ORG Tue Dec 7 16:02:10 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C70F106564A for ; Tue, 7 Dec 2010 16:02:10 +0000 (UTC) (envelope-from martin@lispworks.com) Received: from lwfs1-cam.cam.lispworks.com (mail.lispworks.com [193.34.186.230]) by mx1.freebsd.org (Postfix) with ESMTP id 9C0018FC0A for ; Tue, 7 Dec 2010 16:02:09 +0000 (UTC) Received: from higson.cam.lispworks.com (IDENT:U2FsdGVkX1+P6ZKq26XVKphh60cl3ZqwKF2rrruJ7g4@higson [192.168.1.7]) by lwfs1-cam.cam.lispworks.com (8.14.3/8.14.3) with ESMTP id oB7G26uK005552; Tue, 7 Dec 2010 16:02:06 GMT (envelope-from martin@lispworks.com) Received: from higson.cam.lispworks.com by higson.cam.lispworks.com (8.13.1) id oB7G26pi008334; Tue, 7 Dec 2010 16:02:06 GMT Received: (from martin@localhost) by higson.cam.lispworks.com (8.13.1/8.13.1/Submit) id oB7G26xN008331; Tue, 7 Dec 2010 16:02:06 GMT Date: Tue, 7 Dec 2010 16:02:06 GMT Message-Id: <201012071602.oB7G26xN008331@higson.cam.lispworks.com> From: Martin Simmons To: freebsd-fs@freebsd.org In-reply-to: <4CFE3A8F.9030102@freebsd.org> (message from Andriy Gapon on Tue, 07 Dec 2010 15:45:51 +0200) References: <4CFBD7F9.3000307@ukr.net> <20101206214223.GA6375@icarus.home.lan> <201012071209.oB7C9In9006642@higson.cam.lispworks.com> <4CFE3A8F.9030102@freebsd.org> Subject: Re: How to choose the size of swap? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 16:02:10 -0000 >>>>> On Tue, 07 Dec 2010 15:45:51 +0200, Andriy Gapon said: > > on 07/12/2010 14:09 Martin Simmons said the following: > >>>>>> On Mon, 6 Dec 2010 13:42:23 -0800, Jeremy Chadwick said: > >> > >> And don't forget that (by default) you'll need a /var filesystem that > >> has enough physical room to hold the dump. You can change this using > >> the "dumpdir" option in rc.conf(5) if your existing /var isn't large > >> enough. Just make sure to copy /var/crash to /wherever, and make sure > >> the file/dir permissions are correct + minfree file exists. > > > > Is that working from swap-based dumps now? Last time I looked, the boot > > sequence enables swap before the dump is copied, so it is lost. > > The sequence is that, but the dump should not be destroyed unless the swap gets > actually used during that period. Cool, I'm glad it works sometimes! I had a problem with it in 7.2, but I can't repeat it now in an 8.0 VM. __Martin From owner-freebsd-fs@FreeBSD.ORG Tue Dec 7 16:44:21 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 56596106564A for ; Tue, 7 Dec 2010 16:44:21 +0000 (UTC) (envelope-from a.smith@ukgrid.net) Received: from mx0.ukgrid.net (mx0.ukgrid.net [89.21.28.41]) by mx1.freebsd.org (Postfix) with ESMTP id 1DE6C8FC14 for ; Tue, 7 Dec 2010 16:44:20 +0000 (UTC) Received: from [89.21.28.38] (port=40279 helo=omicron.ukgrid.net) by mx0.ukgrid.net with esmtp (Exim 4.72; FreeBSD) envelope-from a.smith@ukgrid.net envelope-to freebsd-fs@freebsd.org id 1PQ0eK-000BUt-DA; Tue, 07 Dec 2010 16:44:20 +0000 Received: from 39757.net (39757.net [89.107.23.9]) by webmail2.ukgrid.net (Horde Framework) with HTTP; Tue, 07 Dec 2010 16:44:19 +0000 Message-ID: <20101207164419.16106usxfy2h0uo8@webmail2.ukgrid.net> Date: Tue, 07 Dec 2010 16:44:19 +0000 From: a.smith@ukgrid.net To: freebsd-fs@freebsd.org References: <20101202114522.16601wx7crv5zhz4@webmail2.ukgrid.net> <20101207111555.17494gq2v9f0sias@webmail2.ukgrid.net> In-Reply-To: <20101207111555.17494gq2v9f0sias@webmail2.ukgrid.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) H3 (4.3.9) / FreeBSD-8.1 Subject: Re: spurious ZFS "dataset does not exist" errors X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 16:44:21 -0000 This looks a bit similar to this bug: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6805659 With the exception that I actually never created these volumes. Can only guess they are temporarily created by ZFS on my system (perhaps due to the large size of the volume?). From owner-freebsd-fs@FreeBSD.ORG Tue Dec 7 16:49:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2F251065675 for ; Tue, 7 Dec 2010 16:49:00 +0000 (UTC) (envelope-from a.smith@ukgrid.net) Received: from mx0.ukgrid.net (mx0.ukgrid.net [89.21.28.41]) by mx1.freebsd.org (Postfix) with ESMTP id BA1A48FC12 for ; Tue, 7 Dec 2010 16:49:00 +0000 (UTC) Received: from [89.21.28.38] (port=26350 helo=omicron.ukgrid.net) by mx0.ukgrid.net with esmtp (Exim 4.72; FreeBSD) envelope-from a.smith@ukgrid.net envelope-to freebsd-fs@freebsd.org id 1PQ0ip-000BgO-F2; Tue, 07 Dec 2010 16:48:59 +0000 Received: from 39757.net (39757.net [89.107.23.9]) by webmail2.ukgrid.net (Horde Framework) with HTTP; Tue, 07 Dec 2010 16:48:59 +0000 Message-ID: <20101207164859.45873v283owzue04@webmail2.ukgrid.net> Date: Tue, 07 Dec 2010 16:48:59 +0000 From: a.smith@ukgrid.net To: freebsd-fs@freebsd.org References: <20101202114522.16601wx7crv5zhz4@webmail2.ukgrid.net> <20101207111555.17494gq2v9f0sias@webmail2.ukgrid.net> <20101207164419.16106usxfy2h0uo8@webmail2.ukgrid.net> In-Reply-To: <20101207164419.16106usxfy2h0uo8@webmail2.ukgrid.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) H3 (4.3.9) / FreeBSD-8.1 Subject: Re: spurious ZFS "dataset does not exist" errors X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 16:49:01 -0000 Yep was this! Hadn't initially noticed this was a duplicate bug, following the info on the main bug I checked under "/dev/zvol/nas1bkp" and there were the stale files which I have removed and now I get no errors when exporting the pool :) Andy. From owner-freebsd-fs@FreeBSD.ORG Tue Dec 7 17:04:26 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 52178106564A for ; Tue, 7 Dec 2010 17:04:26 +0000 (UTC) (envelope-from a.smith@ukgrid.net) Received: from mx0.ukgrid.net (mx0.ukgrid.net [89.21.28.41]) by mx1.freebsd.org (Postfix) with ESMTP id 153AF8FC0A for ; Tue, 7 Dec 2010 17:04:25 +0000 (UTC) Received: from [89.21.28.38] (port=20558 helo=omicron.ukgrid.net) by mx0.ukgrid.net with esmtp (Exim 4.72; FreeBSD) envelope-from a.smith@ukgrid.net envelope-to freebsd-fs@freebsd.org id 1PQ0xl-000C8T-D2; Tue, 07 Dec 2010 17:04:25 +0000 Received: from 39757.net (39757.net [89.107.23.9]) by webmail2.ukgrid.net (Horde Framework) with HTTP; Tue, 07 Dec 2010 17:04:24 +0000 Message-ID: <20101207170424.12166bgdunb3io00@webmail2.ukgrid.net> Date: Tue, 07 Dec 2010 17:04:24 +0000 From: a.smith@ukgrid.net To: freebsd-fs@freebsd.org References: <20101202114522.16601wx7crv5zhz4@webmail2.ukgrid.net> <20101207111555.17494gq2v9f0sias@webmail2.ukgrid.net> <20101207164419.16106usxfy2h0uo8@webmail2.ukgrid.net> <20101207164859.45873v283owzue04@webmail2.ukgrid.net> In-Reply-To: <20101207164859.45873v283owzue04@webmail2.ukgrid.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Internet Messaging Program (IMP) H3 (4.3.9) / FreeBSD-8.1 Subject: Re: spurious ZFS "dataset does not exist" errors X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Dec 2010 17:04:26 -0000 Ok, actually when I reimport the pool it recreates these files! Under /dev/zvol/mypool I see: iscsi-cctv iscsi-cctv@againp3 iscsi-cctv@anotherp3 iscsi-cctv@again iscsi-cctv@another iscsi-cctvp1 iscsi-cctv@againp1 iscsi-cctv@anotherp1 iscsi-cctvp2 iscsi-cctv@againp2 iscsi-cctv@anotherp2 iscsi-cctvp3 The snapshots of the volume also appear as "snapshot" "snapshotp1" "snapshotp2". If I look at the zvol directory for the original pool (from which I did the zfs send) I also see these multiple p1/p2 snapshot files but the volume itself doesnt have them. Anyway, I guess Im not going to loose to much sleep over it as it wouldnt appear to be anything serious... Andy. From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 01:40:44 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5CDFD1065670 for ; Wed, 8 Dec 2010 01:40:44 +0000 (UTC) (envelope-from gull@gull.us) Received: from mail-ey0-f178.google.com (mail-ey0-f178.google.com [209.85.215.178]) by mx1.freebsd.org (Postfix) with ESMTP id E12B98FC08 for ; Wed, 8 Dec 2010 01:40:43 +0000 (UTC) Received: by eyh5 with SMTP id 5so496728eyh.37 for ; Tue, 07 Dec 2010 17:40:42 -0800 (PST) MIME-Version: 1.0 Received: by 10.14.11.147 with SMTP id 19mr7145180eex.14.1291771091724; Tue, 07 Dec 2010 17:18:11 -0800 (PST) Received: by 10.14.53.67 with HTTP; Tue, 7 Dec 2010 17:18:11 -0800 (PST) X-Originating-IP: [64.81.163.112] In-Reply-To: <7554F69B-0E03-42E3-8BB2-FD7CFE19805A@pean.org> References: <4CF017C8-9D3C-483D-B508-5BF70E32AA14@pean.org> <4CF949B0.5060405@obsysa.net> <20101203210045.GB7429@lor.one-eyed-alien.net> <7554F69B-0E03-42E3-8BB2-FD7CFE19805A@pean.org> Date: Tue, 7 Dec 2010 17:18:11 -0800 Message-ID: From: David Brodbeck To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: Re: zfs snapshots. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 01:40:44 -0000 On Fri, Dec 3, 2010 at 1:25 PM, Peter Ankerst=E5l wrote: >>>> >>> >>> Yes, this is correct. But it still uses up 10MB from the parent >>> filesystem. This is a problem if you use both snapshots and quotas. >>> (and use alot of snapshots) >> >> You many be able to use the refquota attribute instead of quota unless >> you're trying to place quotas on filesystem hiearchies. >> >> -- Brooks > > Oh. Thanks. How could I miss that.. It seems I have to change some of the > "logic" in my setup for this to work as needed but overall this will work= fine. If this is an NFS share, you might want to test what happens if you set a refquota, then exceed it. I know on OpenSolaris there have been some bugs where you get a 'no space left on device' when you try to rm files, in that situation, making it difficult for a user to recover from exceeding their quota. It's possible FreeBSD doesn't suffer from this, but you should test it if it's likely to come up. From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 02:51:03 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DCF60106566B for ; Wed, 8 Dec 2010 02:51:03 +0000 (UTC) (envelope-from spork@bway.net) Received: from xena.bway.net (xena.bway.net [216.220.96.26]) by mx1.freebsd.org (Postfix) with ESMTP id A83F38FC17 for ; Wed, 8 Dec 2010 02:51:03 +0000 (UTC) Received: (qmail 10243 invoked by uid 0); 8 Dec 2010 02:24:20 -0000 Received: from smtp.bway.net (216.220.96.25) by xena.bway.net with (DHE-RSA-AES256-SHA encrypted) SMTP; 8 Dec 2010 02:24:20 -0000 Received: (qmail 10237 invoked by uid 90); 8 Dec 2010 02:24:20 -0000 Received: from unknown (HELO ?10.3.2.41?) (spork@96.57.144.66) by smtp.bway.net with (DHE-RSA-AES256-SHA encrypted) SMTP; 8 Dec 2010 02:24:20 -0000 Date: Tue, 7 Dec 2010 21:24:19 -0500 (EST) From: Charles Sprickman X-X-Sender: spork@hotlap.local To: freebsd-fs@freebsd.org Message-ID: User-Agent: Alpine 2.00 (OSX 1167 2008-08-23) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII Subject: "sharing" hot spares? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 02:51:03 -0000 All, This is probably an odd question... We're extensively using ZFS on a bunch of 1U boxes and packing all the drive bays full. This is all SATA, so I would like to have hot spares available, but without migrating everything to new enclosures, I'm not seeing an easy way to deal with this. Is there anything out there that would allow me to export a few drives from another box as block devices that the other boxes could *temporarily* use as hot spares until someone onsite could physically do a drive swap? I would also really like to share one drive amongst multiple hosts - I am making the likely over-optimistic assumption that I'll be dealing with no more than one drive failure per say, 3 hosts at a time. -hast doesn't seem quite right, as it's more about clustering than just exporting a device. -geom-gate looks like it might fill the bill, but I'm unsure as to whether it would allow an export of one drive to more than one box, and if it did, if it would have any way of letting a second host know the drive is busy. -iscsi seems like it might work as well, but I'm leery of anything not deemed stable/supported enough to be in the base system. Anything else? I'm also totally clueless as to how zfs would deal with having one drive coming over the network while the others are local. I assume it should work. Also if there was a reboot while it was running with a disk from this network spare "pool", could zfs then find the network drive it was using since it will be available later than the local drives? This whole scheme is just something that came to me a few minutes ago, my apologies if it sounds a bit nuts. Just trying to solve an interesting problem. Thanks, Charles From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 03:51:11 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 17E44106564A for ; Wed, 8 Dec 2010 03:51:11 +0000 (UTC) (envelope-from spawk@acm.poly.edu) Received: from acm.poly.edu (acm.poly.edu [128.238.9.200]) by mx1.freebsd.org (Postfix) with ESMTP id AACAC8FC14 for ; Wed, 8 Dec 2010 03:51:10 +0000 (UTC) Received: (qmail 7596 invoked from network); 8 Dec 2010 03:24:28 -0000 Received: from unknown (HELO ?192.168.0.2?) (spawk@96.224.221.101) by acm.poly.edu with CAMELLIA256-SHA encrypted SMTP; 8 Dec 2010 03:24:28 -0000 Message-ID: <4CFEFA69.8020904@acm.poly.edu> Date: Tue, 07 Dec 2010 22:24:25 -0500 From: Boris Kochergin User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.12) Gecko/20101031 Thunderbird/3.1.6 MIME-Version: 1.0 To: Charles Sprickman References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: "sharing" hot spares? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 03:51:11 -0000 On 12/07/10 21:24, Charles Sprickman wrote: > All, > > This is probably an odd question... We're extensively using ZFS on a > bunch of 1U boxes and packing all the drive bays full. This is all > SATA, so I would like to have hot spares available, but without > migrating everything to new enclosures, I'm not seeing an easy way to > deal with this. > > Is there anything out there that would allow me to export a few drives > from another box as block devices that the other boxes could > *temporarily* use as hot spares until someone onsite could physically > do a drive swap? > > I would also really like to share one drive amongst multiple hosts - I > am making the likely over-optimistic assumption that I'll be dealing > with no more than one drive failure per say, 3 hosts at a time. > > -hast doesn't seem quite right, as it's more about clustering than > just exporting a device. > > -geom-gate looks like it might fill the bill, but I'm unsure as to > whether it would allow an export of one drive to more than one box, > and if it did, if it would have any way of letting a second host know > the drive is busy. > > -iscsi seems like it might work as well, but I'm leery of anything not > deemed stable/supported enough to be in the base system. > > Anything else? > > I'm also totally clueless as to how zfs would deal with having one > drive coming over the network while the others are local. I assume it > should work. > > Also if there was a reboot while it was running with a disk from this > network spare "pool", could zfs then find the network drive it was > using since it will be available later than the local drives? > > This whole scheme is just something that came to me a few minutes ago, > my apologies if it sounds a bit nuts. Just trying to solve an > interesting problem. > > Thanks, > > Charles > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" Ahoy. a geom_gate device can indeed be exported to multiple hosts, and ZFS over geom_gate works fine. As for synchronization, you don't get any with geom_gate itself--you can attach a device for writing from multiple hosts. However, I also believe ZFS will not use a disk (unless maybe you force it) if it currently has ZFS metadata on it. If this is the case, you could leave it to ZFS not to use a spare that's already in use by another host. In this scenario, I suppose there might exist a race condition where two machines' disks fail at the same time, and they attempt to use the geom_gate spare at the same time. I'll leave it to you to decide whether to worry about this. -Boris From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 08:29:21 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B4EC9106564A for ; Wed, 8 Dec 2010 08:29:21 +0000 (UTC) (envelope-from canevet@embl.fr) Received: from emblmta1.embl.fr (emblmta1.embl.fr [193.49.43.176]) by mx1.freebsd.org (Postfix) with ESMTP id 54BD38FC17 for ; Wed, 8 Dec 2010 08:29:20 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.59,315,1288566000"; d="asc'?scan'208";a="768274" Received: from unknown (HELO [172.26.15.11]) ([172.26.15.11]) by emblmta1.embl.fr with ESMTP/TLS/DHE-RSA-CAMELLIA256-SHA; 08 Dec 2010 09:19:16 +0100 From: =?ISO-8859-1?Q?Micka=EBl_Can=E9vet?= To: freebsd-fs@freebsd.org Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-4q1nhe6690TwnJEq4n1i" Date: Wed, 08 Dec 2010 09:19:15 +0100 Message-ID: <1291796355.2420.12.camel@pc286.embl.fr> Mime-Version: 1.0 X-Mailer: Evolution 2.30.3 Subject: Zfs and Disc quota exceeded on 8.1-RELEASE X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 08:29:21 -0000 --=-4q1nhe6690TwnJEq4n1i Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hello, When I exceed quota on a ZFS volume on FreeBSD-8.1-RELEASE, I can't remove any file with rm. I know that there is a workaround by using cat /dev/null in a file, but that's not really user friendly and can't work when the volume is exported to Windows Clients via Samba. In tried with FreeBSD-8.1-STABLE-201011 and this bug seams to be solved. Is there a way to apply some patches on FreeBSD-8.1-RELEASE to fix this issue ? Thanks a lot. Micka=C3=ABl --=-4q1nhe6690TwnJEq4n1i Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEABECAAYFAkz/P4MACgkQZjBmN5Hi/YY9TACfZng7Pw8jYCy0HBDD+zUWuBnF E14AoJJh9SFStrqSOLr1agjq5qeGbptG =+cw4 -----END PGP SIGNATURE----- --=-4q1nhe6690TwnJEq4n1i-- From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 09:30:05 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A3E7106564A for ; Wed, 8 Dec 2010 09:30:05 +0000 (UTC) (envelope-from 000.fbsd@quip.cz) Received: from elsa.codelab.cz (elsa.codelab.cz [94.124.105.4]) by mx1.freebsd.org (Postfix) with ESMTP id 329198FC14 for ; Wed, 8 Dec 2010 09:30:03 +0000 (UTC) Received: from elsa.codelab.cz (localhost.codelab.cz [127.0.0.1]) by elsa.codelab.cz (Postfix) with ESMTP id A3E7C19E02F; Wed, 8 Dec 2010 10:30:00 +0100 (CET) Received: from [192.168.1.2] (ip-86-49-61-235.net.upcbroadband.cz [86.49.61.235]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by elsa.codelab.cz (Postfix) with ESMTPSA id 6ADBA19E02E; Wed, 8 Dec 2010 10:29:58 +0100 (CET) Message-ID: <4CFF5015.5030505@quip.cz> Date: Wed, 08 Dec 2010 10:29:57 +0100 From: Miroslav Lachman <000.fbsd@quip.cz> User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.9.1.15) Gecko/20101027 SeaMonkey/2.0.10 MIME-Version: 1.0 To: Charles Sprickman References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org Subject: Re: "sharing" hot spares? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 09:30:05 -0000 Charles Sprickman wrote: > All, > > This is probably an odd question... We're extensively using ZFS on a > bunch of 1U boxes and packing all the drive bays full. This is all SATA, > so I would like to have hot spares available, but without migrating > everything to new enclosures, I'm not seeing an easy way to deal with this. [...] Did you test if ZFS on FreeBSD can handle hot spares? The list time i tried this, ZFS cann't use spares, because in the [Open]Solaris world, there is some background daemon controling the spares, but there is none in the FreeBSD. So ZFS on FreeBSD was not using spares even if it is local drive. Miroslav Lachman From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 15:21:45 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 943C9106564A for ; Wed, 8 Dec 2010 15:21:45 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 096D18FC19 for ; Wed, 8 Dec 2010 15:21:44 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id 7AFB2456B1; Wed, 8 Dec 2010 16:21:43 +0100 (CET) Received: from localhost (pdawidek.whl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id D99D145684; Wed, 8 Dec 2010 16:21:37 +0100 (CET) Date: Wed, 8 Dec 2010 16:21:36 +0100 From: Pawel Jakub Dawidek To: Oliver Fromme Message-ID: <20101208152136.GG1692@garage.freebsd.pl> References: <201012071531.oB7FVEhb047727@lurza.secnetix.de> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="j2AXaZ4YhVcLc+PQ" Content-Disposition: inline In-Reply-To: <201012071531.oB7FVEhb047727@lurza.secnetix.de> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: freebsd-fs@FreeBSD.ORG, Kirk McKusick Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 15:21:45 -0000 --j2AXaZ4YhVcLc+PQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Dec 07, 2010 at 04:31:14PM +0100, Oliver Fromme wrote: > Hi, >=20 > I've bought an OCZ Vertex2 E (120 GB SSD) and installed > FreeBSD i386 stable/8 on it, using UFS (UFS2, to be exact). > I've made sure that the partitions are aligned properly, > and used newfs with 4k fragsize and 32k blocksize. > It works very well so far. >=20 > Now, the only thing missing would be TRIM support for UFS. > I did some research and found the following patch: >=20 > http://people.freebsd.org/~pjd/patches/ufsdel.patch >=20 > Apparently that patch is supposed to make UFS use TRIM, > among other things. However, the patch seems to be rather > old (time stamp says 2007-02-03), and nothing has been > comitted to current or stable so far (I've grepped through > the past 12 months of SVN logs, after TRIM support was > added to ada(4) by mav@). >=20 > So, my question is, are there plans to add TRIM support > to UFS? Is anyone working on it? Or is it already there > and I just overlooked it? I hacked up this patch mostly for Kris and md(4) memory-backed UFS, so on file remove space can be returned to the system. I think you should ask Kirk what to do about that, but I'm afraid my patch can break SU - what if we TRIM, but then panic and on fsck decide to actually use the block? BTW. Have you actually observed any performance degradation without TRIM? I've similar SSDs and from what I tested it somehow can handle wear leveling internally. You can to TRIM entire disk using this simple program below, newfs it and test it. Then fill it with random data, newfs it again, test it and compare results. =3D=3D=3D Makefile: PROG=3D trim LDADD=3D -lgeom NO_MAN=3D WARNS?=3D 6 =2Einclude =3D=3D=3D trim.c: #include #include #include int main(int argc, char *argv[]) { off_t mediasize; int fd; if (argc !=3D 2) errx(1, "usage: %s disk", getprogname()); if ((fd =3D g_open(argv[1], 1)) < 0) err(2, "g_open(%s)", argv[1]); if ((mediasize =3D g_mediasize(fd)) < 0) err(3, "g_mediasize()"); if (g_delete(fd, 0, mediasize) < 0) err(4, "g_delete()"); g_close(fd); exit(0); } --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --j2AXaZ4YhVcLc+PQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAkz/ooAACgkQForvXbEpPzQxgACdFFOUklSHI6O8E+cOZEW2ZSVs 9RcAnjQvx+GY87DCOrnLuVPh1Qz09w/2 =eezJ -----END PGP SIGNATURE----- --j2AXaZ4YhVcLc+PQ-- From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 15:50:35 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 33E2710656E5 for ; Wed, 8 Dec 2010 15:50:35 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-ww0-f50.google.com (mail-ww0-f50.google.com [74.125.82.50]) by mx1.freebsd.org (Postfix) with ESMTP id B40688FC17 for ; Wed, 8 Dec 2010 15:50:34 +0000 (UTC) Received: by wwf26 with SMTP id 26so1266027wwf.31 for ; Wed, 08 Dec 2010 07:50:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :organization:user-agent:mime-version:to:subject:references :in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; bh=oxikw3dxMV6EdkJ+2NVnIxmlatRjOPRTUk8pZuLnhZ4=; b=ncmrGsBlKh7ugGBoHgGONiQ+uj/eRVr0bHUMdgt1l/FhEmFMr88GQxS/A+NBth4vJm JRphqdSYuKAuiEPQIWfbiKrOhZZpolbEEbOcB3pi7d0EcSrYzqhBAm48qH9vf5GNnhH6 z6BChgBiZxamAb2nfoOn6w3udzsu52lIEgWVU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:organization:user-agent:mime-version:to :subject:references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=acnjL1oa67O12SUA2841d6yRB0fWCmyAIijPwjD11s51GNosE1CA4jqnvRjsdLO/1p EVheS5dQoTYkCqA1Ep9GPo0+OUGukeuU93CBCuCe2JpU+R0ksMv6juXlm7pC1ZDzvHv1 070bytQjs3ZY8gU1LQO0kzK0vlbSPKsMA23uE= Received: by 10.227.156.69 with SMTP id v5mr9082117wbw.189.1291823432561; Wed, 08 Dec 2010 07:50:32 -0800 (PST) Received: from centel.dataix.local (adsl-99-19-45-101.dsl.klmzmi.sbcglobal.net [99.19.45.101]) by mx.google.com with ESMTPS id m10sm495544wbc.16.2010.12.08.07.50.30 (version=SSLv3 cipher=RC4-MD5); Wed, 08 Dec 2010 07:50:31 -0800 (PST) Sender: "J. Hellenthal" Message-ID: <4CFFA945.2010104@DataIX.net> Date: Wed, 08 Dec 2010 10:50:29 -0500 From: jhell Organization: http://www.DataIX.net User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.12) Gecko/20101028 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: =?UTF-8?B?TWlja2HDq2wgQ2Fuw6l2ZXQ=?= , "freebsd-fs@freebsd.org >> FreeBSD Filesystems" References: <1291796355.2420.12.camel@pc286.embl.fr> In-Reply-To: <1291796355.2420.12.camel@pc286.embl.fr> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: Re: Zfs and Disc quota exceeded on 8.1-RELEASE X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 15:50:35 -0000 On 12/08/2010 03:19, Mickaël Canévet wrote: > Hello, > > When I exceed quota on a ZFS volume on FreeBSD-8.1-RELEASE, I can't > remove any file with rm. I know that there is a workaround by using > cat /dev/null in a file, but that's not really user friendly and can't > work when the volume is exported to Windows Clients via Samba. > > In tried with FreeBSD-8.1-STABLE-201011 and this bug seams to be solved. > > Is there a way to apply some patches on FreeBSD-8.1-RELEASE to fix this > issue ? I was not able to reproduce this on 8.2-PRERELEASE or later. Can you confirm ? Regards, -- jhell,v From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 15:54:41 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 63E2710656A9 for ; Wed, 8 Dec 2010 15:54:41 +0000 (UTC) (envelope-from mj@feral.com) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id 3037F8FC0C for ; Wed, 8 Dec 2010 15:54:40 +0000 (UTC) Received: from [192.168.1.100] (m206-63.dsl.tsoft.com [198.144.206.63]) by ns1.feral.com (8.14.4/8.14.3) with ESMTP id oB8FSi77003086 for ; Wed, 8 Dec 2010 07:28:44 -0800 (PST) (envelope-from mj@feral.com) Message-ID: <4CFFA42E.3040201@feral.com> Date: Wed, 08 Dec 2010 07:28:46 -0800 From: Matthew Jacob Organization: Feral Software User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <201012071531.oB7FVEhb047727@lurza.secnetix.de> <20101208152136.GG1692@garage.freebsd.pl> In-Reply-To: <20101208152136.GG1692@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Default is to whitelist mail, not delayed by milter-greylist-4.2.6 (ns1.feral.com [192.67.166.1]); Wed, 08 Dec 2010 07:28:45 -0800 (PST) Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 15:54:41 -0000 > BTW. Have you actually observed any performance degradation without > TRIM? I've similar SSDs and from what I tested it somehow can handle > wear leveling internally. You can to TRIM entire disk using this simple > program below, newfs it and test it. Then fill it with random data, > newfs it again, test it and compare results. Yes. I've seen performance degradation. It's not clear that TRIM is the most useful in all cases, partly because it's generally a very slow command and it's not quite clear how well the wear levelling algorithms incorporate TRIM changes. One thing is certainly true is that if SSD drives are for caching only, a SECURE ERASE at each 'mount' time along with the appropriate newfs helps substantially. From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 16:30:06 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C502610656C8 for ; Wed, 8 Dec 2010 16:30:06 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 6E5F38FC19 for ; Wed, 8 Dec 2010 16:30:06 +0000 (UTC) Received: by ywp6 with SMTP id 6so793441ywp.13 for ; Wed, 08 Dec 2010 08:30:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=SvQU0QQGeU8Xr6x/x6ytOe+6ONsfIpI1wuIeZRP1kJU=; b=QdZy43Gdi6AP2oSj1z6SIPE748uME6k20DNB9yVCWcffwqw8Ej1WiRwDKnICi6buP4 DwFtRTyYZO9bOegmFQUNkb3fapzEzc2mZ1wtmtrf2za7YsO/ndCq6BZg3JLdPpyRxa1m 3WMvFAQPnayxzLkaYbLdoWysxupIVdhrYv2cI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=lPMJdRNJO0NzE3irEOpkAM+LDTLXd15BDxZkruNnYvX9JyIUUxNZPq4V6FaMdXbLh0 vqCP2JgkJWNpffzLX6kRwcI3tsu2zeUCkwS421pC0F5LJdrNm4tR8Nhk9KMjmBssfaZt GgxDXfRCFLcT0oXuySSizZ8EDE2A7Ffm183Ls= MIME-Version: 1.0 Received: by 10.90.103.9 with SMTP id a9mr9748543agc.165.1291825805555; Wed, 08 Dec 2010 08:30:05 -0800 (PST) Received: by 10.90.226.2 with HTTP; Wed, 8 Dec 2010 08:30:05 -0800 (PST) In-Reply-To: <4CFFA945.2010104@DataIX.net> References: <1291796355.2420.12.camel@pc286.embl.fr> <4CFFA945.2010104@DataIX.net> Date: Wed, 8 Dec 2010 08:30:05 -0800 Message-ID: From: Freddie Cash To: jhell Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: "freebsd-fs@freebsd.org >> FreeBSD Filesystems" , =?UTF-8?B?TWlja2HDq2wgQ2Fuw6l2ZXQ=?= Subject: Re: Zfs and Disc quota exceeded on 8.1-RELEASE X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 16:30:06 -0000 On Wed, Dec 8, 2010 at 7:50 AM, jhell wrote: > On 12/08/2010 03:19, Micka=C3=ABl Can=C3=A9vet wrote: >> When I exceed quota on a ZFS volume on FreeBSD-8.1-RELEASE, I can't >> remove any file with rm. I know that there is a workaround by using >> cat /dev/null in a file, but that's not really user friendly and can't >> work when the volume is exported to Windows Clients via Samba. >> >> In tried with FreeBSD-8.1-STABLE-201011 and this bug seams to be solved. >> >> Is there a way to apply some patches on FreeBSD-8.1-RELEASE to fix this >> issue ? > > I was not able to reproduce this on 8.2-PRERELEASE or later. Can you > confirm ? Lol. It's right there in the part you quoted, 4 lines up from your quote. = :) "In tried with FreeBSD-8.1-STABLE-201011 and this bug seams to be solved." OP wants to know if there's anyway to backport the fix to 8.1-RELEASE. --=20 Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 16:58:27 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D8321065672; Wed, 8 Dec 2010 16:58:27 +0000 (UTC) (envelope-from olli@lurza.secnetix.de) Received: from lurza.secnetix.de (lurza.secnetix.de [IPv6:2a01:170:102f::2]) by mx1.freebsd.org (Postfix) with ESMTP id BF1878FC18; Wed, 8 Dec 2010 16:58:26 +0000 (UTC) Received: from lurza.secnetix.de (localhost [127.0.0.1]) by lurza.secnetix.de (8.14.3/8.14.3) with ESMTP id oB8Gw9vH010496; Wed, 8 Dec 2010 17:58:24 +0100 (CET) (envelope-from oliver.fromme@secnetix.de) Received: (from olli@localhost) by lurza.secnetix.de (8.14.3/8.14.3/Submit) id oB8Gw9w3010495; Wed, 8 Dec 2010 17:58:09 +0100 (CET) (envelope-from olli) Date: Wed, 8 Dec 2010 17:58:09 +0100 (CET) Message-Id: <201012081658.oB8Gw9w3010495@lurza.secnetix.de> From: Oliver Fromme To: freebsd-fs@FreeBSD.ORG, pjd@FreeBSD.ORG In-Reply-To: <20101208152136.GG1692@garage.freebsd.pl> X-Newsgroups: list.freebsd-fs User-Agent: tin/1.8.3-20070201 ("Scotasay") (UNIX) (FreeBSD/6.4-PRERELEASE-20080904 (i386)) MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.3.5 (lurza.secnetix.de [127.0.0.1]); Wed, 08 Dec 2010 17:58:24 +0100 (CET) Cc: Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 16:58:27 -0000 Pawel Jakub Dawidek wrote: > On Tue, Dec 07, 2010 at 04:31:14PM +0100, Oliver Fromme wrote: > > I've bought an OCZ Vertex2 E (120 GB SSD) and installed > > FreeBSD i386 stable/8 on it, using UFS (UFS2, to be exact). > > I've made sure that the partitions are aligned properly, > > and used newfs with 4k fragsize and 32k blocksize. > > It works very well so far. (I should also mention that I mounted all filesystems from the SSD with the "noatime" option, to reduce writes during normal operation.) > > So, my question is, are there plans to add TRIM support > > to UFS? Is anyone working on it? Or is it already there > > and I just overlooked it? > > I hacked up this patch mostly for Kris and md(4) memory-backed UFS, so > on file remove space can be returned to the system. I see. > I think you should ask Kirk what to do about that, but I'm afraid my > patch can break SU - what if we TRIM, but then panic and on fsck decide > to actually use the block? Oh, you're right. That could be a problem. Maybe it would be better to write a separate tool that performs TRIM commands on areas of the file system that are unused for a while. I also remember that mav@ wrote that the TRIM command is very slow. So, it's probably not feasible to execute it each time some blocks are freed, because it would make the file system much slower and nullify all advantages of the SSD. Just found his comment from r201139: "I have no idea whether it is normal, but for some reason it takes 200ms to handle any TRIM command on this drive, that was making delete extremely slow. But TRIM command is able to accept long list of LBAs and the length of that list seems doesn't affect it's execution time. Implemented request clusting algorithm allowed me to rise delete rate up to reasonable numbers, when many parallel DELETE requests running." > BTW. Have you actually observed any performance degradation without > TRIM? Not yet. My SSD is still very new. It carries only the base system (/home is on a normal 1TB disk), so not many writes happened so far. But as soon as I start doing more write access (buildworld + installworld, updating ports and so on), I expect that performance will degrade over time. I've also heard from several people on various mailing lists that the performance of their SSD drives got worse after some time. That performance degradation is caused by so-called "static wear leveling". The drive will have to move the contents of blocks that are never (or rarely) written to to other blocks, so they can be overwritten, in order to distribute wear equally over all blocks. If a block is known to be unused (which is the case when the drive is new, or after a TRIM command), the contents don't have to be moved, so the write operation is much faster. I think all modern SSD drives use static wear leveling. Without TRIM support in the file system, a work-around is to "newfs -E" the file system when the performance gets too bad. This requires a backup-restore cycle, of course, so it's a somewhat annoying. Another work-around is to leave some space unused, i.e. don't use 20% at the end of the SSD for any file systems, for example. Since those 20% are never written to, they are known to be unused to the SSD's firmware, so it can use them for wear leveling. This will postpone the performance degradation somewhat, but it won't completely avoid it, ultimately. And wasting some space is not a very satisfying solution either. > I've similar SSDs and from what I tested it somehow can handle > wear leveling internally. You can to TRIM entire disk using this simple > program below, newfs it and test it. It does basically the same as "newfs -E", right? > Then fill it with random data, newfs it again, test it and compare > results. Filling it just once will probably not have much of an effect. In fact, wear leveling will probably not kick in if you just fill the whole disk, because all blocks are used equally anyway. The performance degradation will only start to occur after a while (weeks or months) when some blocks are written much more often than others. In this situation, (static) wear leveling will kick in and start moving data in order to re-use seldom-written-to blocks. Best regards Oliver -- Oliver Fromme, secnetix GmbH & Co. KG, Marktplatz 29, 85567 Grafing b. M. Handelsregister: Registergericht Muenchen, HRA 74606, Geschäftsfuehrung: secnetix Verwaltungsgesellsch. mbH, Handelsregister: Registergericht Mün- chen, HRB 125758, Geschäftsführer: Maik Bachmann, Olaf Erb, Ralf Gebhart FreeBSD-Dienstleistungen, -Produkte und mehr: http://www.secnetix.de/bsd "C is quirky, flawed, and an enormous success." -- Dennis M. Ritchie. From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 17:10:45 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 961A91065672 for ; Wed, 8 Dec 2010 17:10:45 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta14.westchester.pa.mail.comcast.net (qmta14.westchester.pa.mail.comcast.net [76.96.59.212]) by mx1.freebsd.org (Postfix) with ESMTP id 403818FC2A for ; Wed, 8 Dec 2010 17:10:44 +0000 (UTC) Received: from omta12.westchester.pa.mail.comcast.net ([76.96.62.44]) by qmta14.westchester.pa.mail.comcast.net with comcast id gfEv1f0010xGWP85EhAlss; Wed, 08 Dec 2010 17:10:45 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta12.westchester.pa.mail.comcast.net with comcast id ghAf1f00W3LrwQ23YhAgqR; Wed, 08 Dec 2010 17:10:42 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 48F2D9B427; Wed, 8 Dec 2010 09:10:38 -0800 (PST) Date: Wed, 8 Dec 2010 09:10:38 -0800 From: Jeremy Chadwick To: Freddie Cash Message-ID: <20101208171038.GA73829@icarus.home.lan> References: <1291796355.2420.12.camel@pc286.embl.fr> <4CFFA945.2010104@DataIX.net> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-fs@freebsd.org >> FreeBSD Filesystems" , =?iso-8859-1?Q?Micka=EBl_Can=E9vet?= Subject: Re: Zfs and Disc quota exceeded on 8.1-RELEASE X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 17:10:45 -0000 On Wed, Dec 08, 2010 at 08:30:05AM -0800, Freddie Cash wrote: > On Wed, Dec 8, 2010 at 7:50 AM, jhell wrote: > > On 12/08/2010 03:19, Mickaël Canévet wrote: > >> When I exceed quota on a ZFS volume on FreeBSD-8.1-RELEASE, I can't > >> remove any file with rm. I know that there is a workaround by using > >> cat /dev/null in a file, but that's not really user friendly and can't > >> work when the volume is exported to Windows Clients via Samba. > >> > >> In tried with FreeBSD-8.1-STABLE-201011 and this bug seams to be solved. > >> > >> Is there a way to apply some patches on FreeBSD-8.1-RELEASE to fix this > >> issue ? > > > > I was not able to reproduce this on 8.2-PRERELEASE or later. Can you > > confirm ? > > Lol. It's right there in the part you quoted, 4 lines up from your quote. :) > > "In tried with FreeBSD-8.1-STABLE-201011 and this bug seams to be solved." > > OP wants to know if there's anyway to backport the fix to 8.1-RELEASE. Probably not. The OP should wait until 8.2-RELEASE happens and then upgrade, otherwise run RELENG_8 (STABLE). 8.2-RELEASE is literally right around the corner. Someone really needs to come up with a defunct template response to people who are running -RELEASE expecting bugs to be fixed; the bugs get fixed in -STABLE, if you want those fixes run -STABLE, otherwise wait until the next -RELEASE comes out. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 17:22:27 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 106EA106566C for ; Wed, 8 Dec 2010 17:22:27 +0000 (UTC) (envelope-from mj@feral.com) Received: from ns1.feral.com (ns1.feral.com [192.67.166.1]) by mx1.freebsd.org (Postfix) with ESMTP id B95F78FC15 for ; Wed, 8 Dec 2010 17:22:26 +0000 (UTC) Received: from [192.168.221.2] (remotevpn [192.168.221.2]) by ns1.feral.com (8.14.4/8.14.3) with ESMTP id oB8HMPJF003666 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Wed, 8 Dec 2010 09:22:25 -0800 (PST) (envelope-from mj@feral.com) Message-ID: <4CFFBEC9.6060405@feral.com> Date: Wed, 08 Dec 2010 09:22:17 -0800 From: Matthew Jacob Organization: Feral Software User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <201012081658.oB8Gw9w3010495@lurza.secnetix.de> In-Reply-To: <201012081658.oB8Gw9w3010495@lurza.secnetix.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender DNS name whitelisted, not delayed by milter-greylist-4.2.6 (ns1.feral.com [192.168.221.1]); Wed, 08 Dec 2010 09:22:26 -0800 (PST) Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 17:22:27 -0000 Perhaps a better solution here is a different filesystem? Since seek time is not an issue for SSDs, establishing large read-only errors with persistent large write journals (a la sprite) in one (large block) locale would be a better fit. From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 17:30:39 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2720C1065675 for ; Wed, 8 Dec 2010 17:30:39 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from out-0.mx.aerioconnect.net (out-0-36.mx.aerioconnect.net [216.240.47.96]) by mx1.freebsd.org (Postfix) with ESMTP id EBC1B8FC28 for ; Wed, 8 Dec 2010 17:30:38 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id oB8H7wMN011701; Wed, 8 Dec 2010 09:07:58 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id BD6812D6011; Wed, 8 Dec 2010 09:07:57 -0800 (PST) Message-ID: <4CFFBB6C.1050400@freebsd.org> Date: Wed, 08 Dec 2010 09:07:56 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: Oliver Fromme References: <201012081658.oB8Gw9w3010495@lurza.secnetix.de> In-Reply-To: <201012081658.oB8Gw9w3010495@lurza.secnetix.de> Content-Type: multipart/mixed; boundary="------------070506050709000605070007" X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: freebsd-fs@freebsd.org, pjd@freebsd.org Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 17:30:39 -0000 This is a multi-part message in MIME format. --------------070506050709000605070007 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit kirk does have some trim patches which he sent to me once.. let me look... hmmm ah here it is.. this may or may not be out of date. I'll let Kirk chime in if he thinks it 's worth it.. I include the email from him as an attachment, hopefully it wont get stripped by the list, but you should both get it.. julian On 12/8/10 8:58 AM, Oliver Fromme wrote: > Pawel Jakub Dawidek wrote: > > On Tue, Dec 07, 2010 at 04:31:14PM +0100, Oliver Fromme wrote: > > > I've bought an OCZ Vertex2 E (120 GB SSD) and installed > > > FreeBSD i386 stable/8 on it, using UFS (UFS2, to be exact). > > > I've made sure that the partitions are aligned properly, > > > and used newfs with 4k fragsize and 32k blocksize. > > > It works very well so far. > > (I should also mention that I mounted all filesystems from > the SSD with the "noatime" option, to reduce writes during > normal operation.) > > > > So, my question is, are there plans to add TRIM support > > > to UFS? Is anyone working on it? Or is it already there > > > and I just overlooked it? > > > > I hacked up this patch mostly for Kris and md(4) memory-backed UFS, so > > on file remove space can be returned to the system. > > I see. > > > I think you should ask Kirk what to do about that, but I'm afraid my > > patch can break SU - what if we TRIM, but then panic and on fsck decide > > to actually use the block? > > Oh, you're right. That could be a problem. > > Maybe it would be better to write a separate tool that > performs TRIM commands on areas of the file system that > are unused for a while. > > I also remember that mav@ wrote that the TRIM command is > very slow. So, it's probably not feasible to execute it > each time some blocks are freed, because it would make the > file system much slower and nullify all advantages of the > SSD. > > Just found his comment from r201139: > "I have no idea whether it is normal, but for some reason it takes 200ms > to handle any TRIM command on this drive, that was making delete extremely > slow. But TRIM command is able to accept long list of LBAs and the length of > that list seems doesn't affect it's execution time. Implemented request > clusting algorithm allowed me to rise delete rate up to reasonable numbers, > when many parallel DELETE requests running." > > > BTW. Have you actually observed any performance degradation without > > TRIM? > > Not yet. My SSD is still very new. It carries only the > base system (/home is on a normal 1TB disk), so not many > writes happened so far. But as soon as I start doing more > write access (buildworld + installworld, updating ports > and so on), I expect that performance will degrade over > time. > > I've also heard from several people on various mailing lists > that the performance of their SSD drives got worse after > some time. > > That performance degradation is caused by so-called "static > wear leveling". The drive will have to move the contents > of blocks that are never (or rarely) written to to other > blocks, so they can be overwritten, in order to distribute > wear equally over all blocks. If a block is known to be > unused (which is the case when the drive is new, or after > a TRIM command), the contents don't have to be moved, so > the write operation is much faster. I think all modern > SSD drives use static wear leveling. > > Without TRIM support in the file system, a work-around is > to "newfs -E" the file system when the performance gets > too bad. This requires a backup-restore cycle, of course, > so it's a somewhat annoying. > > Another work-around is to leave some space unused, i.e. > don't use 20% at the end of the SSD for any file systems, > for example. Since those 20% are never written to, they > are known to be unused to the SSD's firmware, so it can > use them for wear leveling. This will postpone the > performance degradation somewhat, but it won't completely > avoid it, ultimately. And wasting some space is not a > very satisfying solution either. > > > I've similar SSDs and from what I tested it somehow can handle > > wear leveling internally. You can to TRIM entire disk using this simple > > program below, newfs it and test it. > > It does basically the same as "newfs -E", right? > > > Then fill it with random data, newfs it again, test it and compare > > results. > > Filling it just once will probably not have much of an > effect. In fact, wear leveling will probably not kick > in if you just fill the whole disk, because all blocks > are used equally anyway. > > The performance degradation will only start to occur > after a while (weeks or months) when some blocks are > written much more often than others. In this situation, > (static) wear leveling will kick in and start moving > data in order to re-use seldom-written-to blocks. > > Best regards > Oliver > --------------070506050709000605070007 Content-Type: message/rfc822; name="Attached Message" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="Attached Message" Return-Path: X-Original-To: julian@elischer.org Delivered-To: julian@idiom.com X-Client-Authorized: MaGic Cook1e Received: from chez.mckusick.com (chez.mckusick.com [64.81.247.49]) by idiom.com (Postfix) with ESMTP id 04A872D6015 for ; Tue, 3 Nov 2009 20:48:08 -0800 (PST) Received: from chez.mckusick.com (localhost [127.0.0.1]) by chez.mckusick.com (8.14.3/8.14.3) with ESMTP id nA44m6eE095082; Tue, 3 Nov 2009 20:48:08 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <200911040448.nA44m6eE095082@chez.mckusick.com> To: Julian Elischer Subject: Re: UFS2 and TRIM command cc: Scott Long In-reply-to: <4AF0A8A2.4090305@elischer.org> Date: Tue, 03 Nov 2009 20:48:05 -0800 From: Kirk McKusick X-Idiom-Reporting: If this was spam, please report it to http://www.spamcop.net > Date: Tue, 03 Nov 2009 14:03:14 -0800 > From: Julian Elischer > To: Kirk McKusick > CC: Scott Long > Subject: UFS2 and TRIM command > > Kirk, > > You mentioned at BSDCan that you had done some work with the 'trim' > command to tell a drive that a filesystem was no longer intersted > in a particular block. > > can you let us know what the state of that work was and whether you > have done anything more on it? > > thanks, > > Julian Enclosed is my proposed patch that I sent to Poul_henning Kamp. He was working with flash media at the time and wanted notification when blocks were released so he could pre-clear them so they could be used directly when next allocated. i never heard back from him and consequently never followed up on it. ~Kirk =-=-=-= From: Kirk McKusick Date: Wed, 21 May 2008 13:19:18 -0700 To: Poul-Henning Kamp Subject: UFS and BIO_DELETE X-URL: http://WWW.McKusick.COM/ Reply-To: Kirk McKusick I enclose below my proposed patch to add BIO_DELETE to UFS (goes at the end of ffs_blkfree). As I have no way to test it, I am wondering if you could let me know if it works. Also, I am thinking of only enabling it for filesystems mounted with a new flag requesting the behavior since the geteblk is a rather expensive call for the usual no-op case. I did look at just allocating a `struct bio' as a local variable and using that, but it looked like I needed to also come up with a `producer' and/or `consumer' if I wanted to pass it to GEOM directly, so in the end I went with this more expensive solution. If there is an easy way to just pass a bio structure to GEOM, I would much prefer that approach. ~Kirk *** ffs_alloc.c Wed May 21 20:11:04 2008 --- ffs_alloc.c.new Wed May 21 20:10:50 2008 *************** *** 1945,1950 **** --- 1945,1962 ---- ACTIVECLEAR(fs, cg); UFS_UNLOCK(ump); bdwrite(bp); + /* + * Request that the block be cleared. + */ + bp = geteblk(size); + bp->b_iocmd = BIO_DELETE; + bp->b_vp = devvp; + bp->b_blkno = fsbtodb(fs, bno); + bp->b_offset = dbtob(bp->b_blkno); + bp->b_iooffset = bp->b_offset; + bp->b_bcount = size; + BUF_KERNPROC(bp); + BO_STRATEGY(&devvp->v_bufobj, bp); } #ifdef INVARIANTS --------------070506050709000605070007-- From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 21:49:14 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D212E106564A; Wed, 8 Dec 2010 21:49:14 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 32B518FC15; Wed, 8 Dec 2010 21:49:14 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id oB8LnAPe016649 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 8 Dec 2010 23:49:10 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id oB8LnAre042689; Wed, 8 Dec 2010 23:49:10 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id oB8Ln988042688; Wed, 8 Dec 2010 23:49:09 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Wed, 8 Dec 2010 23:49:09 +0200 From: Kostik Belousov To: Julian Elischer Message-ID: <20101208214909.GH33073@deviant.kiev.zoral.com.ua> References: <201012081658.oB8Gw9w3010495@lurza.secnetix.de> <4CFFBB6C.1050400@freebsd.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="v2Uk6McLiE8OV1El" Content-Disposition: inline In-Reply-To: <4CFFBB6C.1050400@freebsd.org> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.2 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_40, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme , Kirk McKusick Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 21:49:15 -0000 --v2Uk6McLiE8OV1El Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 08, 2010 at 09:07:56AM -0800, Julian Elischer wrote: >=20 > From: Kirk McKusick > Date: Wed, 21 May 2008 13:19:18 -0700 > To: Poul-Henning Kamp > Subject: UFS and BIO_DELETE > X-URL: http://WWW.McKusick.COM/ > Reply-To: Kirk McKusick >=20 > I enclose below my proposed patch to add BIO_DELETE to UFS > (goes at the end of ffs_blkfree). As I have no way to test > it, I am wondering if you could let me know if it works. >=20 > Also, I am thinking of only enabling it for filesystems mounted > with a new flag requesting the behavior since the geteblk is a > rather expensive call for the usual no-op case. >=20 > I did look at just allocating a `struct bio' as a local variable > and using that, but it looked like I needed to also come up with a > `producer' and/or `consumer' if I wanted to pass it to GEOM directly, > so in the end I went with this more expensive solution. If there > is an easy way to just pass a bio structure to GEOM, I would much > prefer that approach. >=20 > ~Kirk >=20 >=20 > *** ffs_alloc.c Wed May 21 20:11:04 2008 > --- ffs_alloc.c.new Wed May 21 20:10:50 2008 > *************** > *** 1945,1950 **** > --- 1945,1962 ---- > ACTIVECLEAR(fs, cg); > UFS_UNLOCK(ump); > bdwrite(bp); > + /* > + * Request that the block be cleared. > + */ > + bp =3D geteblk(size); > + bp->b_iocmd =3D BIO_DELETE; > + bp->b_vp =3D devvp; > + bp->b_blkno =3D fsbtodb(fs, bno); > + bp->b_offset =3D dbtob(bp->b_blkno); > + bp->b_iooffset =3D bp->b_offset; > + bp->b_bcount =3D size; > + BUF_KERNPROC(bp); > + BO_STRATEGY(&devvp->v_bufobj, bp); > } > =20 > #ifdef INVARIANTS >=20 The UFS devvp contains the pointer to struct g_consumer in devpp->v_bufobj->bo_private, so g_alloc_bio() and g_io_request() can be used to start BIO_DELETE command without fake bufer allocation. g_io_strategy() may be used as the sample. But, I have a question about the patch. The bitmap block in ffs_blkfree() is handled with delayed write, by bdwrite() call. I think that it is quite possible for BIO_DELETE to be executed before the bitmap block is written and on-disk bit is set in bitmap. Would not this allow for the blocks to be deleted before the bitmap is updated, and potentially before on-disk pointers are cleared that points to the blocks ? SU just rolls back the unfinished dependencies on write, but BIO_DELETE cannot. Also, shouldn't the BIO_DELETE only be issued for real devices, and not the snapshots ? diff --git a/sys/ufs/ffs/ffs_alloc.c b/sys/ufs/ffs/ffs_alloc.c index b740bbb..4ff57ae 100644 --- a/sys/ufs/ffs/ffs_alloc.c +++ b/sys/ufs/ffs/ffs_alloc.c @@ -83,6 +83,8 @@ __FBSDID("$FreeBSD$"); =20 #include =20 +#include + #include #include #include @@ -1850,6 +1852,7 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dephd) u_int cg; u_int8_t *blksfree; struct cdev *dev; + struct bio *bip; =20 cg =3D dtog(fs, bno); if (devvp->v_type =3D=3D VREG) { @@ -1962,6 +1965,16 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dephd) softdep_setup_blkfree(UFSTOVFS(ump), bp, bno, numfrags(fs, size), dephd); bdwrite(bp); + + if (devvp->v_type !=3D VREG) { + bip =3D g_alloc_bio(); + bip->bio_cmd =3D BIO_DELETE; + bip->bio_offset =3D dbtob(fsbtodb(fs, bno)); + bip->bio_done =3D g_destroy_bio; + bip->bio_length =3D size; + g_io_request(bip, + (struct g_consumer *)devvp->v_bufobj.bo_private); + } } =20 #ifdef INVARIANTS --v2Uk6McLiE8OV1El Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAkz//VUACgkQC3+MBN1Mb4hnDwCgqAJkWJ12JeMLqu4+Zq8t3sE9 AnYAnjFi+KlwMkY/sFhicTHejYSOelXv =ngSY -----END PGP SIGNATURE----- --v2Uk6McLiE8OV1El-- From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 22:23:54 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78D291065670; Wed, 8 Dec 2010 22:23:54 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [64.81.247.49]) by mx1.freebsd.org (Postfix) with ESMTP id 2E8518FC2A; Wed, 8 Dec 2010 22:23:53 +0000 (UTC) Received: from chez.mckusick.com (localhost [127.0.0.1]) by chez.mckusick.com (8.14.3/8.14.3) with ESMTP id oB8MNqGV020879; Wed, 8 Dec 2010 14:23:53 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <201012082223.oB8MNqGV020879@chez.mckusick.com> To: Kostik Belousov In-reply-to: <20101208214909.GH33073@deviant.kiev.zoral.com.ua> Date: Wed, 08 Dec 2010 14:23:52 -0800 From: Kirk McKusick Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 22:23:54 -0000 > Date: Wed, 8 Dec 2010 23:49:09 +0200 > From: Kostik Belousov > To: Julian Elischer > Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, > Oliver Fromme , > Kirk McKusick > Subject: Re: TRIM support for UFS? > > On Wed, Dec 08, 2010 at 09:07:56AM -0800, Julian Elischer wrote: > > > > From: Kirk McKusick > > Date: Wed, 21 May 2008 13:19:18 -0700 > > To: Poul-Henning Kamp > > Subject: UFS and BIO_DELETE > > X-URL: http://WWW.McKusick.COM/ > > Reply-To: Kirk McKusick > > > > I enclose below my proposed patch to add BIO_DELETE to UFS > > (goes at the end of ffs_blkfree). As I have no way to test > > it, I am wondering if you could let me know if it works. > > > > Also, I am thinking of only enabling it for filesystems mounted > > with a new flag requesting the behavior since the geteblk is a > > rather expensive call for the usual no-op case. > > > > I did look at just allocating a `struct bio' as a local variable > > and using that, but it looked like I needed to also come up with a > > `producer' and/or `consumer' if I wanted to pass it to GEOM directly, > > so in the end I went with this more expensive solution. If there > > is an easy way to just pass a bio structure to GEOM, I would much > > prefer that approach. > > > > ~Kirk > > > > > > *** ffs_alloc.c Wed May 21 20:11:04 2008 > > --- ffs_alloc.c.new Wed May 21 20:10:50 2008 > > *************** > > *** 1945,1950 **** > > --- 1945,1962 ---- > > ACTIVECLEAR(fs, cg); > > UFS_UNLOCK(ump); > > bdwrite(bp); > > + /* > > + * Request that the block be cleared. > > + */ > > + bp = geteblk(size); > > + bp->b_iocmd = BIO_DELETE; > > + bp->b_vp = devvp; > > + bp->b_blkno = fsbtodb(fs, bno); > > + bp->b_offset = dbtob(bp->b_blkno); > > + bp->b_iooffset = bp->b_offset; > > + bp->b_bcount = size; > > + BUF_KERNPROC(bp); > > + BO_STRATEGY(&devvp->v_bufobj, bp); > > } > > > > #ifdef INVARIANTS > > > The UFS devvp contains the pointer to struct g_consumer in > devpp->v_bufobj->bo_private, so g_alloc_bio() and g_io_request() can be > used to start BIO_DELETE command without fake bufer allocation. > g_io_strategy() may be used as the sample. Thanks for that update to moderize the patch! > But, I have a question about the patch. The bitmap block in ffs_blkfree() > is handled with delayed write, by bdwrite() call. I think that it is > quite possible for BIO_DELETE to be executed before the bitmap block > is written and on-disk bit is set in bitmap. Would not this allow > for the blocks to be deleted before the bitmap is updated, > and potentially before on-disk pointers are cleared that points to > the blocks ? SU just rolls back the unfinished dependencies on write, > but BIO_DELETE cannot. It is safe to do the BIO_DELETE here because at this point we have progressed through the file delete to the point where the inode that claimed this block has been updated on the disk with a NULL pointer. The only step that remains is to update the on-disk bitmap to reflect that the block is available. If the bitmap write is not done before a system crash the only action that will be taken with respect to the freed block is that either background fsck will restore it to the bitmap (SU) or the journal will restore it to the bitmap (SUJ). There is no case where either SU or SUJ will put it back into the file once we have reached this point. > Also, shouldn't the BIO_DELETE only be issued for real devices, > and not the snapshots ? They should also be issued for snapshots. The only time that snapshots give up blocks is when they are deleted. The blocks that they give up at that time are all copies of blocks that were later changed in the filesystem. With the snapshot gone there will be no remaining references to those blocks and they will be made available for reallocation. As such, it would be helpful if they had been TRIMMED. > diff --git a/sys/ufs/ffs/ffs_alloc.c b/sys/ufs/ffs/ffs_alloc.c > index b740bbb..4ff57ae 100644 > --- a/sys/ufs/ffs/ffs_alloc.c > +++ b/sys/ufs/ffs/ffs_alloc.c > @@ -83,6 +83,8 @@ __FBSDID("$FreeBSD$"); > > #include > > +#include > + > #include > #include > #include > @@ -1850,6 +1852,7 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dephd) > u_int cg; > u_int8_t *blksfree; > struct cdev *dev; > + struct bio *bip; > > cg = dtog(fs, bno); > if (devvp->v_type == VREG) { > @@ -1962,6 +1965,16 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dephd) > softdep_setup_blkfree(UFSTOVFS(ump), bp, bno, > numfrags(fs, size), dephd); > bdwrite(bp); > + > + if (devvp->v_type != VREG) { > + bip = g_alloc_bio(); > + bip->bio_cmd = BIO_DELETE; > + bip->bio_offset = dbtob(fsbtodb(fs, bno)); > + bip->bio_done = g_destroy_bio; > + bip->bio_length = size; > + g_io_request(bip, > + (struct g_consumer *)devvp->v_bufobj.bo_private); > + } > } > > #ifdef INVARIANTS The above patch looks good though I would do it unconditionally (e.g., also for snapshots). It seems sensible to make it conditional on a sysctl variable so that folks could experiment with it more easily. And I would leave it off by default as non-SSD disks are unlikely to benefit from it. If it does prove useful for SSD disks then I would make it conditional on a filesystem flag so that it is possible to enable it on a filesystem-by-filesystem basis (e.g., on for filesystems on the SSD and off for the rest). Kirk McKusick From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 22:33:37 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3A110106564A for ; Wed, 8 Dec 2010 22:33:37 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id C21518FC16 for ; Wed, 8 Dec 2010 22:33:36 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id C290B45C8A; Wed, 8 Dec 2010 23:33:34 +0100 (CET) Received: from localhost (89-73-192-49.dynamic.chello.pl [89.73.192.49]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 423C745B36; Wed, 8 Dec 2010 23:33:29 +0100 (CET) Date: Wed, 8 Dec 2010 23:33:27 +0100 From: Pawel Jakub Dawidek To: Kirk McKusick Message-ID: <20101208223327.GD1884@garage.freebsd.pl> References: <20101208214909.GH33073@deviant.kiev.zoral.com.ua> <201012082223.oB8MNqGV020879@chez.mckusick.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="JwB53PgKC5A7+0Ej" Content-Disposition: inline In-Reply-To: <201012082223.oB8MNqGV020879@chez.mckusick.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.6 required=4.5 tests=BAYES_00,RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: freebsd-fs@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 22:33:37 -0000 --JwB53PgKC5A7+0Ej Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 08, 2010 at 02:23:52PM -0800, Kirk McKusick wrote: > The above patch looks good though I would do it unconditionally > (e.g., also for snapshots). It seems sensible to make it conditional > on a sysctl variable so that folks could experiment with it more > easily. And I would leave it off by default as non-SSD disks are > unlikely to benefit from it. If it does prove useful for SSD disks > then I would make it conditional on a filesystem flag so that it > is possible to enable it on a filesystem-by-filesystem basis (e.g., > on for filesystems on the SSD and off for the rest). We already have a flag for disks: DISKFLAG_CANDELETE, which tells if the disk support TRIM or not. Next we should add BIO_GETATTR attribute for DISK class to return true if TRIM is supported. This way UFS can ask if TRIM is supported on mount and don't bother sending BIO_DELETE if it is not supported. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --JwB53PgKC5A7+0Ej Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk0AB7YACgkQForvXbEpPzShTQCdEkP+WCZcl+x7DX2SP/RsxdRH WboAoOAHVCwYe1VMjxRZQcY8ZKrw8vkI =TvTM -----END PGP SIGNATURE----- --JwB53PgKC5A7+0Ej-- From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 22:53:57 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8B982106566B; Wed, 8 Dec 2010 22:53:57 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id 6AB298FC13; Wed, 8 Dec 2010 22:53:56 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id oB8MrqC8020995 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Dec 2010 00:53:52 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id oB8MrqNc043217; Thu, 9 Dec 2010 00:53:52 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id oB8MrqC9043216; Thu, 9 Dec 2010 00:53:52 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 9 Dec 2010 00:53:52 +0200 From: Kostik Belousov To: Kirk McKusick Message-ID: <20101208225352.GK33073@deviant.kiev.zoral.com.ua> References: <20101208214909.GH33073@deviant.kiev.zoral.com.ua> <201012082223.oB8MNqGV020879@chez.mckusick.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="DfnuYBTqzt7sVGu3" Content-Disposition: inline In-Reply-To: <201012082223.oB8MNqGV020879@chez.mckusick.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-2.5 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_20, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 22:53:57 -0000 --DfnuYBTqzt7sVGu3 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 08, 2010 at 02:23:52PM -0800, Kirk McKusick wrote: > > Date: Wed, 8 Dec 2010 23:49:09 +0200 > > From: Kostik Belousov > > To: Julian Elischer > > Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, > > Oliver Fromme , > > Kirk McKusick > > Subject: Re: TRIM support for UFS? > >=20 > > On Wed, Dec 08, 2010 at 09:07:56AM -0800, Julian Elischer wrote: > > > > > > From: Kirk McKusick > > > Date: Wed, 21 May 2008 13:19:18 -0700 > > > To: Poul-Henning Kamp > > > Subject: UFS and BIO_DELETE > > > X-URL: http://WWW.McKusick.COM/ > > > Reply-To: Kirk McKusick > > > > > > I enclose below my proposed patch to add BIO_DELETE to UFS > > > (goes at the end of ffs_blkfree). As I have no way to test > > > it, I am wondering if you could let me know if it works. > > > > > > Also, I am thinking of only enabling it for filesystems mounted > > > with a new flag requesting the behavior since the geteblk is a > > > rather expensive call for the usual no-op case. > > > > > > I did look at just allocating a `struct bio' as a local variable > > > and using that, but it looked like I needed to also come up with a > > > `producer' and/or `consumer' if I wanted to pass it to GEOM directly, > > > so in the end I went with this more expensive solution. If there > > > is an easy way to just pass a bio structure to GEOM, I would much > > > prefer that approach. > > > > > > ~Kirk > > > > > > > > > *** ffs_alloc.c Wed May 21 20:11:04 2008 > > > --- ffs_alloc.c.new Wed May 21 20:10:50 2008 > > > *************** > > > *** 1945,1950 **** > > > --- 1945,1962 ---- > > > ACTIVECLEAR(fs, cg); > > > UFS_UNLOCK(ump); > > > bdwrite(bp); > > > + /* > > > + * Request that the block be cleared. > > > + */ > > > + bp =3D geteblk(size); > > > + bp->b_iocmd =3D BIO_DELETE; > > > + bp->b_vp =3D devvp; > > > + bp->b_blkno =3D fsbtodb(fs, bno); > > > + bp->b_offset =3D dbtob(bp->b_blkno); > > > + bp->b_iooffset =3D bp->b_offset; > > > + bp->b_bcount =3D size; > > > + BUF_KERNPROC(bp); > > > + BO_STRATEGY(&devvp->v_bufobj, bp); > > > } > > >=20 > > > #ifdef INVARIANTS > > > > > The UFS devvp contains the pointer to struct g_consumer in > > devpp->v_bufobj->bo_private, so g_alloc_bio() and g_io_request() can be > > used to start BIO_DELETE command without fake bufer allocation. > > g_io_strategy() may be used as the sample. >=20 > Thanks for that update to moderize the patch! >=20 > > But, I have a question about the patch. The bitmap block in ffs_blkfree= () > > is handled with delayed write, by bdwrite() call. I think that it is > > quite possible for BIO_DELETE to be executed before the bitmap block > > is written and on-disk bit is set in bitmap. Would not this allow > > for the blocks to be deleted before the bitmap is updated, > > and potentially before on-disk pointers are cleared that points to > > the blocks ? SU just rolls back the unfinished dependencies on write, > > but BIO_DELETE cannot. >=20 > It is safe to do the BIO_DELETE here because at this point we have > progressed through the file delete to the point where the inode that > claimed this block has been updated on the disk with a NULL pointer. > The only step that remains is to update the on-disk bitmap to reflect > that the block is available. If the bitmap write is not done before > a system crash the only action that will be taken with respect to > the freed block is that either background fsck will restore it to > the bitmap (SU) or the journal will restore it to the bitmap (SUJ). > There is no case where either SU or SUJ will put it back into the > file once we have reached this point. Thanks for the explanation. On the other hand, can my scenario be real for async mounts ? >=20 > > Also, shouldn't the BIO_DELETE only be issued for real devices, > > and not the snapshots ? >=20 > They should also be issued for snapshots. The only time that snapshots > give up blocks is when they are deleted. The blocks that they give up > at that time are all copies of blocks that were later changed in the > filesystem. With the snapshot gone there will be no remaining references > to those blocks and they will be made available for reallocation. As > such, it would be helpful if they had been TRIMMED. >=20 > > diff --git a/sys/ufs/ffs/ffs_alloc.c b/sys/ufs/ffs/ffs_alloc.c > > index b740bbb..4ff57ae 100644 > > --- a/sys/ufs/ffs/ffs_alloc.c > > +++ b/sys/ufs/ffs/ffs_alloc.c > > @@ -83,6 +83,8 @@ __FBSDID("$FreeBSD$"); > >=20 > > #include > >=20 > > +#include > > + > > #include > > #include > > #include > > @@ -1850,6 +1852,7 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, deph= d) > > u_int cg; > > u_int8_t *blksfree; > > struct cdev *dev; > > + struct bio *bip; > >=20 > > cg =3D dtog(fs, bno); > > if (devvp->v_type =3D=3D VREG) { > > @@ -1962,6 +1965,16 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dep= hd) > > softdep_setup_blkfree(UFSTOVFS(ump), bp, bno, > > numfrags(fs, size), dephd); > > bdwrite(bp); > > + > > + if (devvp->v_type !=3D VREG) { > > + bip =3D g_alloc_bio(); > > + bip->bio_cmd =3D BIO_DELETE; > > + bip->bio_offset =3D dbtob(fsbtodb(fs, bno)); > > + bip->bio_done =3D g_destroy_bio; > > + bip->bio_length =3D size; > > + g_io_request(bip, > > + (struct g_consumer *)devvp->v_bufobj.bo_private); > > + } > > } > >=20 > > #ifdef INVARIANTS >=20 > The above patch looks good though I would do it unconditionally > (e.g., also for snapshots). It seems sensible to make it conditional > on a sysctl variable so that folks could experiment with it more > easily. And I would leave it off by default as non-SSD disks are > unlikely to benefit from it. If it does prove useful for SSD disks > then I would make it conditional on a filesystem flag so that it > is possible to enable it on a filesystem-by-filesystem basis (e.g., > on for filesystems on the SSD and off for the rest). I think it is better to have a flag in superblock instead of the mount option.=20 diff --git a/sbin/dumpfs/dumpfs.c b/sbin/dumpfs/dumpfs.c index 38c05f6..fa562dc 100644 --- a/sbin/dumpfs/dumpfs.c +++ b/sbin/dumpfs/dumpfs.c @@ -253,9 +253,11 @@ dumpfs(const char *name) printf("fs_flags expanded "); if (fsflags & FS_NFS4ACLS) printf("nfsv4acls "); + if (fsflags & FS_TRIM) + printf("trim "); fsflags &=3D ~(FS_UNCLEAN | FS_DOSOFTDEP | FS_NEEDSFSCK | FS_INDEXDIRS | FS_ACLS | FS_MULTILABEL | FS_GJOURNAL | FS_FLAGS_UPDATED | - FS_NFS4ACLS | FS_SUJ); + FS_NFS4ACLS | FS_SUJ | FS_TRIM); if (fsflags !=3D 0) printf("unknown flags (%#x)", fsflags); putchar('\n'); diff --git a/sbin/tunefs/tunefs.c b/sbin/tunefs/tunefs.c index b2ea602..0ed713d 100644 --- a/sbin/tunefs/tunefs.c +++ b/sbin/tunefs/tunefs.c @@ -82,11 +82,13 @@ int main(int argc, char *argv[]) { char *avalue, *jvalue, *Jvalue, *Lvalue, *lvalue, *Nvalue, *nvalue; + char *tvalue; const char *special, *on; const char *name; int active; int Aflag, aflag, eflag, evalue, fflag, fvalue, jflag, Jflag, Lflag; int lflag, mflag, mvalue, Nflag, nflag, oflag, ovalue, pflag, sflag; + int tflag; int svalue, Sflag, Svalue; int ch, found_arg, i; const char *chg[2]; @@ -101,7 +103,8 @@ main(int argc, char *argv[]) evalue =3D fvalue =3D mvalue =3D ovalue =3D svalue =3D Svalue =3D 0; active =3D 0; found_arg =3D 0; /* At least one arg is required. */ - while ((ch =3D getopt(argc, argv, "Aa:e:f:j:J:L:l:m:N:n:o:ps:S:")) !=3D -= 1) + while ((ch =3D getopt(argc, argv, "Aa:e:f:j:J:L:l:m:N:n:o:ps:S:t:")) + !=3D -1) switch (ch) { =20 case 'A': @@ -268,6 +271,18 @@ main(int argc, char *argv[]) Sflag =3D 1; break; =20 + case 't': + found_arg =3D 1; + name =3D "trim"; + tvalue =3D optarg; + if (strcmp(tvalue, "enable") !=3D 0 && + strcmp(tvalue, "disable") !=3D 0) { + errx(10, "bad %s (options are %s)", + name, "`enable' or `disable'"); + } + tflag =3D 1; + break; + default: usage(); } @@ -493,6 +508,24 @@ main(int argc, char *argv[]) sblock.fs_avgfpdir =3D svalue; } } + if (tflag) { + name =3D "issue TRIM to the disk"; + if (strcmp(tvalue, "enable") =3D=3D 0) { + if (sblock.fs_flags & FS_TRIM) + warnx("%s remains unchanged as enabled", name); + else { + sblock.fs_flags |=3D FS_TRIM; + warnx("%s set", name); + } + } else if (strcmp(tvalue, "disable") =3D=3D 0) { + if ((~sblock.fs_flags & FS_TRIM) =3D=3D FS_TRIM) + warnx("%s remains unchanged as disabled", name); + else { + sblock.fs_flags &=3D ~FS_TRIM; + warnx("%s cleared", name); + } + } + } =20 if (sbwrite(&disk, Aflag) =3D=3D -1) goto err; @@ -1011,12 +1044,13 @@ out: void usage(void) { - fprintf(stderr, "%s\n%s\n%s\n%s\n%s\n", + fprintf(stderr, "%s\n%s\n%s\n%s\n%s\n%s\n", "usage: tunefs [-A] [-a enable | disable] [-e maxbpg] [-f avgfilesize]", " [-J enable | disable] [-j enable | disable]",=20 " [-L volname] [-l enable | disable] [-m minfree]", " [-N enable | disable] [-n enable | disable]", -" [-o space | time] [-p] [-s avgfpdir] special | filesystem"); +" [-o space | time] [-p] [-s avgfpdir] [-t enable | disable]", +" special | filesystem"); exit(2); } =20 @@ -1035,6 +1069,8 @@ printfs(void) (sblock.fs_flags & FS_SUJ)? "enabled" : "disabled"); warnx("gjournal: (-J) %s", (sblock.fs_flags & FS_GJOURNAL)? "enabled" : "disabled"); + warnx("trim: (-p) %s",=20 + (sblock.fs_flags & FS_TRIM)? "enabled" : "disabled"); warnx("maximum blocks per file in a cylinder group: (-e) %d", sblock.fs_maxbpg); warnx("average file size: (-f) %d", diff --git a/sys/ufs/ffs/ffs_alloc.c b/sys/ufs/ffs/ffs_alloc.c index b740bbb..ed39aa3 100644 --- a/sys/ufs/ffs/ffs_alloc.c +++ b/sys/ufs/ffs/ffs_alloc.c @@ -83,6 +83,8 @@ __FBSDID("$FreeBSD$"); =20 #include =20 +#include + #include #include #include @@ -1850,6 +1852,7 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dephd) u_int cg; u_int8_t *blksfree; struct cdev *dev; + struct bio *bip; =20 cg =3D dtog(fs, bno); if (devvp->v_type =3D=3D VREG) { @@ -1962,6 +1965,16 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dephd) softdep_setup_blkfree(UFSTOVFS(ump), bp, bno, numfrags(fs, size), dephd); bdwrite(bp); + + if (fs->fs_flags & FS_TRIM) { + bip =3D g_alloc_bio(); + bip->bio_cmd =3D BIO_DELETE; + bip->bio_offset =3D dbtob(fsbtodb(fs, bno)); + bip->bio_done =3D g_destroy_bio; + bip->bio_length =3D size; + g_io_request(bip, + (struct g_consumer *)devvp->v_bufobj.bo_private); + } } =20 #ifdef INVARIANTS diff --git a/sys/ufs/ffs/fs.h b/sys/ufs/ffs/fs.h index ba98ed3..13d9ede 100644 --- a/sys/ufs/ffs/fs.h +++ b/sys/ufs/ffs/fs.h @@ -417,6 +417,7 @@ CTASSERT(sizeof(struct fs) =3D=3D 1376); #define FS_FLAGS_UPDATED 0x0080 /* flags have been moved to new location */ #define FS_NFS4ACLS 0x0100 /* file system has NFSv4 ACLs enabled */ #define FS_INDEXDIRS 0x0200 /* kernel supports indexed directories */ +#define FS_TRIM 0x0400 /* issue BIO_DELETE for deleted blocks */ =20 /* * Macros to access bits in the fs_active array. --DfnuYBTqzt7sVGu3 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0ADH8ACgkQC3+MBN1Mb4hV0gCglghMNOcFLRx0OWWGMIFNtzZU g18AoJEJFYVO8ggUEbTTLLNHbrb4bfyF =NflN -----END PGP SIGNATURE----- --DfnuYBTqzt7sVGu3-- From owner-freebsd-fs@FreeBSD.ORG Wed Dec 8 23:57:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7A030106564A; Wed, 8 Dec 2010 23:57:00 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [64.81.247.49]) by mx1.freebsd.org (Postfix) with ESMTP id E1F028FC08; Wed, 8 Dec 2010 23:56:59 +0000 (UTC) Received: from chez.mckusick.com (localhost [127.0.0.1]) by chez.mckusick.com (8.14.3/8.14.3) with ESMTP id oB8NuxwX040914; Wed, 8 Dec 2010 15:56:59 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <201012082356.oB8NuxwX040914@chez.mckusick.com> To: Kostik Belousov In-reply-to: <20101208225352.GK33073@deviant.kiev.zoral.com.ua> Date: Wed, 08 Dec 2010 15:56:59 -0800 From: Kirk McKusick Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Dec 2010 23:57:00 -0000 > Date: Thu, 9 Dec 2010 00:53:52 +0200 > From: Kostik Belousov > To: Kirk McKusick > Cc: Julian Elischer , freebsd-fs@freebsd.org, > pjd@freebsd.org, Oliver Fromme > Subject: Re: TRIM support for UFS? > > On Wed, Dec 08, 2010 at 02:23:52PM -0800, Kirk McKusick wrote: > > > > It is safe to do the BIO_DELETE here because at this point we have > > progressed through the file delete to the point where the inode that > > claimed this block has been updated on the disk with a NULL pointer. > > Thanks for the explanation. On the other hand, can my scenario be real > for async mounts ? Async mounts are not compatible with SU or SUJ. You are running the filesystem in a mode where there is no guarantee of recovery after a crash. So yes, you may TRIM blocks that are still claimed by on-disk inodes. But that is likely the least of your worries. Since async makes no claims about filesystem consistency after a crash, adding one more way for it to break does not seem like a big deal :-) > > The above patch looks good though I would do it unconditionally > > (e.g., also for snapshots). It seems sensible to make it conditional > > on a sysctl variable so that folks could experiment with it more > > easily. And I would leave it off by default as non-SSD disks are > > unlikely to benefit from it. If it does prove useful for SSD disks > > then I would make it conditional on a filesystem flag so that it > > is possible to enable it on a filesystem-by-filesystem basis (e.g., > > on for filesystems on the SSD and off for the rest). > > I think it is better to have a flag in superblock instead of the mount > option. > > Most of diff deleted... > > diff --git a/sys/ufs/ffs/fs.h b/sys/ufs/ffs/fs.h > index ba98ed3..13d9ede 100644 > --- a/sys/ufs/ffs/fs.h > +++ b/sys/ufs/ffs/fs.h > @@ -417,6 +417,7 @@ CTASSERT(sizeof(struct fs) == 1376); > #define FS_FLAGS_UPDATED 0x0080 /* flags have been moved to new location */ > #define FS_NFS4ACLS 0x0100 /* file system has NFSv4 ACLs enabled */ > #define FS_INDEXDIRS 0x0200 /* kernel supports indexed directories */ > +#define FS_TRIM 0x0400 /* issue BIO_DELETE for deleted blocks */ > > /* > * Macros to access bits in the fs_active array. I approve of this change and am happy to see it (finally) get added. > On Wed, 8 Dec 2010 at 23:33:27 +0100, Pawel Jakub Dawidek wrote: >> On Wed, Dec 08, 2010 at 02:23:52PM -0800, Kirk McKusick wrote: >> The above patch looks good though I would do it unconditionally >> (e.g., also for snapshots). It seems sensible to make it conditional >> on a sysctl variable so that folks could experiment with it more >> easily. And I would leave it off by default as non-SSD disks are >> unlikely to benefit from it. If it does prove useful for SSD disks >> then I would make it conditional on a filesystem flag so that it >> is possible to enable it on a filesystem-by-filesystem basis (e.g., >> on for filesystems on the SSD and off for the rest). > > We already have a flag for disks: DISKFLAG_CANDELETE, which tells if the > disk support TRIM or not. Next we should add BIO_GETATTR attribute for > DISK class to return true if TRIM is supported. This way UFS can ask if > TRIM is supported on mount and don't bother sending BIO_DELETE if it is > not supported. Clearly asking the underlying device is the most automated way to decide whether to do TRIM. Once it is possible to ask the underlying disk if it supports TRIM, your change could be modified to do the querry to the disk and set the FS_TRIM flag accordingly each time that the filesystem is mounted. If it is easy to add a BIO_GETATTR attribute for the DISK class, then we can skip the intermediate stage of using tunefs to enable/disable it. Kirk McKusick From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 07:15:14 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4B25F1065672 for ; Thu, 9 Dec 2010 07:15:14 +0000 (UTC) (envelope-from canevet@embl.fr) Received: from emblmta1.embl.fr (emblmta1.embl.fr [193.49.43.176]) by mx1.freebsd.org (Postfix) with ESMTP id D187E8FC08 for ; Thu, 9 Dec 2010 07:15:13 +0000 (UTC) X-IronPort-AV: E=Sophos;i="4.59,319,1288566000"; d="asc'?scan'208";a="774866" Received: from unknown (HELO [172.26.15.11]) ([172.26.15.11]) by emblmta1.embl.fr with ESMTP/TLS/DHE-RSA-CAMELLIA256-SHA; 09 Dec 2010 08:15:12 +0100 From: =?ISO-8859-1?Q?Micka=EBl_Can=E9vet?= To: jhell In-Reply-To: <4CFFA945.2010104@DataIX.net> References: <1291796355.2420.12.camel@pc286.embl.fr> <4CFFA945.2010104@DataIX.net> Content-Type: multipart/signed; micalg="pgp-sha1"; protocol="application/pgp-signature"; boundary="=-f1rHCIR2/jYAjr0HlrGL" Date: Thu, 09 Dec 2010 08:15:11 +0100 Message-ID: <1291878911.2497.4.camel@pc286.embl.fr> Mime-Version: 1.0 X-Mailer: Evolution 2.30.3 Cc: "freebsd-fs@freebsd.org >> FreeBSD Filesystems" Subject: Re: Zfs and Disc quota exceeded on 8.1-RELEASE X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 07:15:14 -0000 --=-f1rHCIR2/jYAjr0HlrGL Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Wed, 2010-12-08 at 10:50 -0500, jhell wrote: > On 12/08/2010 03:19, Micka=C3=ABl Can=C3=A9vet wrote: > > Hello, > >=20 > > When I exceed quota on a ZFS volume on FreeBSD-8.1-RELEASE, I can't > > remove any file with rm. I know that there is a workaround by using > > cat /dev/null in a file, but that's not really user friendly and can't > > work when the volume is exported to Windows Clients via Samba. > >=20 > > In tried with FreeBSD-8.1-STABLE-201011 and this bug seams to be solved= . > >=20 > > Is there a way to apply some patches on FreeBSD-8.1-RELEASE to fix this > > issue ? >=20 > I was not able to reproduce this on 8.2-PRERELEASE or later. Can you > confirm ? >=20 > Regards, >=20 Hi, Indeed, the bugs seams to be solved in 8.1-STABLE (and so 8.2-PRERELEASE I think), but I'm not sure that I should use this one in production. Is there a fix somewhere for the current "production ready" version of FreeBSD, or should I use a PRERELEASE version of 8.2 ? Thanks --=-f1rHCIR2/jYAjr0HlrGL Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (GNU/Linux) iEYEABECAAYFAk0AgfwACgkQZjBmN5Hi/YZMEQCfXIRpjKkAbxGr0+NTB+7F3VWX 3nMAn11aSZfWWkHOsyNrrzt5hLzHL99a =Er70 -----END PGP SIGNATURE----- --=-f1rHCIR2/jYAjr0HlrGL-- From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 10:34:18 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 055B8106564A; Thu, 9 Dec 2010 10:34:18 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mail.zoral.com.ua (mx0.zoral.com.ua [91.193.166.200]) by mx1.freebsd.org (Postfix) with ESMTP id DBD638FC27; Thu, 9 Dec 2010 10:34:16 +0000 (UTC) Received: from deviant.kiev.zoral.com.ua (root@deviant.kiev.zoral.com.ua [10.1.1.148]) by mail.zoral.com.ua (8.14.2/8.14.2) with ESMTP id oB9AYCGS097299 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Dec 2010 12:34:12 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: from deviant.kiev.zoral.com.ua (kostik@localhost [127.0.0.1]) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4) with ESMTP id oB9AYCQU016138; Thu, 9 Dec 2010 12:34:12 +0200 (EET) (envelope-from kostikbel@gmail.com) Received: (from kostik@localhost) by deviant.kiev.zoral.com.ua (8.14.4/8.14.4/Submit) id oB9AYBCI016137; Thu, 9 Dec 2010 12:34:11 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: deviant.kiev.zoral.com.ua: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 9 Dec 2010 12:34:11 +0200 From: Kostik Belousov To: Kirk McKusick Message-ID: <20101209103411.GL33073@deviant.kiev.zoral.com.ua> References: <20101208225352.GK33073@deviant.kiev.zoral.com.ua> <201012082356.oB8NuxwX040914@chez.mckusick.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="wA9WyeW1yVBM2Q32" Content-Disposition: inline In-Reply-To: <201012082356.oB8NuxwX040914@chez.mckusick.com> User-Agent: Mutt/1.4.2.3i X-Virus-Scanned: clamav-milter 0.95.2 at skuns.kiev.zoral.com.ua X-Virus-Status: Clean X-Spam-Status: No, score=-3.4 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00, DNS_FROM_OPENWHOIS autolearn=no version=3.2.5 X-Spam-Checker-Version: SpamAssassin 3.2.5 (2008-06-10) on skuns.kiev.zoral.com.ua Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 10:34:18 -0000 --wA9WyeW1yVBM2Q32 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Dec 08, 2010 at 03:56:59PM -0800, Kirk McKusick wrote: > > Date: Thu, 9 Dec 2010 00:53:52 +0200 > > From: Kostik Belousov > > To: Kirk McKusick > > Cc: Julian Elischer , freebsd-fs@freebsd.org, > > pjd@freebsd.org, Oliver Fromme > > Subject: Re: TRIM support for UFS? > >=20 > > On Wed, Dec 08, 2010 at 02:23:52PM -0800, Kirk McKusick wrote: > > > > > > It is safe to do the BIO_DELETE here because at this point we have > > > progressed through the file delete to the point where the inode that > > > claimed this block has been updated on the disk with a NULL pointer. > >=20 > > Thanks for the explanation. On the other hand, can my scenario be real > > for async mounts ? >=20 > Async mounts are not compatible with SU or SUJ. You are running the > filesystem in a mode where there is no guarantee of recovery after > a crash. So yes, you may TRIM blocks that are still claimed by on-disk > inodes. But that is likely the least of your worries. Since async makes > no claims about filesystem consistency after a crash, adding one more > way for it to break does not seem like a big deal :-) >=20 > > > The above patch looks good though I would do it unconditionally > > > (e.g., also for snapshots). It seems sensible to make it conditional > > > on a sysctl variable so that folks could experiment with it more > > > easily. And I would leave it off by default as non-SSD disks are > > > unlikely to benefit from it. If it does prove useful for SSD disks > > > then I would make it conditional on a filesystem flag so that it > > > is possible to enable it on a filesystem-by-filesystem basis (e.g., > > > on for filesystems on the SSD and off for the rest). > >=20 > > I think it is better to have a flag in superblock instead of the mount > > option. > > > > Most of diff deleted... > >=20 > > diff --git a/sys/ufs/ffs/fs.h b/sys/ufs/ffs/fs.h > > index ba98ed3..13d9ede 100644 > > --- a/sys/ufs/ffs/fs.h > > +++ b/sys/ufs/ffs/fs.h > > @@ -417,6 +417,7 @@ CTASSERT(sizeof(struct fs) =3D=3D 1376); > > #define FS_FLAGS_UPDATED 0x0080 /* flags have been moved to new locati= on */ > > #define FS_NFS4ACLS 0x0100 /* file system has NFSv4 ACLs enabled */ > > #define FS_INDEXDIRS 0x0200 /* kernel supports indexed directories */ > > +#define FS_TRIM 0x0400 /* issue BIO_DELETE for deleted blocks */ > >=20 > > /* > > * Macros to access bits in the fs_active array. >=20 > I approve of this change and am happy to see it (finally) get added. >=20 > > On Wed, 8 Dec 2010 at 23:33:27 +0100, Pawel Jakub Dawidek wrote: > >> On Wed, Dec 08, 2010 at 02:23:52PM -0800, Kirk McKusick wrote: > >> The above patch looks good though I would do it unconditionally > >> (e.g., also for snapshots). It seems sensible to make it conditional > >> on a sysctl variable so that folks could experiment with it more > >> easily. And I would leave it off by default as non-SSD disks are > >> unlikely to benefit from it. If it does prove useful for SSD disks > >> then I would make it conditional on a filesystem flag so that it > >> is possible to enable it on a filesystem-by-filesystem basis (e.g., > >> on for filesystems on the SSD and off for the rest). > >=20 > > We already have a flag for disks: DISKFLAG_CANDELETE, which tells if the > > disk support TRIM or not. Next we should add BIO_GETATTR attribute for > > DISK class to return true if TRIM is supported. This way UFS can ask if > > TRIM is supported on mount and don't bother sending BIO_DELETE if it is > > not supported. >=20 > Clearly asking the underlying device is the most automated way to decide > whether to do TRIM. Once it is possible to ask the underlying disk if it > supports TRIM, your change could be modified to do the querry to the disk > and set the FS_TRIM flag accordingly each time that the filesystem is > mounted. If it is easy to add a BIO_GETATTR attribute for the DISK class, > then we can skip the intermediate stage of using tunefs to enable/disable= it. I still think that TRIM should be opt-in, at least for the first time. Below is the BIO_GETATTR change integrated. Now, both FS_TRIM should be set and device shall report the GEOM::candelete attribute as true to have BIO_DELETE issued. geom_disk and md(4) report GEOM::candelete. Also, I updated tunefs(8) manpage, missed in the previous version. diff --git a/sbin/dumpfs/dumpfs.c b/sbin/dumpfs/dumpfs.c index 38c05f6..fa562dc 100644 --- a/sbin/dumpfs/dumpfs.c +++ b/sbin/dumpfs/dumpfs.c @@ -253,9 +253,11 @@ dumpfs(const char *name) printf("fs_flags expanded "); if (fsflags & FS_NFS4ACLS) printf("nfsv4acls "); + if (fsflags & FS_TRIM) + printf("trim "); fsflags &=3D ~(FS_UNCLEAN | FS_DOSOFTDEP | FS_NEEDSFSCK | FS_INDEXDIRS | FS_ACLS | FS_MULTILABEL | FS_GJOURNAL | FS_FLAGS_UPDATED | - FS_NFS4ACLS | FS_SUJ); + FS_NFS4ACLS | FS_SUJ | FS_TRIM); if (fsflags !=3D 0) printf("unknown flags (%#x)", fsflags); putchar('\n'); diff --git a/sbin/tunefs/tunefs.8 b/sbin/tunefs/tunefs.8 index a883cd4..c62af94 100644 --- a/sbin/tunefs/tunefs.8 +++ b/sbin/tunefs/tunefs.8 @@ -28,7 +28,7 @@ .\" @(#)tunefs.8 8.2 (Berkeley) 12/11/93 .\" $FreeBSD$ .\" -.Dd March 6, 2010 +.Dd December 9, 2010 .Dt TUNEFS 8 .Os .Sh NAME @@ -51,6 +51,7 @@ .Op Fl p .Op Fl s Ar avgfpdir .Op Fl S Ar size +.Op Fl t Cm enable | disable .Ar special | filesystem .Sh DESCRIPTION The @@ -143,6 +144,10 @@ Specify the expected number of files per directory. .It Fl S Ar size Specify the softdep journal size in bytes. The minimum is 4M. +.It Fl t Cm enable | disable +Turn on/off the TRIM enable flag. +If enabled, and if the underlaying device supports BIO_DELETE +command, kernel will delete the freed blocks. .El .Pp At least one of the above flags is required. diff --git a/sbin/tunefs/tunefs.c b/sbin/tunefs/tunefs.c index b2ea602..0ed713d 100644 --- a/sbin/tunefs/tunefs.c +++ b/sbin/tunefs/tunefs.c @@ -82,11 +82,13 @@ int main(int argc, char *argv[]) { char *avalue, *jvalue, *Jvalue, *Lvalue, *lvalue, *Nvalue, *nvalue; + char *tvalue; const char *special, *on; const char *name; int active; int Aflag, aflag, eflag, evalue, fflag, fvalue, jflag, Jflag, Lflag; int lflag, mflag, mvalue, Nflag, nflag, oflag, ovalue, pflag, sflag; + int tflag; int svalue, Sflag, Svalue; int ch, found_arg, i; const char *chg[2]; @@ -101,7 +103,8 @@ main(int argc, char *argv[]) evalue =3D fvalue =3D mvalue =3D ovalue =3D svalue =3D Svalue =3D 0; active =3D 0; found_arg =3D 0; /* At least one arg is required. */ - while ((ch =3D getopt(argc, argv, "Aa:e:f:j:J:L:l:m:N:n:o:ps:S:")) !=3D -= 1) + while ((ch =3D getopt(argc, argv, "Aa:e:f:j:J:L:l:m:N:n:o:ps:S:t:")) + !=3D -1) switch (ch) { =20 case 'A': @@ -268,6 +271,18 @@ main(int argc, char *argv[]) Sflag =3D 1; break; =20 + case 't': + found_arg =3D 1; + name =3D "trim"; + tvalue =3D optarg; + if (strcmp(tvalue, "enable") !=3D 0 && + strcmp(tvalue, "disable") !=3D 0) { + errx(10, "bad %s (options are %s)", + name, "`enable' or `disable'"); + } + tflag =3D 1; + break; + default: usage(); } @@ -493,6 +508,24 @@ main(int argc, char *argv[]) sblock.fs_avgfpdir =3D svalue; } } + if (tflag) { + name =3D "issue TRIM to the disk"; + if (strcmp(tvalue, "enable") =3D=3D 0) { + if (sblock.fs_flags & FS_TRIM) + warnx("%s remains unchanged as enabled", name); + else { + sblock.fs_flags |=3D FS_TRIM; + warnx("%s set", name); + } + } else if (strcmp(tvalue, "disable") =3D=3D 0) { + if ((~sblock.fs_flags & FS_TRIM) =3D=3D FS_TRIM) + warnx("%s remains unchanged as disabled", name); + else { + sblock.fs_flags &=3D ~FS_TRIM; + warnx("%s cleared", name); + } + } + } =20 if (sbwrite(&disk, Aflag) =3D=3D -1) goto err; @@ -1011,12 +1044,13 @@ out: void usage(void) { - fprintf(stderr, "%s\n%s\n%s\n%s\n%s\n", + fprintf(stderr, "%s\n%s\n%s\n%s\n%s\n%s\n", "usage: tunefs [-A] [-a enable | disable] [-e maxbpg] [-f avgfilesize]", " [-J enable | disable] [-j enable | disable]",=20 " [-L volname] [-l enable | disable] [-m minfree]", " [-N enable | disable] [-n enable | disable]", -" [-o space | time] [-p] [-s avgfpdir] special | filesystem"); +" [-o space | time] [-p] [-s avgfpdir] [-t enable | disable]", +" special | filesystem"); exit(2); } =20 @@ -1035,6 +1069,8 @@ printfs(void) (sblock.fs_flags & FS_SUJ)? "enabled" : "disabled"); warnx("gjournal: (-J) %s", (sblock.fs_flags & FS_GJOURNAL)? "enabled" : "disabled"); + warnx("trim: (-p) %s",=20 + (sblock.fs_flags & FS_TRIM)? "enabled" : "disabled"); warnx("maximum blocks per file in a cylinder group: (-e) %d", sblock.fs_maxbpg); warnx("average file size: (-f) %d", diff --git a/sys/dev/md/md.c b/sys/dev/md/md.c index 6c3484e..c294d1b 100644 --- a/sys/dev/md/md.c +++ b/sys/dev/md/md.c @@ -713,11 +713,12 @@ md_kthread(void *arg) } mtx_unlock(&sc->queue_mtx); if (bp->bio_cmd =3D=3D BIO_GETATTR) { - if (sc->fwsectors && sc->fwheads && + if ((sc->fwsectors && sc->fwheads && (g_handleattr_int(bp, "GEOM::fwsectors", sc->fwsectors) || g_handleattr_int(bp, "GEOM::fwheads", - sc->fwheads))) + sc->fwheads))) || + g_handleattr_int(bp, "GEOM::candelete", 1)) error =3D -1; else error =3D EOPNOTSUPP; diff --git a/sys/geom/geom_disk.c b/sys/geom/geom_disk.c index 25d2e3b..eda702c 100644 --- a/sys/geom/geom_disk.c +++ b/sys/geom/geom_disk.c @@ -297,6 +297,9 @@ g_disk_start(struct bio *bp) } while (bp2 !=3D NULL); break; case BIO_GETATTR: + if (g_handleattr_int(bp, "GEOM::candelete", + (dp->d_flags & DISKFLAG_CANDELETE) !=3D 0)) + break; if (g_handleattr_int(bp, "GEOM::fwsectors", dp->d_fwsectors)) break; else if (g_handleattr_int(bp, "GEOM::fwheads", dp->d_fwheads)) diff --git a/sys/ufs/ffs/ffs_alloc.c b/sys/ufs/ffs/ffs_alloc.c index b740bbb..926229c 100644 --- a/sys/ufs/ffs/ffs_alloc.c +++ b/sys/ufs/ffs/ffs_alloc.c @@ -83,6 +83,8 @@ __FBSDID("$FreeBSD$"); =20 #include =20 +#include + #include #include #include @@ -1850,6 +1852,7 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dephd) u_int cg; u_int8_t *blksfree; struct cdev *dev; + struct bio *bip; =20 cg =3D dtog(fs, bno); if (devvp->v_type =3D=3D VREG) { @@ -1962,6 +1965,16 @@ ffs_blkfree(ump, fs, devvp, bno, size, inum, dephd) softdep_setup_blkfree(UFSTOVFS(ump), bp, bno, numfrags(fs, size), dephd); bdwrite(bp); + + if (ump->um_candelete) { + bip =3D g_alloc_bio(); + bip->bio_cmd =3D BIO_DELETE; + bip->bio_offset =3D dbtob(fsbtodb(fs, bno)); + bip->bio_done =3D g_destroy_bio; + bip->bio_length =3D size; + g_io_request(bip, + (struct g_consumer *)devvp->v_bufobj.bo_private); + } } =20 #ifdef INVARIANTS diff --git a/sys/ufs/ffs/ffs_vfsops.c b/sys/ufs/ffs/ffs_vfsops.c index 72f40da..f578382 100644 --- a/sys/ufs/ffs/ffs_vfsops.c +++ b/sys/ufs/ffs/ffs_vfsops.c @@ -895,6 +895,21 @@ ffs_mountfs(devvp, mp, td) mp->mnt_stat.f_mntonname); #endif } + if ((fs->fs_flags & FS_TRIM) !=3D 0) { + size =3D sizeof(int); + if (g_io_getattr("GEOM::candelete", cp, &size, + &ump->um_candelete) =3D=3D 0) { + if (!ump->um_candelete) + printf( +"WARNING: %s: TRIM flag on fs but disk does not support TRIM\n", + mp->mnt_stat.f_mntonname); + } else { + printf( +"WARNING: %s: TRIM flag on fs but cannot get whether disk supports TRIM\n", + mp->mnt_stat.f_mntonname); + ump->um_candelete =3D 0; + } + } =20 ump->um_mountp =3D mp; ump->um_dev =3D dev; diff --git a/sys/ufs/ffs/fs.h b/sys/ufs/ffs/fs.h index ba98ed3..13d9ede 100644 --- a/sys/ufs/ffs/fs.h +++ b/sys/ufs/ffs/fs.h @@ -417,6 +417,7 @@ CTASSERT(sizeof(struct fs) =3D=3D 1376); #define FS_FLAGS_UPDATED 0x0080 /* flags have been moved to new location */ #define FS_NFS4ACLS 0x0100 /* file system has NFSv4 ACLs enabled */ #define FS_INDEXDIRS 0x0200 /* kernel supports indexed directories */ +#define FS_TRIM 0x0400 /* issue BIO_DELETE for deleted blocks */ =20 /* * Macros to access bits in the fs_active array. diff --git a/sys/ufs/ufs/ufsmount.h b/sys/ufs/ufs/ufsmount.h index 03edd73..b13db40 100644 --- a/sys/ufs/ufs/ufsmount.h +++ b/sys/ufs/ufs/ufsmount.h @@ -95,6 +95,7 @@ struct ufsmount { time_t um_itime[MAXQUOTAS]; /* inode quota time limit */ char um_qflags[MAXQUOTAS]; /* quota specific flags */ int64_t um_savedmaxfilesize; /* XXX - limit maxfilesize */ + int um_candelete; /* devvp supports TRIM */ int (*um_balloc)(struct vnode *, off_t, int, struct ucred *, int, struct = buf **); int (*um_blkatoff)(struct vnode *, off_t, char **, struct buf **); int (*um_truncate)(struct vnode *, off_t, int, struct ucred *, struct thr= ead *); --wA9WyeW1yVBM2Q32 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (FreeBSD) iEYEARECAAYFAk0AsKMACgkQC3+MBN1Mb4gTWgCgr4ImFVQhYac2hVOHZOnTneXv VR0AoKXYQX99T79attXilSYn6/kjX86+ =RLhi -----END PGP SIGNATURE----- --wA9WyeW1yVBM2Q32-- From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 11:50:25 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 75D101065673; Thu, 9 Dec 2010 11:50:25 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id 1DA6A8FC0A; Thu, 9 Dec 2010 11:50:24 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id EF97F45C9C; Thu, 9 Dec 2010 12:50:21 +0100 (CET) Received: from localhost (pdawidek.whl [10.0.1.1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 1A57445CDC; Thu, 9 Dec 2010 12:50:17 +0100 (CET) Date: Thu, 9 Dec 2010 12:50:16 +0100 From: Pawel Jakub Dawidek To: Kostik Belousov Message-ID: <20101209115016.GB1745@garage.freebsd.pl> References: <20101208225352.GK33073@deviant.kiev.zoral.com.ua> <201012082356.oB8NuxwX040914@chez.mckusick.com> <20101209103411.GL33073@deviant.kiev.zoral.com.ua> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="rS8CxjVDS/+yyDmU" Content-Disposition: inline In-Reply-To: <20101209103411.GL33073@deviant.kiev.zoral.com.ua> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-5.9 required=4.5 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.0.4 Cc: Kirk McKusick , freebsd-fs@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 11:50:25 -0000 --rS8CxjVDS/+yyDmU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Dec 09, 2010 at 12:34:11PM +0200, Kostik Belousov wrote: > I still think that TRIM should be opt-in, at least for the first time. > Below is the BIO_GETATTR change integrated. >=20 > Now, both FS_TRIM should be set and device shall report the > GEOM::candelete attribute as true to have BIO_DELETE issued. geom_disk > and md(4) report GEOM::candelete. Also, I updated tunefs(8) manpage, > missed in the previous version. [...] > --- a/sbin/tunefs/tunefs.c > +++ b/sbin/tunefs/tunefs.c [...] > @@ -1035,6 +1069,8 @@ printfs(void) > (sblock.fs_flags & FS_SUJ)? "enabled" : "disabled"); > warnx("gjournal: (-J) %s", > (sblock.fs_flags & FS_GJOURNAL)? "enabled" : "disabled"); > + warnx("trim: (-p) %s",=20 s/-p/-t/ Other than that looks nice and clean. --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --rS8CxjVDS/+yyDmU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk0AwngACgkQForvXbEpPzSWYACgjK/Fte2usUZq27amqDrSun9X 9lUAn1Xa9dvVXhG/gagkUrKl6HVVwgP9 =wUsX -----END PGP SIGNATURE----- --rS8CxjVDS/+yyDmU-- From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 14:30:12 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C8DD106566B for ; Thu, 9 Dec 2010 14:30:12 +0000 (UTC) (envelope-from zeus@relay.ibs.dn.ua) Received: from relay.ibs.dn.ua (relay.ibs.dn.ua [91.216.196.25]) by mx1.freebsd.org (Postfix) with ESMTP id 938558FC08 for ; Thu, 9 Dec 2010 14:30:11 +0000 (UTC) Received: from relay.ibs.dn.ua (localhost [127.0.0.1]) by relay.ibs.dn.ua with ESMTP id oB9EEEpL062741 for ; Thu, 9 Dec 2010 16:14:14 +0200 (EET) Received: (from zeus@localhost) by relay.ibs.dn.ua (8.14.4/8.14.4/Submit) id oB9EECjM062740 for freebsd-fs@freebsd.org; Thu, 9 Dec 2010 16:14:12 +0200 (EET) Date: Thu, 9 Dec 2010 16:14:12 +0200 From: Zeus V Panchenko To: freebsd-fs@freebsd.org Message-ID: <20101209141412.GA62013@relay.ibs.dn.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 8.1-RELEASE X-Editor: GNU Emacs 23.2.1 Subject: what is the best way to use free space on several boxes? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: zeus@ibs.dn.ua List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 14:30:12 -0000 Hi All, we have several FreeBSD 8-STABLE boxes, each has 2x500GB HDD in gmirror UFS dedicated but really it's used less than 30% each box ... so, advice please, what is the best/correct/right way to use the free space on the boxes? nfs? zfs? iscsi? the aim is to have storage place ... -- Zeus V. Panchenko GMT+2 (EET) From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 15:15:25 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5A4151065672 for ; Thu, 9 Dec 2010 15:15:25 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id 12D2E8FC18 for ; Thu, 9 Dec 2010 15:15:24 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PQiDL-0000cD-2J for freebsd-fs@freebsd.org; Thu, 09 Dec 2010 16:15:23 +0100 Received: from cpe-188-129-86-49.dynamic.amis.hr ([188.129.86.49]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 09 Dec 2010 16:15:23 +0100 Received: from ivoras by cpe-188-129-86-49.dynamic.amis.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 09 Dec 2010 16:15:23 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Thu, 09 Dec 2010 16:15:06 +0100 Lines: 32 Message-ID: References: <20101209141412.GA62013@relay.ibs.dn.ua> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: cpe-188-129-86-49.dynamic.amis.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6 In-Reply-To: <20101209141412.GA62013@relay.ibs.dn.ua> Subject: Re: what is the best way to use free space on several boxes? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 15:15:25 -0000 On 12/09/10 15:14, Zeus V Panchenko wrote: > Hi All, > > we have several FreeBSD 8-STABLE boxes, each has 2x500GB HDD in gmirror > UFS dedicated > > but really it's used less than 30% each box ... > > so, advice please, what is the best/correct/right way to use the free > space on the boxes? nfs? zfs? iscsi? > > the aim is to have storage place ... It really depends on how reliable and flexible you need this storage space to be. From one side, you can simply create large files within the existing file systems (e.g. a 300 GB file on each) to serve as virtual disks, export them with iSCSI, import them on another box and create a ZFS raidz volume out of those drives - you get a small amount of fault tolerance here. The bad sides are: you need to have a large "fixed space" file on each of the machines (you can theoretically create sparse files but the downsides are even worse), only one box can see the consolidated space (the ZFS volume), and there are so many layers in this configuration that performance will probably be slow. On the other hand, you can simply NFS-mount a directory from each of the machines under a single tree. This way you can more flexibly share space with existing data on the machines and be faster, but you cannot automatically make use of all of the available space at the same time. Until FUSE is fixed, these are probably the only options. From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 15:42:09 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26BE71065674 for ; Thu, 9 Dec 2010 15:42:09 +0000 (UTC) (envelope-from l.pizzamiglio@bally-wulff.de) Received: from mail2.bally-wulff-berlin.de (mail2.bally-wulff-berlin.de [212.144.118.9]) by mx1.freebsd.org (Postfix) with ESMTP id A746A8FC1A for ; Thu, 9 Dec 2010 15:42:08 +0000 (UTC) Received: from bwex.bally-wulff.de (unknown [192.9.204.106]) by mail2.bally-wulff-berlin.de (Postfix) with ESMTP id 6FF469907F for ; Thu, 9 Dec 2010 16:19:21 +0100 (CET) Received: from [192.9.205.177] ([192.9.205.177]) by bwex.bally-wulff.de with Microsoft SMTPSVC(6.0.3790.4675); Thu, 9 Dec 2010 16:19:21 +0100 Message-ID: <4D00F369.90004@bally-wulff.de> Date: Thu, 09 Dec 2010 16:19:05 +0100 From: Luca Pizzamiglio User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.12) Gecko/20101029 Thunderbird/3.1.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Content-Type: multipart/mixed; boundary="------------010503030304070507020702" X-OriginalArrivalTime: 09 Dec 2010 15:19:22.0051 (UTC) FILETIME=[74D98130:01CB97B4] Subject: Softupdate on UFS and "sometimes" slow flash disk X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 15:42:09 -0000 This is a multi-part message in MIME format. --------------010503030304070507020702 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi list, I'm writing here, hoping this is the right list. We're using a PC with a Flash PATA module (4GB, non properly a SSD). The disk is 2.8 GB used (more or less) and the root partition is around 3.8 GB. During the update/restore procedure (based on restore(8)) I encountered a "disk full message" when the procedure is around 1 GB. After this message, the procedure continues without any other problem. At the end, disk has, as usual, 1 GB available space. I investigate deeply the problem and I monitored the softdep_request_cleanup() function that frees pending blocks. In one case of my scenario it return without freeing nothing, but there are a lot of free pending blocks. After further analysis I discovered that my flash module sometimes is dramatically slow (I don't know what the controller is doing...). softdep_request_cleanup() implements a time control system. Before to free blocks, it invokes ffs_update() in a blocking manner. On my "sometimes" slow flash module this ffs_update() takes more than 2 seconds and softdep_request_cleanup() return without freeing. Other invocations are not so slow and everything is fine. I know that probably there is some problem inside my flash module, but I guess that the initialization of the time control mechanism should be made after the ffs_update(). I propose here a patch, that should correct that behavior. Best regards Luca Pizzamiglio --------------010503030304070507020702 Content-Type: text/plain; name="ffs_softdep.c.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="ffs_softdep.c.patch" Index: ffs_softdep.c =================================================================== --- ffs_softdep.c (revision 3162) +++ ffs_softdep.c (working copy) @@ -5796,7 +5796,6 @@ ump = VTOI(vp)->i_ump; mtx_assert(UFS_MTX(ump), MA_OWNED); needed = fs->fs_cstotal.cs_nbfree + fs->fs_contigsumsize; - starttime = time_second + tickdelay; /* * If we are being called because of a process doing a * copy-on-write, then it is not safe to update the vnode @@ -5809,6 +5808,7 @@ if (error != 0) return (0); } + starttime = time_second + tickdelay; while (fs->fs_pendingblocks > 0 && fs->fs_cstotal.cs_nbfree <= needed) { if (time_second > starttime) return (0); --------------010503030304070507020702-- From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 16:39:12 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3E7711065673 for ; Thu, 9 Dec 2010 16:39:12 +0000 (UTC) (envelope-from gtodd@bellanet.org) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 02AB48FC13 for ; Thu, 9 Dec 2010 16:39:11 +0000 (UTC) Received: by yxh35 with SMTP id 35so1544762yxh.13 for ; Thu, 09 Dec 2010 08:39:11 -0800 (PST) Received: by 10.100.132.11 with SMTP id f11mr6968900and.75.1291911344625; Thu, 09 Dec 2010 08:15:44 -0800 (PST) Received: from wawanesa.iciti.ca (CPE0080c8f208a5-CM001371173cf8.cpe.net.cable.rogers.com [99.246.61.82]) by mx.google.com with ESMTPS id c7sm2113709ana.17.2010.12.09.08.15.42 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 09 Dec 2010 08:15:43 -0800 (PST) Message-ID: <4D010098.4030406@bellanet.org> Date: Thu, 09 Dec 2010 11:15:20 -0500 From: Graham Todd User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100919 Thunderbird/3.1.4 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <20101209141412.GA62013@relay.ibs.dn.ua> In-Reply-To: <20101209141412.GA62013@relay.ibs.dn.ua> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: what is the best way to use free space on several boxes? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 16:39:14 -0000 On 12/09/10 09:14, Zeus V Panchenko wrote: > Hi All, > > we have several FreeBSD 8-STABLE boxes, each has 2x500GB HDD in gmirror > UFS dedicated > > but really it's used less than 30% each box ... > > so, advice please, what is the best/correct/right way to use the free > space on the boxes? nfs? zfs? iscsi? > > the aim is to have storage place ... Have you tried hastd? I believe at this time it only syncs to one remote node but Mikolaj Golub has documented how to build a "stacked" HAST setup out of 2x2 systems. I think the plan is for hast to be able to use multiple nodes Otherwise you could iscsi export files/volumes from each box (i.e. made with truncate or dd) and then use those into a zpool. I'm guessing that if you then nfs exported the file systems from that pool things could be a bit slow though :) From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 17:40:18 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1B40E1065670 for ; Thu, 9 Dec 2010 17:40:18 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (mail.bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id 0035B8FC20 for ; Thu, 9 Dec 2010 17:40:17 +0000 (UTC) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id DB2CD5B8B; Thu, 9 Dec 2010 09:20:58 -0800 (PST) To: zeus@ibs.dn.ua In-reply-to: Your message of "Thu, 09 Dec 2010 16:14:12 +0200." <20101209141412.GA62013@relay.ibs.dn.ua> References: <20101209141412.GA62013@relay.ibs.dn.ua> Comments: In-reply-to Zeus V Panchenko message dated "Thu, 09 Dec 2010 16:14:12 +0200." Date: Thu, 09 Dec 2010 09:20:58 -0800 From: Bakul Shah Message-Id: <20101209172058.DB2CD5B8B@mail.bitblocks.com> Cc: freebsd-fs@freebsd.org Subject: Re: what is the best way to use free space on several boxes? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 17:40:18 -0000 On Thu, 09 Dec 2010 16:14:12 +0200 Zeus V Panchenko wrote: > Hi All, > > we have several FreeBSD 8-STABLE boxes, each has 2x500GB HDD in gmirror > UFS dedicated > > but really it's used less than 30% each box ... > > so, advice please, what is the best/correct/right way to use the free > space on the boxes? nfs? zfs? iscsi? > > the aim is to have storage place ... This is probably more "researchy" than you want and this is probably more suited to a much larger number of nodes but how about some sort of a distributed hash table filesystem? Hash a block address and map it to N different nodes. That way you can add and remove boxes without having to reconfigure. When a block is initially written, its `address' is computed from its contents. One thing you have to worry about is that the amount of available storage can change dynamically. Lots of useful information in the wikipedia entry for this. Even more researchy: even if you move such disks arround to different nodes, or lose some disk blocks, the remaining data on such disks can be reattached. A truly resilient storage system. From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 18:13:41 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E54CB106564A; Thu, 9 Dec 2010 18:13:41 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [64.81.247.49]) by mx1.freebsd.org (Postfix) with ESMTP id 6A8328FC15; Thu, 9 Dec 2010 18:13:41 +0000 (UTC) Received: from chez.mckusick.com (localhost [127.0.0.1]) by chez.mckusick.com (8.14.3/8.14.3) with ESMTP id oB9IDd2H078366; Thu, 9 Dec 2010 10:13:39 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <201012091813.oB9IDd2H078366@chez.mckusick.com> To: Kostik Belousov In-reply-to: <20101209103411.GL33073@deviant.kiev.zoral.com.ua> Date: Thu, 09 Dec 2010 10:13:39 -0800 From: Kirk McKusick Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 18:13:42 -0000 Other than the nit pointed out by Pawel, the diffs look good to me. You should consider adding the -t option to newfs so that the TRIM option can be specified at the time the filesystem is created (as a general rule, anything you can do with tunefs should also be possible with newfs). I agree with your decision to let administrators opt-out of doing TRIM. If experience shows it to be generally useful to have it on, we can change the default to enabled later. If we do change the default to enabled, then we will want to delete the warning about TRIM not being supported by the underlying disk that you added at mount time as we would start getting a lot of them for all the non-SSD disks. Kirk McKusick From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 19:06:18 2010 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 224871065679 for ; Thu, 9 Dec 2010 19:06:18 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) by mx1.freebsd.org (Postfix) with ESMTP id 9D53A8FC26 for ; Thu, 9 Dec 2010 19:06:17 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.4/8.14.4) with ESMTP id oB9In72f094528 for ; Thu, 9 Dec 2010 21:49:07 +0300 (MSK) (envelope-from marck@rinet.ru) Date: Thu, 9 Dec 2010 21:49:07 +0300 (MSK) From: Dmitry Morozovsky To: freebsd-fs@FreeBSD.org Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (woozle.rinet.ru [0.0.0.0]); Thu, 09 Dec 2010 21:49:07 +0300 (MSK) Cc: Subject: stale zpool: how to rename? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 19:06:18 -0000 Dear colleagues, how can I clear stale traces of old pool? [this is on fresh stable/8] Now I have: root@ogre2:~# zpool import pool: og id: 4146846692207551713 state: UNAVAIL status: One or more devices are missing from the system. action: The pool cannot be imported. Attach the missing devices and try again. see: http://www.sun.com/msg/ZFS-8000-3C config: og UNAVAIL insufficient replicas mirror UNAVAIL insufficient replicas da0 UNAVAIL cannot open da1 UNAVAIL cannot open root@ogre2:~# zpool status pool: o state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM o ONLINE 0 0 0 mirror ONLINE 0 0 0 da0h ONLINE 0 0 0 da1h ONLINE 0 0 0 errors: No known data errors root@ogre2:~# zpool import -f og cannot import 'og': no such pool or dataset root@ogre2:~# zpool destroy og cannot open 'og': no such pool Thanks in advance. -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 19:30:41 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AD63E106566B for ; Thu, 9 Dec 2010 19:30:41 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta06.westchester.pa.mail.comcast.net (qmta06.westchester.pa.mail.comcast.net [76.96.62.56]) by mx1.freebsd.org (Postfix) with ESMTP id 5BCDF8FC12 for ; Thu, 9 Dec 2010 19:30:40 +0000 (UTC) Received: from omta07.westchester.pa.mail.comcast.net ([76.96.62.59]) by qmta06.westchester.pa.mail.comcast.net with comcast id h6ew1f0051GhbT8567WhMD; Thu, 09 Dec 2010 19:30:41 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta07.westchester.pa.mail.comcast.net with comcast id h7WV1f0063LrwQ23T7WZUK; Thu, 09 Dec 2010 19:30:40 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 93C7D9B427; Thu, 9 Dec 2010 11:30:27 -0800 (PST) Date: Thu, 9 Dec 2010 11:30:27 -0800 From: Jeremy Chadwick To: Dmitry Morozovsky Message-ID: <20101209193027.GA10784@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@FreeBSD.org Subject: Re: stale zpool: how to rename? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 19:30:41 -0000 On Thu, Dec 09, 2010 at 09:49:07PM +0300, Dmitry Morozovsky wrote: > Dear colleagues, > > how can I clear stale traces of old pool? > > [this is on fresh stable/8] > > Now I have: > > root@ogre2:~# zpool import > pool: og > id: 4146846692207551713 > state: UNAVAIL > status: One or more devices are missing from the system. > action: The pool cannot be imported. Attach the missing > devices and try again. > see: http://www.sun.com/msg/ZFS-8000-3C > config: > > og UNAVAIL insufficient replicas > mirror UNAVAIL insufficient replicas > da0 UNAVAIL cannot open > da1 UNAVAIL cannot open > root@ogre2:~# zpool status > pool: o > state: ONLINE > scrub: none requested > config: > > NAME STATE READ WRITE CKSUM > o ONLINE 0 0 0 > mirror ONLINE 0 0 0 > da0h ONLINE 0 0 0 > da1h ONLINE 0 0 0 > > errors: No known data errors > root@ogre2:~# zpool import -f og > cannot import 'og': no such pool or dataset > root@ogre2:~# zpool destroy og > cannot open 'og': no such pool Try using the "id" of the old pool, not the pool name. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 19:47:48 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 99DE81065670 for ; Thu, 9 Dec 2010 19:47:48 +0000 (UTC) (envelope-from artemb@gmail.com) Received: from mail-qw0-f54.google.com (mail-qw0-f54.google.com [209.85.216.54]) by mx1.freebsd.org (Postfix) with ESMTP id 4B3358FC1C for ; Thu, 9 Dec 2010 19:47:47 +0000 (UTC) Received: by qwj9 with SMTP id 9so2955226qwj.13 for ; Thu, 09 Dec 2010 11:47:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:cc:content-type:content-transfer-encoding; bh=fPwxrxo2t0e/0RCCZiStA7GZt4jk00FYQ7enpcVifxc=; b=GSONPfyPVFWZuQe2JG8p0UN4+0ZQfU3c8SsRJxzBFlqIvBMWdvuUwK3KMZdC7+EAMZ 3CZoWTZGm1QF9KZQ9m4Rba9DU9GiS/KfdFZXUL065/8VjXxtxbVyUmo4bqi1i61qO3Ta Y6npoq2di81VZx7UCQGL4CCPeu2FUJ5z7DFBc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=XEZOqSIoWskCvmbPWc0pmg6qVsZpytkOvaowteUMG4sGRxjtHxGV22TGmS2LbcJbbY dYRGHi3dL1wsPQqLc0+3MQkvOJYkKMUB3SS9ix7lq1de0q3maYHtbGTP0oVKd4OhcQTy eLtOtEiNKBof/m0VYJJnwFxTMigZiwtVrJ558= MIME-Version: 1.0 Received: by 10.229.82.10 with SMTP id z10mr8359821qck.98.1291922210385; Thu, 09 Dec 2010 11:16:50 -0800 (PST) Sender: artemb@gmail.com Received: by 10.220.177.195 with HTTP; Thu, 9 Dec 2010 11:16:50 -0800 (PST) In-Reply-To: References: Date: Thu, 9 Dec 2010 11:16:50 -0800 X-Google-Sender-Auth: M0dq10Xj46lCZemfT7Y5bSTHkSA Message-ID: From: Artem Belevich To: Dmitry Morozovsky Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs@freebsd.org Subject: Re: stale zpool: how to rename? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 19:47:48 -0000 On Thu, Dec 9, 2010 at 10:49 AM, Dmitry Morozovsky wrote: > Dear colleagues, > > how can I clear stale traces of old pool? zpool export ? --Artem > > [this is on fresh stable/8] > > Now I have: > > > root@ogre2:~# zpool import > =A0pool: og > =A0 =A0id: 4146846692207551713 > =A0state: UNAVAIL > status: One or more devices are missing from the system. > action: The pool cannot be imported. Attach the missing > =A0 =A0 =A0 =A0devices and try again. > =A0 see: http://www.sun.com/msg/ZFS-8000-3C > config: > > =A0 =A0 =A0 =A0og =A0 =A0 =A0 =A0 =A0UNAVAIL =A0insufficient replicas > =A0 =A0 =A0 =A0 =A0mirror =A0 =A0UNAVAIL =A0insufficient replicas > =A0 =A0 =A0 =A0 =A0 =A0da0 =A0 =A0 UNAVAIL =A0cannot open > =A0 =A0 =A0 =A0 =A0 =A0da1 =A0 =A0 UNAVAIL =A0cannot open > root@ogre2:~# zpool status > =A0pool: o > =A0state: ONLINE > =A0scrub: none requested > config: > > =A0 =A0 =A0 =A0NAME =A0 =A0 =A0 =A0STATE =A0 =A0 READ WRITE CKSUM > =A0 =A0 =A0 =A0o =A0 =A0 =A0 =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 > =A0 =A0 =A0 =A0 =A0mirror =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0 0 > =A0 =A0 =A0 =A0 =A0 =A0da0h =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0= 0 > =A0 =A0 =A0 =A0 =A0 =A0da1h =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0= 0 > > errors: No known data errors > root@ogre2:~# zpool import -f og > cannot import 'og': no such pool or dataset > root@ogre2:~# zpool destroy og > cannot open 'og': no such pool > > Thanks in advance. > > -- > Sincerely, > D.Marck =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 [DM5020, MCK-RIPE, DM3-RIPN] > [ FreeBSD committer: =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 marck@FreeBSD.org ] > ------------------------------------------------------------------------ > *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** > ------------------------------------------------------------------------ > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Thu Dec 9 21:00:51 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 80593106564A for ; Thu, 9 Dec 2010 21:00:51 +0000 (UTC) (envelope-from carlson39@llnl.gov) Received: from nspiron-1.llnl.gov (nspiron-1.llnl.gov [128.115.41.81]) by mx1.freebsd.org (Postfix) with ESMTP id 6D2D58FC08 for ; Thu, 9 Dec 2010 21:00:51 +0000 (UTC) X-Attachments: None Received: from bagua.llnl.gov (HELO [134.9.197.135]) ([134.9.197.135]) by nspiron-1.llnl.gov with ESMTP; 09 Dec 2010 13:00:51 -0800 Message-ID: <4D014383.2030702@llnl.gov> Date: Thu, 09 Dec 2010 13:00:51 -0800 From: Mike Carlson User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.2.12) Gecko/20101027 Lightning/1.0b2 Thunderbird/3.1.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <20101209141412.GA62013@relay.ibs.dn.ua> <20101209172058.DB2CD5B8B@mail.bitblocks.com> In-Reply-To: <20101209172058.DB2CD5B8B@mail.bitblocks.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: what is the best way to use free space on several boxes? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Dec 2010 21:00:51 -0000 On 12/09/2010 09:20 AM, Bakul Shah wrote: > On Thu, 09 Dec 2010 16:14:12 +0200 Zeus V Panchenko wrote: >> Hi All, >> >> we have several FreeBSD 8-STABLE boxes, each has 2x500GB HDD in gmirror >> UFS dedicated >> >> but really it's used less than 30% each box ... >> >> so, advice please, what is the best/correct/right way to use the free >> space on the boxes? nfs? zfs? iscsi? >> >> the aim is to have storage place ... > This is probably more "researchy" than you want and this is > probably more suited to a much larger number of nodes but how > about some sort of a distributed hash table filesystem? Hash > a block address and map it to N different nodes. That way > you can add and remove boxes without having to reconfigure. > When a block is initially written, its `address' is computed > from its contents. One thing you have to worry about is that > the amount of available storage can change dynamically. > > Lots of useful information in the wikipedia entry for this. > > Even more researchy: even if you move such disks arround to > different nodes, or lose some disk blocks, the remaining data > on such disks can be reattached. A truly resilient storage > system. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > I had played around with MooseFS, which is available in the ports tree, in a small test environment. I would recommend looking at the project and seeing if that would suite your needs. http://www.moosefs.org/ Mike C From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 00:03:33 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C6857106566C; Fri, 10 Dec 2010 00:03:33 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from out-0.mx.aerioconnect.net (out-0-1.mx.aerioconnect.net [216.240.47.61]) by mx1.freebsd.org (Postfix) with ESMTP id 6D19C8FC15; Fri, 10 Dec 2010 00:03:33 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id oBA03WYs023343; Thu, 9 Dec 2010 16:03:32 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id C6CC52D601A; Thu, 9 Dec 2010 16:03:30 -0800 (PST) Message-ID: <4D016E50.101@freebsd.org> Date: Thu, 09 Dec 2010 16:03:28 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: Kirk McKusick References: <201012091813.oB9IDd2H078366@chez.mckusick.com> In-Reply-To: <201012091813.oB9IDd2H078366@chez.mckusick.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 00:03:33 -0000 On 12/9/10 10:13 AM, Kirk McKusick wrote: > Other than the nit pointed out by Pawel, the diffs look good to me. > You should consider adding the -t option to newfs so that the TRIM > option can be specified at the time the filesystem is created (as > a general rule, anything you can do with tunefs should also be > possible with newfs). > > I agree with your decision to let administrators opt-out of doing > TRIM. If experience shows it to be generally useful to have it on, > we can change the default to enabled later. If we do change the > default to enabled, then we will want to delete the warning about TRIM > not being supported by the underlying disk that you added at mount > time as we would start getting a lot of them for all the non-SSD disks. > > Kirk McKusick > I echo the above. I am very happy to see this happen, and if it goes into 8.x I'll try see that we (fusion-io) support it in our FreeBSD driver as soon as we can. Several people have asked why trim is important for flash based devices.. The answer is that 'trimmed' packets get added back to the 'free pool of pages available for a flash device to use and that helps in two ways. firstly the average free space recoverable when you wring out a used block of sectors increases, meaning that you have to garbage collect fewer of them, which means that you have less overhead eating into your performance, and secondly, when you start off after a period of low utilisation, you have more headroom to write into before you need to start garbage collection. it's a win-win.. From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 00:24:00 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BF534106566B; Fri, 10 Dec 2010 00:24:00 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from out-0.mx.aerioconnect.net (out-0-1.mx.aerioconnect.net [216.240.47.61]) by mx1.freebsd.org (Postfix) with ESMTP id 4D2E68FC12; Fri, 10 Dec 2010 00:24:00 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id oBA0NxcT024381; Thu, 9 Dec 2010 16:23:59 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 476302D6011; Thu, 9 Dec 2010 16:23:58 -0800 (PST) Message-ID: <4D01731C.4070802@freebsd.org> Date: Thu, 09 Dec 2010 16:23:56 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: Kostik Belousov References: <20101208225352.GK33073@deviant.kiev.zoral.com.ua> <201012082356.oB8NuxwX040914@chez.mckusick.com> <20101209103411.GL33073@deviant.kiev.zoral.com.ua> In-Reply-To: <20101209103411.GL33073@deviant.kiev.zoral.com.ua> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Kirk McKusick , freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 00:24:00 -0000 On 12/9/10 2:34 AM, Kostik Belousov wrote: [patch] commit when ready.. I'm looking forwards to being able to try it out. From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 00:37:52 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E5EE1065675; Fri, 10 Dec 2010 00:37:52 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (mail.bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id 63B6C8FC0A; Fri, 10 Dec 2010 00:37:52 +0000 (UTC) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 3F7E15B92; Thu, 9 Dec 2010 16:37:49 -0800 (PST) To: Kirk McKusick In-reply-to: Your message of "Thu, 09 Dec 2010 10:13:39 PST." <201012091813.oB9IDd2H078366@chez.mckusick.com> References: <201012091813.oB9IDd2H078366@chez.mckusick.com> Comments: In-reply-to Kirk McKusick message dated "Thu, 09 Dec 2010 10:13:39 -0800." Date: Thu, 09 Dec 2010 16:37:48 -0800 From: Bakul Shah Message-Id: <20101210003749.3F7E15B92@mail.bitblocks.com> Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 00:37:52 -0000 On Thu, 09 Dec 2010 10:13:39 PST Kirk McKusick wrote: > Other than the nit pointed out by Pawel, the diffs look good to me. > You should consider adding the -t option to newfs so that the TRIM > option can be specified at the time the filesystem is created (as > a general rule, anything you can do with tunefs should also be > possible with newfs). > > I agree with your decision to let administrators opt-out of doing > TRIM. If experience shows it to be generally useful to have it on, > we can change the default to enabled later. If we do change the > default to enabled, then we will want to delete the warning about TRIM > not being supported by the underlying disk that you added at mount > time as we would start getting a lot of them for all the non-SSD disks. Would be nice if something like ftrim(fd, offset, size) or trim(path, offsetm size) or TRIM file ioctl is added, to free up blocks undelying a given range in a file. ftruncate can delete blocks at the end but there is no facility to lose blocks in the middle. Mainly handy for virtual disks and databases (and would work nicely with SEEK_DATA, SEEK_HOLE). From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 00:44:56 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F1096106564A for ; Fri, 10 Dec 2010 00:44:56 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta12.westchester.pa.mail.comcast.net (qmta12.westchester.pa.mail.comcast.net [76.96.59.227]) by mx1.freebsd.org (Postfix) with ESMTP id 5C3D88FC24 for ; Fri, 10 Dec 2010 00:44:55 +0000 (UTC) Received: from omta06.westchester.pa.mail.comcast.net ([76.96.62.51]) by qmta12.westchester.pa.mail.comcast.net with comcast id gzjZ1f00816LCl05CCkvtd; Fri, 10 Dec 2010 00:44:55 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta06.westchester.pa.mail.comcast.net with comcast id hCkt1f00B3LrwQ23SCkuxj; Fri, 10 Dec 2010 00:44:55 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 4F5DC9B427; Thu, 9 Dec 2010 16:44:52 -0800 (PST) Date: Thu, 9 Dec 2010 16:44:52 -0800 From: Jeremy Chadwick To: Julian Elischer Message-ID: <20101210004452.GA15765@icarus.home.lan> References: <20101208225352.GK33073@deviant.kiev.zoral.com.ua> <201012082356.oB8NuxwX040914@chez.mckusick.com> <20101209103411.GL33073@deviant.kiev.zoral.com.ua> <4D01731C.4070802@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D01731C.4070802@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Kirk McKusick , pjd@freebsd.org, freebsd-fs@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 00:44:57 -0000 On Thu, Dec 09, 2010 at 04:23:56PM -0800, Julian Elischer wrote: > On 12/9/10 2:34 AM, Kostik Belousov wrote: > > [patch] > > commit when ready.. I'm looking forwards to being able to try it out. And how does one actually verify TRIM is working? Most SSDs I've used do not have a SMART attribute that can help determine this. I would rather not make a bunch of assumptions about Attributes 226, 232, or 233 to help give some indication of it working. For folks to test this, there needs to be some indicator/verification mechanism that it's working. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 00:57:20 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 92BB6106566C; Fri, 10 Dec 2010 00:57:20 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [64.81.247.49]) by mx1.freebsd.org (Postfix) with ESMTP id 372528FC13; Fri, 10 Dec 2010 00:57:20 +0000 (UTC) Received: from chez.mckusick.com (localhost [127.0.0.1]) by chez.mckusick.com (8.14.3/8.14.3) with ESMTP id oBA0vIgT065481; Thu, 9 Dec 2010 16:57:18 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <201012100057.oBA0vIgT065481@chez.mckusick.com> To: Bakul Shah In-reply-to: <20101210003749.3F7E15B92@mail.bitblocks.com> Date: Thu, 09 Dec 2010 16:57:18 -0800 From: Kirk McKusick Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 00:57:20 -0000 > To: Kirk McKusick > cc: Kostik Belousov , freebsd-fs@freebsd.org, > pjd@freebsd.org, Oliver Fromme > Subject: Re: TRIM support for UFS? > Date: Thu, 09 Dec 2010 16:37:48 -0800 > From: Bakul Shah > X-ASK-Info: Message Queued (2010/12/09 16:37:54) > X-ASK-Info: Confirmed by User (2010/12/09 16:39:17) > > Would be nice if something like ftrim(fd, offset, size) or > trim(path, offsetm size) or TRIM file ioctl is added, to free > up blocks undelying a given range in a file. ftruncate can > delete blocks at the end but there is no facility to lose > blocks in the middle. Mainly handy for virtual disks and > databases (and would work nicely with SEEK_DATA, SEEK_HOLE). We have discussed adding such a system call over the years (actually we called it `release' rather than `trim'). If such a call ever does get added, any blocks that it actually manages to release will get passed through to BIO_DELETE by the changes that Kostik is soon to add to the system. Kirk McKusick From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 00:58:48 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1421C106566B for ; Fri, 10 Dec 2010 00:58:48 +0000 (UTC) (envelope-from pjd@garage.freebsd.pl) Received: from mail.garage.freebsd.pl (60.wheelsystems.com [83.12.187.60]) by mx1.freebsd.org (Postfix) with ESMTP id AF67C8FC16 for ; Fri, 10 Dec 2010 00:58:47 +0000 (UTC) Received: by mail.garage.freebsd.pl (Postfix, from userid 65534) id C07DB45C89; Fri, 10 Dec 2010 01:58:45 +0100 (CET) Received: from localhost (89-73-192-49.dynamic.chello.pl [89.73.192.49]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.garage.freebsd.pl (Postfix) with ESMTP id 4BB9B456B1; Fri, 10 Dec 2010 01:58:40 +0100 (CET) Date: Fri, 10 Dec 2010 01:58:38 +0100 From: Pawel Jakub Dawidek To: Bakul Shah Message-ID: <20101210005838.GD1866@garage.freebsd.pl> References: <201012091813.oB9IDd2H078366@chez.mckusick.com> <20101210003749.3F7E15B92@mail.bitblocks.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ylS2wUBXLOxYXZFQ" Content-Disposition: inline In-Reply-To: <20101210003749.3F7E15B92@mail.bitblocks.com> User-Agent: Mutt/1.4.2.3i X-PGP-Key-URL: http://people.freebsd.org/~pjd/pjd.asc X-OS: FreeBSD 9.0-CURRENT amd64 X-Spam-Checker-Version: SpamAssassin 3.0.4 (2005-06-05) on mail.garage.freebsd.pl X-Spam-Level: X-Spam-Status: No, score=-0.1 required=4.5 tests=BAYES_00,OPTING_OUT, RCVD_IN_SORBS_DUL autolearn=no version=3.0.4 Cc: Kirk McKusick , Oliver Fromme , freebsd-fs@freebsd.org Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 00:58:48 -0000 --ylS2wUBXLOxYXZFQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Dec 09, 2010 at 04:37:48PM -0800, Bakul Shah wrote: > On Thu, 09 Dec 2010 10:13:39 PST Kirk McKusick w= rote: > > Other than the nit pointed out by Pawel, the diffs look good to me. > > You should consider adding the -t option to newfs so that the TRIM > > option can be specified at the time the filesystem is created (as > > a general rule, anything you can do with tunefs should also be > > possible with newfs). > >=20 > > I agree with your decision to let administrators opt-out of doing > > TRIM. If experience shows it to be generally useful to have it on, > > we can change the default to enabled later. If we do change the > > default to enabled, then we will want to delete the warning about TRIM > > not being supported by the underlying disk that you added at mount > > time as we would start getting a lot of them for all the non-SSD disks. >=20 > Would be nice if something like ftrim(fd, offset, size) or > trim(path, offsetm size) or TRIM file ioctl is added, to free > up blocks undelying a given range in a file. ftruncate can > delete blocks at the end but there is no facility to lose > blocks in the middle. Mainly handy for virtual disks and > databases (and would work nicely with SEEK_DATA, SEEK_HOLE). libgeom(3) provides: int g_delete(int fd, off_t offset, off_t length); --=20 Pawel Jakub Dawidek http://www.wheelsystems.com pjd@FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! --ylS2wUBXLOxYXZFQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.14 (FreeBSD) iEYEARECAAYFAk0Bez0ACgkQForvXbEpPzTIDQCfV8O/HK/uNsslogvXeWYVfaaO qtsAnRCRdCZTCBUxhBhCCI4EDkwe/oPk =Irz9 -----END PGP SIGNATURE----- --ylS2wUBXLOxYXZFQ-- From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 01:35:15 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B940B106564A for ; Fri, 10 Dec 2010 01:35:15 +0000 (UTC) (envelope-from bakul@bitblocks.com) Received: from mail.bitblocks.com (mail.bitblocks.com [64.142.15.60]) by mx1.freebsd.org (Postfix) with ESMTP id 9E2F18FC14 for ; Fri, 10 Dec 2010 01:35:15 +0000 (UTC) Received: from bitblocks.com (localhost.bitblocks.com [127.0.0.1]) by mail.bitblocks.com (Postfix) with ESMTP id 0C30C5B94; Thu, 9 Dec 2010 17:27:19 -0800 (PST) To: Kirk McKusick In-reply-to: Your message of "Thu, 09 Dec 2010 16:57:18 PST." <201012100057.oBA0vIgT065481@chez.mckusick.com> References: <201012100057.oBA0vIgT065481@chez.mckusick.com> Comments: In-reply-to Kirk McKusick message dated "Thu, 09 Dec 2010 16:57:18 -0800." Date: Thu, 09 Dec 2010 17:27:19 -0800 From: Bakul Shah Message-Id: <20101210012720.0C30C5B94@mail.bitblocks.com> Cc: freebsd-fs@freebsd.org, pjd@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 01:35:15 -0000 On Thu, 09 Dec 2010 16:57:18 PST Kirk McKusick wrote: > > To: Kirk McKusick > > cc: Kostik Belousov , freebsd-fs@freebsd.org, > > pjd@freebsd.org, Oliver Fromme > > Subject: Re: TRIM support for UFS? > > Date: Thu, 09 Dec 2010 16:37:48 -0800 > > From: Bakul Shah > > X-ASK-Info: Message Queued (2010/12/09 16:37:54) > > X-ASK-Info: Confirmed by User (2010/12/09 16:39:17) > > > > Would be nice if something like ftrim(fd, offset, size) or > > trim(path, offsetm size) or TRIM file ioctl is added, to free > > up blocks undelying a given range in a file. ftruncate can > > delete blocks at the end but there is no facility to lose > > blocks in the middle. Mainly handy for virtual disks and > > databases (and would work nicely with SEEK_DATA, SEEK_HOLE). > > We have discussed adding such a system call over the years (actually > we called it `release' rather than `trim'). If such a call ever does > get added, any blocks that it actually manages to release will get > passed through to BIO_DELETE by the changes that Kostik is soon to > add to the system. That is why this seemed like a good time to mention this! Probably most of the logic for this can be borrowed from ftruncate().... Pawel mentions g_delete() but it seems a bit indirect. From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 05:19:57 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0B8661065670; Fri, 10 Dec 2010 05:19:57 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from out-0.mx.aerioconnect.net (out-0-6.mx.aerioconnect.net [216.240.47.66]) by mx1.freebsd.org (Postfix) with ESMTP id A597B8FC17; Fri, 10 Dec 2010 05:19:56 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id oBA5Js1S007869; Thu, 9 Dec 2010 21:19:54 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 6CB3A2D6012; Thu, 9 Dec 2010 21:19:53 -0800 (PST) Message-ID: <4D01B878.4020008@freebsd.org> Date: Thu, 09 Dec 2010 21:19:52 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: Pawel Jakub Dawidek References: <201012091813.oB9IDd2H078366@chez.mckusick.com> <20101210003749.3F7E15B92@mail.bitblocks.com> <20101210005838.GD1866@garage.freebsd.pl> In-Reply-To: <20101210005838.GD1866@garage.freebsd.pl> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Kirk McKusick , Oliver Fromme , freebsd-fs@freebsd.org Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 05:19:57 -0000 On 12/9/10 4:58 PM, Pawel Jakub Dawidek wrote: > On Thu, Dec 09, 2010 at 04:37:48PM -0800, Bakul Shah wrote: >> On Thu, 09 Dec 2010 10:13:39 PST Kirk McKusick wrote: >>> Other than the nit pointed out by Pawel, the diffs look good to me. >>> You should consider adding the -t option to newfs so that the TRIM >>> option can be specified at the time the filesystem is created (as >>> a general rule, anything you can do with tunefs should also be >>> possible with newfs). >>> >>> I agree with your decision to let administrators opt-out of doing >>> TRIM. If experience shows it to be generally useful to have it on, >>> we can change the default to enabled later. If we do change the >>> default to enabled, then we will want to delete the warning about TRIM >>> not being supported by the underlying disk that you added at mount >>> time as we would start getting a lot of them for all the non-SSD disks. >> Would be nice if something like ftrim(fd, offset, size) or >> trim(path, offsetm size) or TRIM file ioctl is added, to free >> up blocks undelying a given range in a file. ftruncate can >> delete blocks at the end but there is no facility to lose >> blocks in the middle. Mainly handy for virtual disks and >> databases (and would work nicely with SEEK_DATA, SEEK_HOLE). > libgeom(3) provides: > > int g_delete(int fd, off_t offset, off_t length); > One of the things that has not been mentioned is that trim is not really 'free' (at least not for us) if you want things to remain trimmed after a reboot. so if I were implementing it I'd want a couple of parameters. 1/ don't bother trimming free space under some size. 2/ does it matter if the trimmed space comes back as garbage after an unclean shutdown? (a hint to the driver, and no, I don't know anyone that supports this yet) (there are security implications to that one but cheap trim (that may come back) is way cheaper than persistent trim to impliment). with these parameters we (fusion-io) could tune our behaviour which can save performance, and also the file system could tune it's min-trim value to see what gives best performance. From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 05:21:38 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 39E12106566C; Fri, 10 Dec 2010 05:21:38 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from out-0.mx.aerioconnect.net (out-0-6.mx.aerioconnect.net [216.240.47.66]) by mx1.freebsd.org (Postfix) with ESMTP id EF7978FC16; Fri, 10 Dec 2010 05:21:37 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id oBA5Lbfo007943; Thu, 9 Dec 2010 21:21:37 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id 19E3C2D6012; Thu, 9 Dec 2010 21:21:35 -0800 (PST) Message-ID: <4D01B8DE.1080601@freebsd.org> Date: Thu, 09 Dec 2010 21:21:34 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: Jeremy Chadwick References: <20101208225352.GK33073@deviant.kiev.zoral.com.ua> <201012082356.oB8NuxwX040914@chez.mckusick.com> <20101209103411.GL33073@deviant.kiev.zoral.com.ua> <4D01731C.4070802@freebsd.org> <20101210004452.GA15765@icarus.home.lan> In-Reply-To: <20101210004452.GA15765@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: Kirk McKusick , pjd@freebsd.org, freebsd-fs@freebsd.org, Oliver Fromme Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 05:21:38 -0000 On 12/9/10 4:44 PM, Jeremy Chadwick wrote: > On Thu, Dec 09, 2010 at 04:23:56PM -0800, Julian Elischer wrote: >> On 12/9/10 2:34 AM, Kostik Belousov wrote: >> >> [patch] >> >> commit when ready.. I'm looking forwards to being able to try it out. > And how does one actually verify TRIM is working? Most SSDs I've used > do not have a SMART attribute that can help determine this. I would > rather not make a bunch of assumptions about Attributes 226, 232, or 233 > to help give some indication of it working. > > For folks to test this, there needs to be some indicator/verification > mechanism that it's working. > On our (fusion-io) cards we do not go via scsi or SATA so we have our own interfaces which can provide all this information.. (tons of sysctls in FreeBSD.. tons of /proc entries in linux) From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 05:45:52 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 08A9B1065670; Fri, 10 Dec 2010 05:45:52 +0000 (UTC) (envelope-from mckusick@mckusick.com) Received: from chez.mckusick.com (chez.mckusick.com [64.81.247.49]) by mx1.freebsd.org (Postfix) with ESMTP id CEBAB8FC08; Fri, 10 Dec 2010 05:45:51 +0000 (UTC) Received: from chez.mckusick.com (localhost [127.0.0.1]) by chez.mckusick.com (8.14.3/8.14.3) with ESMTP id oBA5jkwO027291; Thu, 9 Dec 2010 21:45:46 -0800 (PST) (envelope-from mckusick@chez.mckusick.com) Message-Id: <201012100545.oBA5jkwO027291@chez.mckusick.com> To: Julian Elischer In-reply-to: <4D01B878.4020008@freebsd.org> Date: Thu, 09 Dec 2010 21:45:46 -0800 From: Kirk McKusick Cc: Oliver Fromme , Pawel Jakub Dawidek , freebsd-fs@freebsd.org Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 05:45:52 -0000 > Date: Thu, 09 Dec 2010 21:19:52 -0800 > From: Julian Elischer > To: Pawel Jakub Dawidek > CC: Bakul Shah , Kirk McKusick , > Oliver Fromme , freebsd-fs@freebsd.org > Subject: Re: TRIM support for UFS? > > One of the things that has not been mentioned is that releasing > space in a file is not really 'free' (at least not for us) if you > want things to remain trimmed after a reboot. So if I were > implementing it I'd want a couple of parameters. > > 1/ don't bother trimming free space under some size. > > 2/ does it matter if the trimmed space comes back as garbage > after an unclean shutdown? (a hint to the driver, and no, I don't > know anyone that supports this yet) > (there are security implications to that one but cheap trim (that > may come back) is way cheaper than persistent trim to impliment). > > With these parameters we (fusion-io) could tune our behaviour > which can save performance, and also the file system could tune > it's min-trim value to see what gives best performance. The above is one of the reasons that we never implemented the `release' system call. To be stable after a crash requires either a lot of synchronous writes or a bunch more soft update dependencies. Many of the same issues relate to stability of snapshots across system crashes. In the end we added the dopersistence flag: /* * To ensure the consistency of snapshots across crashes, we must * synchronously write out copied blocks before allowing the * originals to be modified. Because of the rather severe speed * penalty that this imposes, the following flag allows this * crash persistence to be disabled. */ int dopersistence = 0; #ifdef DEBUG #include SYSCTL_INT(_debug, OID_AUTO, dopersistence, CTLFLAG_RW, &dopersistence, 0, ""); #endif /* DEBUG */ As you can see it defaults to off. Kirk McKusick From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 08:36:38 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4EB1B106564A; Fri, 10 Dec 2010 08:36:38 +0000 (UTC) (envelope-from BATV+870d71e9bf4a5d360c11+2665+infradead.org+hch@bombadil.srs.infradead.org) Received: from bombadil.infradead.org (unknown [IPv6:2001:4830:2446:ff00:4687:fcff:fea6:5117]) by mx1.freebsd.org (Postfix) with ESMTP id E1A478FC15; Fri, 10 Dec 2010 08:36:37 +0000 (UTC) Received: from hch by bombadil.infradead.org with local (Exim 4.72 #1 (Red Hat Linux)) id 1PQySb-0004vO-Cc; Fri, 10 Dec 2010 08:36:13 +0000 Date: Fri, 10 Dec 2010 03:36:13 -0500 From: Christoph Hellwig To: Julian Elischer Message-ID: <20101210083613.GA12835@infradead.org> References: <201012091813.oB9IDd2H078366@chez.mckusick.com> <20101210003749.3F7E15B92@mail.bitblocks.com> <20101210005838.GD1866@garage.freebsd.pl> <4D01B878.4020008@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D01B878.4020008@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html Cc: Kirk McKusick , Oliver Fromme , Pawel Jakub Dawidek , freebsd-fs@freebsd.org Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 08:36:38 -0000 On Thu, Dec 09, 2010 at 09:19:52PM -0800, Julian Elischer wrote: > One of the things that has not been mentioned is that trim is not > really 'free' > (at least not for us) if you want things to remain trimmed after a reboot. > so if I were implementing it I'd want a couple of parameters. > > 1/ don't bother trimming free space under some size. > 2/ does it matter if the trimmed space comes back as garbage > after an unclean shutdown? (a hint to the driver, and no, I don't > know anyone that supports this yet) > (there are security implications to that one but cheap trim (that > may come back) is way cheaper than persistent trim to impliment). Please take a look at the SCSI and ATA standards for thin provisioning and TRIM. For SCSI there are EVPD pages with a lot of information about the required trim granularity and alignment. For SCSI an UNMAP or WRITE SAME with the unmap bit set always guarantees deterministic behaviour after a discard of used data, which is optional in ATA, but all SSDs I have access to claim to support it (but at least one didn't do it properly). Both SBC and ATA have a bit that requires any discarded space to be zeroed, which is very important for RAID or virtualization use cases. I would suggest to let the BSD implementation mirror those standards - that's what we've done for Linux. Another interesting angle is that the SCSI UNAMP command and the ATA TRIM command support ranges of blocks to discard, which is a feature that Windows 7 uses a lot on SSDs, given that the overhead of a single TRIM can be quite bad given that it's a non-queueable command. If you want to add ioctls for trimming on raw block devices, the free space on a filesystem, or punching holes please take a look at the existing Linux BLKDISCARD and FITRIM ioctls, as well as the fallocate extension to punch holes that's currently under discussion. From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 10:26:46 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8FBF6106564A for ; Fri, 10 Dec 2010 10:26:46 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from out-0.mx.aerioconnect.net (out-0-11.mx.aerioconnect.net [216.240.47.71]) by mx1.freebsd.org (Postfix) with ESMTP id 7088D8FC1C for ; Fri, 10 Dec 2010 10:26:46 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id oBAAQidC020002; Fri, 10 Dec 2010 02:26:45 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id DEA672D6013; Fri, 10 Dec 2010 02:26:43 -0800 (PST) Message-ID: <4D020062.3030704@freebsd.org> Date: Fri, 10 Dec 2010 02:26:42 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: Christoph Hellwig References: <201012091813.oB9IDd2H078366@chez.mckusick.com> <20101210003749.3F7E15B92@mail.bitblocks.com> <20101210005838.GD1866@garage.freebsd.pl> <4D01B878.4020008@freebsd.org> <20101210083613.GA12835@infradead.org> In-Reply-To: <20101210083613.GA12835@infradead.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: freebsd-fs@freebsd.org Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 10:26:46 -0000 private response. BTW as you seem to be a Linux person how did you end up here? (not a problem, just curious) On 12/10/10 12:36 AM, Christoph Hellwig wrote: > On Thu, Dec 09, 2010 at 09:19:52PM -0800, Julian Elischer wrote: >> One of the things that has not been mentioned is that trim is not >> really 'free' >> (at least not for us) if you want things to remain trimmed after a reboot. >> so if I were implementing it I'd want a couple of parameters. >> >> 1/ don't bother trimming free space under some size. I didn't really explain well, but I meant that this would be a filesystem parameter. SCSI would allow the filesystem to determine it automatically but not all devices are SCSI (or SATA). (e.g. the ones we make at Fusion-IO (www.fusionio.com)) >> 2/ does it matter if the trimmed space comes back as garbage >> after an unclean shutdown? (a hint to the driver, and no, I don't >> know anyone that supports this yet) >> (there are security implications to that one but cheap trim (that >> may come back) is way cheaper than persistent trim to impliment). > Please take a look at the SCSI and ATA standards for thin provisioning > and TRIM. It's been many years since I wrote the scsi code for MACH and BSD (long since handed off to others) and I haven't really followed the scsi spec for a while, but I will follow up and look at these pages. > For SCSI there are EVPD pages with a lot of information about the > required trim granularity and alignment. For SCSI an UNMAP or WRITE > SAME with the unmap bit set always guarantees deterministic behaviour > after a discard of used data, which is optional in ATA, but all SSDs > I have access to claim to support it (but at least one didn't do it > properly). but do they support the option to be nodeterministic about it :-) When you trim a block range you have to store that information somewhere. and that in turn can cause a cascade of problems when you are trying to cope with unclean shutdown. (I work for fusionIO with Jens Axboe and Nick Piggins and my job includes writing code to do exactly that). it takes cpu and ram and doing so detracts from overall performance so if your drive has an option to make trims "non persistant" in the case of power failure you can use that fact to increase the performance in other ways. > Both SBC and ATA have a bit that requires any discarded > space to be zeroed, which is very important for RAID or virtualization > use cases. I would suggest to let the BSD implementation mirror those > standards - that's what we've done for Linux. Another interesting > angle is that the SCSI UNAMP command and the ATA TRIM command support > ranges of blocks to discard, which is a feature that Windows 7 uses > a lot on SSDs, given that the overhead of a single TRIM can be quite bad > given that it's a non-queueable command. yes that's a side effect of some of the hidden complexities of 'trim' I think. > If you want to add ioctls for trimming on raw block devices, the free > space on a filesystem, or punching holes please take a look at the > existing Linux BLKDISCARD and FITRIM ioctls, as well as the fallocate > extension to punch holes that's currently under discussion. I'll leave it to Jens to work on the ioctls for linux, but we have the same driver for Windows, Linux, FreeBSD, OS-X, AIX, ESX, and Solaris with different adaptation layers so what we decide to export is pretty flexible. regards Julian From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 13:34:30 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 641EA1065670 for ; Fri, 10 Dec 2010 13:34:30 +0000 (UTC) (envelope-from mike.barnardq@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id 211FF8FC0C for ; Fri, 10 Dec 2010 13:34:29 +0000 (UTC) Received: by ywp6 with SMTP id 6so2109712ywp.13 for ; Fri, 10 Dec 2010 05:34:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:date:message-id :subject:from:to:content-type; bh=yeLTuDzFd6wz4q4TGkPM178fG8RwvK2XCNEmMqwshOQ=; b=dywtSUZ3cJBD3k9Kwihu5O9dg+oJ8O3YGZaT7DVLy5pX9YxKKO5e9dlRC23K4Mp6KE 5d6fqe+YHoEA0oVJFWgkK+CiZw17zfLUXBpJjS45t2TcAXK0deWjisSFFIgCVdJH8fgo ow9drer4DbE4ghkuMIJW3dMGpkBl++0Wkf/jg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:date:message-id:subject:from:to:content-type; b=mRGvRMMoDgaZwO7afD/2UDw0XxMnMiTIFpshfhaP6boC00nbsy/jm0UGEYm7CVS5Y5 ijRKZEPsE2LBRsezlX39CIz9lVKwmVFVtTHoKaIvB2BNLClKSkcHBI1LNAzmF4dQoG8D MYF35RN9CSe09gtD+moBHX6vcg68QSIckFcBM= MIME-Version: 1.0 Received: by 10.100.164.1 with SMTP id m1mr479520ane.36.1291986353636; Fri, 10 Dec 2010 05:05:53 -0800 (PST) Received: by 10.100.38.18 with HTTP; Fri, 10 Dec 2010 05:05:53 -0800 (PST) Date: Fri, 10 Dec 2010 16:05:53 +0300 Message-ID: From: Mike Barnard To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: HAST role failure X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 13:34:30 -0000 Hi, I have setup HAST on 2 FreeBSD 8.1-STABLE, hastA being primary and hastB secondary. hastctl create mirror works with no problem on hastA, but when I run hastctl role primary shared on the primary, it gives an error and fails: Dec 10 15:47:24 hastA hastd[9911]: [shared] (primary) Unable to open /dev/da0s1g.journal: Operation not permitted. Dec 10 15:47:29 hastA hastd[9888]: [shared] (primary) Worker process exited ungracefully (pid=9911, exitcode=66). My hast.conf looks like this: resource shared { replication memsync on hastA { local /dev/da0s1g.journal remote tcp4://172.19.66.15 } on hastB { local /dev/da0s1g.journal remote tcp4://172.19.66.14 } } kern.securelevel is set to -1. Is it possible to have a hast resource over a journaled device? Any one experienced this? Is there a way I can destroy a HAST resource once the create option is passed to hastctl? -- Mike Of course, you might discount this possibility, but remember that one in a million chances happen 99% of the time. ------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 14:25:51 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CD1E41065673 for ; Fri, 10 Dec 2010 14:25:51 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4E1F08FC0A for ; Fri, 10 Dec 2010 14:25:50 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.4/8.14.4) with ESMTP id oBAEPnBw021572; Fri, 10 Dec 2010 17:25:49 +0300 (MSK) (envelope-from marck@rinet.ru) Date: Fri, 10 Dec 2010 17:25:49 +0300 (MSK) From: Dmitry Morozovsky To: Jeremy Chadwick In-Reply-To: <20101209193027.GA10784@icarus.home.lan> Message-ID: References: <20101209193027.GA10784@icarus.home.lan> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (woozle.rinet.ru [0.0.0.0]); Fri, 10 Dec 2010 17:25:49 +0300 (MSK) Cc: freebsd-fs@freebsd.org Subject: how to get rid of stale pool? [was (incorrect) stale zpool: how to rename?] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 14:25:51 -0000 On Thu, 9 Dec 2010, Jeremy Chadwick wrote: JC> > how can I clear stale traces of old pool? JC> > JC> > [this is on fresh stable/8] JC> > JC> > Now I have: JC> > JC> > root@ogre2:~# zpool import JC> > pool: og JC> > id: 4146846692207551713 JC> > state: UNAVAIL JC> > status: One or more devices are missing from the system. JC> > action: The pool cannot be imported. Attach the missing JC> > devices and try again. JC> > see: http://www.sun.com/msg/ZFS-8000-3C JC> > config: JC> > JC> > og UNAVAIL insufficient replicas JC> > mirror UNAVAIL insufficient replicas JC> > da0 UNAVAIL cannot open JC> > da1 UNAVAIL cannot open JC> > root@ogre2:~# zpool status JC> > pool: o JC> > state: ONLINE JC> > scrub: none requested JC> > config: JC> > JC> > NAME STATE READ WRITE CKSUM JC> > o ONLINE 0 0 0 JC> > mirror ONLINE 0 0 0 JC> > da0h ONLINE 0 0 0 JC> > da1h ONLINE 0 0 0 JC> > JC> > errors: No known data errors JC> > root@ogre2:~# zpool import -f og JC> > cannot import 'og': no such pool or dataset JC> > root@ogre2:~# zpool destroy og JC> > cannot open 'og': no such pool JC> JC> Try using the "id" of the old pool, not the pool name. not much of luck: root@ogre2:~# zpool destroy 4146846692207551713 cannot open '4146846692207551713': name must begin with a letter root@ogre2:~# zpool import -f 4146846692207551713 og cannot import 'og' as 'og': no such pool or dataset root@ogre2:~# zpool import -f 4146846692207551713 ogbad cannot import 'og' as 'ogbad': no such pool or dataset -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 15:35:11 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9711D1065673 for ; Fri, 10 Dec 2010 15:35:11 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-gx0-f172.google.com (mail-gx0-f172.google.com [209.85.161.172]) by mx1.freebsd.org (Postfix) with ESMTP id 50B398FC20 for ; Fri, 10 Dec 2010 15:35:10 +0000 (UTC) Received: by gxk28 with SMTP id 28so2867541gxk.17 for ; Fri, 10 Dec 2010 07:35:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=WTN5gyiH/y5q0gMPktTvfTknGVeG1Ld85nEJzs5IheU=; b=uZMWzTw2poc5icNshDPllEl+9jijUtT50RsrW40jcjBxLG4qUr5e8RD28csyJzuMcQ 3avlM85HT0KbtyK9cekGkNxy/guJKN5wUh7tyN2OCE2fWVB+fkdeK6HLW2zg7GC+VBJV TSQRWuwDJREF+eMxcgAphrq9OeWddlX78QmQw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=gWw89hHL6exyxneeL4bDdAjPnVgNXf6xYvJEX+Wv1RNVQBrtw+mF6WLX3jsRatMBgN STvbhypAKOb8rHb6815OyIpsGHnRcTySKkMrPvU2rH71So6P/tCUQbrrNpQXckcU04SP 4KwcwSl9MHJnEqN8+nJ72v8BKG2N18wqeVei4= MIME-Version: 1.0 Received: by 10.90.103.9 with SMTP id a9mr1329647agc.165.1291995310453; Fri, 10 Dec 2010 07:35:10 -0800 (PST) Received: by 10.90.226.2 with HTTP; Fri, 10 Dec 2010 07:35:10 -0800 (PST) In-Reply-To: References: <20101209193027.GA10784@icarus.home.lan> Date: Fri, 10 Dec 2010 07:35:10 -0800 Message-ID: From: Freddie Cash To: Dmitry Morozovsky Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org Subject: Re: how to get rid of stale pool? [was (incorrect) stale zpool: how to rename?] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 15:35:11 -0000 On Fri, Dec 10, 2010 at 6:25 AM, Dmitry Morozovsky wrote: > On Thu, 9 Dec 2010, Jeremy Chadwick wrote: > JC> > how can I clear stale traces of old pool? zero out the first and last MB of the disk. ZFS stores the GPT and other pool metadata in those two locations. It's nowhere near a MB in size, but if you clear out the first and last MB, you're guaranteed to clear out all traces of ZFS metadata. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 15:39:38 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E322F1065674 for ; Fri, 10 Dec 2010 15:39:38 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-yw0-f54.google.com (mail-yw0-f54.google.com [209.85.213.54]) by mx1.freebsd.org (Postfix) with ESMTP id A0A3D8FC14 for ; Fri, 10 Dec 2010 15:39:38 +0000 (UTC) Received: by ywp6 with SMTP id 6so2201233ywp.13 for ; Fri, 10 Dec 2010 07:39:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=ufHzBncxRevTsikYW+p0QOUujsfzo6YTtIhXENJnmAs=; b=xPkg7mHXlByiV74rLqDb4JnXw/ibZUoRbGjj8g9MYm1fyul+V8r2eJqKCALt3Er3Mt tbporV03D9bSEd4WlFV+LoUgruwRJhQ5RvoJ5MpynDHFNoJhrKtefAuCrFRdrpuetrnq cYtlxrXc81cOCgL1/57czfPodzToeqmGrRGF0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=qYFok58lNuTF4AU+GwiSFqCJh8/LzImBCTitXEVwCARp9UAsVepAwB4FLXGsHrPzhE D1GBxTBCQfhZK6kTpsuS9OtAkr1No74Jx437LVTplh4QrMnm525n/jtdrAu1xODLkLN2 DEkQw2UxXbe1MtHohDxNO70UPMx0M8GXfjJoM= MIME-Version: 1.0 Received: by 10.91.92.19 with SMTP id u19mr1350969agl.111.1291995577760; Fri, 10 Dec 2010 07:39:37 -0800 (PST) Received: by 10.90.226.2 with HTTP; Fri, 10 Dec 2010 07:39:37 -0800 (PST) In-Reply-To: References: Date: Fri, 10 Dec 2010 07:39:37 -0800 Message-ID: From: Freddie Cash To: Mike Barnard Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs@freebsd.org Subject: Re: HAST role failure X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 15:39:39 -0000 On Fri, Dec 10, 2010 at 5:05 AM, Mike Barnard wrote: > I have setup HAST on 2 FreeBSD 8.1-STABLE, hastA being primary and hastB > secondary. > > hastctl create mirror works with no problem on hastA, but when I run hastctl > role primary shared on the primary, it gives an error and fails: > > Dec 10 15:47:24 hastA hastd[9911]: [shared] (primary) Unable to open > /dev/da0s1g.journal: Operation not permitted. > Dec 10 15:47:29 hastA hastd[9888]: [shared] (primary) Worker process exited > ungracefully (pid=9911, exitcode=66). Create hast on the bare partition. Then create the gjournal device on top of the /dev/hast device. Then use the gjournal device for your filesystem. HAST really should be the lowest level in the storage chain. -- Freddie Cash fjwcash@gmail.com From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 16:02:07 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BCB111065670 for ; Fri, 10 Dec 2010 16:02:07 +0000 (UTC) (envelope-from jhellenthal@gmail.com) Received: from mail-fx0-f49.google.com (mail-fx0-f49.google.com [209.85.161.49]) by mx1.freebsd.org (Postfix) with ESMTP id 429518FC18 for ; Fri, 10 Dec 2010 16:02:06 +0000 (UTC) Received: by fxm19 with SMTP id 19so3610864fxm.36 for ; Fri, 10 Dec 2010 08:02:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:sender:message-id:date:from :organization:user-agent:mime-version:to:cc:subject:references :in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; bh=Kqi05quRFvHxc/7AZADrv6voj9rG2UJe/cdPiqwUhQw=; b=oj95D9/01Aq2oNNUO61PaCtDM+bcTk+N+k9IcBZCKZdlzaSJQySKQ4iQe5442zKRUB pSr+egxsPUZGn/xqK/1OXWT63ZWd05bjmuSLUp6pesOt47rhJ1J90FWp6cCUKv7OuMwS XMS20Qh83p2mHzXp39+WPMMfeYBPYr4ilD4Sg= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=sender:message-id:date:from:organization:user-agent:mime-version:to :cc:subject:references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; b=K/Mc6Gh2zEHD5OZeW2aNPBbvEz9CQUOOT7ROdrdKp/WrHLtrR/PHHjJu98TVpiVqjh TVJCokZ/ZykW5bQBW9XLpnpCS8xzdxaa6gLsY5o7eMLrvBXNicuRNXG02T/aqAAcbui0 5eeM7YzjhYgzP8oVRgOTxwkPs9Spl6xj4fJhA= Received: by 10.223.122.133 with SMTP id l5mr1063574far.52.1291996926169; Fri, 10 Dec 2010 08:02:06 -0800 (PST) Received: from centel.dataix.local (adsl-99-19-45-101.dsl.klmzmi.sbcglobal.net [99.19.45.101]) by mx.google.com with ESMTPS id 5sm959559fak.23.2010.12.10.08.02.04 (version=SSLv3 cipher=RC4-MD5); Fri, 10 Dec 2010 08:02:05 -0800 (PST) Sender: "J. Hellenthal" Message-ID: <4D024EFA.7000801@DataIX.net> Date: Fri, 10 Dec 2010 11:02:02 -0500 From: jhell Organization: http://www.DataIX.net User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.9.2.12) Gecko/20101028 Lightning/1.0b1 Thunderbird MIME-Version: 1.0 To: Freddie Cash References: <20101209193027.GA10784@icarus.home.lan> In-Reply-To: X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-fs@freebsd.org, Dmitry Morozovsky Subject: Re: how to get rid of stale pool? [was (incorrect) stale zpool: how to rename?] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 16:02:07 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 12/10/2010 10:35, Freddie Cash wrote: > On Fri, Dec 10, 2010 at 6:25 AM, Dmitry Morozovsky wrote: >> On Thu, 9 Dec 2010, Jeremy Chadwick wrote: >> JC> > how can I clear stale traces of old pool? > > zero out the first and last MB of the disk. > > ZFS stores the GPT and other pool metadata in those two locations. > It's nowhere near a MB in size, but if you clear out the first and > last MB, you're guaranteed to clear out all traces of ZFS metadata. > To add to this as well you might try removing your cache file '/boot/zfs/zpool.cache' after exporting your existing currently working pool. PS: If this file exists after exporting your pool then it holds corrupt information. - -- jhell,v -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJNAk76AAoJEJBXh4mJ2FR+Lm0H+gP/E5nN5K9/M3OTyBIktlI+ J2gut5UgP4mzwnPCg9yB6firIEhag9MG2ZYgpsqA5bP/4jGkbxIOodjn3pmp0fPd 3/WSaYARg0LjwFikruR8L99DdSscLZ1C9yZzKSv/WcTv/BKLlb2KFBIm7nutrY1l pdITJ3MxfBf01oiJGcN9v0VIhp5SEPNWfvIkikk6N+6jdE/odpHajR0QlSOcUVLb bQHkEHRuMess5vPe1+tzL24W9LWjlb/NA1qbxB33R+CgBXymvRlfXYvMu/D76jks LUxK+AVnJ1N6GlVPAeSD94RHfL/mXT66gceE0MRsB0Y5R8177ubAnuQD5MDt7B0= =ZtbJ -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 17:56:21 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35601106564A for ; Fri, 10 Dec 2010 17:56:21 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from out-0.mx.aerioconnect.net (out-0-18.mx.aerioconnect.net [216.240.47.78]) by mx1.freebsd.org (Postfix) with ESMTP id 166F08FC12 for ; Fri, 10 Dec 2010 17:56:20 +0000 (UTC) Received: from idiom.com (postfix@mx0.idiom.com [216.240.32.160]) by out-0.mx.aerioconnect.net (8.13.8/8.13.8) with ESMTP id oBAHuKcu008263; Fri, 10 Dec 2010 09:56:20 -0800 X-Client-Authorized: MaGic Cook1e X-Client-Authorized: MaGic Cook1e Received: from julian-mac.elischer.org (h-67-100-89-137.snfccasy.static.covad.net [67.100.89.137]) by idiom.com (Postfix) with ESMTP id BC85C2D6017; Fri, 10 Dec 2010 09:56:19 -0800 (PST) Message-ID: <4D0269C2.3090608@freebsd.org> Date: Fri, 10 Dec 2010 09:56:18 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.4; en-US; rv:1.9.2.12) Gecko/20101027 Thunderbird/3.1.6 MIME-Version: 1.0 To: Christoph Hellwig References: <201012091813.oB9IDd2H078366@chez.mckusick.com> <20101210003749.3F7E15B92@mail.bitblocks.com> <20101210005838.GD1866@garage.freebsd.pl> <4D01B878.4020008@freebsd.org> <20101210083613.GA12835@infradead.org> <4D020062.3030704@freebsd.org> In-Reply-To: <4D020062.3030704@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.67 on 216.240.47.51 Cc: freebsd-fs@freebsd.org Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 17:56:21 -0000 On 12/10/10 2:26 AM, Julian Elischer wrote: > private response. > BTW as you seem to be a Linux person how did you end up here? > (not a problem, just curious) oops... sorry guys.. missed the CC, but no matter.. there wasn't really private contents. the extra CC was hidden off the end of the mail agent CC field. > Freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Fri Dec 10 18:20:29 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 834E9106566C; Fri, 10 Dec 2010 18:20:29 +0000 (UTC) (envelope-from BATV+870d71e9bf4a5d360c11+2665+infradead.org+hch@bombadil.srs.infradead.org) Received: from bombadil.infradead.org (unknown [IPv6:2001:4830:2446:ff00:4687:fcff:fea6:5117]) by mx1.freebsd.org (Postfix) with ESMTP id DB8878FC17; Fri, 10 Dec 2010 18:20:28 +0000 (UTC) Received: from hch by bombadil.infradead.org with local (Exim 4.72 #1 (Red Hat Linux)) id 1PR7Zb-0001eQ-Vv; Fri, 10 Dec 2010 18:20:04 +0000 Date: Fri, 10 Dec 2010 13:20:03 -0500 From: Christoph Hellwig To: Julian Elischer Message-ID: <20101210182003.GA6201@infradead.org> References: <201012091813.oB9IDd2H078366@chez.mckusick.com> <20101210003749.3F7E15B92@mail.bitblocks.com> <20101210005838.GD1866@garage.freebsd.pl> <4D01B878.4020008@freebsd.org> <20101210083613.GA12835@infradead.org> <4D020062.3030704@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4D020062.3030704@freebsd.org> User-Agent: Mutt/1.5.21 (2010-09-15) X-SRS-Rewrite: SMTP reverse-path rewritten from by bombadil.infradead.org See http://www.infradead.org/rpr.html Cc: freebsd-fs@freebsd.org Subject: Re: TRIM support for UFS? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Dec 2010 18:20:29 -0000 On Fri, Dec 10, 2010 at 02:26:42AM -0800, Julian Elischer wrote: > private response. > BTW as you seem to be a Linux person how did you end up here? > (not a problem, just curious) I might be mostly a Linux person, but I've also done various BSD based customer projects. I've been subscribed for various FreeBSD lists for years to keep my knowledge in this area current. > but do they support the option to be nodeterministic about it :-) Well, it's not an option for the operating system to select. It's something the disk choses to advertise through the ATA IDENTIFY page. > When you trim a block range you have to store that information somewhere. > and that in turn can cause a cascade of problems when you are trying to > cope with unclean shutdown. (I work for fusionIO with Jens Axboe and > Nick Piggins and my job includes writing code to do exactly that). > it takes cpu and ram and doing so detracts from overall performance > so if your drive has an option to make trims "non persistant" in the > case of > power failure you can use that fact to increase the performance in > other ways. Yes, it's simpler, but also not overly useful for many of use cases. Take a look at ftp://ftp.t10.org/t10/document.08/08-347r1.pdf for a slightly older writeup of the issues of indeterminate TRIM behaviour. Depending on how your structure your underlying storage returning determinate blocks is not a huge problem. I've already written two different implementation of TRIM/UNMAP building on top of the XFS hole punching ioctl - the qemu one can be found on the qemu mailinglist for anyone curious enough. With the XFS hole punch ioctl we only need to log changes to the extent btree and the inode core, as well as zeroing out areas of a block that wasn't fully discarded. It boils down to relatively little sequential I/O into the filesystem log, which can be on a separate device for arrays, but you don't even have that problem with flash based devices. > I'll leave it to Jens to work on the ioctls for linux, but we have the > same driver for Windows, Linux, FreeBSD, OS-X, AIX, ESX, and Solaris > with different adaptation layers so what we decide to export is > pretty flexible. The ioctls sit in the Linux block layer or filesystems - the low-level driver just needs to understand the discard type bio - which is pretty similar to the delete bio proposed in this thread. From owner-freebsd-fs@FreeBSD.ORG Sat Dec 11 16:04:39 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AE1BE1065670 for ; Sat, 11 Dec 2010 16:04:39 +0000 (UTC) (envelope-from antinix@gmail.com) Received: from mail-qy0-f175.google.com (mail-qy0-f175.google.com [209.85.216.175]) by mx1.freebsd.org (Postfix) with ESMTP id 69D6D8FC0A for ; Sat, 11 Dec 2010 16:04:39 +0000 (UTC) Received: by qyk8 with SMTP id 8so1761345qyk.13 for ; Sat, 11 Dec 2010 08:04:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:sender:received:from:date :x-google-sender-auth:message-id:subject:cc:content-type; bh=WOdvPTVo8f4i6P8gD5yTPM5jicthnBJKsfSaZWNNvN0=; b=LD4QKZ3UPzgoueyvE2FhLRGbXOjBKcmaEavEKnoOo1DDT7q9EgQtq6aQHT9IwfTJYH Do5sqFQEkjUlXMLzjsrCOGKCSGvADdViLU00nT4Ig52nm1V2Yidz76i3Zb0h0swninoi WX7YGvAE1HEwd58tPvONj0mjmK+jDUvlgdjDc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:from:date:x-google-sender-auth:message-id :subject:cc:content-type; b=sOTsBgtNhtkvL2emsL+OdNJXqNE77ZXM+fTVt4MTE1cBJfmFvEEXe7ahZNzY1QNA0q ch7UDKIVYWwKhBwQk8PMYMZMtvLpsxzRs+Zh80ocLHluDAzY4Tr/e2saPb+KcUMmgb1E IDsZ5sCX3+CZrorfkgeav3/dctPb6lIERnP4w= Received: by 10.229.96.206 with SMTP id i14mr1829530qcn.268.1292082039766; Sat, 11 Dec 2010 07:40:39 -0800 (PST) MIME-Version: 1.0 Sender: antinix@gmail.com Received: by 10.229.215.77 with HTTP; Sat, 11 Dec 2010 07:40:19 -0800 (PST) From: Andrei Kolu Date: Sat, 11 Dec 2010 17:40:19 +0200 X-Google-Sender-Auth: RIuAl8ErrA1cvOfE6Ty40ea-i8o Message-ID: Cc: freebsd-fs@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 Subject: zfs scrub error- vdev.too_small X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Dec 2010 16:04:39 -0000 Stresstesting ZFS: # cd /home/antik/ Create files: # mkfile 128m disk1 # mkfile 128m disk2 Create mirror from virtual devices: # zpool create peegel mirror /home/antik/disk1 /home/antik/disk2 # zpool list NAME SIZE USED AVAIL CAP HEALTH ALTROOT peegel 123M 73.5K 123M 0% ONLINE - # zpool status peegel pool: peegel state: ONLINE scrub: none requested config: NAME STATE READ WRITE CKSUM peegel ONLINE 0 0 0 mirror ONLINE 0 0 0 /home/antik/disk1 ONLINE 0 0 0 /home/antik/disk2 ONLINE 0 0 0 errors: No known data errors Break one virtual disk: # dd if=/dev/random of=/home/antik/disk1 bs=512 count=1 # zpool scrub peegel Last thing I saw on screen was this: ZFS: vdev failure, zpool=peegel type=vdev.too_small < Press a key on the console to reboot, Dec 11 17:29:09 freenas kernel: --> or switch off the system now. Dec 11 17:29:09 freenas kernel: Rebooting... Dec 11 17:29:09 freenas kernel: cpu_reset: Stopping other CPUs Dec 11 17:29:09 freenas kernel: Copyright (c) 1992-2010 The FreeBSD Project. Dec 11 17:29:09 freenas kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 Dec 11 17:29:09 freenas kernel: The Regents of the University of California. All rights reserved. Dec 11 17:29:09 freenas kernel: FreeBSD is a registered trademark of The FreeBSD Foundation. Dec 11 17:29:09 freenas kernel: FreeBSD 8.1-RELEASE-p2 #3: Thu Dec 2 01:50:54 EET 2010 Not a sign about ZFS volumes on my system # mount /dev/da0s1a on / (ufs, local) devfs on /dev (devfs, local, multilabel) /dev/da0s1e on /tmp (ufs, local, soft-updates) /dev/da0s1f on /usr (ufs, local, soft-updates) /dev/da0s1d on /var (ufs, local, soft-updates) Any attempt to scrub pool fails with kernel panic. From owner-freebsd-fs@FreeBSD.ORG Sat Dec 11 19:34:33 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 72B86106566B; Sat, 11 Dec 2010 19:34:33 +0000 (UTC) Date: Sat, 11 Dec 2010 19:34:33 +0000 From: Alexander Best To: freebsd-fs@freebsd.org Message-ID: <20101211193433.GA18970@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Subject: expand_number(3) support for tmpfs -o size and -o maxfilesize X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Dec 2010 19:34:33 -0000 hi there, is it possible to get support for something like this fstab entry? tmpfs /tmp tmpfs rw,mode=1777,size=500m 0 0 tmpfs doesn't seem to handle humanzied numbers that well. df -h reports: Filesystem Size Used Avail Capacity Mounted on tmpfs 17G 12M 17G 0% /tmp cheers. alex -- a13x From owner-freebsd-fs@FreeBSD.ORG Sat Dec 11 20:05:28 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 77EE11065670 for ; Sat, 11 Dec 2010 20:05:28 +0000 (UTC) (envelope-from marck@rinet.ru) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) by mx1.freebsd.org (Postfix) with ESMTP id ECCDE8FC1D for ; Sat, 11 Dec 2010 20:05:27 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.4/8.14.4) with ESMTP id oBBK5Qx9004300; Sat, 11 Dec 2010 23:05:26 +0300 (MSK) (envelope-from marck@rinet.ru) Date: Sat, 11 Dec 2010 23:05:26 +0300 (MSK) From: Dmitry Morozovsky To: Freddie Cash In-Reply-To: Message-ID: References: <20101209193027.GA10784@icarus.home.lan> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.6 (woozle.rinet.ru [0.0.0.0]); Sat, 11 Dec 2010 23:05:26 +0300 (MSK) Cc: freebsd-fs@freebsd.org Subject: Re: how to get rid of stale pool? [was (incorrect) stale zpool: how to rename?] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Dec 2010 20:05:28 -0000 On Fri, 10 Dec 2010, Freddie Cash wrote: FC> On Fri, Dec 10, 2010 at 6:25 AM, Dmitry Morozovsky wrote: FC> > On Thu, 9 Dec 2010, Jeremy Chadwick wrote: FC> > JC> > how can I clear stale traces of old pool? FC> FC> zero out the first and last MB of the disk. FC> FC> ZFS stores the GPT and other pool metadata in those two locations. FC> It's nowhere near a MB in size, but if you clear out the first and FC> last MB, you're guaranteed to clear out all traces of ZFS metadata. well, bot what if I would have keep my existing pool on these disks? ;-) On the other hand, it's a mirror, and not too heavylu used, so I can remove second disk, clean it up, add it back, resilver, then do the same on the first one; however, it seems to be a bit more generic problem: stale pool referring no-longre-accessible devices seem to be unclearable... -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------