From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 02:11:03 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id ED497D86 for ; Sun, 5 Oct 2014 02:11:03 +0000 (UTC) Received: from mail.iXsystems.com (mail.ixsystems.com [12.229.62.4]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "*.ixsystems.com", Issuer "Go Daddy Secure Certification Authority" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C77F1F9F for ; Sun, 5 Oct 2014 02:11:03 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id F08C780909; Sat, 4 Oct 2014 19:11:02 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 33888-01; Sat, 4 Oct 2014 19:11:02 -0700 (PDT) Received: from [10.8.0.26] (unknown [10.8.0.26]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id 7ED96808FF; Sat, 4 Oct 2014 19:11:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1412475062; bh=XFhdi682l5rmT8BpbMAuwcef5rvwP75TDUgvBtG6ReM=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=TopIbjyLdPTLwtj8e0DyHxfb9FmsKn0Bq1Yro/UZCJ8s9gpHpqEtjiF50MhhUBeua KrzdivmPoWLr20y9HYdq+GaoKUMkgL/YEXCRkTp4q5rmxcmLvJg7dEGcp5jUDmam/U lEsaTWlEAiNrMwuQCNc+kZjcsMauSme6hGtmgO6s= Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 8.0 \(1988\)) Subject: Re: FreeBSD support being added to GlusterFS From: Jordan Hubbard In-Reply-To: <73B63510-384B-43F3-AC10-78F429F4A5FF@gluster.org> Date: Sat, 4 Oct 2014 19:11:01 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: References: <0F20AEEC-6244-42BC-815C-1440BBBDE664@mail.turbofuzz.com> <20140629203746.GI34108@ivaldir.etoilebsd.net> <1A58F492-946F-46D4-A19E-2734F368CDAC@mail.turbofuzz.com> <0ABAE2AC-BF1B-4125-ACA9-C6177D013E25@mail.turbofuzz.com> <20140706230910.GA8523@ivaldir.etoilebsd.net> <2F416D06-0A98-4E66-902C-ED0690A4B1C0@ixsystems.com> <20140825211459.GB65120@ivaldir.etoilebsd.net> <73B63510-384B-43F3-AC10-78F429F4A5FF@gluster.org> To: Justin Clift X-Mailer: Apple Mail (2.1988) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 02:11:04 -0000 > On Oct 1, 2014, at 3:33 AM, Justin Clift wrote: >=20 > As a data point, GlusterFS 3.6.0 beta3 has been released now too. :) >=20 > = http://bits.gluster.org/pub/gluster/glusterfs/src/glusterfs-3.6.0beta3.tar= .gz >=20 > Everyone that has time, please test/try it out and report back any = bugs. :D Hi Justin, Given that FreeBSD users interact with 3rd party software through the = ports collection rather than random tarballs, would it be possible to = get the port updated and/or find a long-term maintainer for it so that = successive updates to glusterfs can simply be tracked (and communicated) = through the ports collection? Thanks, - Jordan From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 10:24:47 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2DA8CF99 for ; Sun, 5 Oct 2014 10:24:47 +0000 (UTC) Received: from mail-wg0-x22a.google.com (mail-wg0-x22a.google.com [IPv6:2a00:1450:400c:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C0E382D2 for ; Sun, 5 Oct 2014 10:24:46 +0000 (UTC) Received: by mail-wg0-f42.google.com with SMTP id z12so4423255wgg.13 for ; Sun, 05 Oct 2014 03:24:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=OlUHUiOT3Hxa65nh7sDNm/i817i4/tTzN/EHlG+bI7A=; b=C6Ec6iZ0F0kWA7tfQuElpwfpE3RZNZi0UBRHpd9jjexFjf9uS8iZ3/1OPPpd++O17K MFXpGwuoTbt/Gy48pVR9/BCITc9Axga5mlfYBpyADZ0WAoR5SMwHTIOkf2CAevoLhvFI hz/hM2zAqf863WZN+/VIoolT2PXEvc4tgJVt8WQLzMu6//jKCqI7JQEMW2mTIF5P0uqF vHaSBv5r7yxCOshbGKmQCg7HnYuGa9Re6jduc6PPIrQFPvq4jiwukmSl5ivixhpqE0T3 aY/lDarDpSNRv/KIM7z95EEV64mgNpjgzqCgr/0h8RUuwLjjDkFP+WV7Z2A3p0Wcp4qY kjHw== MIME-Version: 1.0 X-Received: by 10.194.8.232 with SMTP id u8mr22042534wja.64.1412504685132; Sun, 05 Oct 2014 03:24:45 -0700 (PDT) Received: by 10.27.77.135 with HTTP; Sun, 5 Oct 2014 03:24:45 -0700 (PDT) Date: Sun, 5 Oct 2014 12:24:45 +0200 Message-ID: Subject: Unable to create ZVOL (error=6) From: Nikolay Denev To: freebsd-fs Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 10:24:47 -0000 Hi, I have a cron to generate daily snapshots like this: 0 0 * * * root /sbin/zfs snapshot -r zfs/bhyve_volumes@`/bin/date +\%H:\%M:\%S_\%d-\%m-\%Y` And I've noticed that I have the following errors in dmesg: ZFS WARNING: Unable to create ZVOL zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_05-10-2014 (error=6). Snapshot looks ok, it's just that the ZVOL cdev is not created : [12:11][ndenev@nas:~]%zfs get creation zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_05-10-2014 NAME PROPERTY VALUE SOURCE zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_05-10-2014 creation Sun Oct 5 0:00 2014 - Error 6 would be ENXIO. I think this might be due to "volmode=dev" setting, as I've ruled out name lenght, or invalid characters. --Nikolay From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 14:51:27 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 30C3B646 for ; Sun, 5 Oct 2014 14:51:27 +0000 (UTC) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 86248F26 for ; Sun, 5 Oct 2014 14:51:25 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.5/8.14.5) with ESMTP id s95EovMK075584; Sun, 5 Oct 2014 18:50:57 +0400 (MSK) (envelope-from marck@rinet.ru) Date: Sun, 5 Oct 2014 18:50:57 +0400 (MSK) From: Dmitry Morozovsky To: Mikolaj Golub Subject: Re: HAST with broken HDD In-Reply-To: <20141003175439.GA7664@gmail.com> Message-ID: References: <542BC135.1070906@Skynet.be> <542BDDB3.8080805@internetx.com> <542BF853.3040604@internetx.com> <542C019E.2080702@internetx.com> <542C0710.3020402@internetx.com> <97aab72e19d640ebb65c754c858043cc@SERVER.ad.usd-group.com> <20141003175439.GA7664@gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (woozle.rinet.ru [0.0.0.0]); Sun, 05 Oct 2014 18:50:57 +0400 (MSK) Cc: "freebsd-fs@freebsd.org" , Matt Churchyard X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 14:51:27 -0000 On Fri, 3 Oct 2014, Mikolaj Golub wrote: > Disk errors are recorded to syslog. Also error counters are displayed > in `hastctl list' output. There is snmp_hast(3) in base -- a module > for bsnmp to retrieve this statistics via snmp protocol (traps are not > supported though). > > For notifications, the hastd can be configured to execute an arbitrary > command on various HAST events (see description for `exec' in > hast.conf(5)). Unfortunately, it does not have hooks for I/O error > events currently. It might be worth adding though. The problem with > this that it may generate to many events, so some throttling is > needed. And, I it, this should be noted, some kind of error-coalescing or similar before going from "warning" shate (there are some read error, but otherwise the disk is useable, and it would be overly hassle to switch to remote component completely) to "error" state (component is unuseable and needs to be replaced ASAP; drop it from HAST pair, and switchover if needed). Error such as "device lost" is, of course, fatal from the very beginning; but -- how should we interpret, well, sporadic controller resets with the disk coming back and catching syncing again? -- Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------ From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 15:50:55 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9F94E1CA for ; Sun, 5 Oct 2014 15:50:55 +0000 (UTC) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 51DC7793 for ; Sun, 5 Oct 2014 15:50:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id CB40B1472003; Sun, 5 Oct 2014 17:50:45 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 2TjjiEjPR5dy; Sun, 5 Oct 2014 17:50:42 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id A82EC1472001; Sun, 5 Oct 2014 17:50:42 +0200 (CEST) Message-ID: <543168D0.2000705@internetx.com> Date: Sun, 05 Oct 2014 17:50:40 +0200 From: InterNetX - Juergen Gotteswinter Reply-To: jg@internetx.com User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: Dmitry Morozovsky , Mikolaj Golub Subject: Re: HAST with broken HDD References: <542BC135.1070906@Skynet.be> <542BDDB3.8080805@internetx.com> <542BF853.3040604@internetx.com> <542C019E.2080702@internetx.com> <542C0710.3020402@internetx.com> <97aab72e19d640ebb65c754c858043cc@SERVER.ad.usd-group.com> <20141003175439.GA7664@gmail.com> In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: "freebsd-fs@freebsd.org" , Matt Churchyard X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 15:50:55 -0000 Am 05.10.2014 um 16:50 schrieb Dmitry Morozovsky: > On Fri, 3 Oct 2014, Mikolaj Golub wrote: > >> Disk errors are recorded to syslog. Also error counters are displayed >> in `hastctl list' output. There is snmp_hast(3) in base -- a module >> for bsnmp to retrieve this statistics via snmp protocol (traps are not >> supported though). >> >> For notifications, the hastd can be configured to execute an arbitrary >> command on various HAST events (see description for `exec' in >> hast.conf(5)). Unfortunately, it does not have hooks for I/O error >> events currently. It might be worth adding though. The problem with >> this that it may generate to many events, so some throttling is >> needed. > > And, I it, this should be noted, some kind of error-coalescing or similar > before going from "warning" shate (there are some read error, but otherwise the > disk is useable, and it would be overly hassle to switch to remote component > completely) to "error" state (component is unuseable and needs to be replaced > ASAP; drop it from HAST pair, and switchover if needed). > > Error such as "device lost" is, of course, fatal from the very beginning; but > -- how should we interpret, well, sporadic controller resets with the disk > coming back and catching syncing again? > > Hi Dmitry, since HAST is somehow not so different from DRBD, why dont take their way of Error Handling as "Template". DRBD works pretty well and rock solid since years, a well established Solution. HAST got the potencial to become this also, with some improvements. Just my 2 Cents :) From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 15:54:26 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F1374290 for ; Sun, 5 Oct 2014 15:54:26 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id B672E835 for ; Sun, 5 Oct 2014 15:54:26 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 2423020E70898; Sun, 5 Oct 2014 15:54:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id A09E820E70896; Sun, 5 Oct 2014 15:54:17 +0000 (UTC) Message-ID: <8186C4F93E964D41891B3D4B7BA8A260@multiplay.co.uk> From: "Steven Hartland" To: "Nikolay Denev" , "freebsd-fs" References: Subject: Re: Unable to create ZVOL (error=6) Date: Sun, 5 Oct 2014 16:54:13 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 15:54:27 -0000 Just tried to reproduce this on 10.1-BETA2 FreeBSD 10.1-BETA2 #68 r272005M without luck I'm afraid. Steps taken to try to reproduce: mdconfig mdconfig -a -t malloc -s 1G zpool create zfs md0 zfs set volmode=dev zfs zfs create zfs/bhyve_volumes zfs create zfs/bhyve_volumes/freebsd_amd64_10_stable zfs snapshot -r zfs/bhyve_volumes@`/bin/date +\%H:\%M:\%S_\%d-\%m-\%Y` No errors where reported but also no zvols where created with no /dev/zvol existing, which in itself seems broken. I also tried exporting and re-imported the pool still no zvol devs. Are their some other steps your taking? For reference this functionality was added by: https://svnweb.freebsd.org/base?view=revision&revision=264145 Regards Steve ----- Original Message ----- From: "Nikolay Denev" To: "freebsd-fs" Sent: Sunday, October 05, 2014 11:24 AM Subject: Unable to create ZVOL (error=6) > Hi, > > I have a cron to generate daily snapshots like this: > > 0 0 * * * root /sbin/zfs snapshot -r zfs/bhyve_volumes@`/bin/date > +\%H:\%M:\%S_\%d-\%m-\%Y` > > And I've noticed that I have the following errors in dmesg: > > ZFS WARNING: Unable to create ZVOL > zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_05-10-2014 > (error=6). > > Snapshot looks ok, it's just that the ZVOL cdev is not created : > > [12:11][ndenev@nas:~]%zfs get creation > zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_05-10-2014 > > NAME > PROPERTY VALUE SOURCE > > zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_05-10-2014 > creation Sun Oct 5 0:00 2014 - > > Error 6 would be ENXIO. > I think this might be due to "volmode=dev" setting, as I've ruled out > name lenght, or invalid characters. > > > > --Nikolay > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 15:59:05 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BF3BC4FC for ; Sun, 5 Oct 2014 15:59:05 +0000 (UTC) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id 726B4884 for ; Sun, 5 Oct 2014 15:59:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id 5C0081472004; Sun, 5 Oct 2014 17:59:03 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8dVDTF2psi7s; Sun, 5 Oct 2014 17:59:00 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 901C64C4CA14; Sun, 5 Oct 2014 17:59:00 +0200 (CEST) Message-ID: <54316AC2.5040909@internetx.com> Date: Sun, 05 Oct 2014 17:58:58 +0200 From: InterNetX - Juergen Gotteswinter Reply-To: jg@internetx.com Organization: InterNetX GmbH User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: Dmitry Morozovsky , Mikolaj Golub Subject: Re: HAST with broken HDD References: <542BC135.1070906@Skynet.be> <542BDDB3.8080805@internetx.com> <542BF853.3040604@internetx.com> <542C019E.2080702@internetx.com> <542C0710.3020402@internetx.com> <97aab72e19d640ebb65c754c858043cc@SERVER.ad.usd-group.com> <20141003175439.GA7664@gmail.com> <543168D0.2000705@internetx.com> In-Reply-To: <543168D0.2000705@internetx.com> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: "freebsd-fs@freebsd.org" , Matt Churchyard X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 15:59:05 -0000 Am 05.10.2014 um 17:50 schrieb InterNetX - Juergen Gotteswinter: > > > Am 05.10.2014 um 16:50 schrieb Dmitry Morozovsky: >> On Fri, 3 Oct 2014, Mikolaj Golub wrote: >> >>> Disk errors are recorded to syslog. Also error counters are displayed >>> in `hastctl list' output. There is snmp_hast(3) in base -- a module >>> for bsnmp to retrieve this statistics via snmp protocol (traps are not >>> supported though). >>> >>> For notifications, the hastd can be configured to execute an arbitrary >>> command on various HAST events (see description for `exec' in >>> hast.conf(5)). Unfortunately, it does not have hooks for I/O error >>> events currently. It might be worth adding though. The problem with >>> this that it may generate to many events, so some throttling is >>> needed. >> >> And, I it, this should be noted, some kind of error-coalescing or similar >> before going from "warning" shate (there are some read error, but otherwise the >> disk is useable, and it would be overly hassle to switch to remote component >> completely) to "error" state (component is unuseable and needs to be replaced >> ASAP; drop it from HAST pair, and switchover if needed). >> >> Error such as "device lost" is, of course, fatal from the very beginning; but >> -- how should we interpret, well, sporadic controller resets with the disk >> coming back and catching syncing again? >> >> > > Hi Dmitry, > > since HAST is somehow not so different from DRBD, why dont take their > way of Error Handling as "Template". DRBD works pretty well and rock > solid since years, a well established Solution. HAST got the potencial > to become this also, with some improvements. > > Just my 2 Cents :) > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > forgot this one... http://www.drbd.org/users-guide/s-handling-disk-errors.html From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 18:06:19 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A8F0327F for ; Sun, 5 Oct 2014 18:06:19 +0000 (UTC) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.redhat.com", Issuer "DigiCert SHA2 Extended Validation Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7959E776 for ; Sun, 5 Oct 2014 18:06:19 +0000 (UTC) Received: from int-mx13.intmail.prod.int.phx2.redhat.com (int-mx13.intmail.prod.int.phx2.redhat.com [10.5.11.26]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s95I6CFS021972 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Sun, 5 Oct 2014 14:06:12 -0400 Received: from [10.36.6.182] (vpn1-6-182.ams2.redhat.com [10.36.6.182]) by int-mx13.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s95I68Vn004219 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Sun, 5 Oct 2014 14:06:12 -0400 Subject: Re: FreeBSD support being added to GlusterFS Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=windows-1252 From: Justin Clift In-Reply-To: Date: Sun, 5 Oct 2014 19:06:09 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <30FBFCBC-76B6-4504-A8D6-089542A7C198@gluster.org> References: <0F20AEEC-6244-42BC-815C-1440BBBDE664@mail.turbofuzz.com> <20140629203746.GI34108@ivaldir.etoilebsd.net> <1A58F492-946F-46D4-A19E-2734F368CDAC@mail.turbofuzz.com> <0ABAE2AC-BF1B-4125-ACA9-C6177D013E25@mail.turbofuzz.com> <20140706230910.GA8523@ivaldir.etoilebsd.net> <2F416D06-0A98-4E66-902C-ED0690A4B1C0@ixsystems.com> <20140825211459.GB65120@ivaldir.etoilebsd.net> <73B63510-384B-43F3-AC10-78F429F4A5FF@gluster.org> To: Jordan Hubbard X-Scanned-By: MIMEDefang 2.68 on 10.5.11.26 Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 18:06:19 -0000 On 05/10/2014, at 3:11 AM, Jordan Hubbard wrote: >> On Oct 1, 2014, at 3:33 AM, Justin Clift wrote: >>=20 >> As a data point, GlusterFS 3.6.0 beta3 has been released now too. :) >>=20 >> = http://bits.gluster.org/pub/gluster/glusterfs/src/glusterfs-3.6.0beta3.tar= .gz >>=20 >> Everyone that has time, please test/try it out and report back any = bugs. :D >=20 > Hi Justin, >=20 > Given that FreeBSD users interact with 3rd party software through the = ports collection rather than random tarballs, would it be possible to = get the port updated and/or find a long-term maintainer for it so that = successive updates to glusterfs can simply be tracked (and communicated) = through the ports collection? Yep, that sounds like a much better idea. Do you guys have anyone in mind as a potential ports maintainer? :) Regards and best wishes, Justin Clift -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 18:14:39 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C95F73B4 for ; Sun, 5 Oct 2014 18:14:39 +0000 (UTC) Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 646C3853 for ; Sun, 5 Oct 2014 18:14:39 +0000 (UTC) Received: by mail-wg0-f45.google.com with SMTP id m15so4908534wgh.4 for ; Sun, 05 Oct 2014 11:14:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=6MkK1KNDhJBMqRFqciNXXmPDlSv8RlrMtT5sbUyhaKM=; b=W2ar8Gm9SsYXK6hMbD33FZK2WDtwjb0s4HxZBK9oHVMv52k6zF8pMKBTcaaX5uzarB Hcjw1BHHu7KEFX9Hz0HPlwMx6UyYovFfvUqdkvAQwczHoBxX3jiP3cHJclM+wA+7BtaR T8aB9x2vaVBYPizTfwQ9MRh4XcuxHN8uQpM1q2NKYeIHOdCRccLqGKhmjYviuC1r/kkY sshpk3PY81DlJnrJUWA7+ntvEIYoz6bCPEDdfh2KIX0Qm5qrouJmA7FWtDtnQ4+ud84E /d+V/ivJnHl7cuGe/h5OZedX3LDDTnc+naH7D7zb9i1P6NORVSo7oeDi85FUPwanOJmb YCbA== MIME-Version: 1.0 X-Received: by 10.180.91.164 with SMTP id cf4mr13672382wib.3.1412532876346; Sun, 05 Oct 2014 11:14:36 -0700 (PDT) Received: by 10.27.77.135 with HTTP; Sun, 5 Oct 2014 11:14:36 -0700 (PDT) In-Reply-To: <8186C4F93E964D41891B3D4B7BA8A260@multiplay.co.uk> References: <8186C4F93E964D41891B3D4B7BA8A260@multiplay.co.uk> Date: Sun, 5 Oct 2014 20:14:36 +0200 Message-ID: Subject: Re: Unable to create ZVOL (error=6) From: Nikolay Denev To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 18:14:40 -0000 On Sun, Oct 5, 2014 at 5:54 PM, Steven Hartland wrote: > Just tried to reproduce this on 10.1-BETA2 FreeBSD 10.1-BETA2 #68 > r272005M without luck I'm afraid. > > Steps taken to try to reproduce: > mdconfig > mdconfig -a -t malloc -s 1G > zpool create zfs md0 > zfs set volmode=dev zfs > zfs create zfs/bhyve_volumes > zfs create zfs/bhyve_volumes/freebsd_amd64_10_stable > zfs snapshot -r zfs/bhyve_volumes@`/bin/date +\%H:\%M:\%S_\%d-\%m-\%Y` > > No errors where reported but also no zvols where created with no > /dev/zvol existing, which in itself seems broken. > > I also tried exporting and re-imported the pool still no zvol devs. > > Are their some other steps your taking? > > For reference this functionality was added by: > https://svnweb.freebsd.org/base?view=revision&revision=264145 > > Regards > Steve > > ----- Original Message ----- From: "Nikolay Denev" > To: "freebsd-fs" > Sent: Sunday, October 05, 2014 11:24 AM > Subject: Unable to create ZVOL (error=6) > > >> Hi, >> >> I have a cron to generate daily snapshots like this: >> >> 0 0 * * * root /sbin/zfs snapshot -r zfs/bhyve_volumes@`/bin/date >> +\%H:\%M:\%S_\%d-\%m-\%Y` >> >> And I've noticed that I have the following errors in dmesg: >> >> ZFS WARNING: Unable to create ZVOL >> zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_05-10-2014 >> (error=6). >> >> Snapshot looks ok, it's just that the ZVOL cdev is not created : >> >> [12:11][ndenev@nas:~]%zfs get creation >> zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_05-10-2014 >> >> NAME >> PROPERTY VALUE SOURCE >> >> zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_05-10-2014 >> creation Sun Oct 5 0:00 2014 - >> >> Error 6 would be ENXIO. >> I think this might be due to "volmode=dev" setting, as I've ruled out >> name lenght, or invalid characters. >> >> >> >> --Nikolay >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > Hi Steven, I'm not taking other steps, and I'm running "FreeBSD nas.home.lan 10.0-STABLE FreeBSD 10.0-STABLE #13 r270295M: Thu Aug 21 22:05:37 UTC 2014 root@nas.home.lan:/usr/obj/usr/src/sys/NAS amd64" Can you try to set volmode back to "geom", so that you can create a snapshot that creates a cdev in /dev/zvol, and then set it to "dev" again, and try a new snapshot? --Nikolay From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 18:31:05 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 87223870 for ; Sun, 5 Oct 2014 18:31:05 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 49B6B9EB for ; Sun, 5 Oct 2014 18:31:05 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 1B42320E7089D; Sun, 5 Oct 2014 18:31:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id BF23620E7089A; Sun, 5 Oct 2014 18:31:01 +0000 (UTC) Message-ID: <099874C2FEEC483C9453CEF7F2CF3C4F@multiplay.co.uk> From: "Steven Hartland" To: "Nikolay Denev" References: <8186C4F93E964D41891B3D4B7BA8A260@multiplay.co.uk> Subject: Re: Unable to create ZVOL (error=6) Date: Sun, 5 Oct 2014 19:30:57 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="UTF-8"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 18:31:05 -0000 ----- Original Message ----- From: "Nikolay Denev" > I'm not taking other steps, and I'm running "FreeBSD nas.home.lan > 10.0-STABLE FreeBSD 10.0-STABLE #13 r270295M: Thu Aug 21 22:05:37 UTC > 2014 root@nas.home.lan:/usr/obj/usr/src/sys/NAS amd64" > > Can you try to set volmode back to "geom", so that you can create a > snapshot that creates a cdev in /dev/zvol, and then set it to "dev" > again, and try a new snapshot? No change, still no /dev/zvol From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 18:39:06 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 359F8CC2 for ; Sun, 5 Oct 2014 18:39:06 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id EA712A38 for ; Sun, 5 Oct 2014 18:39:05 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 5391620E7089D; Sun, 5 Oct 2014 18:39:05 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: * X-Spam-Status: No, score=2.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 15CF120E7089A; Sun, 5 Oct 2014 18:39:04 +0000 (UTC) Message-ID: <482C298BFC554BC09D1991C0BAEB11A6@multiplay.co.uk> From: "Steven Hartland" To: "Steven Hartland" , "Nikolay Denev" References: <8186C4F93E964D41891B3D4B7BA8A260@multiplay.co.uk> <099874C2FEEC483C9453CEF7F2CF3C4F@multiplay.co.uk> Subject: Re: Unable to create ZVOL (error=6) Date: Sun, 5 Oct 2014 19:38:59 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 18:39:06 -0000 ----- Original Message ----- From: "Steven Hartland" > ----- Original Message ----- > From: "Nikolay Denev" > >> I'm not taking other steps, and I'm running "FreeBSD nas.home.lan >> 10.0-STABLE FreeBSD 10.0-STABLE #13 r270295M: Thu Aug 21 22:05:37 UTC >> 2014 root@nas.home.lan:/usr/obj/usr/src/sys/NAS amd64" >> >> Can you try to set volmode back to "geom", so that you can create a >> snapshot that creates a cdev in /dev/zvol, and then set it to "dev" >> again, and try a new snapshot? > > No change, still no /dev/zvol Are you sure you didn't create the volumes with a -V? As thats what should triggers zvol creation. Can you try updating to the latest stable/10 to see if its still present? Regards Steve From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 18:43:04 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1A9D5D85 for ; Sun, 5 Oct 2014 18:43:04 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id CF682AE4 for ; Sun, 5 Oct 2014 18:43:03 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 4CD4720E7089D; Sun, 5 Oct 2014 18:43:03 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: * X-Spam-Status: No, score=2.0 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id 9575C20E7089A; Sun, 5 Oct 2014 18:43:00 +0000 (UTC) Message-ID: From: "Steven Hartland" To: "Steven Hartland" , "Nikolay Denev" References: <8186C4F93E964D41891B3D4B7BA8A260@multiplay.co.uk> <099874C2FEEC483C9453CEF7F2CF3C4F@multiplay.co.uk> <482C298BFC554BC09D1991C0BAEB11A6@multiplay.co.uk> Subject: Re: Unable to create ZVOL (error=6) Date: Sun, 5 Oct 2014 19:42:56 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 18:43:04 -0000 > ----- Original Message ----- > From: "Steven Hartland" > >> ----- Original Message ----- >> From: "Nikolay Denev" >> >>> I'm not taking other steps, and I'm running "FreeBSD nas.home.lan >>> 10.0-STABLE FreeBSD 10.0-STABLE #13 r270295M: Thu Aug 21 22:05:37 UTC >>> 2014 root@nas.home.lan:/usr/obj/usr/src/sys/NAS amd64" >>> >>> Can you try to set volmode back to "geom", so that you can create a >>> snapshot that creates a cdev in /dev/zvol, and then set it to "dev" >>> again, and try a new snapshot? >> >> No change, still no /dev/zvol > > Are you sure you didn't create the volumes with a -V? As thats what > should triggers zvol creation. > > Can you try updating to the latest stable/10 to see if its still > present? To confirm this, whats the output from: zfs get -r type zfs From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 18:48:04 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F2D9115B for ; Sun, 5 Oct 2014 18:48:03 +0000 (UTC) Received: from mail-wg0-x22d.google.com (mail-wg0-x22d.google.com [IPv6:2a00:1450:400c:c00::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7BC2DB1E for ; Sun, 5 Oct 2014 18:48:03 +0000 (UTC) Received: by mail-wg0-f45.google.com with SMTP id m15so4943301wgh.4 for ; Sun, 05 Oct 2014 11:48:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=u9t6n8boQcEEoUdqNPIgDljQRl+9QWB4ridUXSznHd8=; b=FsVPOkTDFNnKvHZiABr7JgT1zGHCPne5mB7k2ezkmuRtl49o7mgON0j31pAljEYEpl xfI15wJGkPStAZHuzhKlCuKK6J+B2sUxYGi4P4pZhZzPvu0HEsYCshVHXyPAXLmMD+YK WuwHx9J4n18JaPYt0COO4vcmV8MFxp/2lHo0/nARLhXPoplfLfGi0hCPP2lVJDveQK6f abJPwGthTwxAYWc/9qz72+plg6uqhP4sbcI99EwbTB9Lc2H8/6z0dOU/7xTz3nSWK9qI burZAi3V0kd3r7BPEKrwzWmjL6Nz5Wz/q2I6Kn047tE+JGEonE9hR5lv2x9B02HHZrYV qsnQ== MIME-Version: 1.0 X-Received: by 10.181.27.132 with SMTP id jg4mr14532732wid.28.1412534881759; Sun, 05 Oct 2014 11:48:01 -0700 (PDT) Received: by 10.27.77.135 with HTTP; Sun, 5 Oct 2014 11:48:01 -0700 (PDT) In-Reply-To: <482C298BFC554BC09D1991C0BAEB11A6@multiplay.co.uk> References: <8186C4F93E964D41891B3D4B7BA8A260@multiplay.co.uk> <099874C2FEEC483C9453CEF7F2CF3C4F@multiplay.co.uk> <482C298BFC554BC09D1991C0BAEB11A6@multiplay.co.uk> Date: Sun, 5 Oct 2014 20:48:01 +0200 Message-ID: Subject: Re: Unable to create ZVOL (error=6) From: Nikolay Denev To: Steven Hartland Content-Type: text/plain; charset=UTF-8 Cc: freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 18:48:04 -0000 On Sun, Oct 5, 2014 at 8:38 PM, Steven Hartland wrote: > ----- Original Message ----- From: "Steven Hartland" > > >> ----- Original Message ----- From: "Nikolay Denev" >> >>> I'm not taking other steps, and I'm running "FreeBSD nas.home.lan >>> 10.0-STABLE FreeBSD 10.0-STABLE #13 r270295M: Thu Aug 21 22:05:37 UTC >>> 2014 root@nas.home.lan:/usr/obj/usr/src/sys/NAS amd64" >>> >>> Can you try to set volmode back to "geom", so that you can create a >>> snapshot that creates a cdev in /dev/zvol, and then set it to "dev" >>> again, and try a new snapshot? >> >> >> No change, still no /dev/zvol > > > Are you sure you didn't create the volumes with a -V? As thats what > should triggers zvol creation. > > Can you try updating to the latest stable/10 to see if its still > present? > > Regards > Steve Yes, they were created with -V. I seem to have missed that from your first email that you didn't use -V, sorry about that. So I have a zfs/bhyve_volumes dataset, which is with "legacy" mountpoint so it's not mounted anywhere. and below it I'm creating the zvols with -V. The purpose of this dataset was to be used as a parent, so I can do recursive snapshots, etc. It's interesting that I do have some zvols that are snapshots in /dev/zvol, however I'm trying to remember if they were created with older revision, or before I've started fiddling with "volmode" property. I will try to update and see is something changes. --Nikolay Here is the output of the zfs get -r type zfs (but slightly trimmed as it's several screens otherwise): zfs/bhyve_master type volume - zfs/bhyve_master@snap-vm0 type snapshot - zfs/bhyve_volumes type filesystem - zfs/bhyve_volumes@19:46:05_31-08-2014 type snapshot - zfs/bhyve_volumes@01:15:06_01-09-2014 type snapshot - zfs/bhyve_volumes@11:36:54_03-09-2014 type snapshot - zfs/bhyve_volumes@11:04:30_07-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_08-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_09-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_10-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_11-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_12-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_13-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_14-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_15-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_16-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_17-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_18-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_19-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_20-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_21-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_22-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_23-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_24-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_25-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_26-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_27-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_28-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_29-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_30-09-2014 type snapshot - zfs/bhyve_volumes@00:00:00_01-10-2014 type snapshot - zfs/bhyve_volumes@00:00:00_02-10-2014 type snapshot - zfs/bhyve_volumes@00:00:00_03-10-2014 type snapshot - zfs/bhyve_volumes@00:00:00_04-10-2014 type snapshot - zfs/bhyve_volumes@00:00:00_05-10-2014 type snapshot - zfs/bhyve_volumes@12-13-04_05-10-2014 type snapshot - zfs/bhyve_volumes@12-13-59-05-10-2014 type snapshot - zfs/bhyve_volumes@test-snap type snapshot - zfs/bhyve_volumes/benchmark type volume - zfs/bhyve_volumes/benchmark@11:04:30_07-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_08-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_09-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_10-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_11-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_12-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_13-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_14-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_15-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_16-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_17-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_18-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_19-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_20-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_21-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_22-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_23-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_24-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_25-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_26-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_27-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_28-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_29-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_30-09-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_01-10-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_02-10-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_03-10-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_04-10-2014 type snapshot - zfs/bhyve_volumes/benchmark@00:00:00_05-10-2014 type snapshot - zfs/bhyve_volumes/benchmark@12-13-04_05-10-2014 type snapshot - zfs/bhyve_volumes/benchmark@12-13-59-05-10-2014 type snapshot - zfs/bhyve_volumes/benchmark@test-snap type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable type volume - zfs/bhyve_volumes/freebsd_amd64_10_stable@FreeBSD-10.1-PRERELEASE-r270843-30-08-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@19:46:05_31-08-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@01:15:06_01-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@11:36:54_03-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@11:04:30_07-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_08-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_09-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_10-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_11-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_12-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_13-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_14-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_15-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_16-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_17-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_18-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_19-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_20-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_21-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_22-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_23-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_24-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_25-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_26-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_27-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_28-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_29-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_30-09-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_01-10-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_02-10-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_03-10-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_04-10-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@00:00:00_05-10-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@12-13-04_05-10-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@12-13-59-05-10-2014 type snapshot - zfs/bhyve_volumes/freebsd_amd64_10_stable@test-snap type snapshot - From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 20:15:41 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 39111AA6 for ; Sun, 5 Oct 2014 20:15:41 +0000 (UTC) Received: from msa04a.plala.or.jp (msa04.plala.or.jp [IPv6:2400:7800:0:5010::4]) by mx1.freebsd.org (Postfix) with ESMTP id AE005392 for ; Sun, 5 Oct 2014 20:15:40 +0000 (UTC) Received: from i58-95-58-27.s02.a026.ap.plala.or.jp ([58.95.58.27]) by msa04b.plala.or.jp with ESMTP id <20141005200935.HIIA12371.msa04b.plala.or.jp@i58-95-58-27.s02.a026.ap.plala.or.jp>; Mon, 6 Oct 2014 05:09:35 +0900 Date: Mon, 06 Oct 2014 05:09:34 +0900 Message-ID: <864mvigtxd.wl%poyopoyo@puripuri.plala.or.jp> From: poyopoyo@puripuri.plala.or.jp To: Warren Block Subject: Re: Mirrored SSDs for ZIL/SLOG - safety, flushing, capacitors In-Reply-To: References: Mail-Followup-To: Warren Block , javocado , FreeBSD Filesystems User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-2022-JP-2?B?R29qGyQoRCtXGyhC?=) APEL/10.8 Emacs/24.3 (amd64-portbld-freebsd10.0) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-VirusScan: Outbound; msa04b; Mon, 6 Oct 2014 05:09:35 +0900 Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 20:15:41 -0000 At Fri, 3 Oct 2014 05:59:48 -0600 (MDT), Warren Block wrote: > I think the Crucial M550 SSDs have additional backup capacitors also. > However, I'm still skeptical about the value of a small SSD power backup > versus a true UPS for the entire system. I have read an interesting article on this topic: http://www.anandtech.com/show/8528 Micron M600 (128GB, 256GB & 1TB) SSD Review (The Truth About Micron's Power-Loss Protection) The latter part of this article reports Micron's position of this specific function. In my understanding, Micron's consumer-class SSDs such as M550 have very small backup capacitors which would not be worth flushing dirty DRAM cache, but completing ongoing NAND programming. This means while programming at power loss would not corrupt NAND, "successfully written" host write would be lost due to cache loss. It does not provide enough peace of mind required for logging device. It looks like metadata journaling or softupdates for filesystem, not a full journaling implementation. -- kuro From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 21:00:12 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4EBEDF99 for ; Sun, 5 Oct 2014 21:00:12 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3D9A6A60 for ; Sun, 5 Oct 2014 21:00:12 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s95L0Clj005026 for ; Sun, 5 Oct 2014 21:00:12 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <201410052100.s95L0Clj005026@kenobi.freebsd.org> From: bugzilla-noreply@FreeBSD.org To: freebsd-fs@FreeBSD.org Subject: Problem reports for freebsd-fs@FreeBSD.org that need special attention X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 Date: Sun, 05 Oct 2014 21:00:12 +0000 Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 21:00:12 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ----------------+-----------+------------------------------------------------- Needs MFC | 133174 | [msdosfs] [patch] msdosfs must support multibyt Needs MFC | 136470 | [nfs] Cannot mount / in read-only, over NFS Needs MFC | 139651 | [nfs] mount(8): read-only remount of NFS volume Needs MFC | 144447 | [zfs] sharenfs fsunshare() & fsshare_main() non 4 problems total for which you should take action. From owner-freebsd-fs@FreeBSD.ORG Sun Oct 5 21:37:29 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 68687261 for ; Sun, 5 Oct 2014 21:37:29 +0000 (UTC) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E3F8AE2E for ; Sun, 5 Oct 2014 21:37:28 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id C1D864C4CA8D for ; Sun, 5 Oct 2014 23:37:24 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NCyNBKTWtpbL for ; Sun, 5 Oct 2014 23:37:22 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 247274C4CA8A for ; Sun, 5 Oct 2014 23:37:22 +0200 (CEST) Message-ID: <5431BA0E.7090707@internetx.com> Date: Sun, 05 Oct 2014 23:37:18 +0200 From: InterNetX - Juergen Gotteswinter Reply-To: jg@internetx.com, FreeBSD Filesystems User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: FreeBSD Filesystems Subject: Re: Mirrored SSDs for ZIL/SLOG - safety, flushing, capacitors References: <864mvigtxd.wl%poyopoyo@puripuri.plala.or.jp> In-Reply-To: <864mvigtxd.wl%poyopoyo@puripuri.plala.or.jp> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Oct 2014 21:37:29 -0000 Am 05.10.2014 um 22:09 schrieb poyopoyo@puripuri.plala.or.jp: > At Fri, 3 Oct 2014 05:59:48 -0600 (MDT), > Warren Block wrote: >> I think the Crucial M550 SSDs have additional backup capacitors also. >> However, I'm still skeptical about the value of a small SSD power backup >> versus a true UPS for the entire system. > > I have read an interesting article on this topic: > > http://www.anandtech.com/show/8528 > Micron M600 (128GB, 256GB & 1TB) SSD Review > (The Truth About Micron's Power-Loss Protection) > > The latter part of this article reports Micron's position of this > specific function. In my understanding, Micron's consumer-class SSDs > such as M550 have very small backup capacitors which would not be > worth flushing dirty DRAM cache, but completing ongoing NAND > programming. This means while programming at power loss would not > corrupt NAND, "successfully written" host write would be lost due to > cache loss. It does not provide enough peace of mind required for > logging device. > > It looks like metadata journaling or softupdates for filesystem, not a > full journaling implementation. > Nexenta for example recommends Stec SSD, SLC Type. Since the Zil is going to be hammered with writes all the time (mirroring adds another brick to this), its probably a good idea avoid any consumer grade mlc type ssd. look for nexentas / oracles / suns hardware compatibilty lists, to get a clue which brand/type fits good for this job. we are running a nexenta ha cluster since ~3 years, equipped with dell branded pliant lb150s slc ssd (plaint is sandisk nowadays), absolutly flawless. all 4 are still the initially installed flash drives. i dont think that any 150€ ssd will take this for mid/long term, at least performance degradation should start very quickly under zil/zfs conditions. also keep in mind that the zil dont need to be such big, most of the time 16-32gb are more than enaugh. you wont take any profit from huge zil drives. but, like kevin already mentioned before, you really should think about your usecase and if you need dedicated zils. l2arc can be done with every crappy ssd out there, since it wont affect your data consistency if one dies. which protocoll are you going to use to export your shares? ifs its going to be a nfs filer, you should investigate further into this before since you will most likely see pretty bad write performance on the shares and recommendings to disable zil or patch the nfs server to become async. this makes dedicated zils nonsense... cheers, Juergen From owner-freebsd-fs@FreeBSD.ORG Mon Oct 6 00:10:10 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 58091F1 for ; Mon, 6 Oct 2014 00:10:10 +0000 (UTC) Received: from blade.simplesystems.org (blade.simplesystems.org [65.66.246.74]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EF713E80 for ; Mon, 6 Oct 2014 00:10:09 +0000 (UTC) Received: from freddy.simplesystems.org (freddy.simplesystems.org [65.66.246.65]) by blade.simplesystems.org (8.14.4+Sun/8.14.4) with ESMTP id s95Nw5RQ001759; Sun, 5 Oct 2014 18:58:05 -0500 (CDT) Date: Sun, 5 Oct 2014 18:58:05 -0500 (CDT) From: Bob Friesenhahn X-X-Sender: bfriesen@freddy.simplesystems.org To: javocado Subject: Re: Mirrored SSDs for ZIL/SLOG - safety, flushing, capacitors In-Reply-To: Message-ID: References: User-Agent: Alpine 2.01 (GSO 1266 2009-07-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.2 (blade.simplesystems.org [65.66.246.90]); Sun, 05 Oct 2014 18:58:05 -0500 (CDT) Cc: FreeBSD Filesystems X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2014 00:10:10 -0000 On Thu, 2 Oct 2014, javocado wrote: > I'm setting up a SLOG to improve performance on my ZFS-based fileserver. I > plan to use 2 mirrored SSD's to create the log device. Before I continue, > I have some specific questions how to approach which hardware to use. > > Primarily, I'm concerned with what damage/corruption may occur when there's > a power loss. The damage/corruption which might occur due to power loss is primarily dependent on the drive firmware. If the firmware does persist the write then it says it does, then it is not necessary to have a supercap and there is no loss of data. However, peristing the write can be rather slow (and tends to reduce drive lifetime) so SSD firmware often claims to have written the data and defers the write for a while in the hope of having more data to write at once. If the write is deferred, then data will be lost without a supercap. To make matters worse, the drive might not write the data to FLASH in the order requested and so the data on the drive might not be usefully coherent when power returns. A really poor drive will corrupt the data, and might even corrupt data not directly involved in the write requests because the erasure-block size is much larger than the data block size, and because the firmware might decide to move some data around in the background and presumed written data may be left in an inconsistent state. > 2. should I only use SSDs with capacitors? does anyone recommend any? I'm > seeing good things about the Intel s3500 The Intel S3700 would be a better choice for this application since it is optimized for writes. > 4. flushing: does anyone know if (and which) SSDs are respecting and > flushing data to media when told? Likely few do. Unless the vendor specification claims to do so, it is best to assume that they do not. Bob -- Bob Friesenhahn bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/ From owner-freebsd-fs@FreeBSD.ORG Mon Oct 6 08:00:10 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F918349 for ; Mon, 6 Oct 2014 08:00:10 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E6FB2F74 for ; Mon, 6 Oct 2014 08:00:09 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s96809oq053082 for ; Mon, 6 Oct 2014 08:00:09 GMT (envelope-from bugzilla-noreply@freebsd.org) Message-Id: <201410060800.s96809oq053082@kenobi.freebsd.org> From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [FreeBSD Bugzilla] Commit Needs MFC MIME-Version: 1.0 X-Bugzilla-Type: whine X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated Date: Mon, 06 Oct 2014 08:00:09 +0000 Content-Type: text/plain X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2014 08:00:10 -0000 Hi, You have a bug in the "Needs MFC" state which has not been touched in 7 or more days. This email serves as a reminder that you may want to MFC this bug or marked it as completed. In the event you have a longer MFC timeout you may update this bug with a comment and I won't remind you again for 7 days. This reminder is only sent on Mondays. Please file a bug about concerns you may have. This search was scheduled by eadler@FreeBSD.org. (4 bugs) Bug 133174: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=133174 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [msdosfs] [patch] msdosfs must support multibyte international characters in file names Bug 136470: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=136470 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [nfs] Cannot mount / in read-only, over NFS Bug 139651: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=139651 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [nfs] mount(8): read-only remount of NFS volume does not work Bug 144447: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=144447 Severity: Affects Only Me Priority: Normal Hardware: Any Assignee: freebsd-fs@FreeBSD.org Status: Needs MFC Resolution: Summary: [zfs] sharenfs fsunshare() & fsshare_main() non functional From owner-freebsd-fs@FreeBSD.ORG Mon Oct 6 09:59:21 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BDC081BF for ; Mon, 6 Oct 2014 09:59:21 +0000 (UTC) Received: from mail.postbank.bg (mx.postbank.bg [195.242.126.253]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.postbank.bg", Issuer "GeoTrust DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 39A78F26 for ; Mon, 6 Oct 2014 09:59:20 +0000 (UTC) X-AuditID: ac100166-f798a6d000000c98-3d-543264651a39 Received: from sofdc01excv15.postbank.bg ( [10.1.129.38]) (using TLS with cipher AES128-SHA (128/128 bits)) (Client did not present a certificate) by mail.postbank.bg (Eurobank AD BG Outbound mail system) with SMTP id AC.6A.03224.56462345; Mon, 6 Oct 2014 12:44:05 +0300 (EEST) From: "Ivailo A. Tanusheff" To: "freebsd-fs@FreeBSD.org" Subject: ZFS API Thread-Topic: ZFS API Thread-Index: Ac/hSbKM4XfrgsjoRyuCQiNPaof68w== Date: Mon, 6 Oct 2014 09:44:04 +0000 Message-ID: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.1.2.26] Content-Type: text/plain; charset="us-ascii" content-transfer-encoding: quoted-printable MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprEKsWRmVeSWpSXmKPExsXCxdioppuaYhRiMH+JhsWxxz/ZHBg9Znya zxLAGNXAaJOYl5dfkliSqpCSWpxsqxQdkF9ckpSYlx2r4JJZnJyTmJmbWqSkkJliq2SspFCQ k5icmpuaV2KrlFhQkJqXomTHpYABbIDKMvMUUvOS81My89JtlTyD/XUtLEwtdQ2V7BCmWqkp GxonfGXPeLevhbHgK3NFw9u3jA2MU5m7GDk5JARMJP4sec4GYYtJXLi3Hsjm4hASmMMksa7x LhNIgg2oaNvcPWC2iICpxO7PL8CahQUEJJb9nggVF5WYd/ILG4StJ7H99iFWEJtFQEXi/JqV YHFeAV+J50s3MYLYjEDLvp9aA9bLLCAucevJfCaIIwQkluw5D3WcqMTLx/9YIWxZiUffHkPV 60gs2P2JDcLWlli28DUzxHxBiZMzn7BMYBSahWTsLCQts5C0zELSsoCRZRWjZHF+WkqygWGw r3uZgZFeATSK9JLSNzECY3iNAGPaDsY3V5wOMQpwMCrx8EbsMAwRYk0sK67MPcQowcGsJMLr lWgUIsSbklhZlVqUH19UmpNafIgxGRgKE5mlRJPzgeklryTe0MTA0NTEyNDCwMzClDRhJXHe 3zv1QoQE0oFpKTs1tSC1CGYLEwenVAOjgvDvPLUNgav+H9WpY933reXIjv+vfBZPlF3dvG5b sHpq3Lm1kfw+P/+xn9hYM/VCTMRmkZ6Oqo972j7mKrft/+jP0nEwsyPg53uL2RsfPnkgkJEo NpU5ssnhQIyVZkuI2vuvtVK3bz1lc/1fN+3q+z6+J6u0PVfI1luvyw29J7TPZcvu0AnaSizF GYmGWsxFxYkAp/QjmyUDAAA= X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2014 09:59:21 -0000 Dear all, I am looking for an API and documentation about programing some actions on C= /C++ for ZFS management. I have found an library - /usr/src/cddl/contrib/opensolaris/lib/libzfs/commo= n/libzfs.h but this lacks any documentation. I do not want to make a huge development and my skills are not great, so I n= eed some useful API and documentation. Is there such thing available around? What I need is to include some snapshot management techniques :) Regards, Ivailo Tanusheff Disclaimer: This communication is confidential. If you are not the intended recipient, y= ou are hereby notified that any disclosure, copying, distribution or taking= any action in reliance on the contents of this information is strictly proh= ibited and may be unlawful. If you have received this communication by mista= ke, please notify us immediately by responding to this email and then delete= it from your system. Eurobank Bulgaria AD is not responsible for, nor endorses, any opinion, reco= mmendation, conclusion, solicitation, offer or agreement or any information= contained in this communication. Eurobank Bulgaria AD cannot accept any responsibility for the accuracy or co= mpleteness of this message as it has been transmitted over a public network.= If you suspect that the message may have been intercepted or amended, pleas= e call the sender. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 6 13:10:37 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D7D94917 for ; Mon, 6 Oct 2014 13:10:37 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BDA0A919 for ; Mon, 6 Oct 2014 13:10:37 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s96DAbPl035943 for ; Mon, 6 Oct 2014 13:10:37 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 133174] [msdosfs] [patch] msdosfs must support multibyte international characters in file names Date: Mon, 06 Oct 2014 13:10:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: unspecified X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: emaste@freebsd.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2014 13:10:37 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=133174 Ed Maste changed: What |Removed |Added ---------------------------------------------------------------------------- Status|Needs MFC |Issue Resolved CC| |emaste@freebsd.org Resolution|--- |FIXED --- Comment #9 from Ed Maste --- Merged to stable/9 in r230196 -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Mon Oct 6 17:02:26 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CA69FF08 for ; Mon, 6 Oct 2014 17:02:26 +0000 (UTC) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 857E39FB for ; Mon, 6 Oct 2014 17:02:26 +0000 (UTC) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s96H2OFW094803 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Mon, 6 Oct 2014 13:02:24 -0400 (EDT) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.7/8.14.7/Submit) id s96H2ORA094800; Mon, 6 Oct 2014 13:02:24 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21554.52000.530951.15561@khavrinen.csail.mit.edu> Date: Mon, 6 Oct 2014 13:02:24 -0400 From: Garrett Wollman To: Rick Macklem Subject: Re: 9.3 NFS client bug? In-Reply-To: <1265026957.57440085.1412374216862.JavaMail.root@uoguelph.ca> References: <21551.6675.236161.476053@khavrinen.csail.mit.edu> <1265026957.57440085.1412374216862.JavaMail.root@uoguelph.ca> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Mon, 06 Oct 2014 13:02:24 -0400 (EDT) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2014 17:02:26 -0000 < said: > 2 - Try an "oldnfs" mount and see if the old client exhibits the same behaviour. Same behavior both old and new. I'm doing binary search now to try to minimize the size of the workload that exhibits the issue. -GAWollman From owner-freebsd-fs@FreeBSD.ORG Mon Oct 6 23:34:25 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D2C3C924 for ; Mon, 6 Oct 2014 23:34:25 +0000 (UTC) Received: from mail-la0-x236.google.com (mail-la0-x236.google.com [IPv6:2a00:1450:4010:c03::236]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 19C93C53 for ; Mon, 6 Oct 2014 23:34:24 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id gm9so5212394lab.41 for ; Mon, 06 Oct 2014 16:34:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=0uLJjzG/PpQLE8VTHzt49nDIsbK1DKP1UC0kHv4Bl/E=; b=kyVQ5fh7pWGNM5ruLKczZJbBpy6tSjKCJrHyU8NHmzybfvvNAIzVPM4UnuJrBlWt92 e1I6RQ0u7W/rigx8Pvc7zoHQe3vBJ6yjMF7tOe42LafgcPDFBbDq98NMjIcWrG2qW/ko /IzLHEd6o1yRpP7vRXqcpfHKz4rGzCFX19pqTkFWNNeekl+XE2LxktOOxJPEi81WwddQ 7Syb5Tc9nc7eCaA3FsEh+qLEqXaMk5RGTBYc0S5q7NLBdPIvpOpU/AvHCzhVm977H5lP gapq0q1SaHEboXSk7pLx+gkR0ZBgXtmgzZ58zFPRfyXxeDnKN6Y8vQG0FCe3a5hBJufg 6tYw== MIME-Version: 1.0 X-Received: by 10.112.158.170 with SMTP id wv10mr38885lbb.66.1412638462767; Mon, 06 Oct 2014 16:34:22 -0700 (PDT) Received: by 10.152.22.195 with HTTP; Mon, 6 Oct 2014 16:34:22 -0700 (PDT) In-Reply-To: <30FBFCBC-76B6-4504-A8D6-089542A7C198@gluster.org> References: <0F20AEEC-6244-42BC-815C-1440BBBDE664@mail.turbofuzz.com> <20140629203746.GI34108@ivaldir.etoilebsd.net> <1A58F492-946F-46D4-A19E-2734F368CDAC@mail.turbofuzz.com> <0ABAE2AC-BF1B-4125-ACA9-C6177D013E25@mail.turbofuzz.com> <20140706230910.GA8523@ivaldir.etoilebsd.net> <2F416D06-0A98-4E66-902C-ED0690A4B1C0@ixsystems.com> <20140825211459.GB65120@ivaldir.etoilebsd.net> <73B63510-384B-43F3-AC10-78F429F4A5FF@gluster.org> <30FBFCBC-76B6-4504-A8D6-089542A7C198@gluster.org> Date: Tue, 7 Oct 2014 10:34:22 +1100 Message-ID: Subject: Re: FreeBSD support being added to GlusterFS From: Outback Dingo To: Justin Clift Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: freebsd-fs@freebsd.org, Jordan Hubbard X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Oct 2014 23:34:26 -0000 On Mon, Oct 6, 2014 at 5:06 AM, Justin Clift wrote: > On 05/10/2014, at 3:11 AM, Jordan Hubbard wrote: > >> On Oct 1, 2014, at 3:33 AM, Justin Clift wrote: > >> > >> As a data point, GlusterFS 3.6.0 beta3 has been released now too. :) > >> > >> > http://bits.gluster.org/pub/gluster/glusterfs/src/glusterfs-3.6.0beta3.tar.gz > >> > >> Everyone that has time, please test/try it out and report back any > bugs. :D > > > > Hi Justin, > > > > Given that FreeBSD users interact with 3rd party software through the > ports collection rather than random tarballs, would it be possible to get > the port updated and/or find a long-term maintainer for it so that > successive updates to glusterfs can simply be tracked (and communicated) > through the ports collection? > > Yep, that sounds like a much better idea. Do you guys have anyone in > mind as a potential ports maintainer? :) > Okay taking a look at this, Ive updated the port Makefile diff, found on this three to the later glusterfs-freebsd_20140811.tar.bz2 installed cmockery2, from ports as it wasnt a "dependency" in the port Makefile yet. and got this when configure was run on the gluster port. Im building on FreeBSD 10.1-RC in a XenServer VM. === configuring in contrib/argp-standalone (/usr/ports/sysutils/glusterfs/work/glusterfs/contrib/argp-standalone) configure: WARNING: no configuration information is in contrib/argp-standalone GlusterFS configure summary =========================== FUSE client : yes Infiniband verbs : no epoll IO multiplex : no argp-standalone : no fusermount : no readline : yes georeplication : no Linux-AIO : no Enable Debug : no systemtap : yes Block Device xlator : no glupy : yes Use syslog : yes XML output : yes QEMU Block formats : yes Encryption xlator : yes So how do we look so far ? make runs fine, or seems to. However [root@zfsguru /usr/ports/sysutils/glusterfs]# make install ===> Installing for glusterfs-20140811 ===> glusterfs-20140811 depends on file: /usr/local/bin/python2.7 - found ===> glusterfs-20140811 depends on shared library: libargp.so - found (/usr/local/lib/libargp.so.0.0.0) ===> glusterfs-20140811 depends on shared library: libintl.so - found (/usr/local/lib/libintl.so.9) ===> glusterfs-20140811 depends on shared library: libfuse.so - found (/usr/local/lib/libfuse.so.2.9.3) ===> glusterfs-20140811 depends on shared library: libxml2.so - found (/usr/local/lib/libxml2.so.2.9.1) ===> Checking if glusterfs already installed ===> Registering installation for glusterfs-20140811 pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/auth/addr.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/auth/login.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/rpc-transport/socket.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/cluster/afr.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/cluster/dht.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/cluster/distribute.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/cluster/nufa.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/cluster/pump.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/cluster/replicate.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/cluster/stripe.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/cluster/switch.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/debug/error-gen.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/debug/io-stats.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/debug/trace.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/encryption/crypt.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/encryption/rot-13.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/access-control.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/barrier.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/cdc.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/changelog.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/gfid-access.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy/debug-trace.py): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy/debug-trace.pyc): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy/debug-trace.pyo): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy/helloworld.py): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy/helloworld.pyc): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy/helloworld.pyo): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy/negative.py): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy/negative.pyc): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy/negative.pyo): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/index.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/locks.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/mac-compat.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/marker.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/posix-locks.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/prot_client.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/prot_dht.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/prot_server.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/quiesce.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/quota.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/quotad.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/read-only.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/snapview-client.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/snapview-server.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/worm.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/meta.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/mgmt/glusterd.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/mount/api.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/mount/fuse.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/nfs/server.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/performance/io-cache.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/performance/io-threads.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/performance/md-cache.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/performance/open-behind.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/performance/quick-read.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/performance/read-ahead.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/performance/readdir-ahead.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/performance/stat-prefetch.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/performance/write-behind.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/protocol/client.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/protocol/server.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/storage/posix.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/system/posix-acl.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/testing/features/template.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/testing/performance/symlink-cache.so): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/python2.7/site-packages/gluster/gfapi.py): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/python2.7/site-packages/gluster/gfapi.pyc): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/python2.7/site-packages/gluster/gfapi.pyo): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/auth/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/rpc-transport/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/cluster/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/debug/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/encryption/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/glupy/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/features/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/mgmt/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/mount/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/nfs/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/performance/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/protocol/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/storage/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/system/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/testing/features/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/testing/performance/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/testing/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/xlator/): No such file or directory pkg-static: lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5qa2/): No such file or directory > > Regards and best wishes, > > Justin Clift > > -- > GlusterFS - http://www.gluster.org > > An open source, distributed file system scaling to several > petabytes, and handling thousands of clients. > > My personal twitter: twitter.com/realjustinclift > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 09:16:40 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id EC8C9424 for ; Tue, 7 Oct 2014 09:16:40 +0000 (UTC) Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.redhat.com", Issuer "DigiCert SHA2 Extended Validation Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C0534F8E for ; Tue, 7 Oct 2014 09:16:40 +0000 (UTC) Received: from int-mx09.intmail.prod.int.phx2.redhat.com (int-mx09.intmail.prod.int.phx2.redhat.com [10.5.11.22]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s979GX5u021815 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Tue, 7 Oct 2014 05:16:33 -0400 Received: from [10.36.6.183] (vpn1-6-183.ams2.redhat.com [10.36.6.183]) by int-mx09.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s979GT6F023549 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO); Tue, 7 Oct 2014 05:16:30 -0400 Subject: Re: FreeBSD support being added to GlusterFS Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Justin Clift In-Reply-To: Date: Tue, 7 Oct 2014 10:16:30 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: <3FFCED37-C696-4E83-B66C-1B80CD9F7807@gluster.org> References: <0F20AEEC-6244-42BC-815C-1440BBBDE664@mail.turbofuzz.com> <20140629203746.GI34108@ivaldir.etoilebsd.net> <1A58F492-946F-46D4-A19E-2734F368CDAC@mail.turbofuzz.com> <0ABAE2AC-BF1B-4125-ACA9-C6177D013E25@mail.turbofuzz.com> <20140706230910.GA8523@ivaldir.etoilebsd.net> <2F416D06-0A98-4E66-902C-ED0690A4B1C0@ixsystems.com> <20140825211459.GB65120@ivaldir.etoilebsd.net> <73B63510-384B-43F3-AC10-78F429F4A5FF@gluster.org> <30FBFCBC-76B6-4504-A8D6-089542A7C198@gluster.org> To: Outback Dingo X-Scanned-By: MIMEDefang 2.68 on 10.5.11.22 Cc: freebsd-fs@freebsd.org, Jordan Hubbard X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 09:16:41 -0000 On 07/10/2014, at 12:34 AM, Outback Dingo wrote: > Okay taking a look at this, Ive updated the port Makefile diff, found = on this three to the later=20 > glusterfs-freebsd_20140811.tar.bz2 >=20 > installed cmockery2, from ports as it wasnt a "dependency" in the port = Makefile yet. > and got this when configure was run on the gluster port. Im building = on FreeBSD 10.1-RC in a XenServer VM. >=20 > =3D=3D=3D configuring in contrib/argp-standalone = (/usr/ports/sysutils/glusterfs/work/glusterfs/contrib/argp-standalone) > configure: WARNING: no configuration information is in = contrib/argp-standalone >=20 > GlusterFS configure summary > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > FUSE client : yes > Infiniband verbs : no > epoll IO multiplex : no > argp-standalone : no > fusermount : no > readline : yes > georeplication : no > Linux-AIO : no > Enable Debug : no > systemtap : yes > Block Device xlator : no > glupy : yes > Use syslog : yes > XML output : yes > QEMU Block formats : yes > Encryption xlator : yes >=20 > So how do we look so far ?=20 >=20 > make runs fine, or seems to. However >=20 > [root@zfsguru /usr/ports/sysutils/glusterfs]# make install > =3D=3D=3D> Installing for glusterfs-20140811 > =3D=3D=3D> glusterfs-20140811 depends on file: = /usr/local/bin/python2.7 - found > =3D=3D=3D> glusterfs-20140811 depends on shared library: libargp.so = - found (/usr/local/lib/libargp.so.0.0.0) > =3D=3D=3D> glusterfs-20140811 depends on shared library: libintl.so = - found (/usr/local/lib/libintl.so.9) > =3D=3D=3D> glusterfs-20140811 depends on shared library: libfuse.so = - found (/usr/local/lib/libfuse.so.2.9.3) > =3D=3D=3D> glusterfs-20140811 depends on shared library: libxml2.so = - found (/usr/local/lib/libxml2.so.2.9.1) > =3D=3D=3D> Checking if glusterfs already installed > =3D=3D=3D> Registering installation for glusterfs-20140811 > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/auth/addr.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/auth/login.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/rpc-transport/socket.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/cluster/afr.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/cluster/dht.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/cluster/distribute.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/cluster/nufa.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/cluster/pump.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/cluster/replicate.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/cluster/stripe.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/cluster/switch.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/debug/error-gen.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/debug/io-stats.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/debug/trace.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/encryption/crypt.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/encryption/rot-13.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/access-control.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/barrier.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/cdc.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/changelog.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/gfid-access.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy/debug-trace.py): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy/debug-trace.pyc): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy/debug-trace.pyo): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy/helloworld.py): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy/helloworld.pyc): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy/helloworld.pyo): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy/negative.py): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy/negative.pyc): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy/negative.pyo): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/index.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/locks.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/mac-compat.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/marker.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/posix-locks.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/prot_client.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/prot_dht.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/prot_server.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/quiesce.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/quota.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/quotad.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/read-only.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/snapview-client.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/snapview-server.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/worm.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/meta.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/mgmt/glusterd.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/mount/api.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/mount/fuse.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/nfs/server.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/performance/io-cache.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/performance/io-threads.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/performance/md-cache.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/performance/open-behind.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/performance/quick-read.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/performance/read-ahead.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/performance/readdir-ahead.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/performance/stat-prefetch.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/performance/write-behind.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/protocol/client.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/protocol/server.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/storage/posix.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/system/posix-acl.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/testing/features/template.so): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/testing/performance/symlink-cache.so): No such file or = directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/python2.7/sit= e-packages/gluster/gfapi.py): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/python2.7/sit= e-packages/gluster/gfapi.pyc): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/python2.7/sit= e-packages/gluster/gfapi.pyo): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/auth/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/rpc-transport/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/cluster/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/debug/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/encryption/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/glupy/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/features/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/mgmt/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/mount/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/nfs/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/performance/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/protocol/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/storage/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/system/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/testing/features/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/testing/performance/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/testing/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/xlator/): No such file or directory > pkg-static: = lstat(/usr/ports/sysutils/glusterfs/work/stage/usr/local/lib/glusterfs/3.5= qa2/): No such file or directory Heh, that definitely doesn't look right. ;) I think Harsha is still on holiday until later this week. So there = might be a few days delay until he gets back to us on this. Regards and best wishes, Justin Clift -- GlusterFS - http://www.gluster.org An open source, distributed file system scaling to several petabytes, and handling thousands of clients. My personal twitter: twitter.com/realjustinclift From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 12:04:44 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B3C9FE28 for ; Tue, 7 Oct 2014 12:04:44 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id E7D9B5DF for ; Tue, 7 Oct 2014 12:04:43 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArYEAE/VM1SDaFve/2dsb2JhbABfDoQrBIJ/0TcCgSQBe4QEAQEEIwRSGxgCAg0ZAlkGE4g+rGGVCgEXgSyOZDQHgneBVAWYdYV6kFyDf4MjXCEvgUiBAgEBAQ X-IronPort-AV: E=Sophos;i="5.04,670,1406606400"; d="scan'208";a="159895168" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 07 Oct 2014 08:03:33 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id D212CB4031; Tue, 7 Oct 2014 08:03:33 -0400 (EDT) Date: Tue, 7 Oct 2014 08:03:33 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <1109402573.59384867.1412683413852.JavaMail.root@uoguelph.ca> In-Reply-To: <21554.52000.530951.15561@khavrinen.csail.mit.edu> Subject: Re: 9.3 NFS client bug? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 12:04:44 -0000 Garrett Wollman wrote: > < said: > > > 2 - Try an "oldnfs" mount and see if the old client exhibits the > > same behaviour. > > Same behavior both old and new. I'm doing binary search now to try > to > minimize the size of the workload that exhibits the issue. > > -GAWollman > By ay chance is your ZFS server allocating a ZFS inode (whatever they call it) with a fileno/inode# that doesn't fit in 32 bits? (There is a diagnostic printf w.r.t. this for NFSv4, but not NFSv3, so you might try an NFSv4 mount. The printf starts with "NFSv4 fileid > 32..".) Since d_fileno in "struct dirent" is 32bits, NFS just truncates the 64bit fileid to 32bits (or fills the high order 32bits with 0s for the server). ZFS is known to generate fileids with non-zero high order 32bits->busted. (And not at all easy to fix. I actually have a somewhat hackish idea for a way to make 64bit d_fileno values work without busting backwards compatibility. I think I'll post that to freebsd-fs@ and see what others think?) rick From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 13:37:34 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 150D61DB for ; Tue, 7 Oct 2014 13:37:34 +0000 (UTC) Received: from mail1.postbank.bg (mx.postbank.bg [195.242.126.253]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.postbank.bg", Issuer "GeoTrust DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 23AB6FF0 for ; Tue, 7 Oct 2014 13:37:32 +0000 (UTC) X-AuditID: ac100165-f793c6d000000db7-1c-5433ec91e0e6 Received: from sofdc01excv14.postbank.bg ( [10.1.129.37]) (using TLS with cipher AES128-SHA (128/128 bits)) (Client did not present a certificate) by mail1.postbank.bg (Eurobank AD BG Outbound mail system) with SMTP id 99.96.03511.19CE3345; Tue, 7 Oct 2014 16:37:22 +0300 (EEST) From: "Ivailo A. Tanusheff" To: "freebsd-fs@FreeBSD.org" Subject: RE: ZFS API Thread-Topic: ZFS API Thread-Index: Ac/hSbKM4XfrgsjoRyuCQiNPaof68wA6cgCg Date: Tue, 7 Oct 2014 13:37:20 +0000 Message-ID: <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> References: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> In-Reply-To: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.1.2.26] Content-Type: text/plain; charset="us-ascii" content-transfer-encoding: quoted-printable MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrBKsWRmVeSWpSXmKPExsXCxdioqjvpjXGIwelz7BbHHv9kc2D0mPFp PksAY1QDo01iXl5+SWJJqkJKanGyrVJ0QH5xSVJiXnasgktmcXJOYmZuapGSQmaKrZKxkkJB TmJyam5qXomtUmJBQWpeipIdlwIGsAEqy8xTSM1Lzk/JzEu3VfIM9te1sDC11DVUskOYaqWm bGic8JU9Y8KnPWwFE/gr2roOsTUwHufpYuTkkBAwkZh67zo7hC0mceHeejYQW0hgLpPEoz+5 IDYbUM22uXuYQGwRAVOJ3Z9fMIPYwgIiEheen2aBiItKzDv5hQ3CNpL4u3AimM0ioCJxZ3kP YxcjBwevgK/EuV0WEON9Jbbf2ckKYnMK+Em0PupkBLEZgU74fmoN2CpmAXGJW0/mM0GcJiCx ZM95ZghbVOLl43+sELasxKNvj6HqdSQW7P7EBmFrSyxb+BqsnldAUOLkzCcsExhFZiEZOwtJ yywkLbOQtCxgZFnFKFmcn5aSbGAY7OteZmCoVwCNOL2k9E2MwHhfI8CYuoPxxRWnQ4wCHIxK PLwMusYhQqyJZcWVuYcYJTiYlUR4970ACvGmJFZWpRblxxeV5qQWH2JMBgbORGYp0eR8YCrK K4k3NDEwNDUxMrQwMLMwJU1YSZz39069ECGBdGAKy05NLUgtgtnCxMEp1cCYd8KMI88grj1E 58HyaSIinqEfw7Zeun5wd+tJUdaX3/2uPTdRWK25R0qDbVb/X4uPyu3TLt89c/Hb/9vTN/9m OPTH785pazm33w3n/+yRqtBlYLhz8/TD28fj2P22m6v8jb2oVbL12Tk50xlT0/JFmQLY80yv rH567WfJ+sAT/DJt2X5akueSlFiKMxINtZiLihMBGleycjsDAAA= X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 13:37:34 -0000 Dear all, Is there really no one that knows/uses ZFS API? That sounds ... well, scary. Regards, Ivailo -----Original Message----- From: owner-freebsd-fs@freebsd.org [mailto:owner-freebsd-fs@freebsd.org] On= Behalf Of Ivailo A. Tanusheff Sent: Monday, October 06, 2014 12:44 PM To: freebsd-fs@FreeBSD.org Subject: ZFS API Dear all, I am looking for an API and documentation about programing some actions on C= /C++ for ZFS management. I have found an library - /usr/src/cddl/contrib/opensolaris/lib/libzfs/commo= n/libzfs.h but this lacks any documentation. I do not want to make a huge development and my skills are not great, so I n= eed some useful API and documentation. Is there such thing available around? What I need is to include some snapshot management techniques :) Regards, Ivailo Tanusheff Disclaimer: This communication is confidential. If you are not the intended recipient, y= ou are hereby notified that any disclosure, copying, distribution or taking= any action in reliance on the contents of this information is strictly proh= ibited and may be unlawful. If you have received this communication by mista= ke, please notify us immediately by responding to this email and then delete= it from your system. Eurobank Bulgaria AD is not responsible for, nor endorses, any opinion, reco= mmendation, conclusion, solicitation, offer or agreement or any information= contained in this communication. Eurobank Bulgaria AD cannot accept any responsibility for the accuracy or co= mpleteness of this message as it has been transmitted over a public network.= If you suspect that the message may have been intercepted or amended, pleas= e call the sender. _______________________________________________ freebsd-fs@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-fs To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" Disclaimer: This communication is confidential. If you are not the intended recipient, y= ou are hereby notified that any disclosure, copying, distribution or taking= any action in reliance on the contents of this information is strictly proh= ibited and may be unlawful. If you have received this communication by mista= ke, please notify us immediately by responding to this email and then delete= it from your system. Eurobank Bulgaria AD is not responsible for, nor endorses, any opinion, reco= mmendation, conclusion, solicitation, offer or agreement or any information= contained in this communication. Eurobank Bulgaria AD cannot accept any responsibility for the accuracy or co= mpleteness of this message as it has been transmitted over a public network.= If you suspect that the message may have been intercepted or amended, pleas= e call the sender. From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 14:08:52 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 10C5AD8 for ; Tue, 7 Oct 2014 14:08:52 +0000 (UTC) Received: from mail.pingpong.net (mail2.pingpong.net [79.136.116.202]) by mx1.freebsd.org (Postfix) with ESMTP id 93B785FE for ; Tue, 7 Oct 2014 14:08:51 +0000 (UTC) Received: from [10.0.0.191] (citron2.pingpong.net [195.178.173.68]) (using TLSv1 with cipher ECDHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.pingpong.net (Postfix) with ESMTPSA id C123E34B7; Tue, 7 Oct 2014 15:59:54 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_0CEDD9D2-C07C-4EC6-956A-888BC7307CEB"; protocol="application/pgp-signature"; micalg=pgp-sha1 Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.6\)) Subject: Re: ZFS API From: Palle Girgensohn In-Reply-To: <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> Date: Tue, 7 Oct 2014 15:59:53 +0200 Message-Id: <757ED2D6-0585-43C1-B827-FB349045246A@pingpong.net> References: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> To: "Ivailo A. Tanusheff" X-Mailer: Apple Mail (2.1878.6) Cc: "freebsd-fs@FreeBSD.org" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 14:08:52 -0000 --Apple-Mail=_0CEDD9D2-C07C-4EC6-956A-888BC7307CEB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii zfs is all command line, AFAIK. I have never heard of anyone who uses = libs for zfs admin. It will probably work, just never heard anyone who = has had the need for it? What are you trying to achieve. 7 okt 2014 kl. 15:37 skrev Ivailo A. Tanusheff : > Dear all, >=20 > Is there really no one that knows/uses ZFS API? > That sounds ... well, scary. >=20 > Regards, > Ivailo >=20 > -----Original Message----- > From: owner-freebsd-fs@freebsd.org = [mailto:owner-freebsd-fs@freebsd.org] On Behalf Of Ivailo A. Tanusheff > Sent: Monday, October 06, 2014 12:44 PM > To: freebsd-fs@FreeBSD.org > Subject: ZFS API >=20 > Dear all, >=20 > I am looking for an API and documentation about programing some = actions on C/C++ for ZFS management. > I have found an library - = /usr/src/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h but this = lacks any documentation. > I do not want to make a huge development and my skills are not great, = so I need some useful API and documentation. > Is there such thing available around? > What I need is to include some snapshot management techniques :) >=20 > Regards, >=20 > Ivailo Tanusheff >=20 >=20 > Disclaimer: >=20 > This communication is confidential. If you are not the intended = recipient, you are hereby notified that any disclosure, copying, = distribution or taking any action in reliance on the contents of this = information is strictly prohibited and may be unlawful. If you have = received this communication by mistake, please notify us immediately by = responding to this email and then delete it from your system. > Eurobank Bulgaria AD is not responsible for, nor endorses, any = opinion, recommendation, conclusion, solicitation, offer or agreement or = any information contained in this communication. > Eurobank Bulgaria AD cannot accept any responsibility for the accuracy = or completeness of this message as it has been transmitted over a public = network. If you suspect that the message may have been intercepted or = amended, please call the sender. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >=20 > Disclaimer: >=20 > This communication is confidential. If you are not the intended = recipient, you are hereby notified that any disclosure, copying, = distribution or taking any action in reliance on the contents of this = information is strictly prohibited and may be unlawful. If you have = received this communication by mistake, please notify us immediately by = responding to this email and then delete it from your system. > Eurobank Bulgaria AD is not responsible for, nor endorses, any = opinion, recommendation, conclusion, solicitation, offer or agreement or = any information contained in this communication. > Eurobank Bulgaria AD cannot accept any responsibility for the accuracy = or completeness of this message as it has been transmitted over a public = network. If you suspect that the message may have been intercepted or = amended, please call the sender. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" --Apple-Mail=_0CEDD9D2-C07C-4EC6-956A-888BC7307CEB Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iQEcBAEBAgAGBQJUM/HZAAoJEIhV+7FrxBJDwK4H/0UUF+vM3zOjyLeI5SZDrT98 SpqbiBDY0p7keZ+8yEjsxxqSQCOTQlbe7TjTO65zQbuh54E4dpJYVxOWbOQ2AHiN VJXYslBT/3EnmjS9F5t6qYtD+1fxslQ+O7EUa9buXxqui43cM6CBBh4bsOzGR9Gr NRBBCJmZcnsPNCNIwxcaEObPTx9vRsxS3YlAyHuO1RQwU4+AbRsYGxC6bMgAR9iX HNpzx/rlpNoScIUVEcOYDkfkv/tqiG923DFPnakkizTH6IvvJX54lKOxuHHWw218 cXd1NsuOg+t0cyx+MKOvZbXPKs7bVMIeYWkpvBefjIsV6dT2GpQYMvXE6SY7C5g= =xWYn -----END PGP SIGNATURE----- --Apple-Mail=_0CEDD9D2-C07C-4EC6-956A-888BC7307CEB-- From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 14:15:55 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 26BE18E4 for ; Tue, 7 Oct 2014 14:15:55 +0000 (UTC) Received: from mail.postbank.bg (mx.postbank.bg [195.242.126.253]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mail.postbank.bg", Issuer "GeoTrust DV SSL CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5C699785 for ; Tue, 7 Oct 2014 14:15:53 +0000 (UTC) X-AuditID: ac100166-f798a6d000000c98-c1-5433f595f506 Received: from sofdc01excv16.postbank.bg ( [10.1.129.39]) (using TLS with cipher AES128-SHA (128/128 bits)) (Client did not present a certificate) by mail.postbank.bg (Eurobank AD BG Outbound mail system) with SMTP id 1A.78.03224.695F3345; Tue, 7 Oct 2014 17:15:50 +0300 (EEST) From: "Ivailo A. Tanusheff" To: Palle Girgensohn Subject: RE: ZFS API Thread-Topic: ZFS API Thread-Index: Ac/hSbKM4XfrgsjoRyuCQiNPaof68wA6cgCg///Ur4D//8mDsA== Date: Tue, 7 Oct 2014 14:15:49 +0000 Message-ID: <1422065A4E115F409E22C1EC9EDAFBA4604B8D@sofdc01exc02.postbank.bg> References: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> <757ED2D6-0585-43C1-B827-FB349045246A@pingpong.net> In-Reply-To: <757ED2D6-0585-43C1-B827-FB349045246A@pingpong.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.1.2.26] Content-Type: text/plain; charset="us-ascii" content-transfer-encoding: quoted-printable MIME-Version: 1.0 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrPKsWRmVeSWpSXmKPExsXCxdiorjvtq3GIwfMePotjj3+yWdxb+YTN gcljxqf5LB6X1nSxBjBFNTDaJObl5ZcklqQqpKQWJ9sqRQfkF5ckJeZlxyq4ZBYn5yRm5qYW KSlkptgqGSspFOQkJqfmpuaV2ColFhSk5qUo2XEpYAAboLLMPIXUvOT8lMy8dFslz2B/XQsL U0tdQyU7hKlWasqGxglf2TP2bE8oeCJbsfrdVdYGxlMSXYycHBICJhJPt75mhLDFJC7cW8/W xcjFISQwh0ni3oSFrCAJNqCibXP3MIHYIgJaEo8nrGEGsZkFTCVurTsP1iwsICJx4flpFoga UYl5J78ADeIAsp0kZn9QAQmzCKhITHs5FayEV8BXomfPXKhdxxkl7i2dCzafU8BB4vmMX2Dz GYEO+n5qDRPELnGJW0/mM0EcKiCxZM95ZghbVOLl43+sELasxKNvj6HqdSQW7P7EBmFrSyxb +JoZYrGgxMmZT1gmMIrOQjJ2FpKWWUhaZiFpWcDIsopRsjg/LSXZwDDY173MwEivABqLeknp mxiByWGNAGPaDsY3V5wOMQpwMCrx8K7QMg4RYk0sK67MPcQowcGsJMI75T1QiDclsbIqtSg/ vqg0J7X4EGMyMHgmMkuJJucDE1deSbyhiYGhqYmRoYWBmYUpacJK4ry/d+qFCAmkA5Nbdmpq QWoRzBYmDk6pBsa5ZY4tWVtKLpn36zPteOXzyUBhof7ByXpKK7ft4LksIsC55sm9k1V+hW8Z Foh4uci2Mdy1bwn88Mc7ce6NH34342JbxO6tkww/ImHneVQhq0VMK1TUnk3efMHe1or60wWW fKzR/ikWN2SbDx70ECiYPz+vt3Lfr/cKFQI8564XPG46LeLKpcRSnJFoqMVcVJwIAP7ZqdlS AwAA Cc: "freebsd-fs@FreeBSD.org" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 14:15:55 -0000 I want to achieve an automated snapshot creation/removal trough C written da= emon by myself. Obvious there is an API, but without proper documentation. Regards, Ivo -----Original Message----- From: Palle Girgensohn [mailto:girgen@pingpong.net] Sent: Tuesday, October 07, 2014 5:00 PM To: Ivailo A. Tanusheff Cc: freebsd-fs@FreeBSD.org Subject: Re: ZFS API zfs is all command line, AFAIK. I have never heard of anyone who uses libs f= or zfs admin. It will probably work, just never heard anyone who has had the= need for it? What are you trying to achieve. 7 okt 2014 kl. 15:37 skrev Ivailo A. Tanusheff : > Dear all, > > Is there really no one that knows/uses ZFS API? > That sounds ... well, scary. > > Regards, > Ivailo > > -----Original Message----- > From: owner-freebsd-fs@freebsd.org [mailto:owner-freebsd-fs@freebsd.org] O= n Behalf Of Ivailo A. Tanusheff > Sent: Monday, October 06, 2014 12:44 PM > To: freebsd-fs@FreeBSD.org > Subject: ZFS API > > Dear all, > > I am looking for an API and documentation about programing some actions on= C/C++ for ZFS management. > I have found an library - /usr/src/cddl/contrib/opensolaris/lib/libzfs/com= mon/libzfs.h but this lacks any documentation. > I do not want to make a huge development and my skills are not great, so I= need some useful API and documentation. > Is there such thing available around? > What I need is to include some snapshot management techniques :) > > Regards, > > Ivailo Tanusheff > > > Disclaimer: > > This communication is confidential. If you are not the intended recipient,= you are hereby notified that any disclosure, copying, distribution or takin= g any action in reliance on the contents of this information is strictly pro= hibited and may be unlawful. If you have received this communication by mist= ake, please notify us immediately by responding to this email and then delet= e it from your system. > Eurobank Bulgaria AD is not responsible for, nor endorses, any opinion, re= commendation, conclusion, solicitation, offer or agreement or any informatio= n contained in this communication. > Eurobank Bulgaria AD cannot accept any responsibility for the accuracy or= completeness of this message as it has been transmitted over a public netwo= rk. If you suspect that the message may have been intercepted or amended, pl= ease call the sender. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > Disclaimer: > > This communication is confidential. If you are not the intended recipient,= you are hereby notified that any disclosure, copying, distribution or takin= g any action in reliance on the contents of this information is strictly pro= hibited and may be unlawful. If you have received this communication by mist= ake, please notify us immediately by responding to this email and then delet= e it from your system. > Eurobank Bulgaria AD is not responsible for, nor endorses, any opinion, re= commendation, conclusion, solicitation, offer or agreement or any informatio= n contained in this communication. > Eurobank Bulgaria AD cannot accept any responsibility for the accuracy or= completeness of this message as it has been transmitted over a public netwo= rk. If you suspect that the message may have been intercepted or amended, pl= ease call the sender. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" Disclaimer: This communication is confidential. If you are not the intended recipient, y= ou are hereby notified that any disclosure, copying, distribution or taking= any action in reliance on the contents of this information is strictly proh= ibited and may be unlawful. If you have received this communication by mista= ke, please notify us immediately by responding to this email and then delete= it from your system. Eurobank Bulgaria AD is not responsible for, nor endorses, any opinion, reco= mmendation, conclusion, solicitation, offer or agreement or any information= contained in this communication. Eurobank Bulgaria AD cannot accept any responsibility for the accuracy or co= mpleteness of this message as it has been transmitted over a public network.= If you suspect that the message may have been intercepted or amended, pleas= e call the sender. From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 14:34:57 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 2CD7243B for ; Tue, 7 Oct 2014 14:34:57 +0000 (UTC) Received: from mxout01.bytecamp.net (mxout01.bytecamp.net [212.204.60.217]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D6C4A9A0 for ; Tue, 7 Oct 2014 14:34:56 +0000 (UTC) Received: by mxout01.bytecamp.net (Postfix, from userid 1001) id 1EACD315121; Tue, 7 Oct 2014 16:24:54 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=bytecamp.net; h=message-id:date:from:mime-version:to:subject:references:in-reply-to:content-type:content-transfer-encoding; s=20140709; bh=N70qOLOEuttD1YLY90gJwKFC5Cw=; b=KYlSjpnBG4/USfOz8t3HMayUixR/0BEyId34XnGV35mSepRzM3hs2TgaPy6yg28V/smJeSWVr14B5VEm023ZTq7ELk3IJIzlHyB4HjXnqPupaXO0GbI8jz1KtYkfEDgKwD00D34Zq8vSghAZL0KBnneRT1NV1a71kV2ICvVRzM8= Received: from mail.bytecamp.net (mailstore.bytecamp.net [212.204.60.20]) by mxout01.bytecamp.net (Postfix) with ESMTP id CE340315163 for ; Tue, 7 Oct 2014 16:24:53 +0200 (CEST) Received: (qmail 55768 invoked by uid 89); 7 Oct 2014 16:24:53 +0200 Received: from stella.bytecamp.net (HELO ?212.204.60.37?) (rs%bytecamp.net@212.204.60.37) by mail.bytecamp.net with CAMELLIA256-SHA encrypted SMTP; 7 Oct 2014 16:24:53 +0200 Message-ID: <5433F7B5.8090604@bytecamp.net> Date: Tue, 07 Oct 2014 16:24:53 +0200 From: Robert Schulze Organization: bytecamp GmbH User-Agent: Mozilla/5.0 (X11; Linux i686; rv:17.0) Gecko/20130330 Thunderbird/17.0.5 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: ZFS API References: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> <757ED2D6-0585-43C1-B827-FB349045246A@pingpong.net> <1422065A4E115F409E22C1EC9EDAFBA4604B8D@sofdc01exc02.postbank.bg> In-Reply-To: <1422065A4E115F409E22C1EC9EDAFBA4604B8D@sofdc01exc02.postbank.bg> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 14:34:57 -0000 Am 07.10.2014 16:15, schrieb Ivailo A. Tanusheff: > I want to achieve an automated snapshot creation/removal trough C written daemon by myself. > Obvious there is an API, but without proper documentation. KISS. This can easily be done with any scripting language/shell, no need to code that in C. with kind regards, Robert Schulze From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 14:36:43 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3DCF55B7 for ; Tue, 7 Oct 2014 14:36:43 +0000 (UTC) Received: from mail-ie0-f176.google.com (mail-ie0-f176.google.com [209.85.223.176]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0815B9C8 for ; Tue, 7 Oct 2014 14:36:42 +0000 (UTC) Received: by mail-ie0-f176.google.com with SMTP id rp18so5442915iec.21 for ; Tue, 07 Oct 2014 07:36:42 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:message-id:date:from:user-agent:mime-version:to :subject:references:in-reply-to:content-type :content-transfer-encoding; bh=RuagVHGEQDkTVelNyBPtaZUZOEDO1eESz0XHQmJMEzU=; b=KhBoSrvmN6I+4vyvyT/RxnC8ZruFOybMC177gjnpnlvhNRyetsWa2VKRsJEHuAPxoV 815vIV56oPlEBR6auBNT94E8O/6oINwTb42MKplQftt1uvbzitf6V9dQ61Ef5YR4Oa64 1usuN4E5sb949B+3/PZzC3e1QEe5mC4FNCmLQptmbf1p2pY3I1fXAlyEpoiEgk0o0sz+ QecjH2XIOzaka02bXScbzo4RQ1mbd9h7S6tAbUo4OYHESSSivATir1fDdy0Aoyz5Jk89 B0TWa6qSOimHa7Vvd1l5oQPZk8ncDbxGh0Coirl+XAKcp0MuritzTGiXNJlG3rV3xXhX 0LWQ== X-Gm-Message-State: ALoCoQnyWAJEdZWIVZQh4oyxx3ux1UJHuzjznaRFRRPYt+bz3GzuS2FM4bPcUW5aSC3sYnpudq7r X-Received: by 10.50.61.243 with SMTP id t19mr32960064igr.20.1412692601893; Tue, 07 Oct 2014 07:36:41 -0700 (PDT) Received: from kateleycoimac.local ([63.231.252.189]) by mx.google.com with ESMTPSA id qc6sm6404916igb.17.2014.10.07.07.36.41 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 07 Oct 2014 07:36:41 -0700 (PDT) Message-ID: <5433FA77.5010605@kateley.com> Date: Tue, 07 Oct 2014 09:36:39 -0500 From: Linda Kateley User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: ZFS API References: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> In-Reply-To: <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 14:36:43 -0000 You can get what you need in the open-zfs.org community Linda On 10/7/14, 8:37 AM, Ivailo A. Tanusheff wrote: > Dear all, > > Is there really no one that knows/uses ZFS API? > That sounds ... well, scary. > > Regards, > Ivailo > > -----Original Message----- > From: owner-freebsd-fs@freebsd.org [mailto:owner-freebsd-fs@freebsd.org] On Behalf Of Ivailo A. Tanusheff > Sent: Monday, October 06, 2014 12:44 PM > To: freebsd-fs@FreeBSD.org > Subject: ZFS API > > Dear all, > > I am looking for an API and documentation about programing some actions on C/C++ for ZFS management. > I have found an library - /usr/src/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h but this lacks any documentation. > I do not want to make a huge development and my skills are not great, so I need some useful API and documentation. > Is there such thing available around? > What I need is to include some snapshot management techniques :) > > Regards, > > Ivailo Tanusheff > > > Disclaimer: > > This communication is confidential. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication by mistake, please notify us immediately by responding to this email and then delete it from your system. > Eurobank Bulgaria AD is not responsible for, nor endorses, any opinion, recommendation, conclusion, solicitation, offer or agreement or any information contained in this communication. > Eurobank Bulgaria AD cannot accept any responsibility for the accuracy or completeness of this message as it has been transmitted over a public network. If you suspect that the message may have been intercepted or amended, please call the sender. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > Disclaimer: > > This communication is confidential. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication by mistake, please notify us immediately by responding to this email and then delete it from your system. > Eurobank Bulgaria AD is not responsible for, nor endorses, any opinion, recommendation, conclusion, solicitation, offer or agreement or any information contained in this communication. > Eurobank Bulgaria AD cannot accept any responsibility for the accuracy or completeness of this message as it has been transmitted over a public network. If you suspect that the message may have been intercepted or amended, please call the sender. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 16:22:28 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A9872661 for ; Tue, 7 Oct 2014 16:22:28 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 82D81984 for ; Tue, 7 Oct 2014 16:22:28 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 70686B9BB; Tue, 7 Oct 2014 12:22:27 -0400 (EDT) From: John Baldwin To: freebsd-fs@freebsd.org Subject: Re: 9.3 NFS client bug? Date: Tue, 7 Oct 2014 11:18:00 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20140415; KDE/4.5.5; amd64; ; ) References: <1109402573.59384867.1412683413852.JavaMail.root@uoguelph.ca> In-Reply-To: <1109402573.59384867.1412683413852.JavaMail.root@uoguelph.ca> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201410071118.00364.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 07 Oct 2014 12:22:27 -0400 (EDT) Cc: Garrett Wollman X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 16:22:28 -0000 On Tuesday, October 07, 2014 8:03:33 am Rick Macklem wrote: > Garrett Wollman wrote: > > < > said: > > > > > 2 - Try an "oldnfs" mount and see if the old client exhibits the > > > same behaviour. > > > > Same behavior both old and new. I'm doing binary search now to try > > to > > minimize the size of the workload that exhibits the issue. > > > > -GAWollman > > > By ay chance is your ZFS server allocating a ZFS inode (whatever they > call it) with a fileno/inode# that doesn't fit in 32 bits? > (There is a diagnostic printf w.r.t. this for NFSv4, but not NFSv3, > so you might try an NFSv4 mount. The printf starts with "NFSv4 fileid > 32..".) > > Since d_fileno in "struct dirent" is 32bits, NFS just truncates the > 64bit fileid to 32bits (or fills the high order 32bits with 0s for the server). > ZFS is known to generate fileids with non-zero high order 32bits->busted. > (And not at all easy to fix. I actually have a somewhat hackish idea for > a way to make 64bit d_fileno values work without busting backwards compatibility. > I think I'll post that to freebsd-fs@ and see what others think?) Also, does it exhibit if you NFS export a UFS filesystem instead of ZFS? -- John Baldwin From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 21:06:06 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A9A4340D; Tue, 7 Oct 2014 21:06:06 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 5EB93EFB; Tue, 7 Oct 2014 21:06:05 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: ArUEAIVVNFSDaFve/2dsb2JhbABfDoNTWASDAMkaCoZ5VAKBJwF7hAMBAQEDAQEBASAEJyALBRYYAgINGQIpAQkmBggHBAEcBIgVCA2tN5UFAReBLI43EAEBGzQHgneBVAWWNIJBgUmEMTyQIIN/gyNcIS8HgQAIFyKBAgEBAQ X-IronPort-AV: E=Sophos;i="5.04,673,1406606400"; d="scan'208";a="160020817" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 07 Oct 2014 17:06:02 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id B8E39B4097; Tue, 7 Oct 2014 17:06:02 -0400 (EDT) Date: Tue, 7 Oct 2014 17:06:02 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <603326314.59995969.1412715962746.JavaMail.root@uoguelph.ca> In-Reply-To: <1109402573.59384867.1412683413852.JavaMail.root@uoguelph.ca> Subject: Re: 9.3 NFS client bug? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 21:06:06 -0000 I wrote: > Garrett Wollman wrote: > > < > said: > > > > > 2 - Try an "oldnfs" mount and see if the old client exhibits the > > > same behaviour. > > > > Same behavior both old and new. I'm doing binary search now to try > > to > > minimize the size of the workload that exhibits the issue. > > > > -GAWollman > > > By ay chance is your ZFS server allocating a ZFS inode (whatever they > call it) with a fileno/inode# that doesn't fit in 32 bits? > (There is a diagnostic printf w.r.t. this for NFSv4, but not NFSv3, > so you might try an NFSv4 mount. The printf starts with "NFSv4 > fileid > 32..".) > Oops, I've realized that this diagnostic won't help if the server is a FreeBSD one, since the FreeBSD server just sets the high order 32bits to 0s and not what the file system returns. (va_filerev is a "long", so it is only 64bits on some arches, yikes.) You could try what jhb@ suggested (try a UFS file system) or just use a different ZFS volume that isn't very full, if you think the 32bit restriction could have been exceeded. Btw, I tried your bonnie++ command on my test machines, but they failed part way through because they couldn't allocate boundary tags. (At least now I can reproduce that easily, although it is the M_NOWAIT case where the thread just loops in the kernel.) rick > Since d_fileno in "struct dirent" is 32bits, NFS just truncates the > 64bit fileid to 32bits (or fills the high order 32bits with 0s for > the server). > ZFS is known to generate fileids with non-zero high order > 32bits->busted. > (And not at all easy to fix. I actually have a somewhat hackish idea > for > a way to make 64bit d_fileno values work without busting backwards > compatibility. > I think I'll post that to freebsd-fs@ and see what others think?) > > rick > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 21:50:08 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C1D32DF0; Tue, 7 Oct 2014 21:50:08 +0000 (UTC) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 81FA93E8; Tue, 7 Oct 2014 21:50:08 +0000 (UTC) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s97Lo61K006251 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Tue, 7 Oct 2014 17:50:06 -0400 (EDT) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.7/8.14.7/Submit) id s97Lo611006248; Tue, 7 Oct 2014 17:50:06 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21556.24590.269326.188103@khavrinen.csail.mit.edu> Date: Tue, 7 Oct 2014 17:50:06 -0400 From: Garrett Wollman To: Rick Macklem Subject: Re: 9.3 NFS client bug? In-Reply-To: <603326314.59995969.1412715962746.JavaMail.root@uoguelph.ca> References: <1109402573.59384867.1412683413852.JavaMail.root@uoguelph.ca> <603326314.59995969.1412715962746.JavaMail.root@uoguelph.ca> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Tue, 07 Oct 2014 17:50:06 -0400 (EDT) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 21:50:08 -0000 < said: >> By ay chance is your ZFS server allocating a ZFS inode (whatever they >> call it) with a fileno/inode# that doesn't fit in 32 bits? Nope. (And I want to emphasize that Ubuntu clients don't exhibit the problem, so whatever it is, it doesn't appear to be related to the backing filesystem on the server.) > Btw, I tried your bonnie++ command on my test machines, but they failed > part way through because they couldn't allocate boundary tags. (At least > now I can reproduce that easily, although it is the M_NOWAIT case where > the thread just loops in the kernel.) You really need to get some server-class hardware. Perhaps the Foundation can help? I'm currently using the following command, which seems to reproduce the problem reliably on my client and server: bonnie++ -s 0 -n 144:0:0:1:16384 -u 4294967294 -D This minimizes the amount of temporary storage required. -GAWollman From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 22:02:16 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 548076B8; Tue, 7 Oct 2014 22:02:16 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 0AB957E8; Tue, 7 Oct 2014 22:02:15 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqEEAGliNFSDaFve/2dsb2JhbABfhD2DANBxgSkBe4QtBFI1Ag0ZAl+IUa1LlQ8BF4EsjmQ0gn6BVAWYdYV6g0SHZ4kwg38hgXeBAgEBAQ X-IronPort-AV: E=Sophos;i="5.04,673,1406606400"; d="scan'208";a="158862998" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 07 Oct 2014 18:01:50 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id E54F5B40D4; Tue, 7 Oct 2014 18:01:50 -0400 (EDT) Date: Tue, 7 Oct 2014 18:01:50 -0400 (EDT) From: Rick Macklem To: FreeBSD Filesystems Message-ID: <134743690.60044002.1412719310927.JavaMail.root@uoguelph.ca> Subject: RFC: hackish way to support 64bit d_fileno values MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.209] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: Kirk McKusick , Gleb Kurtsou X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 22:02:16 -0000 Hi, Some file systems (ZFS, NFS) can support 64bit filenos (i-node #s) and in fact I think ZFS will just do so when/if the file system gets large enough. As such, I think FreeBSD needs to somehow support these someday. Changing d_fileno to 64bits the obvious way (defining it as uint64_t) is fraught with challenges. For example, even if UFS copies the on-disk format to this structure, the directory offsets all change and even move to different block numbers for cases where the on-disk directory is tightly packed. As such, I've thought up this (somewhat hackish;-) alternative that I think might allow this to be done without introducing serious backwards compatibility (POLA) violations. - Define a new field I'll call d_filenohigh, that can live at the end of "struct dirent" (after d_name). Define a flag for d_type called DT_FILENOEXT as 0x80, that could be or'd into d_type to indicate that d_filenohigh exists. (Since d_filenohigh lives after d_name, it would actually have to be accessed via a macro of accessor function, but I'll just call it d_filenohigh for simplicity. It would be a uint32_t properly aligned and positioned after the end of d_name. It would be included in d_reclen when it exists.) Now, file systems like UFS which use 32bit filenos wouldn't change at all. They would just generate "struct dirent"s like they do now. File systems that use 64bit filenos (ZFS, NFS, ..) would put the high order 32bits of their fileno in d_dilenohigh and set DT_FILENOEXT. (They could do it always or only when the high order 32bits are non-zero.) The NFS server could check for DT_FILENOEXT set and handle the high order 32bits if it is set. getdirentries() { and compatibility code for Linux... } would just clear the DT_FINENOEXT flag. (ZFS, NFS dirents would appear less densely packed, but I don't think that should matter, so long as they still obey the "don't let a struct dirent straddle 2 DIRBLKSIZ blocks".) getdirentries64() would be implemented that would return directory structures that look like: struct dirent64 { uint64_t d_fileno; uint16_t d_reclen; uint8_t d_type; uint8_t d_namlen; char d_name[MAXNAMELEN + 1]; /* actually variable length */ uint64_t d_diroff; }; (I think POSIX will allow the size of d_fileno to be whatever the OS chooses.) d_diroff - This holds the directory offset "cookie" for the directory entry in the underlying file system this entry was created from. (For example, the byte offset in the UFS directory for UFS or...) --> This field can be used to get to the correct position in the directory via lseek(). I think "basep" isn't sufficient, since the size of each directory entry isn't the same as on-disk, so I think you need one for every directory entry. In turn, telldir(), seekdir() should be able to use these values. (They need to be defined as returning and using uint64_t instead of "long". Hopefully this can be done without too much pain?) I haven't looked at the libc functions yet, so I don't know if there are gotchas. File systems could generate this new "struct dirent64" natively (some MNTK_xx flag could indicate this) and then the old getdirentries(2) would have to do the copying/conversion to old "struct dirent". It could leave "gaps" (ie d_reclen wouldn't change) so the directory offsets remain the same as for the underlying file system that generated them. For file systems that don't generate this new format, getdirentries64(2) would have to do the copying out to the new format, including filling in d_diroff correctly. At some point, I think getdirentries64(2) could be renamed getdirentries(2) and getdirentries(2) renamed to ogetdirentries(2) along with a renaming of the old and new "struct dirent"s. I believe this transition could be delayed until the basic changes had settled in. Comments anyone? rick ps: Gleb, if you weren't the guy who already did some 64bit ino_t stuff, please let me know, so I can forward this to the right guy. From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 22:16:26 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7F76A945; Tue, 7 Oct 2014 22:16:26 +0000 (UTC) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 369138F5; Tue, 7 Oct 2014 22:16:25 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aq8EAMJkNFSDaFve/2dsb2JhbABfDoQrBIMA0HECgScBe4QDAQEBAwEjVhsYAgINGQJZBhOINgitNpUTAReBLI5PFTQHgneBVAWRTo0hkFyDf4MjXCEvgQYBHwQegQIBAQE X-IronPort-AV: E=Sophos;i="5.04,673,1406606400"; d="scan'208";a="158864915" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 07 Oct 2014 18:16:24 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id EA971B4060; Tue, 7 Oct 2014 18:16:24 -0400 (EDT) Date: Tue, 7 Oct 2014 18:16:24 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <654363730.60050348.1412720184948.JavaMail.root@uoguelph.ca> In-Reply-To: <21556.24590.269326.188103@khavrinen.csail.mit.edu> Subject: Re: 9.3 NFS client bug? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 22:16:26 -0000 Garrett Wollman wrote: > < said: > > >> By ay chance is your ZFS server allocating a ZFS inode (whatever > >> they > >> call it) with a fileno/inode# that doesn't fit in 32 bits? > > Nope. (And I want to emphasize that Ubuntu clients don't exhibit the > problem, so whatever it is, it doesn't appear to be related to the > backing filesystem on the server.) > Well, not many things in FreeBSD break when the i-node#s get truncated. fts(3) will complain about a possible loop in the directory structure, but I'm not aware of much else. I have no idea what might break in the Ubuntu client for this case. All I'm saying is that "it works for Ubuntu" suggests that the bug is in the FreeBSD client (or FreeBSD port of bonnie++), but doesn't guarantee that there isn't a server side problem being exposed by the FreeBSD client. rick ps: I would like to hear what happens when you test against a UFS volume. > > Btw, I tried your bonnie++ command on my test machines, but they > > failed > > part way through because they couldn't allocate boundary tags. (At > > least > > now I can reproduce that easily, although it is the M_NOWAIT case > > where > > the thread just loops in the kernel.) > > You really need to get some server-class hardware. Perhaps the > Foundation can help? > > I'm currently using the following command, which seems to reproduce > the problem reliably on my client and server: > > bonnie++ -s 0 -n 144:0:0:1:16384 -u 4294967294 -D > > This minimizes the amount of temporary storage required. > > -GAWollman > From owner-freebsd-fs@FreeBSD.ORG Tue Oct 7 23:01:46 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5771470D; Tue, 7 Oct 2014 23:01:46 +0000 (UTC) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 18AE0DAF; Tue, 7 Oct 2014 23:01:45 +0000 (UTC) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s97N1iTN006634 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Tue, 7 Oct 2014 19:01:44 -0400 (EDT) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.7/8.14.7/Submit) id s97N1irN006631; Tue, 7 Oct 2014 19:01:44 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21556.28888.523838.856998@khavrinen.csail.mit.edu> Date: Tue, 7 Oct 2014 19:01:44 -0400 From: Garrett Wollman To: Rick Macklem Subject: Re: 9.3 NFS client bug? In-Reply-To: <654363730.60050348.1412720184948.JavaMail.root@uoguelph.ca> References: <21556.24590.269326.188103@khavrinen.csail.mit.edu> <654363730.60050348.1412720184948.JavaMail.root@uoguelph.ca> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Tue, 07 Oct 2014 19:01:44 -0400 (EDT) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 07 Oct 2014 23:01:46 -0000 < said: > ps: I would like to hear what happens when you test against a UFS volume. We don't have any UFS volumes, and this 96-drive file server is not likely ever to have any. FWIW, I now have a packet capture of the complete test, taken from the server side, if anyone wants to look at it in detail. The inner loop of the "sequential delete" benchmark looks like this (slightly simplified): while(1) { file_ent = readdir(d); if(file_ent == NULL) break; if(file_ent->d_name[0] != '.') { if(unlink(file_ent->d_name)) { fprintf(stderr, "Can't delete file %s\n", file_ent->d_name); return -1; } count++; } } closedir(d); This makes me wonder if there isn't some issue with the NFS VOP_READDIR losing its place in the presence of interleaved deletes. fts(3) won't see this as it reads the whole directory at once (for sorting, if requested) before invoking the callback on each entry. -GAWollman From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 00:40:59 2014 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6881C1F9; Wed, 8 Oct 2014 00:40:59 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id F2EBC9AD; Wed, 8 Oct 2014 00:40:55 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s980enXK027532 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 8 Oct 2014 00:40:54 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s980enD1027531; Wed, 8 Oct 2014 00:40:49 GMT (envelope-from swills) Date: Wed, 8 Oct 2014 00:40:48 +0000 From: Steve Wills To: current@FreeBSD.org, fs@FreeBSD.org Subject: zfs hang Message-ID: <20141008004045.GA24762@mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Wed, 08 Oct 2014 00:40:54 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=HEADER_FROM_DIFFERENT_DOMAINS autolearn=unavailable autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mouf.net X-Virus-Scanned: clamav-milter 0.98.3 at mouf.net X-Virus-Status: Clean X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 00:40:59 -0000 Hi, Not sure which thread this belongs to, but I have a zfs hang on one of my boxes running r272152. Running procstat -kka looks like: http://pastebin.com/szZZP8Tf My zpool commands seem to be hung in spa_errlog_lock while others are hung in zfs_lookup. Suggestions? Thanks, Steve From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 01:20:43 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4B1F485A; Wed, 8 Oct 2014 01:20:43 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 019B8D13; Wed, 8 Oct 2014 01:20:42 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aq8EACmQNFSDaFve/2dsb2JhbABfDoQrBIMA0HICgSMBe4QEAQEEIwRSGxgCAg0ZAlkGE4g+rQqVDwEXgSyOTxU0B4J3gVQFkU6NIZBcg3+DI1whL4EGAR8EHoECAQEB X-IronPort-AV: E=Sophos;i="5.04,674,1406606400"; d="scan'208";a="160057608" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 07 Oct 2014 21:20:41 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 1977DB4081; Tue, 7 Oct 2014 21:20:41 -0400 (EDT) Date: Tue, 7 Oct 2014 21:20:41 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <1402889337.60119616.1412731241078.JavaMail.root@uoguelph.ca> In-Reply-To: <21556.24590.269326.188103@khavrinen.csail.mit.edu> Subject: Re: 9.3 NFS client bug? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 01:20:43 -0000 Garrett Wollman wrote: > < said: > > >> By ay chance is your ZFS server allocating a ZFS inode (whatever > >> they > >> call it) with a fileno/inode# that doesn't fit in 32 bits? > > Nope. (And I want to emphasize that Ubuntu clients don't exhibit the > problem, so whatever it is, it doesn't appear to be related to the > backing filesystem on the server.) > > > Btw, I tried your bonnie++ command on my test machines, but they > > failed > > part way through because they couldn't allocate boundary tags. (At > > least > > now I can reproduce that easily, although it is the M_NOWAIT case > > where > > the thread just loops in the kernel.) > > You really need to get some server-class hardware. Perhaps the > Foundation can help? > > I'm currently using the following command, which seems to reproduce > the problem reliably on my client and server: > > bonnie++ -s 0 -n 144:0:0:1:16384 -u 4294967294 -D > > This minimizes the amount of temporary storage required. > > -GAWollman > I took a look at the bonnie++ sources and it does the removal of the files in a loop like while ((dp = readdir(dir)) != NULL) unlink(dp->d_name); As far as I know, this has never worked correctly for FreeBSD. The unlink() invalidates the directory offset cookies and then it has trouble finding the next entry. To make the above loop work correctly for FreeBSD, it needs to be re-written to start at the beginning of the directory after each unlink(). I can't remember why it was coded to invalidate the offset cookies, since they should still be ok after an unlink() for UFS, but maybe other file systems break. Maybe jhb@ can remember more about this? OpenBSD doesn't use the buffer cache for NFS Readdir, but I can't remember if it also suffers from this issue. Sorry, but I hadn't run into this for a while, so I didn't spot it, rick From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 01:24:23 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D527B901; Wed, 8 Oct 2014 01:24:23 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 8B400DB8; Wed, 8 Oct 2014 01:24:23 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Aq8EAEmRNFSDaFve/2dsb2JhbABfDoQrBIMA0HICgSMBe4QEAQEEIwRSGxgCAg0ZAlkGE4g+rQiVDwEXgSyOZDQHgneBVAWzSoMjXCEvgUiBAgEBAQ X-IronPort-AV: E=Sophos;i="5.04,674,1406606400"; d="scan'208";a="160057976" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 07 Oct 2014 21:24:22 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 5CC31B4037; Tue, 7 Oct 2014 21:24:22 -0400 (EDT) Date: Tue, 7 Oct 2014 21:24:22 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <1991198781.60122149.1412731462373.JavaMail.root@uoguelph.ca> In-Reply-To: <21556.28888.523838.856998@khavrinen.csail.mit.edu> Subject: Re: 9.3 NFS client bug? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 01:24:23 -0000 Garrett Wollman wrote: > < said: > > > ps: I would like to hear what happens when you test against a UFS > > volume. > > We don't have any UFS volumes, and this 96-drive file server is not > likely ever to have any. > > FWIW, I now have a packet capture of the complete test, taken from > the > server side, if anyone wants to look at it in detail. > > The inner loop of the "sequential delete" benchmark looks like this > (slightly simplified): > > while(1) > { > file_ent = readdir(d); > if(file_ent == NULL) > break; > if(file_ent->d_name[0] != '.') > { > if(unlink(file_ent->d_name)) > { > fprintf(stderr, "Can't delete file %s\n", > file_ent->d_name); > return -1; > } > count++; > } > } > closedir(d); > > This makes me wonder if there isn't some issue with the NFS > VOP_READDIR losing its place in the presence of interleaved deletes. > fts(3) won't see this as it reads the whole directory at once (for > sorting, if requested) before invoking the callback on each entry. > Yep. As noted in the other reply, an unlink() inside a loop on readdir() has never worked for NFS on FreeBSD, as far as I know. It's related to the handling of directory offset cookies, but I can't remember all the details. rick > -GAWollman > > From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 05:56:33 2014 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id F134B903; Wed, 8 Oct 2014 05:56:32 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id E4F02A99; Wed, 8 Oct 2014 05:56:31 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id IAA22718; Wed, 08 Oct 2014 08:56:24 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1XbkEZ-0000Io-Qj; Wed, 08 Oct 2014 08:56:23 +0300 Message-ID: <5434D1CE.8010801@FreeBSD.org> Date: Wed, 08 Oct 2014 08:55:26 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: Steve Wills Subject: Re: zfs hang References: <20141008004045.GA24762__48659.9047123038$1412728878$gmane$org@mouf.net> In-Reply-To: <20141008004045.GA24762__48659.9047123038$1412728878$gmane$org@mouf.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: current@FreeBSD.org, fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 05:56:33 -0000 On 08/10/2014 03:40, Steve Wills wrote: > Hi, > > Not sure which thread this belongs to, but I have a zfs hang on one of my boxes > running r272152. Running procstat -kka looks like: > > http://pastebin.com/szZZP8Tf > > My zpool commands seem to be hung in spa_errlog_lock while others are hung in > zfs_lookup. Suggestions? There are several threads in zio_wait. If this is their permanent state then there is some problem with I/O somewhere below ZFS. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 08:28:24 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A39DBBB9 for ; Wed, 8 Oct 2014 08:28:24 +0000 (UTC) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5D983AC3 for ; Wed, 8 Oct 2014 08:28:23 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id EE9469DE650; Wed, 8 Oct 2014 10:28:19 +0200 (CEST) Subject: Re: ZFS API Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <5433F7B5.8090604@bytecamp.net> Date: Wed, 8 Oct 2014 10:28:18 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <7BD1DA91-6667-4183-B4C2-297FA1602703@sarenet.es> References: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> <757ED2D6-0585-43C1-B827-FB349045246A@pingpong.net> <1422065A4E115F409E22C1EC9EDAFBA4604B8D@sofdc01exc02.postbank.bg> <5433F7B5.8090604@bytecamp.net> To: Robert Schulze X-Mailer: Apple Mail (2.1283) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 08:28:24 -0000 On Oct 7, 2014, at 4:24 PM, Robert Schulze wrote: > Am 07.10.2014 16:15, schrieb Ivailo A. Tanusheff: >> I want to achieve an automated snapshot creation/removal trough C = written daemon by myself. >> Obvious there is an API, but without proper documentation. >=20 > KISS. >=20 > This can easily be done with any scripting language/shell, no need to = code that in C. It depends on your goals. Relying on scripts running commands and = parsing their output is not what I would call sound software design. So much can go wrong. If you are just putting = together some tools do to some light work it's fine, though. The problem with the API is that, as far as I know, it is not stable. It = hasn't been defined as a valid interface to control ZFS, but just a convenience library and interface between the command = line tools and the underlying system. Hence, it can change completely even with an apparently minor update, so relying = on it is even worse than playing command parsing. It would be great to have a reliable API. Borja. From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 13:12:54 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3EB9A838 for ; Wed, 8 Oct 2014 13:12:54 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2683BD50 for ; Wed, 8 Oct 2014 13:12:54 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s98DCsO3014691 for ; Wed, 8 Oct 2014 13:12:54 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 159251] [zfs] [request]: add FLETCHER4 as DEDUP hash option Date: Wed, 08 Oct 2014 13:12:53 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: JeanFrancois.Boeuf@Gmail.com X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 13:12:54 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D159251 Jean-Fran=C3=A7ois BOEUF changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |JeanFrancois.Boeuf@Gmail.co | |m --- Comment #2 from Jean-Fran=C3=A7ois BOEUF = --- not relevant anymore : fletcher4 is the default (since r259477). Can be clo= sed ? --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 15:00:56 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 43882DAC for ; Wed, 8 Oct 2014 15:00:56 +0000 (UTC) Received: from spindle.one-eyed-alien.net (spindle.one-eyed-alien.net [199.48.129.229]) by mx1.freebsd.org (Postfix) with ESMTP id 1B872B5C for ; Wed, 8 Oct 2014 15:00:55 +0000 (UTC) Received: by spindle.one-eyed-alien.net (Postfix, from userid 3001) id CFD695A9F25; Wed, 8 Oct 2014 15:00:54 +0000 (UTC) Date: Wed, 8 Oct 2014 15:00:54 +0000 From: Brooks Davis To: Borja Marcos Subject: Re: ZFS API Message-ID: <20141008150054.GD25906@spindle.one-eyed-alien.net> References: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> <757ED2D6-0585-43C1-B827-FB349045246A@pingpong.net> <1422065A4E115F409E22C1EC9EDAFBA4604B8D@sofdc01exc02.postbank.bg> <5433F7B5.8090604@bytecamp.net> <7BD1DA91-6667-4183-B4C2-297FA1602703@sarenet.es> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hwvH6HDNit2nSK4j" Content-Disposition: inline In-Reply-To: <7BD1DA91-6667-4183-B4C2-297FA1602703@sarenet.es> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 15:00:56 -0000 --hwvH6HDNit2nSK4j Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Oct 08, 2014 at 10:28:18AM +0200, Borja Marcos wrote: >=20 > On Oct 7, 2014, at 4:24 PM, Robert Schulze wrote: >=20 > > Am 07.10.2014 16:15, schrieb Ivailo A. Tanusheff: > >> I want to achieve an automated snapshot creation/removal trough C writ= ten daemon by myself. > >> Obvious there is an API, but without proper documentation. > >=20 > > KISS. > >=20 > > This can easily be done with any scripting language/shell, no need to c= ode that in C. >=20 > It depends on your goals. Relying on scripts running commands and parsing= their output is not what I would call > sound software design. So much can go wrong. If you are just putting toge= ther some tools do to some light work it's fine, though. It's worth noting that the ZFS command line tools are explicitly designed to enable relible scripting if you use arguments like -H (no headers) and -o (extract specific columns). I talked about this a bit at the end of this presentation: http://2011.eurobsdcon.org/papers/davis/system-mgmt-zfs-notes.pdf -- Brooks --hwvH6HDNit2nSK4j Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlQ1UaYACgkQXY6L6fI4GtRRHgCbBeP17L4DSP2GldjmY/jIY4mr 8SMAn1h0WZRjM313zyoR27Zngh4gWBgA =yzQC -----END PGP SIGNATURE----- --hwvH6HDNit2nSK4j-- From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 15:14:31 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D21CF3C2; Wed, 8 Oct 2014 15:14:31 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 889F1D40; Wed, 8 Oct 2014 15:14:31 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 20E849DCE9A; Wed, 8 Oct 2014 17:04:38 +0200 (CEST) Subject: Re: ZFS API Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset="utf8"; X-Pgp-Agent: GPGMail (null) From: Borja Marcos In-Reply-To: <20141008150054.GD25906@spindle.one-eyed-alien.net> Date: Wed, 8 Oct 2014 17:04:31 +0200 Content-Transfer-Encoding: 8bit Message-Id: <3A396786-4010-4934-AA12-9DBC87392D99@sarenet.es> References: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> <757ED2D6-0585-43C1-B827-FB349045246A@pingpong.net> <1422065A4E115F409E22C1EC9EDAFBA4604B8D@sofdc01exc02.postbank.bg> <5433F7B5.8090604@bytecamp.net> <7BD1DA91-6667-4183-B4C2-297FA1602703@sarenet.es> <20141008150054.GD25906@spindle.one-eyed-alien.net> To: Brooks Davis X-Mailer: Apple Mail (2.1283) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 15:14:31 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Oct 8, 2014, at 5:00 PM, Brooks Davis wrote: It depends on your goals. Relying on scripts running commands and parsing their output is not what I would call sound software design. So much can go wrong. If you are just putting together some tools do to some light work it's fine, though. It's worth noting that the ZFS command line tools are explicitly designed to enable relible scripting if you use arguments like -H (no headers) and -o (extract specific columns). I talked about this a bit at the end of this presentation: http://2011.eurobsdcon.org/papers/davis/system-mgmt-zfs-notes.pdf I know, again it's not exactly the optimal way to do it. Call me old-fashioned :) Borja. -----BEGIN PGP SIGNATURE----- Comment: GPGTools - http://gpgtools.org iEYEARECAAYFAlQ1UoQACgkQULpVo4XWgJ8tggCeN9nKboZm3jTkxIDXaQscY7EO mgoAn1OIxQVK5LrzZL7kj7hTvuBwitkA =30bv -----END PGP SIGNATURE----- From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 15:25:20 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 1D1DA7F8; Wed, 8 Oct 2014 15:25:20 +0000 (UTC) Received: from khavrinen.csail.mit.edu (khavrinen.csail.mit.edu [IPv6:2001:470:8b2d:1e1c:21b:21ff:feb8:d7b0]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "khavrinen.csail.mit.edu", Issuer "Client CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CE303E6D; Wed, 8 Oct 2014 15:25:19 +0000 (UTC) Received: from khavrinen.csail.mit.edu (localhost [127.0.0.1]) by khavrinen.csail.mit.edu (8.14.7/8.14.7) with ESMTP id s98FPIBs012951 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL CN=khavrinen.csail.mit.edu issuer=Client+20CA); Wed, 8 Oct 2014 11:25:18 -0400 (EDT) (envelope-from wollman@khavrinen.csail.mit.edu) Received: (from wollman@localhost) by khavrinen.csail.mit.edu (8.14.7/8.14.7/Submit) id s98FPHhm012948; Wed, 8 Oct 2014 11:25:17 -0400 (EDT) (envelope-from wollman) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <21557.22365.961980.709081@khavrinen.csail.mit.edu> Date: Wed, 8 Oct 2014 11:25:17 -0400 From: Garrett Wollman To: Rick Macklem Subject: Re: 9.3 NFS client bug? In-Reply-To: <1402889337.60119616.1412731241078.JavaMail.root@uoguelph.ca> References: <21556.24590.269326.188103@khavrinen.csail.mit.edu> <1402889337.60119616.1412731241078.JavaMail.root@uoguelph.ca> X-Mailer: VM 7.17 under 21.4 (patch 22) "Instant Classic" XEmacs Lucid X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (khavrinen.csail.mit.edu [127.0.0.1]); Wed, 08 Oct 2014 11:25:18 -0400 (EDT) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 15:25:20 -0000 < said: > As far as I know, this has never worked correctly for FreeBSD. The > unlink() invalidates the directory offset cookies and then it has > trouble finding the next entry. > To make the above loop work correctly for FreeBSD, it needs to be > re-written to start at the beginning of the directory after each > unlink(). How about instead we fix FreeBSD to work properly? Clearly it is not impossible since the Linux NFS client does work. What exactly is the issue? (Forgive me, I know very little about how VOP_READDIR works under the hood.) -GAWollman From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 15:50:38 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 141CA741; Wed, 8 Oct 2014 15:50:38 +0000 (UTC) Received: from mail.iXsystems.com (mail.ixsystems.com [12.229.62.4]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "*.ixsystems.com", Issuer "Go Daddy Secure Certification Authority" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E448021F; Wed, 8 Oct 2014 15:50:37 +0000 (UTC) Received: from localhost (mail.ixsystems.com [10.2.55.1]) by mail.iXsystems.com (Postfix) with ESMTP id 60C178082D; Wed, 8 Oct 2014 08:50:37 -0700 (PDT) Received: from mail.iXsystems.com ([10.2.55.1]) by localhost (mail.ixsystems.com [10.2.55.1]) (maiad, port 10024) with ESMTP id 15519-06; Wed, 8 Oct 2014 08:50:37 -0700 (PDT) Received: from [10.20.30.117] (125.sub-70-197-4.myvzw.com [70.197.4.125]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.iXsystems.com (Postfix) with ESMTPSA id CF03A807EA; Wed, 8 Oct 2014 08:50:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=ixsystems.com; s=newknight0; t=1412783426; bh=TIShFg8jfG1jItLcE/uAdEwKn36sFc/R52lG2KZUmgM=; h=Subject:From:In-Reply-To:Date:Cc:References:To; b=gNldonAHgMYP/AwIJHCB3kTcNtcxOyIZWpW3C6SaQ2RHyM1XPI7LDr4zDJBgDTZQX rTcdJBjqHeUrmy1OlvDCUxIxvi2WRu5l4h8mtYP6VhpYtSod7qShFTv8RcIouB/lW/ OIViaciR9ltiVcWAjqzP7ZEBKZTz8Ga6Hlm8UoGg= Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 8.0 \(1990.1\)) Subject: Re: ZFS API From: Jordan Hubbard In-Reply-To: <3A396786-4010-4934-AA12-9DBC87392D99@sarenet.es> Date: Wed, 8 Oct 2014 08:50:23 -0700 Content-Transfer-Encoding: quoted-printable Message-Id: <8F48CD13-B7BD-4261-8CE4-BBB59CAC473C@ixsystems.com> References: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> <1422065A4E115F409E22C1EC9EDAFBA4604B35@sofdc01exc02.postbank.bg> <757ED2D6-0585-43C1-B827-FB349045246A@pingpong.net> <1422065A4E115F409E22C1EC9EDAFBA4604B8D@sofdc01exc02.postbank.bg> <5433F7B5.8090604@bytecamp.net> <7BD1DA91-6667-4183-B4C2-297FA1602703@sarenet.es> <20141008150054.GD25906@spindle.one-eyed-alien.net> <3A396786-4010-4934-AA12-9DBC87392D99@sarenet.es> To: Borja Marcos X-Mailer: Apple Mail (2.1990.1) Cc: freebsd-fs@freebsd.org, Brooks Davis X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 15:50:38 -0000 > On Oct 8, 2014, at 8:04 AM, Borja Marcos wrote: >=20 > I know, again it's not exactly the optimal way to do it. Call me = old-fashioned :) I can=E2=80=99t help but feel that if even half the energy that has gone = into this discussion were instead directed towards actually *addressing = the problem*, the problem would be well on its way towards being = addressed right now. In other words, the code does not care about this philosophical = discussion. If someone wants to have a robust, well-supported API for = ZFS, then they (the person who wants it, ideally, since motivation is = 9/10ths of the battle) should work on making that happen. The OpenZFS project has been pretty good about accepting contributions. = Its biggest challenge is simply that there aren=E2=80=99t enough people = volunteering to contribute. My sensors detect a potential volunteer. :) - Jordan From owner-freebsd-fs@FreeBSD.ORG Wed Oct 8 17:44:19 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A2C3029C for ; Wed, 8 Oct 2014 17:44:19 +0000 (UTC) Received: from mail-yh0-x22a.google.com (mail-yh0-x22a.google.com [IPv6:2607:f8b0:4002:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 654FB14C for ; Wed, 8 Oct 2014 17:44:19 +0000 (UTC) Received: by mail-yh0-f42.google.com with SMTP id t59so4216111yho.29 for ; Wed, 08 Oct 2014 10:44:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:cc:content-type; bh=idVqZyB65tQWEwUx24yyCo8GA/ADBQPTDZ7spzZCcls=; b=r2PfeUgRvY5LRoNqbOE0cxmSfaVEylr7coA1L/MMS4G5ykgV/vQ6Xs1QnVym5VnGOu y0evhr37guI0j4BA2IM/EBWWCmszGxGoMXnyDjFUI89IzIOmfk4HuT9MSFj/19N+f8sW sm01OJk4SI1bg3QexzZpDv8krYhPD+8Ij349P7g5sUALxz9WkAZ8e+miMfbQpk8CcRPE 8vyCVa4K5L8yHaBLwKRsSswhNDs1BUO+KtpsO1ozb3WHjSIDsLSHwjE0tdN9k7UfoThZ H4VGzP8TFR0YFQnjq/Vkk9QaMYNhw1vGWTVQQBPj4Ca6opR4pY6g8idjqr2fuTpCspZ0 cRpA== MIME-Version: 1.0 X-Received: by 10.236.134.202 with SMTP id s50mr15904962yhi.4.1412790258482; Wed, 08 Oct 2014 10:44:18 -0700 (PDT) Sender: kmacybsd@gmail.com Received: by 10.170.82.197 with HTTP; Wed, 8 Oct 2014 10:44:18 -0700 (PDT) In-Reply-To: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> References: <1422065A4E115F409E22C1EC9EDAFBA46044E4@sofdc01exc02.postbank.bg> Date: Wed, 8 Oct 2014 10:44:18 -0700 X-Google-Sender-Auth: I0PJ54OHfFBefZQz6G7HCxfZiQA Message-ID: Subject: Re: ZFS API From: "K. Macy" To: "Ivailo A. Tanusheff" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "freebsd-fs@FreeBSD.org" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Oct 2014 17:44:19 -0000 On Mon, Oct 6, 2014 at 2:44 AM, Ivailo A. Tanusheff wrote: > Dear all, > > I am looking for an API and documentation about programing some actions on > C/C++ for ZFS management. > I have found an library - > /usr/src/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h but this lacks > any documentation. > I do not want to make a huge development and my skills are not great, so I > need some useful API and documentation. > Is there such thing available around? > What I need is to include some snapshot management techniques :) > > Regards, > > Ivailo Tanusheff > > Solaris' ndmpd snapshots the volumes in question prior to doing a back up. It isn't what you asked for, but it is an example of other applications using libzfs. That file works almost unmodified in the (incomplete) FreeBSD ndmpd port. https://github.com/illumos/illumos-gate/blob/master/usr/src/cmd/ndmpd/ndmp/ndmpd_zfs.c If you intend on being a frequent correspondent on public mailing lists you might consider using a webmail account to avoid issues with spam etc. Most of all, the confidentiality footers don't make a lot of sense in this context. Cheers. -K > > Disclaimer: > > This communication is confidential. If you are not the intended recipient, > you are hereby notified that any disclosure, copying, distribution or > taking any action in reliance on the contents of this information is > strictly prohibited and may be unlawful. If you have received this > communication by mistake, please notify us immediately by responding to > this email and then delete it from your system. > Eurobank Bulgaria AD is not responsible for, nor endorses, any opinion, > recommendation, conclusion, solicitation, offer or agreement or any > information contained in this communication. > Eurobank Bulgaria AD cannot accept any responsibility for the accuracy or > completeness of this message as it has been transmitted over a public > network. If you suspect that the message may have been intercepted or > amended, please call the sender. > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > From owner-freebsd-fs@FreeBSD.ORG Thu Oct 9 00:43:37 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 023888B1; Thu, 9 Oct 2014 00:43:37 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id AC1387C4; Thu, 9 Oct 2014 00:43:35 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqIEAPHYNVSDaFve/2dsb2JhbABfDoQvgwDPYwKBGwF7hAQBAQQjBFIbGAICDRkCWQYTiD6xPpRbAReBLI5kNAeCd4FUBZ5zg0WDLI1ugyNcIS+BSIECAQEB X-IronPort-AV: E=Sophos;i="5.04,681,1406606400"; d="scan'208";a="160265195" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 08 Oct 2014 20:43:34 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id E5C60B405F; Wed, 8 Oct 2014 20:43:33 -0400 (EDT) Date: Wed, 8 Oct 2014 20:43:33 -0400 (EDT) From: Rick Macklem To: Garrett Wollman Message-ID: <1713580100.60978206.1412815413928.JavaMail.root@uoguelph.ca> In-Reply-To: <21557.22365.961980.709081@khavrinen.csail.mit.edu> Subject: Re: 9.3 NFS client bug? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Oct 2014 00:43:37 -0000 Garrett Wollman wrote: > < said: > > > As far as I know, this has never worked correctly for FreeBSD. The > > unlink() invalidates the directory offset cookies and then it has > > trouble finding the next entry. > > To make the above loop work correctly for FreeBSD, it needs to be > > re-written to start at the beginning of the directory after each > > unlink(). > > How about instead we fix FreeBSD to work properly? Clearly it is not > impossible since the Linux NFS client does work. What exactly is the > issue? (Forgive me, I know very little about how VOP_READDIR works > under the hood.) > > -GAWollman > > Well, I've never looked at Linux or OpenSolaris to see how they handle these things, but here's a couple of ways I am aware of that could fix this: 1 - There is a "cookie_verifier" defined for NFS, which is a 64bit value that is supposed to change whenever the directory offset cookies are no longer valid. Implementing this requires something like: - add an attribute or new VOP_xxx() for this cookie_verfier - fix every file system to handle it --> This requires a good knowledge of the underlying file system, since it needs to change when the directory_offset_cookies are stale (I believe this is when objects are added to a directory for UFS. Have no idea for ZFS, etc.) - It needs to be stored on-disk (in the i-node or similar) since it is supposed to survive server crashes. The value for this cookie_verifier is in the readdir reply and then the client sends it in subsequent requests so that the server can reply with an error if the cookie_verifier refers to "stale" directory offset cookies. --> Unfortunately some servers haven't supported this correctly for a long time and it is difficult for clients to recover from the error. RFC-3530 strongly recommends that directory offset cookies not be allowed to become stale, but I don't know how to do this for UFS, ZFS, ... (There is still an ancient comment in the server code about the check being too strict for Solaris 2.5 clients. When was Solaris 2.5 released?;-) All in all, a mess. As such, the FreeBSD client assumes that the cookies are no longer valid when it sees the modify time on the directory change (guess what happens every time an entry is unlink'd from the directory). Unless not only the FreeBSD servers but most/all other servers (a lot of old BSD servers and I believe others are broken) are fixed, the client can't really depend on this to determine if directory offset cookies are still valid. (Can you now see why this has never been fixed?) 2 - Have readdir(3) do what fts(3) does. Read the entire directory into the user address space on the first readdir() after opendir() and then subsequent readdir() calls just return directory entries from memory (avoiding further getdirentries(2) calls that don't work correctly because of potentially stale directory offset cookies). --> This one is probably straightforward, but may eat a lot of address space for apps. that opendir(), readdir() a lot of large directories. This might be fine or it might break a bunch of apps that run out of address space and do more harm than the broken case of removing entries in a readdir() loop (which can be easily coded around)? If someone else knows of a better way to fix this (maybe what Linux or Solaris does) please post, because I don't think either 1 or 2 above is a good plan. rick ps: This is my recollection of the problem, but I haven't worked on it in quite a while. From owner-freebsd-fs@FreeBSD.ORG Thu Oct 9 01:10:55 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 18EAFEC9 for ; Thu, 9 Oct 2014 01:10:55 +0000 (UTC) Received: from mail-la0-x22a.google.com (mail-la0-x22a.google.com [IPv6:2a00:1450:4010:c03::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8FF1DA57 for ; Thu, 9 Oct 2014 01:10:54 +0000 (UTC) Received: by mail-la0-f42.google.com with SMTP id mk6so212456lab.29 for ; Wed, 08 Oct 2014 18:10:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=EAnyspPFcDv96klXIY/tS0TW6eUA4V/Qa/Qwt2xOF7o=; b=dqGlZM74rO84XwFPjUGLChD126MAotmjGFXGcDq+QFxXoI6AnjyRG0VquVpqXWyfRQ zUYWWeD0wgnklwq7XpkJXOQsdoLnfJuSgLh56IJlFW7JilQCbdQvtGOl0wzu8Kc/orfF BlKHk7EablVmWQVgqbvjfh1csc5uqFjPexB7g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=EAnyspPFcDv96klXIY/tS0TW6eUA4V/Qa/Qwt2xOF7o=; b=V2+TIKrdrgyH7MAsDdCXdkEp+yoEkCo4FN6JQTsJfGZYVeMn6bHpf5/6emh+WPC5uU tgYE83NwTNd9jxtLF4ilc3N6Wd6m8oFfN2njQ3evG31anw9XIW0ZhjAbi/hh0rwokvyw 0IfIdzXWmEupGIBjKgxlFmXjQzyuwAZ621Ox53pgdvjHyULCclZew6n0DTjOmDgHTAc0 EHQVr8rh8VvCcoeY8pheKmzaSCouFAwGsjEWYKGSK/J7pHl8hfb7Mgls90J7V4pSGWMG W4/F6u4p7Iwm7bgsR3wP5N5NNEAJ94xxw6mLBKXp6DA2Sxle0+BVH8/oZAfoAMCVlYNR ONnA== X-Gm-Message-State: ALoCoQnhmEzR00SZY4+qbFt8dQUXORidbPHu9GxeA0XzsBGDXeCWoFkBsugPQxGUFe7b5DON0dWv MIME-Version: 1.0 X-Received: by 10.152.19.37 with SMTP id b5mr11506641lae.43.1412817052268; Wed, 08 Oct 2014 18:10:52 -0700 (PDT) Received: by 10.25.155.132 with HTTP; Wed, 8 Oct 2014 18:10:52 -0700 (PDT) In-Reply-To: <1422065A4E115F409E22C1EC9EDAFBA4604F7E@sofdc01exc02.postbank.bg> References: <1422065A4E115F409E22C1EC9EDAFBA4604F7E@sofdc01exc02.postbank.bg> Date: Wed, 8 Oct 2014 18:10:52 -0700 Message-ID: Subject: Re: [OpenZFS Developer] ZFS API From: Matthew Ahrens To: "Ivailo A. Tanusheff" Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 Cc: "developer@open-zfs.org" , freebsd-fs X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Oct 2014 01:10:55 -0000 The CLI is the supported, stable interface for ZFS administrative operations. libzfs_core is on its way to being a stable API, but is incomplete. Snapshot creation and destruction are there, though. See libzfs_core.c. There's some more info about its future direction here: http://blog.delphix.com/matt/2012/01/17/the-future-of-libzfs/ libzfs will probably never be a stable API, but it is complete, and plenty of people have built tools around it. But you must be very wary of its sharp edges. --matt On Wed, Oct 8, 2014 at 12:40 AM, Ivailo A. Tanusheff wrote: > Dear all, > > > > I am looking for an API and documentation about programing some actions on > C/C++ for ZFS management. > > I have found an library on FreeBSD - > /usr/src/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h but this lacks > any documentation. > > I do not want to make a huge development and my skills are not great, so I > need some useful API and documentation. > > Is there such thing available around? > > What I need is to include some snapshot management techniques in my > daemons :) > > > > Regards, > > > > Ivailo Tanusheff > > > > Disclaimer: This communication is confidential. If you are not the > intended recipient, you are hereby notified that any disclosure, copying, > distribution or taking any action in reliance on the contents of this > information is strictly prohibited and may be unlawful. If you have > received this communication by mistake, please notify us immediately by > responding to this email and then delete it from your system. Eurobank > Bulgaria AD is not responsible for, nor endorses, any opinion, > recommendation, conclusion, solicitation, offer or agreement or any > information contained in this communication. Eurobank Bulgaria AD cannot > accept any responsibility for the accuracy or completeness of this message > as it has been transmitted over a public network. If you suspect that the > message may have been intercepted or amended, please call the sender. > > _______________________________________________ > developer mailing list > developer@open-zfs.org > http://lists.open-zfs.org/mailman/listinfo/developer > > From owner-freebsd-fs@FreeBSD.ORG Thu Oct 9 07:41:39 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5B329D03 for ; Thu, 9 Oct 2014 07:41:39 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 428B830C for ; Thu, 9 Oct 2014 07:41:39 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s997fclq046270 for ; Thu, 9 Oct 2014 07:41:38 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 159251] [zfs] [request]: add FLETCHER4 as DEDUP hash option Date: Thu, 09 Oct 2014 07:41:38 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: vermaden@interia.pl X-Bugzilla-Status: In Discussion X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Oct 2014 07:41:39 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=159251 --- Comment #3 from vermaden@interia.pl --- > not relevant anymore : fletcher4 is the default (since r259477). Can be closed ? Check the message again. Its not about: # zfs set checksum=fletcher4 Its about: # zfs set checksum=fletcher4,verify ... which still does not work: # zfs set checksum=fletcher4,verify sys/TEST cannot set property for 'sys/TEST': 'checksum' must be one of 'on | off | fletcher2 | fletcher4 | sha256' Regards, vermaden -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Thu Oct 9 09:22:54 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6AAD430F for ; Thu, 9 Oct 2014 09:22:54 +0000 (UTC) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 28A8AEFD for ; Thu, 9 Oct 2014 09:22:53 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop04.sare.net (Postfix) with ESMTPSA id 14C559DC4F8 for ; Thu, 9 Oct 2014 11:22:50 +0200 (CEST) From: Borja Marcos Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Subject: zfs send -R not replicating holds Date: Thu, 9 Oct 2014 11:22:45 +0200 Message-Id: <55079EE9-CA03-4FCD-AA07-2087E31673E9@sarenet.es> To: "freebsd-fs@FreeBSD.org Filesystems" Mime-Version: 1.0 (Apple Message framework v1283) X-Mailer: Apple Mail (2.1283) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Oct 2014 09:22:54 -0000 Hello, I've noticed that a zfs send -R does not actually replicate holds which = is, in my opinion, surprising in the wrong way, as the=20 manual for "zfs send" says: "-R Generate a replication stream package, which will = replicate the specified filesystem, and all descendent file = systems, up to the named snapshot. When received, all properties, = snap- shots, descendent file systems, and clones are = preserved." Reading that I would assume that a hold is a kind of property (unless of = course we limit the definition of "property" strictly to the properties handled by "zfs = get/set").=20 I am playing with ZFS dataset replication and holds seem like an ideal = way to mark snapshots for preservation (for example, in a "time machine" style). In that case, = it would be great if the=20 process of replicating a snapshot to a backup server and marking it with = a hold was atomic rather than doing the replication and applying the relevant holds later. Is there any provision for this, is it an oversight or is there any = reason I am missing not to do it that way? (apart from possible zfs send stream format incompatibilities) Borja. From owner-freebsd-fs@FreeBSD.ORG Thu Oct 9 12:36:44 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 94FC7F65 for ; Thu, 9 Oct 2014 12:36:44 +0000 (UTC) Received: from frv189.fwdcdn.com (frv189.fwdcdn.com [212.42.77.189]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5594D7B3 for ; Thu, 9 Oct 2014 12:36:44 +0000 (UTC) Received: from [10.10.1.23] (helo=frv199.fwdcdn.com) by frv189.fwdcdn.com with esmtp ID 1XcCxI-000AcQ-OE for freebsd-fs@freebsd.org; Thu, 09 Oct 2014 15:36:28 +0300 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=ukr.net; s=ffe; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-Id:To:Subject:From:Date; bh=3stpP77wWO1GkckNndRnp2yxDgjsUvquQpoeNbhsGxg=; b=YYw4ZZTJ5ZbDRtmc217L2lrFHQcvfzHLbr3YRA2kpsdivg8FWwF9tDQ7NW8E/Gdy+x+M+nka8UeXC0TqB4x277+TvKGzNX602QQsCMMjFaZeuiUtuPeClnCDLrmg1PjRBDikoiZ1tU7soBMql5FL2lJLYnJJs/knsNxYCgo2SPI=; Received: from [10.10.10.38] (helo=frv38.fwdcdn.com) by frv199.fwdcdn.com with smtp ID 1XcCx6-0009qX-OL for freebsd-fs@freebsd.org; Thu, 09 Oct 2014 15:36:16 +0300 Date: Thu, 09 Oct 2014 15:36:16 +0300 From: Paul Subject: Question about metadata in ARC To: freebsd-fs@freebsd.org X-Mailer: mail.ukr.net 5.0 Message-Id: <1412858175.581495697.syuqcm4o@frv38.fwdcdn.com> MIME-Version: 1.0 Received: from devgs@ukr.net by frv38.fwdcdn.com; Thu, 09 Oct 2014 15:36:16 +0300 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: binary Content-Disposition: inline X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Oct 2014 12:36:44 -0000 Cheers. Recently on production servers we have discovered strange ARC behavior. Up until now we weren't using readdir() too ofter, only on occasion. Now some of our daemons periodically call readdir() on many thousands of folders. On average a time between two readdir() calls on same directory is small: ~10-15 seconds. But for some reason 99% of the calls miss the cache. Directories never have more than 3 files in them. One of them, the lock file, is never removed and stays on file system. Other one or two files almost immediately unliked, after being created. Scan of directory using readdir(), before unlinking those files always takes roughly 10-12 milliseconds. I have figured it's because directory metadata is getting pumped out of ARC. The question is why so quickly? Why is on the other hand, metadata needed for stat() stays in cache much much longer. Literally hours. stat()-ing file once in few hours always hits the cache. And we stat() millions of dirrefent files per day. So I imagine there is two kinds of metadata: directory data blocks (hash table, according to wiki) and data blocks where stat()'s metadata is stored. Why is stat() metadata lives so much longer than directory metadata? Or better say, why is directory metadata rejected so quickly? Is there a way to configure it otherwise? I want to show you my little test case. Environment: The test is performed on production server with 128G RAM and max ARC size set to 90G We are running FreeBSD 11.0-CURRENT #3 r260625 Stats from top are: ARC: 86G Total, 1884M MFU, 77G MRU, 54M Anon, 2160M Header, 5149M Other Average disk busyness is 40% 6 CPU cores with hypertherading (12 virtual cores) are 25% busy on average Setup: To setup test case I have created test directory and spawned 5000 files using: # for a in {1..5000}; do touch ${RANDOM}_test_file_name_${RANDOM}_$a; done And saved their names to temporary file. # ls > /tmp/testfiles Testing: Then I waited an hour (for cached metadata to expire from cache) and did two things: 1) scan directory using plain ls # time ls >| /dev/null; ls -G >| /dev/null 0,03s user 0,05s system 5% cpu 1,472 total 2) stat files by their names, that were retrieved from temporary file created earlier # time stat `cat /tmp/lstest`>| /dev/null stat `cat /tmp/lstest` >| /dev/null 0,16s user 0,19s system 99% cpu 1,481 total Then I waited one minute and repeated two above actions: 1) # time ls >| /dev/null; ls -G >| /dev/null 0,03s user 0,04s system 5% cpu 1,327 total 2) # time stat `cat /tmp/lstest`>| /dev/null stat `cat /tmp/lstest` >| /dev/null 0,16s user 0,19s system 99% cpu 0,351 total As you can see in case (1) majority of time is waiting for disk ie cache miss. In case (2) CPU time is in majority. I did many more experiments and came to conclusion that directory metadata is removed from cache almost immediately. Sometimes it takes 5 seconds, sometimes it's even 1 second, rarely it's 10 or more seconds. While on the other hand when I ran stat `cat /tmp/lstest` hours later I still had total time around 350ms. So, how can I configure ZFS to reduce cache misses when reading directories? There is another problem I think related to my issue. Periodical unlink() of non-existent files also takes 10 to 12 milliseconds. While stat() + unlink() (if file exists) always takes no more than tens of microseconds! Paul, Thanks. From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 01:27:34 2014 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D39D5446; Fri, 10 Oct 2014 01:27:34 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 94DB0A4B; Fri, 10 Oct 2014 01:27:34 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s9A1RRRF078213 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 10 Oct 2014 01:27:32 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s9A1RRNH078212; Fri, 10 Oct 2014 01:27:27 GMT (envelope-from swills) Date: Fri, 10 Oct 2014 01:27:27 +0000 From: Steve Wills To: Andriy Gapon Subject: Re: zfs hang Message-ID: <20141010012724.GD79158@mouf.net> References: <20141008004045.GA24762__48659.9047123038$1412728878$gmane$org@mouf.net> <5434D1CE.8010801@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <5434D1CE.8010801@FreeBSD.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Fri, 10 Oct 2014 01:27:33 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=HEADER_FROM_DIFFERENT_DOMAINS autolearn=unavailable autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mouf.net X-Virus-Scanned: clamav-milter 0.98.3 at mouf.net X-Virus-Status: Clean Cc: current@freebsd.org, fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 01:27:34 -0000 On Wed, Oct 08, 2014 at 08:55:26AM +0300, Andriy Gapon wrote: > On 08/10/2014 03:40, Steve Wills wrote: > > Hi, > > > > Not sure which thread this belongs to, but I have a zfs hang on one of my boxes > > running r272152. Running procstat -kka looks like: > > > > http://pastebin.com/szZZP8Tf > > > > My zpool commands seem to be hung in spa_errlog_lock while others are hung in > > zfs_lookup. Suggestions? > > There are several threads in zio_wait. If this is their permanent state then > there is some problem with I/O somewhere below ZFS. Thanks for the feedback. It seems one of my disks is dying, I rebooted and it came up OK, but today I got: panic: I/O to pool 'rpool' appears to be hung on vdev guid ..... at '/dev/ada0p3' I have screenshots and backtrace if anyone is interested. Dying drives shouldn't cause panic, right? Steve From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 01:35:18 2014 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B3D6263A; Fri, 10 Oct 2014 01:35:18 +0000 (UTC) Received: from smtp1.multiplay.co.uk (smtp1.multiplay.co.uk [85.236.96.35]) by mx1.freebsd.org (Postfix) with ESMTP id 77A04B2B; Fri, 10 Oct 2014 01:35:18 +0000 (UTC) Received: by smtp1.multiplay.co.uk (Postfix, from userid 65534) id 389C420E709A4; Fri, 10 Oct 2014 01:35:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.multiplay.co.uk X-Spam-Level: ** X-Spam-Status: No, score=2.2 required=8.0 tests=AWL,BAYES_00,DOS_OE_TO_MX, FSL_HELO_NON_FQDN_1,RDNS_DYNAMIC,STOX_REPLY_TYPE autolearn=no version=3.3.1 Received: from r2d2 (82-69-141-170.dsl.in-addr.zen.co.uk [82.69.141.170]) by smtp1.multiplay.co.uk (Postfix) with ESMTPS id B797620E709A2; Fri, 10 Oct 2014 01:35:14 +0000 (UTC) Message-ID: From: "Steven Hartland" To: "Steve Wills" , "Andriy Gapon" References: <20141008004045.GA24762__48659.9047123038$1412728878$gmane$org@mouf.net> <5434D1CE.8010801@FreeBSD.org> <20141010012724.GD79158@mouf.net> Subject: Re: zfs hang Date: Fri, 10 Oct 2014 02:35:14 +0100 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 Cc: current@freebsd.org, fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 01:35:18 -0000 ----- Original Message ----- From: "Steve Wills" To: "Andriy Gapon" Cc: ; Sent: Friday, October 10, 2014 2:27 AM Subject: Re: zfs hang > On Wed, Oct 08, 2014 at 08:55:26AM +0300, Andriy Gapon wrote: >> On 08/10/2014 03:40, Steve Wills wrote: >> > Hi, >> > >> > Not sure which thread this belongs to, but I have a zfs hang on one of my boxes >> > running r272152. Running procstat -kka looks like: >> > >> > http://pastebin.com/szZZP8Tf >> > >> > My zpool commands seem to be hung in spa_errlog_lock while others are hung in >> > zfs_lookup. Suggestions? >> >> There are several threads in zio_wait. If this is their permanent state then >> there is some problem with I/O somewhere below ZFS. > > Thanks for the feedback. It seems one of my disks is dying, I rebooted and it > came up OK, but today I got: > > panic: I/O to pool 'rpool' appears to be hung on vdev guid ..... at '/dev/ada0p3' > > I have screenshots and backtrace if anyone is interested. Dying drives > shouldn't cause panic, right? Its the deadman timer kicking in so yes, thats expected. The following sysctls control this behaviour if you want to try and recover: vfs.zfs.deadman_synctime_ms: 1000000 vfs.zfs.deadman_checktime_ms: 5000 vfs.zfs.deadman_enabled: 1 Regards Steve From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 02:03:08 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E782B8C1 for ; Fri, 10 Oct 2014 02:03:08 +0000 (UTC) Received: from mail-la0-x234.google.com (mail-la0-x234.google.com [IPv6:2a00:1450:4010:c03::234]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 74D8CD71 for ; Fri, 10 Oct 2014 02:03:08 +0000 (UTC) Received: by mail-la0-f52.google.com with SMTP id hz20so2362037lab.11 for ; Thu, 09 Oct 2014 19:03:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=JRk/QYhFIkLTR0v9PIk5Gjs7iqvj6g4S2b5Xm61S5Sg=; b=ajOPg+E8KEt4Gr6ikAwTnUC4+zgZQ5a3QxoMVu0MK1GwTXNDq+ayf1qAKYWGdrf9ie qGo951C/0PDZnBAJNMub0mB/kIUaLw5Era8PxxkINY9QuqDfWGbzKM7WEfeuIuW3nwX1 DAc2URnO7XqFUtjEPJqAHlmL6dPGnwgBFnDLDssrImTulZE2h47BGmJajvf3CnGz/7cH 6fLWD85doLgmBsU0lSUuJ4wHL9bFm4aE0ZS3fKOgMqoLe5osVJ/SMuSCYvurZB5HuM1r /3hLqe2gJ/F1Hpum7zfdj7MKIJKwhNnAfLBZA3FCLMo2/9f4nAWJpUUsLo/ZQVRj/JWL /gPg== MIME-Version: 1.0 X-Received: by 10.152.163.66 with SMTP id yg2mr1361413lab.38.1412906585979; Thu, 09 Oct 2014 19:03:05 -0700 (PDT) Received: by 10.25.87.72 with HTTP; Thu, 9 Oct 2014 19:03:05 -0700 (PDT) Date: Thu, 9 Oct 2014 19:03:05 -0700 Message-ID: Subject: fsck / GEOM / Editing files in mirror partitions separately From: Stephan Wehner To: freebsd-fs@freebsd.org Content-Type: text/plain; charset=UTF-8 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 02:03:09 -0000 Hello there, I have a FreeBSD 10.0-RELEASE-p7 system with system specifics: $ mount /dev/mirror/gm0s1a on / (ufs, local, journaled soft-updates) devfs on /dev (devfs, local, multilabel) /dev/mirror/gm0s1b on /home (ufs, local, journaled soft-updates) procfs on /proc (procfs, local) $ gmirror status Name Status Components mirror/gm0 COMPLETE ada0 (ACTIVE) ada1 (ACTIVE) $ gpart show => 63 3907029104 mirror/gm0 MBR (1.8T) 63 3907029042 1 freebsd [active] (1.8T) 3907029105 62 - free - (31K) => 0 3907029042 mirror/gm0s1 BSD (1.8T) 0 2456848384 1 freebsd-ufs (1.1T) 2456848384 1433600000 2 freebsd-ufs (684G) 3890448384 16580658 4 freebsd-swap (7.9G) Computer was rebooted by simply turning it off. On reboot, I got message "error aborting boot enter full pathname or shell or return for /bin/sh" First question: Is it normal that the filesystem will not survive a power loss, so that manual intervention is needed? I thought UFS would be more robust. Now I did something that may have been a bad idea. I ran fsck on both partitions of the mirror separately, I think they were called ad4s1 and ad6s1. For both it reported some problems to fix, and I did Y until they were declared "CLEAN." Then I mounted each partition separately, and made the same changes to a single text file (the file was identical in each partition). Then rebooted, and now the system is running fine. Second question. Is that mirror in good shape? Is there a way to test? Third question. Does one even run fsck on partitions that are then controlled by geom? This shows in dmesg: GEOM_MIRROR: Device mirror/gm0 launched (1/2). GEOM_MIRROR: Device gm0: rebuilding provider ada0. GEOM_MIRROR: Device gm0: rebuilding provider ada0 finished. GEOM_MIRROR: Device mirror/gm0 launched (2/2). Thanks, Stephan From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 04:02:37 2014 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5957891C; Fri, 10 Oct 2014 04:02:37 +0000 (UTC) Received: from mouf.net (mouf.net [IPv6:2607:fc50:0:4400:216:3eff:fe69:33b3]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "mouf.net", Issuer "mouf.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id F2BE0B0B; Fri, 10 Oct 2014 04:02:36 +0000 (UTC) Received: from mouf.net (swills@mouf [199.48.129.64]) by mouf.net (8.14.5/8.14.5) with ESMTP id s9A42TmG080514 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 10 Oct 2014 04:02:34 GMT (envelope-from swills@mouf.net) Received: (from swills@localhost) by mouf.net (8.14.5/8.14.5/Submit) id s9A42TIP080513; Fri, 10 Oct 2014 04:02:29 GMT (envelope-from swills) Date: Fri, 10 Oct 2014 04:02:29 +0000 From: Steve Wills To: Steven Hartland Subject: Re: zfs hang Message-ID: <20141010040228.GI79158@mouf.net> References: <20141008004045.GA24762__48659.9047123038$1412728878$gmane$org@mouf.net> <5434D1CE.8010801@FreeBSD.org> <20141010012724.GD79158@mouf.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.23 (2014-03-12) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mouf.net [199.48.129.64]); Fri, 10 Oct 2014 04:02:34 +0000 (UTC) X-Spam-Status: No, score=0.0 required=4.5 tests=HEADER_FROM_DIFFERENT_DOMAINS autolearn=unavailable autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mouf.net X-Virus-Scanned: clamav-milter 0.98.3 at mouf.net X-Virus-Status: Clean Cc: fs@freebsd.org, current@freebsd.org, Andriy Gapon X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 04:02:37 -0000 On Fri, Oct 10, 2014 at 02:35:14AM +0100, Steven Hartland wrote: > > ----- Original Message ----- > From: "Steve Wills" > To: "Andriy Gapon" > Cc: ; > Sent: Friday, October 10, 2014 2:27 AM > Subject: Re: zfs hang > > > > On Wed, Oct 08, 2014 at 08:55:26AM +0300, Andriy Gapon wrote: > >> On 08/10/2014 03:40, Steve Wills wrote: > >> > Hi, > >> > > >> > Not sure which thread this belongs to, but I have a zfs hang on one of my boxes > >> > running r272152. Running procstat -kka looks like: > >> > > >> > http://pastebin.com/szZZP8Tf > >> > > >> > My zpool commands seem to be hung in spa_errlog_lock while others are hung in > >> > zfs_lookup. Suggestions? > >> > >> There are several threads in zio_wait. If this is their permanent state then > >> there is some problem with I/O somewhere below ZFS. > > > > Thanks for the feedback. It seems one of my disks is dying, I rebooted and it > > came up OK, but today I got: > > > > panic: I/O to pool 'rpool' appears to be hung on vdev guid ..... at '/dev/ada0p3' > > > > I have screenshots and backtrace if anyone is interested. Dying drives > > shouldn't cause panic, right? > > Its the deadman timer kicking in so yes, thats expected. > > The following sysctls control this behaviour if you want to try and recover: > vfs.zfs.deadman_synctime_ms: 1000000 > vfs.zfs.deadman_checktime_ms: 5000 > vfs.zfs.deadman_enabled: 1 Ah, ok. This pool has two disks, mirrored. I think one of them is dying, the BIOS gives a SMART error on startup, but it still uses the disk fine. From what I read of the zfs deadman design, it's for when the controller is acting up. So I'm confused. Maybe this means both disks are dying? Steve From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 04:44:09 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 3CF55CB4 for ; Fri, 10 Oct 2014 04:44:09 +0000 (UTC) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 1AC66E4E for ; Fri, 10 Oct 2014 04:44:08 +0000 (UTC) Received: from Julian-MBP3.local (50-196-156-133-static.hfc.comcastbusiness.net [50.196.156.133]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id s9A4i0jE011963 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Thu, 9 Oct 2014 21:44:05 -0700 (PDT) (envelope-from julian@freebsd.org) Message-ID: <5437640A.6020000@freebsd.org> Date: Fri, 10 Oct 2014 12:43:54 +0800 From: Julian Elischer User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: Stephan Wehner , freebsd-fs@freebsd.org Subject: Re: fsck / GEOM / Editing files in mirror partitions separately References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 04:44:09 -0000 On 10/10/14, 10:03 AM, Stephan Wehner wrote: > Hello there, > > I have a FreeBSD 10.0-RELEASE-p7 system with system specifics: > > $ mount > /dev/mirror/gm0s1a on / (ufs, local, journaled soft-updates) > devfs on /dev (devfs, local, multilabel) > /dev/mirror/gm0s1b on /home (ufs, local, journaled soft-updates) > procfs on /proc (procfs, local) > > $ gmirror status > Name Status Components > mirror/gm0 COMPLETE ada0 (ACTIVE) > ada1 (ACTIVE) > > $ gpart show > => 63 3907029104 mirror/gm0 MBR (1.8T) > 63 3907029042 1 freebsd [active] (1.8T) > 3907029105 62 - free - (31K) > > => 0 3907029042 mirror/gm0s1 BSD (1.8T) > 0 2456848384 1 freebsd-ufs (1.1T) > 2456848384 1433600000 2 freebsd-ufs (684G) > 3890448384 16580658 4 freebsd-swap (7.9G) > > Computer was rebooted by simply turning it off. On reboot, I got > message "error aborting boot enter full pathname or shell or return > for /bin/sh" > > First question: Is it normal that the filesystem will not survive a > power loss, so that manual intervention is needed? > I thought UFS would be more robust. > > Now I did something that may have been a bad idea. > > I ran fsck on both partitions of the mirror separately, I think they > were called ad4s1 and ad6s1. For both it reported some problems to > fix, and I did Y until they were declared "CLEAN." > > Then I mounted each partition separately, and made the same changes to > a single text file (the file was identical in each partition). but you PROBABLY have different metadata blocks i.e. the timestamps will be different, and inodes may have been allocated in different places etc. > Then rebooted, and now the system is running fine. > > Second question. Is that mirror in good shape? Is there a way to test? I doubt it is in good shape though it may THINK it's in good shape. to test.. boot of a third device (USB?) and compare the drives. you will probably find differences.. (your choice of how to compare the physical devices..) > > Third question. Does one even run fsck on partitions that are then > controlled by geom? > > This shows in dmesg: > > GEOM_MIRROR: Device mirror/gm0 launched (1/2). > GEOM_MIRROR: Device gm0: rebuilding provider ada0. > GEOM_MIRROR: Device gm0: rebuilding provider ada0 finished. > GEOM_MIRROR: Device mirror/gm0 launched (2/2). how long did it take the do the above? might it have copied one drive to another? if so then you are ok.. (I do not use gmirror myself) > > Thanks, > > Stephan > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 06:30:03 2014 Return-Path: Delivered-To: fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 47F62BAE; Fri, 10 Oct 2014 06:30:03 +0000 (UTC) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 1F5B29E1; Fri, 10 Oct 2014 06:30:01 +0000 (UTC) Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua [212.40.38.100]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id JAA26255; Fri, 10 Oct 2014 09:29:59 +0300 (EEST) (envelope-from avg@FreeBSD.org) Received: from localhost ([127.0.0.1]) by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD)) id 1XcTiA-0009I6-Qk; Fri, 10 Oct 2014 09:29:58 +0300 Message-ID: <54377C99.1000406@FreeBSD.org> Date: Fri, 10 Oct 2014 09:28:41 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Thunderbird/31.1.2 MIME-Version: 1.0 To: Steve Wills Subject: Re: zfs hang References: <20141008004045.GA24762__48659.9047123038$1412728878$gmane$org@mouf.net> <5434D1CE.8010801@FreeBSD.org> <20141010012724.GD79158@mouf.net> In-Reply-To: <20141010012724.GD79158@mouf.net> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Cc: current@FreeBSD.org, fs@FreeBSD.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 06:30:03 -0000 On 10/10/2014 04:27, Steve Wills wrote: > Dying drives > shouldn't cause panic, right? There can be different shades of dying. Returning errors is one thing, hanging is a different thing. There is a good reason why systems panics in that case. I am surprised though that the hang was not detected in the lower layers like ahci driver or CAM. -- Andriy Gapon From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 08:27:16 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C9855E18 for ; Fri, 10 Oct 2014 08:27:16 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B0FBD786 for ; Fri, 10 Oct 2014 08:27:16 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s9A8RGIR096248 for ; Fri, 10 Oct 2014 08:27:16 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 159251] [zfs] [request]: add FLETCHER4 as DEDUP hash option Date: Fri, 10 Oct 2014 08:27:15 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: delphij@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: freebsd-fs@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_status cc resolution Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 08:27:16 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=159251 Xin LI changed: What |Removed |Added ---------------------------------------------------------------------------- Status|In Discussion |Issue Resolved CC| |delphij@FreeBSD.org Resolution|--- |Feature Proposal Rejected --- Comment #4 from Xin LI --- This feature is intentionally removed in OpenSolaris build 128 because fletcher4 is not endianness safe and caused problems for cross-endianness. On the other hand the collision rate is fairly high, making it not suitable as a dedup checksum. We may add some other faster checksum algorithms in the future but it's unlikely that fletcher4 will ever be added back as a dedup checksum. -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 08:27:34 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6B0F0E88 for ; Fri, 10 Oct 2014 08:27:34 +0000 (UTC) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 52D6878D for ; Fri, 10 Oct 2014 08:27:34 +0000 (UTC) Received: from bugs.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.14.9/8.14.9) with ESMTP id s9A8RYFE096415 for ; Fri, 10 Oct 2014 08:27:34 GMT (envelope-from bugzilla-noreply@freebsd.org) From: bugzilla-noreply@freebsd.org To: freebsd-fs@FreeBSD.org Subject: [Bug 159251] [zfs] [request]: add FLETCHER4 as DEDUP hash option Date: Fri, 10 Oct 2014 08:27:34 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: 8.2-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Only Me X-Bugzilla-Who: delphij@FreeBSD.org X-Bugzilla-Status: Issue Resolved X-Bugzilla-Priority: Normal X-Bugzilla-Assigned-To: delphij@FreeBSD.org X-Bugzilla-Target-Milestone: --- X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: assigned_to Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 08:27:34 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=159251 Xin LI changed: What |Removed |Added ---------------------------------------------------------------------------- Assignee|freebsd-fs@FreeBSD.org |delphij@FreeBSD.org -- You are receiving this mail because: You are the assignee for the bug. From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 08:39:28 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 91EAC19D for ; Fri, 10 Oct 2014 08:39:28 +0000 (UTC) Received: from smtp.unix-experience.fr (62-210-206-43.rev.poneytelecom.eu [62.210.206.43]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 26B8E895 for ; Fri, 10 Oct 2014 08:39:26 +0000 (UTC) Received: from smtp.unix-experience.fr (unknown [192.168.200.21]) by smtp.unix-experience.fr (Postfix) with ESMTP id 5A6A8F8BA for ; Fri, 10 Oct 2014 08:39:19 +0000 (UTC) X-Virus-Scanned: scanned by unix-experience.fr Received: from smtp.unix-experience.fr ([192.168.200.21]) by smtp.unix-experience.fr (smtp.unix-experience.fr [192.168.200.21]) (amavisd-new, port 10024) with ESMTP id YBShxrZA9Jse for ; Fri, 10 Oct 2014 08:39:17 +0000 (UTC) Received: from mail.unix-experience.fr (unknown [192.168.200.1]) by smtp.unix-experience.fr (Postfix) with ESMTPSA id 4BF8BF89D for ; Fri, 10 Oct 2014 08:39:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=unix-experience.fr; s=uxselect; t=1412930357; bh=fIGKFf2FpJSapA7W2D+XOfUK95vQOcd0ceu3dLaV81E=; h=Date:From:Subject:To; b=laVogqWGLhsVpXqmlXG2i90CfKBb99Q9e2mumuCo1DUagCsVyeA6Ql2T4eabsJHPG +1kCAAP90FVbj6ecnPWgnwKdhl7It637VXslFI8dWuxUplGNn8KBtguzvE3QAY6A+I Pkv2wRUphztOvfzVk8UBk+S7iWSHecktd5BiHJGY= Mime-Version: 1.0 Date: Fri, 10 Oct 2014 08:39:16 +0000 Message-ID: <93722b83a95e50f3fae349ecc0cd254d@mail.unix-experience.fr> X-Mailer: RainLoop/1.6.9.161 From: "=?utf-8?B?TG/Dr2MgQmxvdA==?=" Subject: NFSv4 nobody issue To: freebsd-fs@freebsd.org Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: Quoted-Printable X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 08:39:28 -0000 Hello @freebsd-fs,=0A i'm trying to do jail hosting over NFSv4 with ezjai= l and i'm experimenting an issue that i can't resolve. When i extract bas= e.txz (with ezjail) or i set nobody user on a file, i have this error:=0A= =0A chown nobody:nobody /usr/jails/fulljail/mnt/=0A No name and/or group = mapping for uid,gid:(65534,65534)=0A chown: /usr/jails/fulljail/mnt/: Ope= ration not permitted=0A=0A No problem if i set:=0A chown mysql:nobody /us= r/jails/fulljail/mnt/=0A=0A Problem appears on all files.=0A=0A On my ZFS= +NFSv4 server i do a dataset, exported in NFS=0A=0A /etc/exports:=0A V4: = /=0A=0A zfs get sharenfs pool/jails:=0A -network=3D10.99.99.0 -mask=3D255= .255.255.0 -maproot=3Droot=0A=0A nfsuserd and nfsv4_server_enable=3DYES o= n both client and server, plus nfsbcd on client.=0A=0A On the client here= is the fstab entry=0A 10.99.99.99:/pool/jails /usr/jails nfs rw,nfsv4 0 = 0=0A=0A What i'm doing wrong ?=0A=0A Thanks in advance=0A Regards,=0A=0A = Lo=C3=AFc Blot,=0A UNIX Systems, Network and Security Engineer=0A http://= www.unix-experience.fr From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 11:51:48 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A40DBBCE for ; Fri, 10 Oct 2014 11:51:48 +0000 (UTC) Received: from esa-annu.net.uoguelph.ca (esa-annu.mail.uoguelph.ca [131.104.91.36]) by mx1.freebsd.org (Postfix) with ESMTP id 2AC87D15 for ; Fri, 10 Oct 2014 11:51:47 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqwEAHrHN1SDaFve/2dsb2JhbABgg2FYBIMCyEEKhnlUAoEeAXuEAwEBAQMBAQEBIAQnIAsFFhgCAg0HEgIpAQkmBggHBAEcBIgVCA2tZJUHAQEBAQEBAQMBAQEBAQEBAQEZgSyORwEBGzQHGIJfgVQFljuEDIQyPJApg36EEyEvB4EIOYECAQEB X-IronPort-AV: E=Sophos;i="5.04,691,1406606400"; d="scan'208";a="160524552" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-annu.net.uoguelph.ca with ESMTP; 10 Oct 2014 07:51:40 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id B843DB40AC; Fri, 10 Oct 2014 07:51:40 -0400 (EDT) Date: Fri, 10 Oct 2014 07:51:40 -0400 (EDT) From: Rick Macklem To: =?utf-8?B?TG/Dr2M=?= Blot Message-ID: <1738545148.62071361.1412941900737.JavaMail.root@uoguelph.ca> In-Reply-To: <93722b83a95e50f3fae349ecc0cd254d@mail.unix-experience.fr> Subject: Re: NFSv4 nobody issue MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.203] X-Mailer: Zimbra 7.2.6_GA_2926 (ZimbraWebClient - FF3.0 (Win)/7.2.6_GA_2926) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 11:51:48 -0000 Loic Blot wrote: > Hello @freebsd-fs, > i'm trying to do jail hosting over NFSv4 with ezjail and i'm > experimenting an issue that i can't resolve. When i extract > base.txz (with ezjail) or i set nobody user on a file, i have this > error: >=20 > chown nobody:nobody /usr/jails/fulljail/mnt/ > No name and/or group mapping for uid,gid:(65534,65534) > chown: /usr/jails/fulljail/mnt/: Operation not permitted >=20 > No problem if i set: > chown mysql:nobody /usr/jails/fulljail/mnt/ >=20 > Problem appears on all files. >=20 Do you have a user by the name of "nobody" in your password database? (NFSv4 uses names and not numbers on the wire, so no name-->no mapping and chown can't be done.) rick > On my ZFS+NFSv4 server i do a dataset, exported in NFS >=20 > /etc/exports: > V4: / >=20 > zfs get sharenfs pool/jails: > -network=3D10.99.99.0 -mask=3D255.255.255.0 -maproot=3Droot >=20 > nfsuserd and nfsv4_server_enable=3DYES on both client and server, plus > nfsbcd on client. >=20 > On the client here is the fstab entry > 10.99.99.99:/pool/jails /usr/jails nfs rw,nfsv4 0 0 >=20 > What i'm doing wrong ? >=20 > Thanks in advance > Regards, >=20 > Lo=C3=AFc Blot, > UNIX Systems, Network and Security Engineer > http://www.unix-experience.fr > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" From owner-freebsd-fs@FreeBSD.ORG Fri Oct 10 12:03:11 2014 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id B66C1127 for ; Fri, 10 Oct 2014 12:03:11 +0000 (UTC) Received: from wonkity.com (wonkity.com [67.158.26.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "wonkity.com", Issuer "wonkity.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 61E31E55 for ; Fri, 10 Oct 2014 12:03:11 +0000 (UTC) Received: from wonkity.com (localhost [127.0.0.1]) by wonkity.com (8.14.9/8.14.9) with ESMTP id s9AC38IM068405 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 10 Oct 2014 06:03:08 -0600 (MDT) (envelope-from wblock@wonkity.com) Received: from localhost (wblock@localhost) by wonkity.com (8.14.9/8.14.9/Submit) with ESMTP id s9AC38f6068402; Fri, 10 Oct 2014 06:03:08 -0600 (MDT) (envelope-from wblock@wonkity.com) Date: Fri, 10 Oct 2014 06:03:08 -0600 (MDT) From: Warren Block To: Stephan Wehner Subject: Re: fsck / GEOM / Editing files in mirror partitions separately In-Reply-To: Message-ID: References: User-Agent: Alpine 2.11 (BSF 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (wonkity.com [127.0.0.1]); Fri, 10 Oct 2014 06:03:08 -0600 (MDT) Cc: freebsd-fs@freebsd.org X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 10 Oct 2014 12:03:11 -0000 On Thu, 9 Oct 2014, Stephan Wehner wrote: > Hello there, > > I have a FreeBSD 10.0-RELEASE-p7 system with system specifics: > > $ mount > /dev/mirror/gm0s1a on / (ufs, local, journaled soft-updates) > devfs on /dev (devfs, local, multilabel) > /dev/mirror/gm0s1b on /home (ufs, local, journaled soft-updates) > procfs on /proc (procfs, local) > > $ gmirror status > Name Status Components > mirror/gm0 COMPLETE ada0 (ACTIVE) > ada1 (ACTIVE) > > $ gpart show > => 63 3907029104 mirror/gm0 MBR (1.8T) > 63 3907029042 1 freebsd [active] (1.8T) > 3907029105 62 - free - (31K) > > => 0 3907029042 mirror/gm0s1 BSD (1.8T) > 0 2456848384 1 freebsd-ufs (1.1T) > 2456848384 1433600000 2 freebsd-ufs (684G) > 3890448384 16580658 4 freebsd-swap (7.9G) > > Computer was rebooted by simply turning it off. On reboot, I got > message "error aborting boot enter full pathname or shell or return > for /bin/sh" > > First question: Is it normal that the filesystem will not survive a > power loss, so that manual intervention is needed? > I thought UFS would be more robust. fsck is often required when filesystem has been stopped without a clean shutdown. This is to prevent corruption from writing to a non-consistent filesystem. > Now I did something that may have been a bad idea. > > I ran fsck on both partitions of the mirror separately, I think they > were called ad4s1 and ad6s1. For both it reported some problems to > fix, and I did Y until they were declared "CLEAN." Why? A mirror is a single entity. Writes are sent to both drives, and the whole point is that the two drives are identical. Using them individually should be avoided. > Then rebooted, and now the system is running fine. > > Second question. Is that mirror in good shape? Is there a way to test? That's an interesting question. I don't know if there is a way to force gmirror to check the mirror in depth. Normally, I think it just keeps track of whether the drives are synchronized. With the mirror stopped, the contents of both drives should be identical except for the last block, where the gmirror metadata is stored. > Third question. Does one even run fsck on partitions that are then > controlled by geom? Well, yes. But if RAID is involved, go through the RAID geom rather than around it. So fsck the filesystems through the mirror, not on each disk. Let gmirror keep them synchronized. > This shows in dmesg: > > GEOM_MIRROR: Device mirror/gm0 launched (1/2). > GEOM_MIRROR: Device gm0: rebuilding provider ada0. > GEOM_MIRROR: Device gm0: rebuilding provider ada0 finished. > GEOM_MIRROR: Device mirror/gm0 launched (2/2). It found an unsynchronized mirror and rebuilt it, copying one entire drive to another. This can also be forced manually with 'gmirror rebuild'. From owner-freebsd-fs@FreeBSD.ORG Sat Oct 11 21:39:16 2014 Return-Path: Delivered-To: freebsd-fs@FreeBSd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 91FA3F94 for ; Sat, 11 Oct 2014 21:39:16 +0000 (UTC) Received: from woozle.rinet.ru (woozle.rinet.ru [195.54.192.68]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14D73B5B for ; Sat, 11 Oct 2014 21:39:12 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by woozle.rinet.ru (8.14.5/8.14.5) with ESMTP id s9BLcrVt030082 for ; Sun, 12 Oct 2014 01:38:53 +0400 (MSK) (envelope-from marck@rinet.ru) Date: Sun, 12 Oct 2014 01:38:53 +0400 (MSK) From: Dmitry Morozovsky To: freebsd-fs@FreeBSd.org Subject: Snapshots and what not to snapshot Message-ID: User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) X-NCC-RegID: ru.rinet X-OpenPGP-Key-ID: 6B691B03 MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (woozle.rinet.ru [0.0.0.0]); Sun, 12 Oct 2014 01:38:53 +0400 (MSK) X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Oct 2014 21:39:16 -0000 Colleagues, reading some last threads I'm starting to think again about the problem I thought about for many times, but invent nothing but crude hacks: it would be great to have a mechanism to exclude some subtrees from recursive snapshots; the model is like: you have some tree of ZFS file systems, like pool/path/r pool/path/jails pool/path/jails/j1 pool/path/jails/j1/obj .. pool/path/persistent pool/path/obj or something alike. To have the ability to make consistent backup, one would use ``zfs snapshot -r'' but -- before using zfs send or other replication machanisms it would be feasible to remove snapshots of not-so-important filesystems. For now, the kludge I could see is to set on these some artificial property like org.freebsd:nodump or similar, then traverse zfs list with this attribute and delete non-needed snapshots. Maybe somewhere there are more elegant solutions? Sincerely, D.Marck [DM5020, MCK-RIPE, DM3-RIPN] [ FreeBSD committer: marck@FreeBSD.org ] ------------------------------------------------------------------------ *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck@rinet.ru *** ------------------------------------------------------------------------