From owner-freebsd-fs@freebsd.org Mon Apr 23 21:00:54 2018 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A41F2FAB5C9 for ; Mon, 23 Apr 2018 21:00:54 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-QB1-obe.outbound.protection.outlook.com (mail-eopbgr660052.outbound.protection.outlook.com [40.107.66.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT TLS CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 3761B83EF7 for ; Mon, 23 Apr 2018 21:00:53 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM (52.132.66.153) by YQBPR0101MB1060.CANPRD01.PROD.OUTLOOK.COM (52.132.66.159) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.696.13; Mon, 23 Apr 2018 21:00:52 +0000 Received: from YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::893c:efc2:d71f:945a]) by YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM ([fe80::893c:efc2:d71f:945a%13]) with mapi id 15.20.0696.017; Mon, 23 Apr 2018 21:00:52 +0000 From: Rick Macklem To: "freebsd-fs@freebsd.org" , "karli@inparadise.se" Subject: Re: pNFS server completed Thread-Topic: pNFS server completed Thread-Index: AQHT2n6KBZhPilerAEebAsjWoa5XxaQOAmgAgADSTVg= Date: Mon, 23 Apr 2018 21:00:52 +0000 Message-ID: References: , <1524471526.9560.51.camel@inparadise.se> In-Reply-To: <1524471526.9560.51.camel@inparadise.se> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: freebsd.org; dkim=none (message not signed) header.d=none;freebsd.org; dmarc=none action=none header.from=uoguelph.ca; x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; YQBPR0101MB1060; 7:qz6J6/g11MwOFha86gcH3byZoLqDNM2PiOZTAMf+YSdmfocjS1fagY9QXlHsdZBKeXNXxby2uJcT/30w10e+YWLKWh6gKIuwvHApDBLHZL3aKCL/yVI+RVELTnHSLSPRnEMhqftWu2He1VtvvK9MgyBF5ekkA/Z0jdZXk4tlJ9GRiJ90WlsVisRkArmAKoZDMQquNMOiznmUQHN7m6zDkviwgL5tSlvu92t1sIMkD8Nd+Awlgb/19Cfb3hIdmhPQ x-ms-exchange-antispam-srfa-diagnostics: SOS; x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(8989080)(4534165)(4627221)(201703031133081)(201702281549075)(8990040)(5600026)(2017052603328)(7153060)(7193020); SRVR:YQBPR0101MB1060; x-ms-traffictypediagnostic: YQBPR0101MB1060: x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(158342451672863)(5213294742642); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040522)(2401047)(5005006)(8121501046)(93006095)(93001095)(10201501046)(3231232)(944501410)(52105095)(3002001)(6041310)(20161123560045)(201703131423095)(201702281529075)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123562045)(20161123558120)(20161123564045)(6072148)(201708071742011); SRVR:YQBPR0101MB1060; BCL:0; PCL:0; RULEID:; SRVR:YQBPR0101MB1060; x-forefront-prvs: 06515DA04B x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(39380400002)(346002)(366004)(376002)(396003)(377424004)(229853002)(6306002)(9686003)(6506007)(33656002)(102836004)(55016002)(59450400001)(53936002)(26005)(7696005)(2906002)(6246003)(74316002)(3280700002)(7116003)(305945005)(76176011)(5660300001)(74482002)(5250100002)(2501003)(3660700001)(2900100001)(8676002)(8936002)(81166006)(110136005)(966005)(316002)(25786009)(3480700004)(478600001)(786003)(476003)(6436002)(11346002)(86362001)(186003)(446003); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR0101MB1060; H:YQBPR0101MB1042.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; MLV:sfv; x-microsoft-antispam-message-info: zsDkGnVgUsnE6BHixmc7G9lnQfcVSWIZFCONHZ7PfjDLfhKaQel/MapuJy5zi2tiBF2o7rqNGZZn/kryBphnLU7ox1VOGsGLR3tUZoUiNjcKzfKQy4zQlNvz7vplXK8ZjODKtAFzXkyky3RzXuBCd2eca4/PQNMkSUdwzF8biJsIzNc715Ma6SWogLvkf2tf spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Office365-Filtering-Correlation-Id: 77d6a201-f3d9-4ca5-59f2-08d5a95d4ca6 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: 77d6a201-f3d9-4ca5-59f2-08d5a95d4ca6 X-MS-Exchange-CrossTenant-originalarrivaltime: 23 Apr 2018 21:00:52.5347 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR0101MB1060 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Apr 2018 21:00:55 -0000 Karli Sj=F6berg wrote: >On Sun, 2018-04-22 at 21:18 +0000, Rick Macklem wrote: >> Hi, >> >> I have now updated the projects/pnfs-planb-server subtree of svn with >> what >> I believe is a complete pNFS server. It now handles mirrored Data >> Servers (DSs) >> and recovery of a failed mirrored DS while the service is >> operational. >> It handle non-pNFS clients (such as NFSv3 mounts) via having the >> MetaData >> Server act as a proxy for Read/Write operations. >> For pNFS capable NFSv4.1 clients, the Read/Write operations are done >> directly >> on the appropriate DS(s), avoiding the MetaData Server (which act >> like a normal >> NFS server except for pNFS capable clients). >> >> If you are interested in more information or doing testing, see: >> http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt >> >> I hope that this should be ready for head/current with some >> additional testing. >> I will post here asking the "collective" about this when I think it >> is ready. >> >> If there are any volunteers on the doc side who would like to turn >> the above >> text file into a useful document, that would be greatly appreciated. >> >> Good luck with it, if you test it, rick > >Hi Rick! > >I think this sounds amazing to have a distributed file server natively >in FreeBSD, awesome! > >One question that comes to mind is about the MDS: wouldn=B4t that be a >SPOF?=20 Yes, the MDS is just like a regular NFS server on FreeBSD. It is a single p= oint of failure. >Maybe that has already been discussed and explained and I=B4ve just >missed it, but would be nice to know. And if it is a SPOF, are there >any plans of also mirroring MDS's as well? Perhaps a DS can be MDS to, >something like that? No plans w.r.t. mirroring the MDS. The pNFS client using Flexible File Layo= ut knows to write to both mirrors (and the MDS can do so acting as a proxy for a client). However, NFS clients don't know how to do NFS operations on multiple NFS se= rvers. As such, any mirroring would have to be implemented "behind the curtain" and I'll admit I don't know how to do that. I did try using a clustered file system (GlusterFS) as a backend when I fir= st implemented a pNFS server two years ago, but that was a failure, due to terrible performance. (Using a clustered file system as a storage backend w= ould make multiple NFS servers/MDSs more feasible, but it didn't work.) rick