From owner-freebsd-current@freebsd.org Sat Nov 26 23:19:29 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9453AC575DD for ; Sat, 26 Nov 2016 23:19:29 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2on0056.outbound.protection.outlook.com [207.46.100.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "Microsoft IT SSL SHA2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 33F66929; Sat, 26 Nov 2016 23:19:28 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from YQBPR01MB0180.CANPRD01.PROD.OUTLOOK.COM (10.169.141.138) by YQBPR01MB0177.CANPRD01.PROD.OUTLOOK.COM (10.169.141.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.747.13; Sat, 26 Nov 2016 23:19:19 +0000 Received: from YQBPR01MB0180.CANPRD01.PROD.OUTLOOK.COM ([10.169.141.138]) by YQBPR01MB0180.CANPRD01.PROD.OUTLOOK.COM ([10.169.141.138]) with mapi id 15.01.0747.015; Sat, 26 Nov 2016 23:19:19 +0000 From: Rick Macklem To: Konstantin Belousov CC: Alan Somers , FreeBSD CURRENT Subject: Re: NFSv4 performance degradation with 12.0-CURRENT client Thread-Topic: NFSv4 performance degradation with 12.0-CURRENT client Thread-Index: AQHSRhIPtzL8Jj0C/0q6PJtKKLQgMaDn2GqAgAA8DvaAAGR2gIAAPUCRgACtAACAAEDk+4AAF32AgAIr5lw= Date: Sat, 26 Nov 2016 23:19:19 +0000 Message-ID: References: <20161124090811.GO54029@kib.kiev.ua> <20161125084106.GX54029@kib.kiev.ua> , <20161125135725.GD54029@kib.kiev.ua> In-Reply-To: <20161125135725.GD54029@kib.kiev.ua> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=rmacklem@uoguelph.ca; x-ms-office365-filtering-correlation-id: d7cbc0f0-5feb-476c-dec9-08d41652a64e x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(22001);SRVR:YQBPR01MB0177; x-microsoft-exchange-diagnostics: 1; YQBPR01MB0177; 7:MLbEy7c2EqOeZIQcMF9hHAqv0yfPWZ2tdmmAQ30VuprSfgRd2TOtTgP4g6Ak64gVCdAdE5qAnbu6sGwyRNgdtG4RfrXJ6P57Xo5Q22wJCTLkTa8wtG68LCwKeA8KEL8QHvlb33H7iYuHRg5Gu6TzdhrPZUmNGlXsjXI6v6fK8E4lYwErbEBaI08M65C1dwYrbs2Dhgp2FRWnST3Geo2cAw7sX063ypRrCH+0NeZGUDOgkizBqIU0yTmw+2K484SuB24fGuvDzyVYYgVi2yrPQl3uPX2JhQSRtJJhD89cN+/px2G7mhrLPIL5vSbm4Wl0H0C2rSQEsrRsQFqYgPETxUENwjl04Mh0G18OrF0xEppgyoEAl2S4/44NeDf5+MnIQNbaHrB0hOSNIw8ZtoLMCSukopu5anFUM1masB9WoMnYnrHjASWNk9MeA/ztaN765eLhRVVsvDqbQZmSVOpocA== x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6060326)(6040361)(6045199)(2401047)(5005006)(8121501046)(3002001)(10201501046)(6041248)(6061324)(20161123560025)(20161123555025)(20161123564025)(20161123562025)(2016111802025)(6043046)(6042181); SRVR:YQBPR01MB0177; BCL:0; PCL:0; RULEID:; SRVR:YQBPR01MB0177; x-forefront-prvs: 0138CD935C x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(6009001)(7916002)(24454002)(189002)(199003)(74316002)(74482002)(305945005)(189998001)(9686002)(3280700002)(7846002)(102836003)(105586002)(5660300001)(8936002)(81166006)(68736007)(106116001)(81156014)(106356001)(8676002)(97736004)(3660700001)(2950100002)(92566002)(76176999)(101416001)(1411001)(50986999)(6916009)(86362001)(39060400001)(110136003)(93886004)(2906002)(122556002)(54356999)(7696004)(33656002)(4326007)(77096006)(38730400001)(2900100001)(6506003)(229853002); DIR:OUT; SFP:1101; SCL:1; SRVR:YQBPR01MB0177; H:YQBPR01MB0180.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-originalarrivaltime: 26 Nov 2016 23:19:19.8047 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-Transport-CrossTenantHeadersStamped: YQBPR01MB0177 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Nov 2016 23:19:29 -0000 Konstantin Belousov wrote: [stuff snipped] >I thought that the issue was in tracking any opens and mmaps, but from thi= s >reply it is not that clear. Do you need callback when all opens and mmaps >have ended, or only opens and mmaps for write ? If later, we already have >a suitable mechanism VOP_ADD_WRITECOUNT(). Not quite. The NFSv4 client needs to Close the NFSv4 Open after all I/O on the file has been done. This applies to both reads and writes. Since mmap'd files can generate I/O after the VOP_CLOSE(), the NFSv4 client can't do the NFSv4 Close in VOP_CLOSE(). Since it can't do it then, it waits until VOP_INACTIVE() to do the NFSv4 Cl= ose. This might be improved by: - A flag that indicates that an open file descriptor has been mmap()d, whic= h VOP_CLOSE() could check to decide if it can do the NFSv4 Close. (ie. It could do the NFSv4 Close if the file descriptor hasn't been mmap(= )d.) - If the system knows when mmap()d I/O is done (the process has terminated?= ), it could do a VOP_MMAP_IO_DONE() and the NFSv4 client would do the NFSv4 = Close in it. --> I don't know if this is feasible and I suspect if it could be done, t= hat it would usually happen just before VOP_INACTIVE(). { This case of nullfs ca= ching was an exception, I think? } Does this clarify it? rick