From owner-freebsd-current@freebsd.org Thu Apr 11 04:41:38 2019 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id BF581157891A for ; Thu, 11 Apr 2019 04:41:37 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from CAN01-TO1-obe.outbound.protection.outlook.com (mail-eopbgr670060.outbound.protection.outlook.com [40.107.67.60]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (Client CN "mail.protection.outlook.com", Issuer "GlobalSign Organization Validation CA - SHA256 - G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 893FF749CE; Thu, 11 Apr 2019 04:41:36 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from QB1PR01MB3537.CANPRD01.PROD.OUTLOOK.COM (52.132.84.84) by QB1PR01MB3874.CANPRD01.PROD.OUTLOOK.COM (52.132.88.222) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1792.14; Thu, 11 Apr 2019 04:41:34 +0000 Received: from QB1PR01MB3537.CANPRD01.PROD.OUTLOOK.COM ([fe80::9c13:e118:131c:f3fd]) by QB1PR01MB3537.CANPRD01.PROD.OUTLOOK.COM ([fe80::9c13:e118:131c:f3fd%2]) with mapi id 15.20.1792.009; Thu, 11 Apr 2019 04:41:34 +0000 From: Rick Macklem To: Mateusz Guzik CC: "kib@freebsd.org" , "freebsd-current@FreeBSD.org" Subject: Re: Do the pidhashtbl_locks added by r340742 need to be sx locks? Thread-Topic: Do the pidhashtbl_locks added by r340742 need to be sx locks? Thread-Index: AQHU8AP7fWJXG48RU0WTxwDO79KhQ6Y2MZSAgAArjKw= Date: Thu, 11 Apr 2019 04:41:34 +0000 Message-ID: References: , In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: e34e1c18-2047-4770-ce72-08d6be37fa04 x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600139)(711020)(4605104)(2017052603328)(7193020); SRVR:QB1PR01MB3874; x-ms-traffictypediagnostic: QB1PR01MB3874: x-microsoft-antispam-prvs: x-forefront-prvs: 00046D390F x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(136003)(346002)(366004)(39860400002)(396003)(376002)(189003)(199004)(71200400001)(106356001)(305945005)(2906002)(476003)(68736007)(1411001)(74482002)(53936002)(186003)(71190400001)(8936002)(97736004)(229853002)(6436002)(7696005)(8676002)(52536014)(33656002)(81156014)(486006)(81166006)(105586002)(478600001)(74316002)(6246003)(5660300002)(14454004)(316002)(786003)(76176011)(86362001)(256004)(55016002)(102836004)(6506007)(99286004)(46003)(25786009)(446003)(4326008)(14444005)(54906003)(11346002)(6916009)(9686003); DIR:OUT; SFP:1101; SCL:1; SRVR:QB1PR01MB3874; H:QB1PR01MB3537.CANPRD01.PROD.OUTLOOK.COM; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1; received-spf: None (protection.outlook.com: uoguelph.ca does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: n3a2zvZ46gYT8njBce0xcFziD1CjRlfO4XEIrq2YbHlXIrOgdigF3UcM57YR2VtIVgbCnSMbn7A383eBJGLd3RycQshQOJlTq0sIazaC5kyFAOjNBB+w3t7wuqblkVMROS3XwIka7gx7Fh0HQlw9xqpEV2icz6c0934F9mRYbrSJa6ryMAsZnPpoEL8GtjIk14u8xEXRUUIJkqXBBWRPgYo8APlte8T1nEw+3jQbXLWNTXRUfQN45OGX290dnTenhgeQzUerOKCt9pXw7DwZWxDcB7DvFZfs88zgswIee1PUuEUgoAD5SuYS9vxcJjRub5XGtPSgWCeDliCCszO6CYU9pn+AmSI0u4yF/rpnhXOI04IBVkSpGCg+t30BvkDWJG/vbhAVcPrLfuQrRqebP1E3wS/rWT1SP0GbuME+lZY= Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: uoguelph.ca X-MS-Exchange-CrossTenant-Network-Message-Id: e34e1c18-2047-4770-ce72-08d6be37fa04 X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Apr 2019 04:41:34.5730 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: be62a12b-2cad-49a1-a5fa-85f4f3156a7d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-Transport-CrossTenantHeadersStamped: QB1PR01MB3874 X-Rspamd-Queue-Id: 893FF749CE X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; spf=pass (mx1.freebsd.org: domain of rmacklem@uoguelph.ca designates 40.107.67.60 as permitted sender) smtp.mailfrom=rmacklem@uoguelph.ca X-Spamd-Result: default: False [0.00 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.85)[-0.852,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:40.107.0.0/16]; NEURAL_HAM_LONG(-0.61)[-0.610,0]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[uoguelph.ca]; TO_DN_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.77)[0.774,0]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MX_GOOD(-0.01)[cached: mx2.hc184-76.ca.iphmx.com]; RCVD_IN_DNSWL_NONE(0.00)[60.67.107.40.list.dnswl.org : 127.0.3.0]; FREEMAIL_TO(0.00)[gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Apr 2019 04:41:38 -0000 Mateusz Guzik wrote: >On 4/11/19, Rick Macklem wrote: >> Hi, >> >> I finally got around to looking at what effect replacing pfind_locked() >> with >> pfind() has for the NFSv4 client and it is broken. >> >> The problem is that the NFS code needs to call some variant of "pfind()" >> while >> holding a mutex lock. The current _pfind() code uses the pidhashtbl_lock= s, >> which are "sx" locks. >> >> There are a few ways to fix this: >> 1 - Create a custom version of _pfind() for the NFS client with the sx_X= () >> calls >> removed, plus replace the locking of allproc_lock with locking of = all >> the >> pidhashtbl_locks, so that the "sx" locks are acquired before the >> mutex. >> --> Not very efficient, but since it is only done once/sec, I can = live >> with it. >> 2 - Similar to the above, but still lock the allproc_lock and use a loop= of >> FOREACH_PROC_IN_SYSTEM(p) instead of a hash list for the pid in the >> custom pfind(). (I don't know if this would be preferable to lockin= g >> all >> the pidhashtbl_locks for other users of pfind()?) >> 3 - Convert the pidhashtbl_locks to mutexes. Then the NFS client doesn't >> need >> to acquire any proc related locks and it just works. >> I can't see anywhere that "sleeps" while holding the pidhashtbl_loc= ks, >> so I >> think they can be converted, although I haven't tried it yet? >> >> From my perspective, #3 seems the better solution. >> What do others think? >> > >Changing the lock type to rwlock may be doable and worthwhile on its own, >but I don't think it would constitute the right fix. > >Preferably there would be an easy to use mechanism which allows >registering per-process callbacks. Currently it can be somewhat emulated >with EVENTHANDLERs, but it would give calls for all exiting processes, not >only the ones of interest. Then there would be no need to periodically >scan as you would always get notified on process exit. Long ago when I first did the NFSv4 code for OpenBSD2.6, I had a callback f= unction pointer in "struct proc" which the NFS code set non-null to get a callback. { The code still has remnants of that because it still has nfscl_cleanup_co= mmon(), which was code shared by that callback and this approach which was used = for the Mac OS X port, where I couldn't change "struct proc". } I have never added anything like that for FreeBSD, but I suppose we could l= ook at doing it that way. To be honest, since the current code works fine and can be difficult to tes= t well, I hesitate to change to using a callback. >Note the current code does not ref processes it is interested in any >manner and just performs a timestamp check to see if it got the one it >expected (with pid reuse in mind). > >So I think a temporary hack which will do the trick will take the current >approach further: rely on struct proc being type-stable (i.e. never being >freed) and also store the pointer. You can always safely PROC_LOCK it, do >checks to see the proc is alive and has the right timestamp... Hmm, so you are saying that every element of the proc_zone always has a val= id p_mtx field in it that can be safely PROC_LOCK()'d no matter if the element refers to a process at that time? I would also need help with the code to determine if the structure refers t= o a process that currently exists with the same pid and creation time. I suppose saving "p" with the lock/open owner string and then doing what yo= u suggest is possible, but it would take some work. For now, I can just grab all the pidhashtbl_locks once/sec and fix head so = it works. rick