From owner-freebsd-current@FreeBSD.ORG Thu Jun 18 00:03:57 2009 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 799B7106566B; Thu, 18 Jun 2009 00:03:57 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (cl-162.ewr-01.us.sixxs.net [IPv6:2001:4830:1200:a1::2]) by mx1.freebsd.org (Postfix) with ESMTP id 189248FC12; Thu, 18 Jun 2009 00:03:56 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.3/8.14.3) with ESMTP id n5I048JR046030; Wed, 17 Jun 2009 19:04:08 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.3/8.14.3/Submit) id n5I048KI046029; Wed, 17 Jun 2009 19:04:08 -0500 (CDT) (envelope-from brooks) Date: Wed, 17 Jun 2009 19:04:08 -0500 From: Brooks Davis To: freebsd-hackers@freebsd.org, freebsd-current@freebsd.org Message-ID: <20090618000407.GA43514@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="vkogqOf2sHV7VnPd" Content-Disposition: inline User-Agent: Mutt/1.5.17 (2007-11-01) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Wed, 17 Jun 2009 19:04:08 -0500 (CDT) Cc: Subject: CFT: final patches for NGROUPS>>16 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Jun 2009 00:03:57 -0000 --vkogqOf2sHV7VnPd Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Please find attached three patches which result in raising NGROUPS to 1024, making programs in the base system immune to changing values of NGROUPS, and pave the way for NGROUPS to be boot time configurable. The first two patches (ngroups-catman.diff and ngroups-posix.diff) are userland cleanups. The third (ngroups-main.diff) is the core of the change and primairly in the kernel. It is strongly based on work contributed by Isilon Systems. The proposed commit messages appear to the bottom of this message. I've been running the basic code on my FreeBSD laptop for a while and I've tested basic NFS mounts, but more areas need testing before this hits a release. Key things to hit include: - NFS mount and export - IPFW uid, gid, and jail rules - portalfs (ideally portalfs should be extended to not truncate groups) In practice, expect appliations to not care about this change unless they are run with more than 16 groups. If that happens, calls to getgroups() will fail if they use statically sized buffers recompilation should fix them. Please test, review, etc. I'd like to get this in before code freeze if at all possible. -- Brooks ngroups-catman.diff: When checking if we can write to a file, use access() instead of a manual permission check based on stat output. Also, get rid of the executability check since it is not used. MFC after: 2 weeks ngroups-posix.diff In preparation for raising NGROUPS and NGROUPS_MAX, change base system callers of getgroups(), getgrouplist(), and setgroups() to allocate buffers dynamically. Specifically, allocate a buffer of size sysconf(_SC_NGROUPS_MAX)+1 (+2 in a few cases to allow for overflow). This (or similar gymnastics) is required for the code to actually follow the POSIX.1-2008 specification where {NGROUPS_MAX} may differ at runtime and where getgroups may return {NGROUPS_MAX}+1 results on systems like FreeBSD which include the primary group. In id(1), don't pointlessly add the primary group to the list of all groups, it is always the first result from getgroups(). In principle the old code was more portable, but this was only done in one of the two places where getgroups() was called to the overall effect was pointless. Document the actual POSIX requirements in the getgroups(2) and setgroups(2) manpages. We do not yet support a dynamic NGROUPS, but we may in the future. MFC after: 2 weeks ngroups-main.diff: Rework the credential code to support larger values of NGROUPS and NGROUPS_MAX, eliminate ABI dependencies on them, and raise the to 1024 and 1023 respectively. (Previously they were equal, but under a close reading of POSIX, NGROUPS_MAX was defined to be too large by 1 since it is the number of supplemental groups, not total number of groups.) The bulk of the change consists of converting the struct ucred member cr_groups from a static array to a pointer. Do the equivalent in kinfo_proc. Introduce new interfaces crcopysafe() and crsetgroups() for duplicating a process credential before modifying it and for setting group lists respectively. Both interfaces take care for the details of allocating groups array. crsetgroups() takes care of truncating the group list to the current maximum (NGROUPS) if necessary. In the future, crsetgroups() may be responsible for insuring invariants such as sorting the supplemental groups to allow groupmember() to be implemented as a binary search. Because we can not change struct xucred without breaking application ABIs, we leave it alone and introduce a new XU_NGROUPS value which is always 16 and is to be used or NGRPS as appropriate for things such as NFS which need to use no more than 16 groups. When feasible, truncate the group list rather than generating an error. Minor changes: - Reduce the number of hand rolled versions of groupmember(). - Do not assign to both cr_gid and cr_groups[0]. - Modify ipfw to cache ucreds instead of part of their contents since they are immutable once referenced by more than one entity. Submitted by: Isilon Systems X-MFC after: never --vkogqOf2sHV7VnPd Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (FreeBSD) iD8DBQFKOYR3XY6L6fI4GtQRAlOkAJwKRcS2PcUsUAyGurwQZMgAyM5npQCgkGR8 u6SdpLhMZAQgQ7+RXhyJ5FU= =TZms -----END PGP SIGNATURE----- --vkogqOf2sHV7VnPd--