From owner-freebsd-security@FreeBSD.ORG Tue Feb 10 12:27:21 2015 Return-Path: Delivered-To: freebsd-security@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 0F2C0405; Tue, 10 Feb 2015 12:27:21 +0000 (UTC) Received: from ironport2-out.teksavvy.com (ironport2-out.teksavvy.com [206.248.154.181]) by mx1.freebsd.org (Postfix) with ESMTP id 9B9E465B; Tue, 10 Feb 2015 12:27:20 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AnsOAPOG1lRFpa1E/2dsb2JhbABbghVxAVFaAYMBrSYBAQEBAQEGg2iOVROFaAMCgQ9EAQEBAQEBfIQ2FR4iATUCBQ0BCAsCCwMCAQIBJzEBBwEBiCkNuCWVcQEBAQcCAR+BIYRjh2eBREkZAQOCUoFCBYongxGBY4NQg1eDU4RyimuBRSKEDCAxAQGBAQEHF4EgAQEB X-IPAS-Result: AnsOAPOG1lRFpa1E/2dsb2JhbABbghVxAVFaAYMBrSYBAQEBAQEGg2iOVROFaAMCgQ9EAQEBAQEBfIQ2FR4iATUCBQ0BCAsCCwMCAQIBJzEBBwEBiCkNuCWVcQEBAQcCAR+BIYRjh2eBREkZAQOCUoFCBYongxGBY4NQg1eDU4RyimuBRSKEDCAxAQGBAQEHF4EgAQEB X-IronPort-AV: E=Sophos;i="5.09,536,1418101200"; d="scan'208";a="110104683" Received: from 69-165-173-68.dsl.teksavvy.com (HELO porter.razorfever.net) ([69.165.173.68]) by ironport2-out.teksavvy.com with ESMTP/TLS/DHE-RSA-AES256-SHA; 10 Feb 2015 07:26:11 -0500 Received: from [127.0.0.1] (localhost [127.0.0.1]) by porter.razorfever.net (8.14.4/8.14.4) with ESMTP id t1ACQ7mh075976; Tue, 10 Feb 2015 07:26:08 -0500 (EST) (envelope-from 482254ac@razorfever.net) Message-ID: <54D9F8DF.7070904@razorfever.net> Date: Tue, 10 Feb 2015 07:26:07 -0500 From: "Derek (freebsd lists)" <482254ac@razorfever.net> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-security@FreeBSD.org Subject: [patch] libcrypt & friends - modular crypt format support in /etc/login.conf Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Cc: delphij@FreeBSD.org, John-Mark Gurney , "A.J. Kehoe IV \(Nanoman\)" X-BeenThere: freebsd-security@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Security issues \[members-only posting\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Feb 2015 12:27:21 -0000 Hello! I've been working on this for a while, and I've produced a patch that does a few things with the base system: 1. allows modular crypt to be specified as passwd_format in /etc/login.conf - this allows setting the algorithm *and rounds*, i.e. $2b$10$ for users of varying classes. - this will allow any future algorithms and parameters supported by crypt(3) to be supported by the tools around login.conf 2. introduces a new api, crypt_makesalt which will generate an appropriate salt for any algorithm selected 3. updates userland to use this API, and removes totally the {crypt_set_format, login_setcryptfmt, login_getcryptfmt} APIs 4. switches crypt algorithms to use thread-local storage, so the good old global crypt buffer is thread-local 5. includes a bunch of new test vectors for libcrypt ATF tests There are references to previous discussions/patches/etc here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182518 http://docs.freebsd.org/cgi/getmsg.cgi?fetch=168499+0+/usr/local/www/db/text/2013/freebsd-current/20131006.freebsd-current http://docs.freebsd.org/cgi/getmsg.cgi?fetch=361757+0+/usr/local/www/db/text/2014/freebsd-current/20140112.freebsd-current And most recent discussion here: http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1751919+0+archive/2014/freebsd-current/20140716.freebsd-current Anyways, I've put a bunch of work into this, and am anxious to actually get this accepted into -HEAD. What more can I do at this point? A patch against current is in the original PR/"bug": https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182518 Thanks, Derek From owner-freebsd-security@FreeBSD.ORG Wed Feb 11 02:19:13 2015 Return-Path: Delivered-To: freebsd-security@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C6DCB3A1; Wed, 11 Feb 2015 02:19:13 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id A230FD76; Wed, 11 Feb 2015 02:19:13 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1B2JBsP040337 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 10 Feb 2015 18:19:12 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1B2JBOW040336; Tue, 10 Feb 2015 18:19:11 -0800 (PST) (envelope-from jmg) Date: Tue, 10 Feb 2015 18:19:10 -0800 From: John-Mark Gurney To: "Derek (freebsd lists)" <482254ac@razorfever.net> Subject: [CFR] Re: [patch] libcrypt & friends - modular crypt format support in /etc/login.conf Message-ID: <20150211021910.GQ1953@funkthat.com> References: <54D9F8DF.7070904@razorfever.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54D9F8DF.7070904@razorfever.net> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Tue, 10 Feb 2015 18:19:12 -0800 (PST) Cc: freebsd-security@freebsd.org, "A.J. Kehoe IV \(Nanoman\)" , delphij@freebsd.org X-BeenThere: freebsd-security@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Security issues \[members-only posting\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Feb 2015 02:19:14 -0000 Derek (freebsd lists) wrote this message on Tue, Feb 10, 2015 at 07:26 -0500: > I've been working on this for a while, and I've produced a patch > that does a few things with the base system: > > 1. allows modular crypt to be specified as passwd_format in > /etc/login.conf > - this allows setting the algorithm *and rounds*, i.e. $2b$10$ > for users of varying classes. > - this will allow any future algorithms and parameters > supported by crypt(3) to be supported by the tools around login.conf > > 2. introduces a new api, crypt_makesalt which will generate an > appropriate salt for any algorithm selected > > 3. updates userland to use this API, and removes totally the > {crypt_set_format, login_setcryptfmt, login_getcryptfmt} APIs > > 4. switches crypt algorithms to use thread-local storage, so the > good old global crypt buffer is thread-local > > 5. includes a bunch of new test vectors for libcrypt ATF tests > > > There are references to previous discussions/patches/etc here: > > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=182518 > > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=168499+0+/usr/local/www/db/text/2013/freebsd-current/20131006.freebsd-current > > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=361757+0+/usr/local/www/db/text/2014/freebsd-current/20140112.freebsd-current > > > And most recent discussion here: > > http://docs.freebsd.org/cgi/getmsg.cgi?fetch=1751919+0+archive/2014/freebsd-current/20140716.freebsd-current > > > Anyways, I've put a bunch of work into this, and am anxious to > actually get this accepted into -HEAD. > > > > What more can I do at this point? I finally got around to reviewing this... For the tests, we should probably add an invalid password test for each format... We need man pages for the new function... I guess this new man page would be a good place to document all the modular formats in more detail.. what is in crypt(3) isn't that useful... Also, crypt(3) should have an xref to crypt_makesalt... Other than those, unless someone objects, I'll commit it... -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." From owner-freebsd-security@FreeBSD.ORG Sat Feb 14 17:33:17 2015 Return-Path: Delivered-To: freebsd-security@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 436BF4D7; Sat, 14 Feb 2015 17:33:17 +0000 (UTC) Received: from thyme.infocus-llc.com (thyme.infocus-llc.com [199.15.120.10]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1A420D5A; Sat, 14 Feb 2015 17:33:16 +0000 (UTC) Received: from draco.over-yonder.net (c-75-65-60-66.hsd1.ms.comcast.net [75.65.60.66]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by thyme.infocus-llc.com (Postfix) with ESMTPSA id A465537B5A0; Sat, 14 Feb 2015 11:33:14 -0600 (CST) Received: by draco.over-yonder.net (Postfix, from userid 100) id 3kkzF26vVMzTP; Sat, 14 Feb 2015 11:33:10 -0600 (CST) Date: Sat, 14 Feb 2015 11:33:10 -0600 From: "Matthew D. Fuller" To: "Derek (freebsd lists)" <482254ac@razorfever.net> Subject: Re: [patch] libcrypt & friends - modular crypt format support in /etc/login.conf Message-ID: <20150214173310.GD37668@over-yonder.net> References: <54D9F8DF.7070904@razorfever.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54D9F8DF.7070904@razorfever.net> X-Editor: vi X-OS: FreeBSD User-Agent: Mutt/1.5.23-fullermd.4 (2014-03-12) X-Virus-Scanned: clamav-milter 0.98.6 at thyme.infocus-llc.com X-Virus-Status: Clean Cc: freebsd-security@FreeBSD.org, John-Mark Gurney , "A.J. Kehoe IV \(Nanoman\)" , delphij@FreeBSD.org X-BeenThere: freebsd-security@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Security issues \[members-only posting\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Feb 2015 17:33:17 -0000 On Tue, Feb 10, 2015 at 07:26:07AM -0500 I heard the voice of Derek (freebsd lists), and lo! it spake thus: > > 2. introduces a new api, crypt_makesalt which will generate an > appropriate salt for any algorithm selected It has been an endlessly-repeated source of pain to me that there isn't a standard API for this, and it's just been into the wound[0] that there isn't even a NON-standard one, and so I have to guess and re-implement any time I want to use crypt(3) for anything except /etc/passwd. Of course, I want it in non-C, but one problem at a time... If you accomplish nothing else with this, I'll happily fall at your feet just for this 8-} [0] By a hydraulic press, I think. -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream. From owner-freebsd-security@FreeBSD.ORG Sat Feb 14 23:17:16 2015 Return-Path: Delivered-To: freebsd-security@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 81588C05; Sat, 14 Feb 2015 23:17:16 +0000 (UTC) Received: from mx1.stack.nl (relay02.stack.nl [IPv6:2001:610:1108:5010::104]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "mailhost.stack.nl", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 42BE328A; Sat, 14 Feb 2015 23:17:16 +0000 (UTC) Received: from snail.stack.nl (snail.stack.nl [IPv6:2001:610:1108:5010::131]) by mx1.stack.nl (Postfix) with ESMTP id 320ED358C5F; Sun, 15 Feb 2015 00:17:13 +0100 (CET) Received: by snail.stack.nl (Postfix, from userid 1677) id 1FD2528494; Sun, 15 Feb 2015 00:17:13 +0100 (CET) Date: Sun, 15 Feb 2015 00:17:13 +0100 From: Jilles Tjoelker To: "Derek (freebsd lists)" <482254ac@razorfever.net> Subject: Re: [patch] libcrypt & friends - modular crypt format support in /etc/login.conf Message-ID: <20150214231712.GA1360@stack.nl> References: <54D9F8DF.7070904@razorfever.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54D9F8DF.7070904@razorfever.net> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-security@FreeBSD.org, John-Mark Gurney , "A.J. Kehoe IV \(Nanoman\)" , delphij@FreeBSD.org X-BeenThere: freebsd-security@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Security issues \[members-only posting\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Feb 2015 23:17:16 -0000 On Tue, Feb 10, 2015 at 07:26:07AM -0500, Derek (freebsd lists) wrote: > I've been working on this for a while, and I've produced a patch > that does a few things with the base system: > 1. allows modular crypt to be specified as passwd_format in > /etc/login.conf > - this allows setting the algorithm *and rounds*, i.e. $2b$10$ > for users of varying classes. > - this will allow any future algorithms and parameters > supported by crypt(3) to be supported by the tools around login.conf OK. > 2. introduces a new api, crypt_makesalt which will generate an > appropriate salt for any algorithm selected I like the idea. > 3. updates userland to use this API, and removes totally the > {crypt_set_format, login_setcryptfmt, login_getcryptfmt} APIs Removing API functions completely requires a SHLIB_MAJOR bump. I think this can be avoided by replacing the functions with a stub instead, so they would behave as if the default always applied and not allow changes to it. > 4. switches crypt algorithms to use thread-local storage, so the > good old global crypt buffer is thread-local This uses quite a bit of memory for each thread created, even if it does not call crypt() at all. Fortunately, libcrypt is not commonly used. Given that crypt() has never been thread-safe, consider implementing crypt_r() as in glibc and leaving crypt() thread-unsafe. Thread-local storage via pthread_key_create() (one key for libcrypt) is still "magic" but reduces the memory waste for threads that do not call crypt(). > 5. includes a bunch of new test vectors for libcrypt ATF tests OK. Some remarks about the code: lib/libcrypt/crypt.c crypt_makesalt() > b64_from_24bit((uint8_t) rand_buf[2], (uint8_t) rand_buf[1], (uint8_t) > rand_buf[0], diff, (int *) &(diff), &out); All these casts can be avoided by making the affected variables the proper type in the first place. The cast of &diff causes a strict-aliasing violation and is definitely wrong on 64-bit big-endian systems. rand_buf is a salt, not a secret, so clearing it afterwards is unnecessary. Consider memcpy() and adding '\0' afterward instead of strncpy(). It seems unnecessary to clear the buffer completely. -- Jilles Tjoelker From owner-freebsd-security@FreeBSD.ORG Sat Feb 14 23:42:51 2015 Return-Path: Delivered-To: freebsd-security@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 7B90A62A; Sat, 14 Feb 2015 23:42:51 +0000 (UTC) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "gold.funkthat.com", Issuer "gold.funkthat.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F03B783; Sat, 14 Feb 2015 23:42:50 +0000 (UTC) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.14.5/8.14.5) with ESMTP id t1ENghQs000860 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 14 Feb 2015 15:42:43 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.14.5/8.14.5/Submit) id t1ENghQ7000859; Sat, 14 Feb 2015 15:42:43 -0800 (PST) (envelope-from jmg) Date: Sat, 14 Feb 2015 15:42:43 -0800 From: John-Mark Gurney To: Jilles Tjoelker Subject: Re: [patch] libcrypt & friends - modular crypt format support in /etc/login.conf Message-ID: <20150214234243.GX1953@funkthat.com> References: <54D9F8DF.7070904@razorfever.net> <20150214231712.GA1360@stack.nl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150214231712.GA1360@stack.nl> X-Operating-System: FreeBSD 9.1-PRERELEASE amd64 X-PGP-Fingerprint: 54BA 873B 6515 3F10 9E88 9322 9CB1 8F74 6D3F A396 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.5.21 (2010-09-15) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.2.7 (gold.funkthat.com [127.0.0.1]); Sat, 14 Feb 2015 15:42:43 -0800 (PST) Cc: delphij@freebsd.org, freebsd-security@freebsd.org, "Derek \(freebsd lists\)" <482254ac@razorfever.net>, "A.J. Kehoe IV \(Nanoman\)" X-BeenThere: freebsd-security@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "Security issues \[members-only posting\]" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Feb 2015 23:42:51 -0000 Jilles Tjoelker wrote this message on Sun, Feb 15, 2015 at 00:17 +0100: > On Tue, Feb 10, 2015 at 07:26:07AM -0500, Derek (freebsd lists) wrote: > > 3. updates userland to use this API, and removes totally the > > {crypt_set_format, login_setcryptfmt, login_getcryptfmt} APIs > > Removing API functions completely requires a SHLIB_MAJOR bump. I think > this can be avoided by replacing the functions with a stub instead, so > they would behave as if the default always applied and not allow changes > to it. It shouldn't be hard to support crypt_{get,set}_format, since default is supported for find_format... As for login_getcryptfmt, I can't find it... and if we keep crypt_{get,set}_format, we can keep login_setcryptfmt, but just mark them as deprecated... > > 4. switches crypt algorithms to use thread-local storage, so the > > good old global crypt buffer is thread-local > > This uses quite a bit of memory for each thread created, even if it does > not call crypt() at all. Fortunately, libcrypt is not commonly used. And not linked against normally, so, I don't see an issue... > Given that crypt() has never been thread-safe, consider implementing > crypt_r() as in glibc and leaving crypt() thread-unsafe. We should go full thread safe, though that requirese some work on most of the functions, as it appears that only sha256 and sha512 are safe... > Thread-local storage via pthread_key_create() (one key for libcrypt) is > still "magic" but reduces the memory waste for threads that do not call > crypt(). With the way the crypt is pluggable, sharing storage between implementations doesn't seem doable... Also, I just realized that crypt_sha256 and crypt_sha512 are not safe in their use of __thread... As the buffer isn't static, if the same thread calls again, it could be previously returns memory gets free'd by the realloc call... > rand_buf is a salt, not a secret, so clearing it afterwards is > unnecessary. > > Consider memcpy() and adding '\0' afterward instead of strncpy(). It > seems unnecessary to clear the buffer completely. I had thought of both of these before, and agree that the salt is not a secret (it is kept hidden), but, it leaks information, and _makesalt is called so rarely, that saving the time doesn't make sense... So, I'd prefer to keep the code as is WRT these points.. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."