From owner-freebsd-stable@freebsd.org Wed Mar 9 13:31:15 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3D62BAC8CD9 for ; Wed, 9 Mar 2016 13:31:15 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from resqmta-po-04v.sys.comcast.net (resqmta-po-04v.sys.comcast.net [IPv6:2001:558:fe16:19:96:114:154:163]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (Client CN "Bizanga Labs SMTP Client Certificate", Issuer "Bizanga Labs CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 160F0796 for ; Wed, 9 Mar 2016 13:31:14 +0000 (UTC) (envelope-from jdc@koitsu.org) Received: from resomta-po-07v.sys.comcast.net ([96.114.154.231]) by resqmta-po-04v.sys.comcast.net with comcast id TpXC1s0074zp9eg01pXDoy; Wed, 09 Mar 2016 13:31:13 +0000 Received: from koitsu.org ([69.181.142.213]) by resomta-po-07v.sys.comcast.net with comcast id TpXC1s0074cTVs501pXCu8; Wed, 09 Mar 2016 13:31:12 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id DD6BC1AF153; Wed, 9 Mar 2016 05:31:11 -0800 (PST) Date: Wed, 9 Mar 2016 05:31:11 -0800 From: Jeremy Chadwick To: freebsd-stable@freebsd.org Cc: cgreen@sentex.net, mike@sentex.net Subject: Re: svn commit: r296462 - in stable/9: crypto/openssl/crypto/bio crypto/openssl/crypto/bn crypto/openssl/doc/apps crypto/openssl/ssl secure/usr.bin/openssl/man Message-ID: <20160309133111.GA1035@icarus.home.lan> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=comcast.net; s=q20140121; t=1457530273; bh=7ZArI5zwCU5iygqSa5QvP+TkuQSfeU7JtrYyRrSvdy0=; h=Received:Received:Received:Date:From:To:Subject:Message-ID: MIME-Version:Content-Type; b=D2OODr0PhXw+D6T0iAbaW187/pmkAbtzocbI8h9ArOmn/D0nYWu30nGzohuq0S/37 UwnSjaaTn0ZJnYKq4bYRh2KtDI8Gj6mKfygzZeJlC1pg/B7KND+3rBIDSfF6ig3KpB tRnOGdC7TODC8ls4P9cgci1RFpF4JUqaO3rdsYQvd+gO1IZ8w8IlWHzyG6DcKA1Qu2 nBGUenp/brjN9Lt5u5sEsTE+jGwRSjRBtbjsm1Tv1C0DSOHYPuj+qyr5BBEz08a9YU 5bqXW9AF2AkbwlIcjGTK1+6ZbLB2iSlY4KRHFXAMAkvtLC/k0C/WC6TrZ4s7G2NN+A GefrjrzCDUboQ== X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Mar 2016 13:31:15 -0000 (Please keep me CC'd as I'm not subscribed to -stable) r296462 is either not ABI-compatible, or if it truly is, it breaks internal behavioural compatibility with libcrypto/libssl in some way. Building the below programs (fetchmail + postfix) from ports directly (i.e. source) **does not** fix the problem. Hope the gdb in fetchmail helps narrow down where the problem is. Don't ask me for "bt full" output, as it's pointless since none of the system libs are built with -g/-g3/-ggdb. I have no problems with SSH (unlike Mike), but that means very little given configuration differences and setups. Rolling back to r296461 (i.e. svn up -r296461) rectifies the problem fully. If jkim@ et al need a box running r296462 w/ full root to troubleshoot this, let me know and I can set one up. Might take a day or two though. $ fetchmail -a -v fetchmail: removing stale lockfile fetchmail: 6.3.26 querying mambo.koitsu.org (protocol IMAP) at Wed 9 Mar 04:55:16 2016: poll started Trying to connect to 104.238.183.73/993...connected. fetchmail: Server certificate: fetchmail: Issuer Organisation: koitsu.org fetchmail: Issuer CommonName: mambo.koitsu.org fetchmail: Subject CommonName: mambo.koitsu.org fetchmail: mambo.koitsu.org key fingerprint: F4:35:18:75:88:92:BF:1C:82:14:9E:17:EC:7E:3D:1C fetchmail: mambo.koitsu.org fingerprints match. fetchmail: Server certificate: fetchmail: Issuer Organisation: koitsu.org fetchmail: Issuer CommonName: mambo.koitsu.org fetchmail: Subject CommonName: mambo.koitsu.org Segmentation fault: 11 (core dumped) $ gdb /usr/local/bin/fetchmail fetchmail.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"...(no debugging symbols found)... Core was generated by `fetchmail'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/local/lib/libintl.so.8...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libintl.so.8 Reading symbols from /usr/lib/libopie.so.7...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libopie.so.7 Reading symbols from /lib/libcrypt.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypt.so.5 Reading symbols from /lib/libkvm.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libkvm.so.5 Reading symbols from /usr/lib/libcom_err.so.5...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcom_err.so.5 Reading symbols from /usr/lib/libssl.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.6 Reading symbols from /lib/libcrypto.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypto.so.6 Reading symbols from /usr/lib/libgssapi.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgssapi.so.10 Reading symbols from /usr/lib/libheimntlm.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libheimntlm.so.10 Reading symbols from /usr/lib/libkrb5.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libkrb5.so.10 Reading symbols from /usr/lib/libhx509.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libhx509.so.10 Reading symbols from /usr/lib/libasn1.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libasn1.so.10 Reading symbols from /usr/lib/libroken.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libroken.so.10 Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/local/lib/libiconv.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/local/lib/libiconv.so.2 Reading symbols from /lib/libmd.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libmd.so.5 Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x0000000801616774 in BN_mod_exp_mont_consttime () from /lib/libcrypto.so.6 (gdb) bt #0 0x0000000801616774 in BN_mod_exp_mont_consttime () from /lib/libcrypto.so.6 #1 0x00000008015f79f7 in DH_OpenSSL () from /lib/libcrypto.so.6 #2 0x00000008012c8d25 in ssl3_send_client_key_exchange () from /usr/lib/libssl.so.6 #3 0x00000008012cc0ab in ssl3_connect () from /usr/lib/libssl.so.6 #4 0x00000008012c7d04 in ssl23_connect () from /usr/lib/libssl.so.6 #5 0x00000000004052bf in ?? () #6 0x000000000040e360 in ?? () #7 0x000000000040813d in ?? () #8 0x000000000040a69a in ?? () #9 0x0000000000404e01 in ?? () #10 0x000000080065c000 in ?? () #11 0x0000000000000000 in ?? () (gdb) q Also tried to send mail to myself locally, as postfix's smtp(8) links to libcrypt/libssl/libcrypto. Bzzt, nope: pid 5046 (smtp), uid 125: exited on signal 11 Mar 9 04:49:38 icarus postfix/master[802]: daemon started -- version 3.1.0, configuration /usr/local/etc/postfix Mar 9 04:54:38 icarus postfix/pickup[5043]: 1835D1AF150: uid=1000 from= Mar 9 04:54:38 icarus postfix/cleanup[5044]: 1835D1AF150: message-id=<20160309125438.GA5033@icarus.home.lan> Mar 9 04:54:38 icarus postfix/qmgr[804]: 1835D1AF150: from=, size=631, nrcpt=1 (queue active) Mar 9 04:54:38 icarus postfix/qmgr[804]: warning: private/smtp socket: malformed response Mar 9 04:54:38 icarus postfix/qmgr[804]: warning: transport smtp failure -- see a previous warning/fatal/panic logfile record for the problem description Mar 9 04:54:38 icarus postfix/master[802]: warning: process /usr/local/libexec/postfix/smtp pid 5046 killed by signal 11 Mar 9 04:54:38 icarus postfix/master[802]: warning: /usr/local/libexec/postfix/smtp: bad command startup -- throttling Mar 9 04:54:38 icarus postfix/error[5048]: 1835D1AF150: to=, relay=none, delay=0.5, delays=0.05/0.44/0/0.01, dsn=4.3.0, status=deferred (unknown mail transport error) -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB |