From owner-freebsd-stable@freebsd.org Fri Sep 6 23:55:04 2019 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0276BDB6B2 for ; Fri, 6 Sep 2019 23:55:04 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from smtp-out-so.shaw.ca (smtp-out-so.shaw.ca [64.59.136.139]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 46QDv71hplz3KXg for ; Fri, 6 Sep 2019 23:54:58 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from spqr.komquats.com ([70.67.125.17]) by shaw.ca with ESMTPA id 6O40iKiNKIhW96O42iuFDd; Fri, 06 Sep 2019 17:54:54 -0600 X-Authority-Analysis: v=2.3 cv=FcFJO626 c=1 sm=1 tr=0 a=VFtTW3WuZNDh6VkGe7fA3g==:117 a=VFtTW3WuZNDh6VkGe7fA3g==:17 a=kj9zAlcOel0A:10 a=J70Eh1EUuV4A:10 a=EkcXrb_YAAAA:8 a=-i4I5E_jAAAA:8 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=J9wUqfX70oTMe43JHpkA:9 a=CjuIK1q_8ugA:10 a=LK5xJRSDVpKd5WXXoEvA:22 a=YQreJwxzuLcQAHRr27xt:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 a=pHzHmUro8NiASowvMSCR:22 a=nt3jZW36AmriUCFCBwmW:22 Received: from slippy.cwsent.com (slippy8 [10.2.2.6]) by spqr.komquats.com (Postfix) with ESMTPS id 26A058CC; Fri, 6 Sep 2019 16:54:48 -0700 (PDT) Received: from slippy.cwsent.com (localhost [127.0.0.1]) by slippy.cwsent.com (8.15.2/8.15.2) with ESMTP id x86Nslvd003751; Fri, 6 Sep 2019 16:54:47 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Received: from slippy (cy@localhost) by slippy.cwsent.com (8.15.2/8.15.2/Submit) with ESMTP id x86Nskfg003748; Fri, 6 Sep 2019 16:54:46 -0700 (PDT) (envelope-from Cy.Schubert@cschubert.com) Message-Id: <201909062354.x86Nskfg003748@slippy.cwsent.com> X-Authentication-Warning: slippy.cwsent.com: cy owned process doing -bs X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.7.1 Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Harlan Stenn cc: Cy Schubert , =?UTF-8?Q?Trond_Endrest=c3=b8l?= , freebsd-stable@freebsd.org Subject: Re: ntpd doesn't like ASLR on stable/12 post-r350672 In-reply-to: <9b0c95de-2d0e-89b4-32e6-63ec5af729b4@nwtime.org> References: <201909060639.x866dL68090189@slippy.cwsent.com> <9b0c95de-2d0e-89b4-32e6-63ec5af729b4@nwtime.org> Comments: In-reply-to Harlan Stenn message dated "Fri, 06 Sep 2019 10:51:46 -0700." Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Fri, 06 Sep 2019 16:54:46 -0700 X-CMAE-Envelope: MS4wfOZyr82Hy8MErF/jfvgiHh/3CSw8pkMPA8XXx1KhxZ/fQ+bxz0c2SBCTLpH/oiLdwUVv7qa8Eji/auMlLxPQFQz1UoIu70icuRuAhnwhCSJmo96Qhrwk KFfhtMfJc6wYhY8lM0VX11rQN6S3mZoY4Ycg1+rww4vs3ckWAjBr5LNK/UVR3ywKnR04GQ1NGfD8gX25xC9BgJG/u7hgQL+gs/YvOmHg9SsKAe7SHptt2t0+ sWmUZ4ttFWu1ywwprTrLsQ== X-Rspamd-Queue-Id: 46QDv71hplz3KXg X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=none; spf=none (mx1.freebsd.org: domain of cy.schubert@cschubert.com has no SPF policy when checking 64.59.136.139) smtp.mailfrom=cy.schubert@cschubert.com X-Spamd-Result: default: False [-4.82 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; HAS_REPLYTO(0.00)[Cy.Schubert@cschubert.com]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; HAS_XAW(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLYTO_EQ_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.988,0]; RCVD_IN_DNSWL_NONE(0.00)[139.136.59.64.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_SPF_NA(0.00)[]; RECEIVED_SPAMHAUS_PBL(0.00)[17.125.67.70.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.11]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:6327, ipnet:64.59.128.0/20, country:CA]; RCVD_TLS_LAST(0.00)[]; IP_SCORE(-2.23)[ip: (-5.58), ipnet: 64.59.128.0/20(-3.10), asn: 6327(-2.41), country: CA(-0.09)]; FROM_EQ_ENVFROM(0.00)[] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2019 23:55:04 -0000 In message <9b0c95de-2d0e-89b4-32e6-63ec5af729b4@nwtime.org>, Harlan Stenn writ es: > Hi folks, > > It's easy for us to customize the default value for DFLT_RLIMIT_STACK on > a per-version and per-OS basis. We do this in the "configure" machinery. > > The defaults are expected to be "generally sane", and were likely chosen > a relatively long time ago. > > What would you like the value(s) to be for what versions of FreeBSD? > > H > > On 9/5/2019 11:39 PM, Cy Schubert wrote: > > In message . > > inf > > o>, =?UTF-8?Q?Trond_Endrest=C3=B8l?= writes: > >> Hi, > >> > >> I'm running stable/12 with ASLR enabled in /etc/sysctl.conf: > >> > >> kern.elf64.aslr.enable=1 > >> kern.elf64.aslr.pie_enable=1 > >> kern.elf32.aslr.enable=1 > >> kern.elf32.aslr.pie_enable=1 > >> > >> After upgrading to anything after r350672, now at r351450, ntpd > >> refuses to start at boot. > >> > >> Aug 24 21:25:42 HOSTNAME ntpd[5618]: ntpd 4.2.8p12-a (1): Sta > rti > >> ng > >> Aug 24 21:25:43 HOSTNAME kernel: [406] pid 5619 (ntpd), jid 0, > ui > >> d 123: exited on signal 11 > >> > >> Disabling ASLR, kern.elf64.aslr.enable=0, before starting ntpd > >> manually is a workaround, but this is not viable in the long run. > >> > >> I tried changing command="/usr/sbin/${name}" to > >> command="/usr/bin/proccontrol -m aslr -s disable /usr/sbin/${name}" in > >> /etc/rc.d/ntpd, but that didn't go well. > > > > For now, until this can be solved, add this to your rc.conf: > > > > ntpd_prepend="/usr/bin/proccontrol -m aslr -s disable" > > > >> > >> Running ntpd through gdb while ASLR was enabled, I narrowed it down to > >> /usr/src/contrib/ntp/ntpd/ntpd.c:1001 > >> > >> ntp_rlimit(RLIMIT_STACK, DFLT_RLIMIT_STACK * 4096, 4096, "4k"); > >> > >> which calls /usr/src/contrib/ntp/ntpd/ntp_config.c:5211 and proceeds > >> to /usr/src/contrib/ntp/ntpd/ntp_config.c:5254 > >> > >> if (-1 == getrlimit(RLIMIT_STACK, &rl)) { > >> > >> Single stepping from this point gave me: > >> > >> ==== > >> > >> (gdb) s > >> _thr_rtld_set_flag (mask=1) at /usr/src/lib/libthr/thread/thr_rtld.c:171 > >> 171 { > >> (gdb) > >> 176 return (0); > >> (gdb) > >> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/t > hr_ > >> rtld.c:115 > >> 115 { > >> (gdb) > >> 120 curthread = _get_curthread(); > >> (gdb) > >> _get_curthread () at /usr/src/lib/libthr/arch/amd64/include/pthread_md.h:9 > 7 > >> 97 return (TCB_GET64(tcb_thread)); > >> (gdb) > >> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/t > hr_ > >> rtld.c:121 > >> 121 SAVE_ERRNO(); > >> (gdb) > >> 124 THR_CRITICAL_ENTER(curthread); > >> (gdb) > >> _thr_rwlock_tryrdlock (rwlock=, flags=0) at /usr/src/lib/li > bth > >> r/thread/thr_umtx.h:192 > >> 192 (rwlock->rw_flags & URWLOCK_PREFER_READER) != 0) > >> (gdb) > >> 191 if ((flags & URWLOCK_PREFER_READER) != 0 || > >> (gdb) > >> 197 while (!(state & wrflags)) { > >> (gdb) > >> 201 if (atomic_cmpset_acq_32(&rwlock->rw_state, state, > st > >> ate + 1)) > >> (gdb) > >> atomic_cmpset_int (dst=, expect=, src=1) at > /us > >> r/obj/usr/src/amd64.amd64/tmp/usr/include/machine/atomic.h:220 > >> 220 ATOMIC_CMPSET(int); > >> (gdb) > >> _thr_rwlock_tryrdlock (rwlock=, flags=0) at /usr/src/lib/li > bth > >> r/thread/thr_umtx.h:201 > >> 201 if (atomic_cmpset_acq_32(&rwlock->rw_state, state, > st > >> ate + 1)) > >> (gdb) > >> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/t > hr_ > >> rtld.c:127 > >> 127 curthread->rdlock_count++; > >> (gdb) > >> 128 RESTORE_ERRNO(); > >> (gdb) > >> 129 } > >> (gdb) > >> _thr_rtld_clr_flag (mask=1) at /usr/src/lib/libthr/thread/thr_rtld.c:181 > >> 181 { > >> (gdb) > >> 182 return (0); > >> (gdb) > >> _thr_rtld_lock_release (lock=0x80180d200) at /usr/src/lib/libthr/thread/th > r_r > >> tld.c:150 > >> 150 { > >> (gdb) > >> _get_curthread () at /usr/src/lib/libthr/arch/amd64/include/pthread_md.h:9 > 7 > >> 97 return (TCB_GET64(tcb_thread)); > >> (gdb) > >> _thr_rtld_lock_release (lock=0x80180d200) at /usr/src/lib/libthr/thread/th > r_r > >> tld.c:157 > >> 157 SAVE_ERRNO(); > >> (gdb) > >> 160 state = l->lock.rw_state; > >> (gdb) > >> 161 if (_thr_rwlock_unlock(&l->lock) == 0) { > >> (gdb) > >> _thr_rwlock_unlock (rwlock=0x80180d200) at /usr/src/lib/libthr/thread/thr_ > umt > >> x.h:249 > >> 249 state = rwlock->rw_state; > >> (gdb) > >> 250 if ((state & URWLOCK_WRITE_OWNER) != 0) { > >> (gdb) > >> 256 if (__predict_false(URWLOCK_READER_COUNT(s > tat > >> e) == 0)) > >> (gdb) > >> 260 URWLOCK_READER_COUNT(state) == 1)) > >> { > >> (gdb) > >> 259 URWLOCK_READ_WAITERS)) != 0 && > >> (gdb) > >> 262 state, state - 1)) > >> (gdb) > >> 261 if (atomic_cmpset_rel_32(&rwlock-> > rw_ > >> state, > >> (gdb) > >> atomic_cmpset_int (dst=, expect=, src=0) at > /us > >> r/obj/usr/src/amd64.amd64/tmp/usr/include/machine/atomic.h:220 > >> 220 ATOMIC_CMPSET(int); > >> (gdb) > >> _thr_rwlock_unlock (rwlock=0x80180d200) at /usr/src/lib/libthr/thread/thr_ > umt > >> x.h:261 > >> 261 if (atomic_cmpset_rel_32(&rwlock-> > rw_ > >> state, > >> (gdb) > >> _thr_rtld_lock_release (lock=) at /usr/src/lib/libthr/threa > d/t > >> hr_rtld.c:162 > >> 162 if ((state & URWLOCK_WRITE_OWNER) == 0) > >> (gdb) > >> 163 curthread->rdlock_count--; > >> (gdb) > >> 164 THR_CRITICAL_LEAVE(curthread); > >> (gdb) > >> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:2 > 71 > >> 271 if (!THR_IN_CRITICAL(curthread)) { > >> (gdb) > >> 272 check_deferred_signal(curthread); > >> (gdb) > >> check_deferred_signal (curthread=0x80864b000) at /usr/src/lib/libthr/threa > d/t > >> hr_sig.c:332 > >> 332 if (__predict_true(curthread->deferred_siginfo.si_signo == > 0 > >> || > >> (gdb) > >> 351 } > >> (gdb) > >> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:2 > 73 > >> 273 check_suspend(curthread); > >> (gdb) > >> check_suspend (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_si > g.c > >> :358 > >> 358 if (__predict_true((curthread->flags & > >> (gdb) > >> 401 } > >> (gdb) > >> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:2 > 74 > >> 274 check_cancel(curthread, NULL); > >> (gdb) > >> check_cancel (curthread=0x80864b000, ucp=0x0) at /usr/src/lib/libthr/threa > d/t > >> hr_sig.c:283 > >> 283 if (__predict_true(!curthread->cancel_pending || > >> (gdb) > >> _thr_ast (curthread=) at /usr/src/lib/libthr/thread/thr_sig > .c: > >> 276 > >> 276 } > >> (gdb) > >> _thr_rtld_lock_release (lock=) at /usr/src/lib/libthr/threa > d/t > >> hr_rtld.c:166 > >> 166 RESTORE_ERRNO(); > >> (gdb) > >> 167 } > >> (gdb) > >> getrlimit () at getrlimit.S:3 > >> 3 RSYSCALL(getrlimit) > >> (gdb) > >> ntp_rlimit (rl_what=, rl_value=204800, rl_scale= out > >>> , rl_sstr=) at /usr/src/contrib/ntp/ntpd/ntp_config.c:5257 > >> 5257 if (rl_value > rl.rlim_max) { > >> (gdb) > >> 5264 rl.rlim_cur = rl_value; > >> (gdb) > >> 5265 if (-1 == setrlimit(RLIMIT_STACK, &rl)) { > >> (gdb) > >> _thr_rtld_set_flag (mask=1) at /usr/src/lib/libthr/thread/thr_rtld.c:171 > >> 171 { > >> (gdb) > >> 176 return (0); > >> (gdb) > >> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/t > hr_ > >> rtld.c:115 > >> 115 { > >> (gdb) > >> 120 curthread = _get_curthread(); > >> (gdb) > >> _get_curthread () at /usr/src/lib/libthr/arch/amd64/include/pthread_md.h:9 > 7 > >> 97 return (TCB_GET64(tcb_thread)); > >> (gdb) > >> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/t > hr_ > >> rtld.c:121 > >> 121 SAVE_ERRNO(); > >> (gdb) > >> 124 THR_CRITICAL_ENTER(curthread); > >> (gdb) > >> _thr_rwlock_tryrdlock (rwlock=, flags=0) at /usr/src/lib/li > bth > >> r/thread/thr_umtx.h:192 > >> 192 (rwlock->rw_flags & URWLOCK_PREFER_READER) != 0) > >> (gdb) > >> 191 if ((flags & URWLOCK_PREFER_READER) != 0 || > >> (gdb) > >> 197 while (!(state & wrflags)) { > >> (gdb) > >> 201 if (atomic_cmpset_acq_32(&rwlock->rw_state, state, > st > >> ate + 1)) > >> (gdb) > >> atomic_cmpset_int (dst=, expect=, src=1) at > /us > >> r/obj/usr/src/amd64.amd64/tmp/usr/include/machine/atomic.h:220 > >> 220 ATOMIC_CMPSET(int); > >> (gdb) > >> _thr_rwlock_tryrdlock (rwlock=, flags=0) at /usr/src/lib/li > bth > >> r/thread/thr_umtx.h:201 > >> 201 if (atomic_cmpset_acq_32(&rwlock->rw_state, state, > st > >> ate + 1)) > >> (gdb) > >> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/t > hr_ > >> rtld.c:127 > >> 127 curthread->rdlock_count++; > >> (gdb) > >> 128 RESTORE_ERRNO(); > >> (gdb) > >> 129 } > >> (gdb) > >> _thr_rtld_clr_flag (mask=1) at /usr/src/lib/libthr/thread/thr_rtld.c:181 > >> 181 { > >> (gdb) > >> 182 return (0); > >> (gdb) > >> _thr_rtld_lock_release (lock=0x80180d200) at /usr/src/lib/libthr/thread/th > r_r > >> tld.c:150 > >> 150 { > >> (gdb) > >> _get_curthread () at /usr/src/lib/libthr/arch/amd64/include/pthread_md.h:9 > 7 > >> 97 return (TCB_GET64(tcb_thread)); > >> (gdb) > >> _thr_rtld_lock_release (lock=0x80180d200) at /usr/src/lib/libthr/thread/th > r_r > >> tld.c:157 > >> 157 SAVE_ERRNO(); > >> (gdb) > >> 160 state = l->lock.rw_state; > >> (gdb) > >> 161 if (_thr_rwlock_unlock(&l->lock) == 0) { > >> (gdb) > >> _thr_rwlock_unlock (rwlock=0x80180d200) at /usr/src/lib/libthr/thread/thr_ > umt > >> x.h:249 > >> 249 state = rwlock->rw_state; > >> (gdb) > >> 250 if ((state & URWLOCK_WRITE_OWNER) != 0) { > >> (gdb) > >> 256 if (__predict_false(URWLOCK_READER_COUNT(s > tat > >> e) == 0)) > >> (gdb) > >> 260 URWLOCK_READER_COUNT(state) == 1)) { > >> (gdb) > >> 259 URWLOCK_READ_WAITERS)) != 0 && > >> (gdb) > >> 262 state, state - 1)) > >> (gdb) > >> 261 if (atomic_cmpset_rel_32(&rwlock-> > rw_ > >> state, > >> (gdb) > >> atomic_cmpset_int (dst=, expect=, src=0) at > /us > >> r/obj/usr/src/amd64.amd64/tmp/usr/include/machine/atomic.h:220 > >> 220 ATOMIC_CMPSET(int); > >> (gdb) > >> _thr_rwlock_unlock (rwlock=0x80180d200) at /usr/src/lib/libthr/thread/thr_ > umt > >> x.h:261 > >> 261 if (atomic_cmpset_rel_32(&rwlock-> > rw_ > >> state, > >> (gdb) > >> _thr_rtld_lock_release (lock=) at /usr/src/lib/libthr/threa > d/t > >> hr_rtld.c:162 > >> 162 if ((state & URWLOCK_WRITE_OWNER) == 0) > >> (gdb) > >> 163 curthread->rdlock_count--; > >> (gdb) > >> 164 THR_CRITICAL_LEAVE(curthread); > >> (gdb) > >> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:2 > 71 > >> 271 if (!THR_IN_CRITICAL(curthread)) { > >> (gdb) > >> 272 check_deferred_signal(curthread); > >> (gdb) > >> check_deferred_signal (curthread=0x80864b000) at /usr/src/lib/libthr/threa > d/t > >> hr_sig.c:332 > >> 332 if > >> (__predict_true(curthread->deferred_siginfo.si_signo == 0 || > >> (gdb) > >> 351 } > >> (gdb) > >> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:2 > 73 > >> 273 check_suspend(curthread); > >> (gdb) > >> check_suspend (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_si > g.c > >> :358 > >> 358 if (__predict_true((curthread->flags & > >> (gdb) > >> 401 } > >> (gdb) > >> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:2 > 74 > >> 274 check_cancel(curthread, NULL); > >> (gdb) > >> check_cancel (curthread=0x80864b000, ucp=0x0) at /usr/src/lib/libthr/threa > d/t > >> hr_sig.c:283 > >> 283 if (__predict_true(!curthread->cancel_pending || > >> (gdb) > >> _thr_ast (curthread=) at /usr/src/lib/libthr/thread/thr_sig > .c: > >> 276 > >> 276 } > >> (gdb) > >> _thr_rtld_lock_release (lock=) at /usr/src/lib/libthr/threa > d/t > >> hr_rtld.c:166 > >> 166 RESTORE_ERRNO(); > >> (gdb) > >> 167 } > >> (gdb) > >> setrlimit () at setrlimit.S:3 > >> 3 RSYSCALL(setrlimit) > >> (gdb) > >> > >> Program received signal SIGSEGV, Segmentation fault. > >> setrlimit () at setrlimit.S:3 > >> 3 RSYSCALL(setrlimit) > >> (gdb) > >> > >> Program terminated with signal SIGSEGV, Segmentation fault. > >> The program no longer exists. > >> (gdb) q > >> > >> ==== > >> > >> I'm sorry for the long post. Is there anything (else) I can do to > >> further narrow it down? > > > > I've been able to confirm that kib@'s hunch regarding the gap is correct. > > > > Use the workaround until this can be solved. > > > > A better workaround is, for people who wish to use ASLR, is to add this to /etc/ntp.conf: rlimit memlock 64 Currently the default is 32 MB. This sets the limit to 64. -- Cheers, Cy Schubert FreeBSD UNIX: Web: http://www.FreeBSD.org The need of the many outweighs the greed of the few.