Date: Fri, 6 Sep 2019 10:51:46 -0700 From: Harlan Stenn <stenn@nwtime.org> To: Cy Schubert <Cy.Schubert@cschubert.com>, =?UTF-8?Q?Trond_Endrest=c3=b8l?= <trond.endrestol@ximalas.info> Cc: freebsd-stable@freebsd.org Subject: Re: ntpd doesn't like ASLR on stable/12 post-r350672 Message-ID: <9b0c95de-2d0e-89b4-32e6-63ec5af729b4@nwtime.org> In-Reply-To: <201909060639.x866dL68090189@slippy.cwsent.com> References: <alpine.BSF.2.21.99999.352.1908242135380.6386@enterprise.ximalas.info> <201909060639.x866dL68090189@slippy.cwsent.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi folks,
It's easy for us to customize the default value for DFLT_RLIMIT_STACK on
a per-version and per-OS basis. We do this in the "configure" machinery.
The defaults are expected to be "generally sane", and were likely chosen
a relatively long time ago.
What would you like the value(s) to be for what versions of FreeBSD?
H
On 9/5/2019 11:39 PM, Cy Schubert wrote:
> In message <alpine.BSF.2.21.99999.352.1908242135380.6386@enterprise.ximalas.
> inf
> o>, =?UTF-8?Q?Trond_Endrest=C3=B8l?= writes:
>> Hi,
>>
>> I'm running stable/12 with ASLR enabled in /etc/sysctl.conf:
>>
>> kern.elf64.aslr.enable=1
>> kern.elf64.aslr.pie_enable=1
>> kern.elf32.aslr.enable=1
>> kern.elf32.aslr.pie_enable=1
>>
>> After upgrading to anything after r350672, now at r351450, ntpd
>> refuses to start at boot.
>>
>> Aug 24 21:25:42 <ntp.notice> HOSTNAME ntpd[5618]: ntpd 4.2.8p12-a (1): Starti
>> ng
>> Aug 24 21:25:43 <kern.info> HOSTNAME kernel: [406] pid 5619 (ntpd), jid 0, ui
>> d 123: exited on signal 11
>>
>> Disabling ASLR, kern.elf64.aslr.enable=0, before starting ntpd
>> manually is a workaround, but this is not viable in the long run.
>>
>> I tried changing command="/usr/sbin/${name}" to
>> command="/usr/bin/proccontrol -m aslr -s disable /usr/sbin/${name}" in
>> /etc/rc.d/ntpd, but that didn't go well.
>
> For now, until this can be solved, add this to your rc.conf:
>
> ntpd_prepend="/usr/bin/proccontrol -m aslr -s disable"
>
>>
>> Running ntpd through gdb while ASLR was enabled, I narrowed it down to
>> /usr/src/contrib/ntp/ntpd/ntpd.c:1001
>>
>> ntp_rlimit(RLIMIT_STACK, DFLT_RLIMIT_STACK * 4096, 4096, "4k");
>>
>> which calls /usr/src/contrib/ntp/ntpd/ntp_config.c:5211 and proceeds
>> to /usr/src/contrib/ntp/ntpd/ntp_config.c:5254
>>
>> if (-1 == getrlimit(RLIMIT_STACK, &rl)) {
>>
>> Single stepping from this point gave me:
>>
>> ====
>>
>> (gdb) s
>> _thr_rtld_set_flag (mask=1) at /usr/src/lib/libthr/thread/thr_rtld.c:171
>> 171 {
>> (gdb)
>> 176 return (0);
>> (gdb)
>> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/thr_
>> rtld.c:115
>> 115 {
>> (gdb)
>> 120 curthread = _get_curthread();
>> (gdb)
>> _get_curthread () at /usr/src/lib/libthr/arch/amd64/include/pthread_md.h:97
>> 97 return (TCB_GET64(tcb_thread));
>> (gdb)
>> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/thr_
>> rtld.c:121
>> 121 SAVE_ERRNO();
>> (gdb)
>> 124 THR_CRITICAL_ENTER(curthread);
>> (gdb)
>> _thr_rwlock_tryrdlock (rwlock=<optimized out>, flags=0) at /usr/src/lib/libth
>> r/thread/thr_umtx.h:192
>> 192 (rwlock->rw_flags & URWLOCK_PREFER_READER) != 0)
>> (gdb)
>> 191 if ((flags & URWLOCK_PREFER_READER) != 0 ||
>> (gdb)
>> 197 while (!(state & wrflags)) {
>> (gdb)
>> 201 if (atomic_cmpset_acq_32(&rwlock->rw_state, state, st
>> ate + 1))
>> (gdb)
>> atomic_cmpset_int (dst=<optimized out>, expect=<optimized out>, src=1) at /us
>> r/obj/usr/src/amd64.amd64/tmp/usr/include/machine/atomic.h:220
>> 220 ATOMIC_CMPSET(int);
>> (gdb)
>> _thr_rwlock_tryrdlock (rwlock=<optimized out>, flags=0) at /usr/src/lib/libth
>> r/thread/thr_umtx.h:201
>> 201 if (atomic_cmpset_acq_32(&rwlock->rw_state, state, st
>> ate + 1))
>> (gdb)
>> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/thr_
>> rtld.c:127
>> 127 curthread->rdlock_count++;
>> (gdb)
>> 128 RESTORE_ERRNO();
>> (gdb)
>> 129 }
>> (gdb)
>> _thr_rtld_clr_flag (mask=1) at /usr/src/lib/libthr/thread/thr_rtld.c:181
>> 181 {
>> (gdb)
>> 182 return (0);
>> (gdb)
>> _thr_rtld_lock_release (lock=0x80180d200) at /usr/src/lib/libthr/thread/thr_r
>> tld.c:150
>> 150 {
>> (gdb)
>> _get_curthread () at /usr/src/lib/libthr/arch/amd64/include/pthread_md.h:97
>> 97 return (TCB_GET64(tcb_thread));
>> (gdb)
>> _thr_rtld_lock_release (lock=0x80180d200) at /usr/src/lib/libthr/thread/thr_r
>> tld.c:157
>> 157 SAVE_ERRNO();
>> (gdb)
>> 160 state = l->lock.rw_state;
>> (gdb)
>> 161 if (_thr_rwlock_unlock(&l->lock) == 0) {
>> (gdb)
>> _thr_rwlock_unlock (rwlock=0x80180d200) at /usr/src/lib/libthr/thread/thr_umt
>> x.h:249
>> 249 state = rwlock->rw_state;
>> (gdb)
>> 250 if ((state & URWLOCK_WRITE_OWNER) != 0) {
>> (gdb)
>> 256 if (__predict_false(URWLOCK_READER_COUNT(stat
>> e) == 0))
>> (gdb)
>> 260 URWLOCK_READER_COUNT(state) == 1))
>> {
>> (gdb)
>> 259 URWLOCK_READ_WAITERS)) != 0 &&
>> (gdb)
>> 262 state, state - 1))
>> (gdb)
>> 261 if (atomic_cmpset_rel_32(&rwlock->rw_
>> state,
>> (gdb)
>> atomic_cmpset_int (dst=<optimized out>, expect=<optimized out>, src=0) at /us
>> r/obj/usr/src/amd64.amd64/tmp/usr/include/machine/atomic.h:220
>> 220 ATOMIC_CMPSET(int);
>> (gdb)
>> _thr_rwlock_unlock (rwlock=0x80180d200) at /usr/src/lib/libthr/thread/thr_umt
>> x.h:261
>> 261 if (atomic_cmpset_rel_32(&rwlock->rw_
>> state,
>> (gdb)
>> _thr_rtld_lock_release (lock=<optimized out>) at /usr/src/lib/libthr/thread/t
>> hr_rtld.c:162
>> 162 if ((state & URWLOCK_WRITE_OWNER) == 0)
>> (gdb)
>> 163 curthread->rdlock_count--;
>> (gdb)
>> 164 THR_CRITICAL_LEAVE(curthread);
>> (gdb)
>> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:271
>> 271 if (!THR_IN_CRITICAL(curthread)) {
>> (gdb)
>> 272 check_deferred_signal(curthread);
>> (gdb)
>> check_deferred_signal (curthread=0x80864b000) at /usr/src/lib/libthr/thread/t
>> hr_sig.c:332
>> 332 if (__predict_true(curthread->deferred_siginfo.si_signo == 0
>> ||
>> (gdb)
>> 351 }
>> (gdb)
>> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:273
>> 273 check_suspend(curthread);
>> (gdb)
>> check_suspend (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c
>> :358
>> 358 if (__predict_true((curthread->flags &
>> (gdb)
>> 401 }
>> (gdb)
>> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:274
>> 274 check_cancel(curthread, NULL);
>> (gdb)
>> check_cancel (curthread=0x80864b000, ucp=0x0) at /usr/src/lib/libthr/thread/t
>> hr_sig.c:283
>> 283 if (__predict_true(!curthread->cancel_pending ||
>> (gdb)
>> _thr_ast (curthread=<optimized out>) at /usr/src/lib/libthr/thread/thr_sig.c:
>> 276
>> 276 }
>> (gdb)
>> _thr_rtld_lock_release (lock=<optimized out>) at /usr/src/lib/libthr/thread/t
>> hr_rtld.c:166
>> 166 RESTORE_ERRNO();
>> (gdb)
>> 167 }
>> (gdb)
>> getrlimit () at getrlimit.S:3
>> 3 RSYSCALL(getrlimit)
>> (gdb)
>> ntp_rlimit (rl_what=<optimized out>, rl_value=204800, rl_scale=<optimized out
>>> , rl_sstr=<optimized out>) at /usr/src/contrib/ntp/ntpd/ntp_config.c:5257
>> 5257 if (rl_value > rl.rlim_max) {
>> (gdb)
>> 5264 rl.rlim_cur = rl_value;
>> (gdb)
>> 5265 if (-1 == setrlimit(RLIMIT_STACK, &rl)) {
>> (gdb)
>> _thr_rtld_set_flag (mask=1) at /usr/src/lib/libthr/thread/thr_rtld.c:171
>> 171 {
>> (gdb)
>> 176 return (0);
>> (gdb)
>> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/thr_
>> rtld.c:115
>> 115 {
>> (gdb)
>> 120 curthread = _get_curthread();
>> (gdb)
>> _get_curthread () at /usr/src/lib/libthr/arch/amd64/include/pthread_md.h:97
>> 97 return (TCB_GET64(tcb_thread));
>> (gdb)
>> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/thr_
>> rtld.c:121
>> 121 SAVE_ERRNO();
>> (gdb)
>> 124 THR_CRITICAL_ENTER(curthread);
>> (gdb)
>> _thr_rwlock_tryrdlock (rwlock=<optimized out>, flags=0) at /usr/src/lib/libth
>> r/thread/thr_umtx.h:192
>> 192 (rwlock->rw_flags & URWLOCK_PREFER_READER) != 0)
>> (gdb)
>> 191 if ((flags & URWLOCK_PREFER_READER) != 0 ||
>> (gdb)
>> 197 while (!(state & wrflags)) {
>> (gdb)
>> 201 if (atomic_cmpset_acq_32(&rwlock->rw_state, state, st
>> ate + 1))
>> (gdb)
>> atomic_cmpset_int (dst=<optimized out>, expect=<optimized out>, src=1) at /us
>> r/obj/usr/src/amd64.amd64/tmp/usr/include/machine/atomic.h:220
>> 220 ATOMIC_CMPSET(int);
>> (gdb)
>> _thr_rwlock_tryrdlock (rwlock=<optimized out>, flags=0) at /usr/src/lib/libth
>> r/thread/thr_umtx.h:201
>> 201 if (atomic_cmpset_acq_32(&rwlock->rw_state, state, st
>> ate + 1))
>> (gdb)
>> _thr_rtld_rlock_acquire (lock=0x80180d200) at /usr/src/lib/libthr/thread/thr_
>> rtld.c:127
>> 127 curthread->rdlock_count++;
>> (gdb)
>> 128 RESTORE_ERRNO();
>> (gdb)
>> 129 }
>> (gdb)
>> _thr_rtld_clr_flag (mask=1) at /usr/src/lib/libthr/thread/thr_rtld.c:181
>> 181 {
>> (gdb)
>> 182 return (0);
>> (gdb)
>> _thr_rtld_lock_release (lock=0x80180d200) at /usr/src/lib/libthr/thread/thr_r
>> tld.c:150
>> 150 {
>> (gdb)
>> _get_curthread () at /usr/src/lib/libthr/arch/amd64/include/pthread_md.h:97
>> 97 return (TCB_GET64(tcb_thread));
>> (gdb)
>> _thr_rtld_lock_release (lock=0x80180d200) at /usr/src/lib/libthr/thread/thr_r
>> tld.c:157
>> 157 SAVE_ERRNO();
>> (gdb)
>> 160 state = l->lock.rw_state;
>> (gdb)
>> 161 if (_thr_rwlock_unlock(&l->lock) == 0) {
>> (gdb)
>> _thr_rwlock_unlock (rwlock=0x80180d200) at /usr/src/lib/libthr/thread/thr_umt
>> x.h:249
>> 249 state = rwlock->rw_state;
>> (gdb)
>> 250 if ((state & URWLOCK_WRITE_OWNER) != 0) {
>> (gdb)
>> 256 if (__predict_false(URWLOCK_READER_COUNT(stat
>> e) == 0))
>> (gdb)
>> 260 URWLOCK_READER_COUNT(state) == 1)) {
>> (gdb)
>> 259 URWLOCK_READ_WAITERS)) != 0 &&
>> (gdb)
>> 262 state, state - 1))
>> (gdb)
>> 261 if (atomic_cmpset_rel_32(&rwlock->rw_
>> state,
>> (gdb)
>> atomic_cmpset_int (dst=<optimized out>, expect=<optimized out>, src=0) at /us
>> r/obj/usr/src/amd64.amd64/tmp/usr/include/machine/atomic.h:220
>> 220 ATOMIC_CMPSET(int);
>> (gdb)
>> _thr_rwlock_unlock (rwlock=0x80180d200) at /usr/src/lib/libthr/thread/thr_umt
>> x.h:261
>> 261 if (atomic_cmpset_rel_32(&rwlock->rw_
>> state,
>> (gdb)
>> _thr_rtld_lock_release (lock=<optimized out>) at /usr/src/lib/libthr/thread/t
>> hr_rtld.c:162
>> 162 if ((state & URWLOCK_WRITE_OWNER) == 0)
>> (gdb)
>> 163 curthread->rdlock_count--;
>> (gdb)
>> 164 THR_CRITICAL_LEAVE(curthread);
>> (gdb)
>> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:271
>> 271 if (!THR_IN_CRITICAL(curthread)) {
>> (gdb)
>> 272 check_deferred_signal(curthread);
>> (gdb)
>> check_deferred_signal (curthread=0x80864b000) at /usr/src/lib/libthr/thread/t
>> hr_sig.c:332
>> 332 if
>> (__predict_true(curthread->deferred_siginfo.si_signo == 0 ||
>> (gdb)
>> 351 }
>> (gdb)
>> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:273
>> 273 check_suspend(curthread);
>> (gdb)
>> check_suspend (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c
>> :358
>> 358 if (__predict_true((curthread->flags &
>> (gdb)
>> 401 }
>> (gdb)
>> _thr_ast (curthread=0x80864b000) at /usr/src/lib/libthr/thread/thr_sig.c:274
>> 274 check_cancel(curthread, NULL);
>> (gdb)
>> check_cancel (curthread=0x80864b000, ucp=0x0) at /usr/src/lib/libthr/thread/t
>> hr_sig.c:283
>> 283 if (__predict_true(!curthread->cancel_pending ||
>> (gdb)
>> _thr_ast (curthread=<optimized out>) at /usr/src/lib/libthr/thread/thr_sig.c:
>> 276
>> 276 }
>> (gdb)
>> _thr_rtld_lock_release (lock=<optimized out>) at /usr/src/lib/libthr/thread/t
>> hr_rtld.c:166
>> 166 RESTORE_ERRNO();
>> (gdb)
>> 167 }
>> (gdb)
>> setrlimit () at setrlimit.S:3
>> 3 RSYSCALL(setrlimit)
>> (gdb)
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> setrlimit () at setrlimit.S:3
>> 3 RSYSCALL(setrlimit)
>> (gdb)
>>
>> Program terminated with signal SIGSEGV, Segmentation fault.
>> The program no longer exists.
>> (gdb) q
>>
>> ====
>>
>> I'm sorry for the long post. Is there anything (else) I can do to
>> further narrow it down?
>
> I've been able to confirm that kib@'s hunch regarding the gap is correct.
>
> Use the workaround until this can be solved.
>
>
--
Harlan Stenn <stenn@nwtime.org>
http://networktimefoundation.org - be a member!
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?9b0c95de-2d0e-89b4-32e6-63ec5af729b4>