Date: Fri, 11 Apr 2014 13:23:00 -0400 (EDT) From: Daniel Eischen <deischen@freebsd.org> To: Konstantin Belousov <kostikbel@gmail.com> Cc: freebsd-hackers@freebsd.org, Karl Pielorz <kpielorz_lst@tdx.co.uk> Subject: Re: Stuck CLOSED sockets / sshd / zombies... Message-ID: <Pine.GSO.4.64.1404111320250.23129@sea.ntplx.net> In-Reply-To: <20140411160628.GV21331@kib.kiev.ua> References: <20140409084951.GE21331@kib.kiev.ua> <2A722BB3B12E0D80CA9FF075@Mail-PC.tdx.co.uk> <20140409111917.GH21331@kib.kiev.ua> <851413886E3982D2CCFEA9D9@Mail-PC.tdx.co.uk> <20140410184855.GP21331@kib.kiev.ua> <211BD03C086DDB1A07FDF036@Mail-PC.tdx.co.uk> <20140411131649.GR21331@kib.kiev.ua> <652B8CA4866C0B9E4650430B@Mail-PC.tdx.co.uk> <20140411141526.GT21331@kib.kiev.ua> <464979E8F6FCBD7EA7DAA38B@Mail-PC.tdx.co.uk> <20140411160628.GV21331@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 11 Apr 2014, Konstantin Belousov wrote: > On Fri, Apr 11, 2014 at 03:57:00PM +0100, Karl Pielorz wrote: >> >> >> --On 11 April 2014 17:15 +0300 Konstantin Belousov <kostikbel@gmail.com> >> wrote: >> >>> So my suspicious idea seems to be true. From the ldd output, libc >>> appears before libthr in the global order, so libc sigaction() symbol >>> is resolved before libthr interposer. The result is that libthr wrapper >>> thr_sighandler() for the signal handlers is not installed as the >>> recepient of the kernel signal, which prevents libthr locks for rtld >>> from working properly. >> >> Ok, I can just about follow that ;) >> >>> To confirm or deny my theory, please apply the patch below, in addition to >>> the previous patch, and rebuild sshd only, >>> # cd src/secure/usr.sbin/sshd && make clean all install >>> The patch tilts the order of initialization, for my build I got >>> sandy% ldd /usr/sbin/sshd >>> ... >>> libz.so.6 => /lib/libz.so.6 (0x802f0d000) >>> libthr.so.3 => /lib/libthr.so.3 (0x803123000) >>> libc.so.7 => /lib/libc.so.7 (0x803348000) >>> libldns.so.5 => /usr/lib/private/libldns.so.5 (0x8036d1000) >>> libmd.so.6 => /lib/libmd.so.6 (0x803926000) >>> which could be enough to prevent the bug. >>> >>> Please retest and report. >> >> Ok, patched the makefile - rebuilt / installed sshd restarted (which has >> the same initialisation order as yours above), it and ran the security scan >> against it. >> >> *This does indeed seem to fix the problem* >> >> The scan completes, and there are no stuck 'urdlck' sshd's - and no socket >> sitting around in CLOSE_WAIT or CLOSED - thanks! :) >> >> I re-ran the scan a couple of times more to be sure, with the same result - >> no zombies or anything. > Great. > >> >> Is this situation likely to be repeated anywhere else on the system? Or is >> it likely just to be specific to sshd? > Well. > > The issue with libthr so relying on interposition of libc has already bitten > us more than once. The biggest practical consequence of it is that libthr > cannot be dynamically loaded, it must be linked to the main binary for the > whole construct to work. This means that any program big enough to load > plugins at runtime must link to libthr if it might need to load plugin > linked to libthr. Recently, perl and other programs from ports started > doing just that. > > But this is first time I see interposing so broken due to wrong linking > order, esp. in the base system. > > The correct solution is to merge libthr into libc. Some neccessary > preparations were already done, but the main work did not started yet. > This is huge efforts, and it probably should be coordinated with some > other ABI changes planed for libthr to support process-shared locks. Eek, no, I don't think that is necessary. When we go to using real structs instead of pointers for synchronization types (mutex, CV) in libthr, then I don't think there will be a problem. -- DE
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.GSO.4.64.1404111320250.23129>