Date: Fri, 11 Apr 2014 09:35:10 -0700 (PDT) From: "Chris H" <bsd-lists@bsdforge.com> To: "Konstantin Belousov" <kostikbel@gmail.com> Cc: freebsd-hackers@freebsd.org, Karl Pielorz <kpielorz_lst@tdx.co.uk> Subject: Re: Stuck CLOSED sockets / sshd / zombies... Message-ID: <ac803d5945b233d6413dee2344361b80.authenticated@ultimatedns.net> In-Reply-To: <20140411160628.GV21331@kib.kiev.ua> References: <20140409084951.GE21331@kib.kiev.ua> <2A722BB3B12E0D80CA9FF075@Mail-PC.tdx.co.uk> <20140409111917.GH21331@kib.kiev.ua> <851413886E3982D2CCFEA9D9@Mail-PC.tdx.co.uk> <20140410184855.GP21331@kib.kiev.ua> <211BD03C086DDB1A07FDF036@Mail-PC.tdx.co.uk> <20140411131649.GR21331@kib.kiev.ua> <652B8CA4866C0B9E4650430B@Mail-PC.tdx.co.uk> <20140411141526.GT21331@kib.kiev.ua> <464979E8F6FCBD7EA7DAA38B@Mail-PC.tdx.co.uk> <20140411160628.GV21331@kib.kiev.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Fri, Apr 11, 2014 at 03:57:00PM +0100, Karl Pielorz wrote: >> >> >> --On 11 April 2014 17:15 +0300 Konstantin Belousov <kostikbel@gmail.com> >> wrote: >> >> > So my suspicious idea seems to be true. From the ldd output, libc >> > appears before libthr in the global order, so libc sigaction() symbol >> > is resolved before libthr interposer. The result is that libthr wrapper >> > thr_sighandler() for the signal handlers is not installed as the >> > recepient of the kernel signal, which prevents libthr locks for rtld >> > from working properly. >> >> Ok, I can just about follow that ;) >> >> > To confirm or deny my theory, please apply the patch below, in addition to >> > the previous patch, and rebuild sshd only, >> ># cd src/secure/usr.sbin/sshd && make clean all install >> > The patch tilts the order of initialization, for my build I got >> > sandy% ldd /usr/sbin/sshd >> > ... >> > libz.so.6 => /lib/libz.so.6 (0x802f0d000) >> > libthr.so.3 => /lib/libthr.so.3 (0x803123000) >> > libc.so.7 => /lib/libc.so.7 (0x803348000) >> > libldns.so.5 => /usr/lib/private/libldns.so.5 (0x8036d1000) >> > libmd.so.6 => /lib/libmd.so.6 (0x803926000) >> > which could be enough to prevent the bug. >> > >> > Please retest and report. >> >> Ok, patched the makefile - rebuilt / installed sshd restarted (which has >> the same initialisation order as yours above), it and ran the security scan >> against it. >> >> *This does indeed seem to fix the problem* >> >> The scan completes, and there are no stuck 'urdlck' sshd's - and no socket >> sitting around in CLOSE_WAIT or CLOSED - thanks! :) >> >> I re-ran the scan a couple of times more to be sure, with the same result - >> no zombies or anything. > Great. > >> >> Is this situation likely to be repeated anywhere else on the system? Or is >> it likely just to be specific to sshd? > Well. > > The issue with libthr so relying on interposition of libc has already bitten > us more than once. The biggest practical consequence of it is that libthr > cannot be dynamically loaded, it must be linked to the main binary for the > whole construct to work. This means that any program big enough to load > plugins at runtime must link to libthr if it might need to load plugin > linked to libthr. Recently, perl and other programs from ports started > doing just that. > > But this is first time I see interposing so broken due to wrong linking > order, esp. in the base system. > > The correct solution is to merge libthr into libc. Some neccessary > preparations were already done, but the main work did not started yet. > This is huge efforts, and it probably should be coordinated with some > other ABI changes planed for libthr to support process-shared locks. Sorry. Second time around. I used the wrong email address. :P This may be wishful thinking, or I might have even overlooked the obvious. Would it make any sense to create 2 versions of libthr, one with libc, and one without? At least that would overcome the /possible/ ordering issue associated with this sort of thing. Then it's simply a matter of choosing which one to load. --Chris > > Anyway, for now, this patch, possibly enhanced to only link with > libpthread when kerberos is used, should be good enough. >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ac803d5945b233d6413dee2344361b80.authenticated>