Date: Sun, 13 Nov 2016 14:47:04 +0100 From: Henri Hennebert <hlh@restart.be> To: Andriy Gapon <avg@FreeBSD.org>, freebsd-stable@FreeBSD.org Cc: Konstantin Belousov <kib@FreeBSD.org> Subject: Re: Freebsd 11.0 RELEASE - ZFS deadlock Message-ID: <c3d570fe-ba04-d678-1fd2-a19d7a59e416@restart.be> In-Reply-To: <cf0fc1e3-b621-074e-1351-4dd89d980ddd@restart.be> References: <0c223160-b76f-c635-bb15-4a068ba7efe7@restart.be> <aaf2df40-b0df-2141-9ed8-5b947d8d5a33@FreeBSD.org> <43c9d4d4-1995-5626-d70a-f92a5b456629@FreeBSD.org> <a14d508d-351f-71f4-c7cc-ac73dbcde357@restart.be> <9d1f9a76-5a8d-6eca-9a50-907d55099847@FreeBSD.org> <6bc95dce-31e1-3013-bfe3-7c2dd80f9d1e@restart.be> <e4878992-a362-3f12-e743-8efa1347cabf@FreeBSD.org> <23a66749-f138-1f1a-afae-c775f906ff37@restart.be> <8e7547ef-87f7-7fab-6f45-221e8cea1989@FreeBSD.org> <6d991cea-b420-531e-12cc-001e4aeed66b@restart.be> <67f2e8bd-bff0-f808-7557-7dabe5cad78c@FreeBSD.org> <1cb09c54-5f0e-2259-a41a-fefe76b4fe8b@restart.be> <d25c8035-b710-5de9-ebe3-7990b2d0e3b1@FreeBSD.org> <9f20020b-e2f1-862b-c3fc-dc6ff94e301e@restart.be> <c1b7aa94-1f1d-7edd-8764-adb72fdc053c@FreeBSD.org> <599c5a5b-aa08-2030-34f3-23ff19d09a9b@restart.be> <32686283-948a-6faf-7ded-ed8fcd23affb@FreeBSD.org> <cf0fc1e3-b621-074e-1351-4dd89d980ddd@restart.be>
next in thread | previous in thread | raw e-mail | index | archive | help
On 11/13/2016 14:28, Henri Hennebert wrote:
> This 2 threads are innd processes. In core.txt.4:
>
> 8 14789 29165 0 24 4 40040 6612 zfs DN - 0:00.00 [innd]
> 8 29165 1 0 20 0 42496 6888 select Ds - 0:01.33 [innd]
> 8 49778 29165 0 24 4 40040 6900 zfs DN - 0:00.00 [innd]
> 8 82034 29165 0 24 4 132 0 zfs DN - 0:00.00 [innd]
>
> the corresponding info treads are:
>
> 687 Thread 101243 (PID=49778: innd) sched_switch
> (td=0xfffff800b642b500, newtd=0xfffff8000285ea00, flags=<value optimized
> out>) at /usr/src/sys/kern/sched_ule.c:1973
> 681 Thread 101147 (PID=14789: innd) sched_switch
> (td=0xfffff80065f4e500, newtd=0xfffff8000285f000, flags=<value optimized
> out>) at /usr/src/sys/kern/sched_ule.c:1973
> 669 Thread 101250 (PID=82034: innd) sched_switch
> (td=0xfffff800b6429000, newtd=0xfffff8000285ea00, flags=<value optimized
> out>) at /usr/src/sys/kern/sched_ule.c:1973
> 665 Thread 101262 (PID=29165: innd) sched_switch
> (td=0xfffff800b6b54a00, newtd=0xfffff8000285ea00, flags=<value optimized
> out>) at /usr/src/sys/kern/sched_ule.c:1973
>
In case it may help, I have a look at innd. This processes use 2 execv:
one to execute /bin/sh and the other to execute itself:
/*
** Re-exec ourselves.
*/
static const char *
CCxexec(char *av[])
{
char *innd;
char *p;
int i;
if (CCargv == NULL)
return "1 no argv!";
innd = concatpath(innconf->pathbin, "innd");
/* Get the pathname. */
p = av[0];
if (*p == '\0' || strcmp(p, "innd") == 0)
CCargv[0] = innd;
else
return "1 Bad value";
#ifdef DO_PERL
PLmode(Mode, OMshutdown, av[0]);
#endif
#ifdef DO_PYTHON
PYmode(Mode, OMshutdown, av[0]);
#endif
JustCleanup();
syslog(L_NOTICE, "%s execv %s", LogName, CCargv[0]);
/* Close all fds to protect possible fd leaking accross successive
innds. */
for (i=3; i<30; i++)
close(i);
execv(CCargv[0], CCargv);
syslog(L_FATAL, "%s cant execv %s %m", LogName, CCargv[0]);
_exit(1);
/* NOTREACHED */
return "1 Exit failed";
}
The culprit may be /usr/local/news/bin/innd,
remember that find is locked in /usr/local/news/bin
Henri
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c3d570fe-ba04-d678-1fd2-a19d7a59e416>