Date: Mon, 7 Mar 2011 15:35:59 +0000 From: Alexander Best <arundel@freebsd.org> To: Chagin Dmitry <dchagin@freebsd.org> Cc: freebsd-emulation@freebsd.org Subject: Re: regarding issues with futexes and multiple threads Message-ID: <20110307153559.GA94499@freebsd.org> In-Reply-To: <20110304174637.GA8522@dchagin.static.corbina.ru> References: <20110303212051.GA54978@freebsd.org> <20110304061328.GA4073@dchagin.static.corbina.ru> <20110304110843.GA65337@freebsd.org> <20110304174637.GA8522@dchagin.static.corbina.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri Mar 4 11, Chagin Dmitry wrote:
> On Fri, Mar 04, 2011 at 11:08:43AM +0000, Alexander Best wrote:
> > On Fri Mar 4 11, Chagin Dmitry wrote:
> > > On Thu, Mar 03, 2011 at 09:20:51PM +0000, Alexander Best wrote:
> > > > hi there,
> > > >
> > > > i've been investigating this issue for quite a while now and simply wanted to
> > > > post some measurements i did.
> > > >
> > > > basically there's a massive slowdown when a process triggers a lot of threads
> > > > which are doing futex operations. the following example is from darren hart's
> > > > futex test suite (performance/). the slowdown seems to SMP related. here are
> > > > the measurements running with SMP enabled and with kern.smp.disabled=1:
> > > >
> > >
> > > 1) slowdown is expected behaviour.
> >
> > even if the slowdown is to be expected, looking at the screenshot with the
> > top(1) output it seems obvious that there are issues with the current futex
> > implementation.
> >
>
> why do you think it's a problem with the futexes?
i added the following debug printf's to linux_futex.c:
diff --git a/sys/compat/linux/linux_futex.c b/sys/compat/linux/linux_futex.c
index b69936c..3d5576f 100644
--- a/sys/compat/linux/linux_futex.c
+++ b/sys/compat/linux/linux_futex.c
@@ -510,6 +510,8 @@ linux_sys_futex(struct thread *td, struct linux_sys_futex_args *args)
}
error = futex_wait(f, wp, args->timeout, args->val3);
+ if (error)
+ printf("LINUX_FUTEX_WAIT_BITSET finished with error = %d\n", error);
break;
case LINUX_FUTEX_WAKE:
@@ -541,6 +543,8 @@ linux_sys_futex(struct thread *td, struct linux_sys_futex_args *args)
}
td->td_retval[0] = futex_wake(f, args->val, args->val3);
futex_put(f, NULL);
+ if (error)
+ printf("LINUX_FUTEX_WAKE_BITSET finished with error = %d\n", error);
break;
case LINUX_FUTEX_CMP_REQUEUE:
@@ -601,6 +605,8 @@ linux_sys_futex(struct thread *td, struct linux_sys_futex_args *args)
td->td_retval[0] = futex_requeue(f, args->val, f2, nrwake);
futex_put(f2, NULL);
futex_put(f, NULL);
+ if (error)
+ printf("LINUX_FUTEX_CMP_REQUEUE finished with error = %d\n", error);
break;
case LINUX_FUTEX_WAKE_OP:
@@ -664,6 +670,8 @@ linux_sys_futex(struct thread *td, struct linux_sys_futex_args *args)
futex_put(f2, NULL);
futex_put(f, NULL);
td->td_retval[0] = ret;
+ if (error)
+ printf("LINUX_FUTEX_WAKE_OP finished with error = %d\n", error);
break;
case LINUX_FUTEX_LOCK_PI:
...then i ran chrome with some flash website. this was the dmesg output (well
part of it):
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAITL_IBNIUTXS_EFTU TfEiXn_iWsAhIeTd_ BwIiTtShE Te rfrionri s=h ed6 0wi
th error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 60
LINUX_FUTEX_WAIT_BITSET finished with error = 4
LINUX_FUTEX_WAIT_BITSET finisLhIeNdU Xw_iFtUhT EeXr_rWoArI T=_ B4IT
SET finished with erroLrI N=U X4_F
UTEX_WAIT_BITSET finLiIsNhUeXd_ FwUiTtEhX _eWrArIoTr_ B=I T4SE
T finished with errLoIrN U=X _4FU
TEX_WAIT_BITSET finLiIsNhUeXd_ FwUiTtEhX _eWrArIoTr_ B=I T4SE
T finished with erLrIoNrU X=_ F4UT
EX_WAIT_BITSET finiLsIhNeUdX _wFiUtThE Xe_rWrAoIrT _=B I4TS
ET finished with error = 4
... running the futex testsuite with #threads 1 and 32 showed no error = *
output, but i'll try running the suite with more testcases in a minute.
cheers.
alex
>
> > > 2) is it possible to run this test suite on FreeBSD?
> >
> > i used emulators/linux_dist-gentoo-stage3 to compile the testsuite and then ran
> > it natively under freebsd. if you want to i can send you the file. it's
> >
> > futex_wait: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped
> >
>
> ah, tests directly calls sys_futex syscall, so can't be used for umtx/futex comparison :)
> It would be great to have the same set of tests to compare the linuxulator threads
> and the native.
>
> --
> Have fun!
> chd
--
a13x
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110307153559.GA94499>