From owner-freebsd-emulation@FreeBSD.ORG Mon Mar 7 15:35:59 2011 Return-Path: Delivered-To: freebsd-emulation@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 1233) id 36AE11065672; Mon, 7 Mar 2011 15:35:59 +0000 (UTC) Date: Mon, 7 Mar 2011 15:35:59 +0000 From: Alexander Best To: Chagin Dmitry Message-ID: <20110307153559.GA94499@freebsd.org> References: <20110303212051.GA54978@freebsd.org> <20110304061328.GA4073@dchagin.static.corbina.ru> <20110304110843.GA65337@freebsd.org> <20110304174637.GA8522@dchagin.static.corbina.ru> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20110304174637.GA8522@dchagin.static.corbina.ru> Cc: freebsd-emulation@freebsd.org Subject: Re: regarding issues with futexes and multiple threads X-BeenThere: freebsd-emulation@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Development of Emulators of other operating systems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Mar 2011 15:35:59 -0000 On Fri Mar 4 11, Chagin Dmitry wrote: > On Fri, Mar 04, 2011 at 11:08:43AM +0000, Alexander Best wrote: > > On Fri Mar 4 11, Chagin Dmitry wrote: > > > On Thu, Mar 03, 2011 at 09:20:51PM +0000, Alexander Best wrote: > > > > hi there, > > > > > > > > i've been investigating this issue for quite a while now and simply wanted to > > > > post some measurements i did. > > > > > > > > basically there's a massive slowdown when a process triggers a lot of threads > > > > which are doing futex operations. the following example is from darren hart's > > > > futex test suite (performance/). the slowdown seems to SMP related. here are > > > > the measurements running with SMP enabled and with kern.smp.disabled=1: > > > > > > > > > > 1) slowdown is expected behaviour. > > > > even if the slowdown is to be expected, looking at the screenshot with the > > top(1) output it seems obvious that there are issues with the current futex > > implementation. > > > > why do you think it's a problem with the futexes? i added the following debug printf's to linux_futex.c: diff --git a/sys/compat/linux/linux_futex.c b/sys/compat/linux/linux_futex.c index b69936c..3d5576f 100644 --- a/sys/compat/linux/linux_futex.c +++ b/sys/compat/linux/linux_futex.c @@ -510,6 +510,8 @@ linux_sys_futex(struct thread *td, struct linux_sys_futex_args *args) } error = futex_wait(f, wp, args->timeout, args->val3); + if (error) + printf("LINUX_FUTEX_WAIT_BITSET finished with error = %d\n", error); break; case LINUX_FUTEX_WAKE: @@ -541,6 +543,8 @@ linux_sys_futex(struct thread *td, struct linux_sys_futex_args *args) } td->td_retval[0] = futex_wake(f, args->val, args->val3); futex_put(f, NULL); + if (error) + printf("LINUX_FUTEX_WAKE_BITSET finished with error = %d\n", error); break; case LINUX_FUTEX_CMP_REQUEUE: @@ -601,6 +605,8 @@ linux_sys_futex(struct thread *td, struct linux_sys_futex_args *args) td->td_retval[0] = futex_requeue(f, args->val, f2, nrwake); futex_put(f2, NULL); futex_put(f, NULL); + if (error) + printf("LINUX_FUTEX_CMP_REQUEUE finished with error = %d\n", error); break; case LINUX_FUTEX_WAKE_OP: @@ -664,6 +670,8 @@ linux_sys_futex(struct thread *td, struct linux_sys_futex_args *args) futex_put(f2, NULL); futex_put(f, NULL); td->td_retval[0] = ret; + if (error) + printf("LINUX_FUTEX_WAKE_OP finished with error = %d\n", error); break; case LINUX_FUTEX_LOCK_PI: ...then i ran chrome with some flash website. this was the dmesg output (well part of it): LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAITL_IBNIUTXS_EFTU TfEiXn_iWsAhIeTd_ BwIiTtShE Te rfrionri s=h ed6 0wi th error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 60 LINUX_FUTEX_WAIT_BITSET finished with error = 4 LINUX_FUTEX_WAIT_BITSET finisLhIeNdU Xw_iFtUhT EeXr_rWoArI T=_ B4IT SET finished with erroLrI N=U X4_F UTEX_WAIT_BITSET finLiIsNhUeXd_ FwUiTtEhX _eWrArIoTr_ B=I T4SE T finished with errLoIrN U=X _4FU TEX_WAIT_BITSET finLiIsNhUeXd_ FwUiTtEhX _eWrArIoTr_ B=I T4SE T finished with erLrIoNrU X=_ F4UT EX_WAIT_BITSET finiLsIhNeUdX _wFiUtThE Xe_rWrAoIrT _=B I4TS ET finished with error = 4 ... running the futex testsuite with #threads 1 and 32 showed no error = * output, but i'll try running the suite with more testcases in a minute. cheers. alex > > > > 2) is it possible to run this test suite on FreeBSD? > > > > i used emulators/linux_dist-gentoo-stage3 to compile the testsuite and then ran > > it natively under freebsd. if you want to i can send you the file. it's > > > > futex_wait: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.9, not stripped > > > > ah, tests directly calls sys_futex syscall, so can't be used for umtx/futex comparison :) > It would be great to have the same set of tests to compare the linuxulator threads > and the native. > > -- > Have fun! > chd -- a13x