From owner-freebsd-arch@freebsd.org Wed Jan 6 17:08:43 2016 Return-Path: Delivered-To: freebsd-arch@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 41C8BA65A5F for ; Wed, 6 Jan 2016 17:08:43 +0000 (UTC) (envelope-from bright@mu.org) Received: from elvis.mu.org (elvis.mu.org [IPv6:2001:470:1f05:b76::196]) by mx1.freebsd.org (Postfix) with ESMTP id 36C15186F; Wed, 6 Jan 2016 17:08:43 +0000 (UTC) (envelope-from bright@mu.org) Received: from Alfreds-MacBook-Pro-2.local (unknown [IPv6:2601:645:8001:cee1:d5d7:5a09:788d:3b60]) by elvis.mu.org (Postfix) with ESMTPSA id 95765345A921; Wed, 6 Jan 2016 09:08:42 -0800 (PST) Subject: Re: Expected taskqueue behavior To: John Baldwin , freebsd-arch@freebsd.org References: <9174153.lS3QljxB7A@ralph.baldwin.cx> From: Alfred Perlstein Message-ID: <568D4A1A.5080204@mu.org> Date: Wed, 6 Jan 2016 09:08:42 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:38.0) Gecko/20100101 Thunderbird/38.5.0 MIME-Version: 1.0 In-Reply-To: <9174153.lS3QljxB7A@ralph.baldwin.cx> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jan 2016 17:08:43 -0000 On 1/5/16 7:19 AM, John Baldwin wrote: > Given this bit of code: > > void > cpu_hog(void *context, int pending) > { > > for (;;) > ; > } > > static struct taskqueue *tq; > static struct task t; > > void > mod_init(void) > { > int i; > > TASK_INIT(&t, 0, cpu_hog, NULL); > tq = taskqueue_create("test", M_WAITOK, taskqueue_thread_enqueue, > &tq); > taskqueue_start_threads(&tq, mp_ncpu, PRI_MAX_IDLE, "test taskq"); > > for (i = 0; i < mp_ncpu; i++) { > taskqueue_enqueue(tq, &t); > pause("foo", 1); > } > } > > How many threads would you expect to be busy? The taskqueue(9) manpage > does not describe what happens when you queue a task that is already > executing. Currently our implementation will allow the task to execute > on a second thread (or a third, fourth, fifth, etc.). I think this > behavior is probably "surprising" as I expect most folks assume that > a task won't be executed concurrently and rely on this assumption to > avoid the need for certain types of serialization in task handlers. > > Linux workqueues explicitly document work-items as being non-reentrant. > > From https://www.kernel.org/doc/Documentation/workqueue.txt > > > Note that the flag WQ_NON_REENTRANT no longer exists as all workqueues > are now non-reentrant - any work item is guaranteed to be executed by > at most one worker system-wide at any given time. > > > Windows work queues also forbid this though indirectly becuase they > forbid extra queueing (which bumps 'pending' on FreeBSD) outright. > > From https://msdn.microsoft.com/en-us/library/windows/hardware/ff564587%28v=vs.85%29.aspx > > > Do not call IoQueueWorkItem or IoQueueWorkItemEx to queue a work item that > is already in the queue. In checked builds, this error causes a bug check. In > retail builds, the error is not detected but can cause corruption of system > data structures. > > > I think our current behavior is "surprising" and is a bug from when > taskqueue thread pools were added (the case simply wasn't handled). > > At a minimum we should document the expected behavior (and fix it if > its wrong). If we don't change our current behavior then the Linux > KPI workqueue implementation will need additional changes as it can > no longer simply call taskqueue_enqueue(). > Agreed. I had to debug numerous problems due to this in a Javascript application where the behavior was similar. -Alfred