From owner-freebsd-drivers@FreeBSD.ORG Thu May 30 06:04:58 2013 Return-Path: Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 380973FE; Thu, 30 May 2013 06:04:58 +0000 (UTC) (envelope-from oritm@mellanox.com) Received: from eu1sys200aog118.obsmtp.com (eu1sys200aog118.obsmtp.com [207.126.144.145]) by mx1.freebsd.org (Postfix) with ESMTP id 28447F6; Thu, 30 May 2013 06:04:56 +0000 (UTC) Received: from MTLCAS02.mtl.com ([193.47.165.155]) (using TLSv1) by eu1sys200aob118.postini.com ([207.126.147.11]) with SMTP ID DSNKUabsAim/O9uegauP85lK/le4VNY4cmE6@postini.com; Thu, 30 May 2013 06:04:57 UTC Received: from MTLDAG01.mtl.com ([10.0.8.75]) by MTLCAS02.mtl.com ([10.0.8.72]) with mapi id 14.03.0123.003; Thu, 30 May 2013 09:04:49 +0300 From: Orit Moskovich To: John Baldwin , "freebsd-drivers@freebsd.org" Subject: RE: taskqueues Thread-Topic: taskqueues Thread-Index: Ac5XuKI6XhgBsdJeSxm0tXVuhEIwAQEs0ywAACKnorA= Date: Thu, 30 May 2013 06:04:48 +0000 Message-ID: <981733489AB3BD4DB24B48340F53E0A55B0D6417@MTLDAG01.mtl.com> References: <981733489AB3BD4DB24B48340F53E0A55B0D5206@MTLDAG01.mtl.com> <201305291156.29230.jhb@freebsd.org> In-Reply-To: <201305291156.29230.jhb@freebsd.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.0.13.1] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 May 2013 06:04:58 -0000 Also, is it possible to set affinity of a task in a taskqueue? From what I = understood, each interrupt thread handling taskqueues and the ithread of th= e bus_setup_intr is a single thread, not one per cpu. What possibilities I have if I need to defer work from a filter routine to = multiple tasks, and schedule them to work on a specific core? -----Original Message----- From: John Baldwin [mailto:jhb@freebsd.org]=20 Sent: Wednesday, May 29, 2013 09:17 PM To: freebsd-drivers@freebsd.org Cc: Orit Moskovich Subject: Re: taskqueues On Thursday, May 23, 2013 9:47:14 am Orit Moskovich wrote: > Hi, >=20 > Can you please specify the difference between interrupt threads and=20 > regular kernel threads in the context of the different default taskqueues? > Meaning, I saw that the taskqueue taskqueue_thread is actually a=20 > kernel thread running the function taskqueue_thread_loop, > And the 3 other default taskqueues are working with ithreads. >=20 > Which of the above preempts the other? What should I use if=20 > performance is critical (something equivalent to Linux softirq...)? ithreads use the most important priority range: #define PRI_MIN (0) /* Highest priority. */ #define PRI_MAX (255) /* Lowest priority. */ #define PRI_MIN_ITHD (PRI_MIN) #define PRI_MAX_ITHD (PRI_MIN_REALTIME - 1) #define PI_REALTIME (PRI_MIN_ITHD + 0) #define PI_AV (PRI_MIN_ITHD + 4) #define PI_NET (PRI_MIN_ITHD + 8) #define PI_DISK (PRI_MIN_ITHD + 12) #define PI_TTY (PRI_MIN_ITHD + 16) #define PI_DULL (PRI_MIN_ITHD + 20) #define PI_SOFT (PRI_MIN_ITHD + 24) #define PI_SWI(x) (PI_SOFT + (x) * RQ_PPQ) For example, an INTR_TYPE_NET interrupt will generally run in an ithread wi= th the PI_NET priority. For software interrupt threads the priority is gen= erally PI_SWI(SWI_xxx). The SWI_xxx constants are: /* * Software interrupt numbers in priority order. The priority determines * the priority of the corresponding interrupt thread. */ #define SWI_TTY 0 #define SWI_NET 1 #define SWI_CAMBIO 2 #define SWI_VM 3 #define SWI_CLOCK 4 #define SWI_TQ_FAST 5 #define SWI_TQ 6 #define SWI_TQ_GIANT 6 For taskqueues you have three global taskqueues to choose from: taskqueue_fast: runs at PI_SWI(SWI_TQ_FAST) =3D 44 taskqueue_swi: runs at PI_SWI(SWI_TQ) =3D 48 taskqueue_thread: runs at PWAIT =3D 108 However, you can create your own thread-backed taskqueue that can run at wh= atever priority you choose. There is nothing really special about having t= he taskqueue be an SWI vs a normal thread. If you wanted a taskqueue that = ran at PI_NET you could do: TASKQUEUE_DEFINE(foo, taskqueue_thread_enqueue, &taskqueue_foo, taskqueue_start_threads(&taskqueue_foo, 1, PI_NET, "foo taskq"); If you needed to queue tasks from an interrupt filter you would want to use= a fast taskqueue instead: TASKQUEUE_FAST_DEFINE(< same args as above >) -- John Baldwin