Date: Sun, 21 Aug 2016 15:42:26 +0100 From: "Robert N. M. Watson" <rwatson@FreeBSD.org> To: Adrian Chadd <adrian@freebsd.org> Cc: "Andrey V. Elsukov" <ae@freebsd.org>, FreeBSD Net <freebsd-net@freebsd.org> Subject: Re: svn commit: r304313 - head/sys/net Message-ID: <28FA9F29-FA29-4547-875D-0734DA120636@FreeBSD.org> In-Reply-To: <CAJ-Vmonu3ZSKZtZZMa68FAS9nDdiriE3iv81cjoW%2Bk3mHKO2=g@mail.gmail.com> References: <201608172021.u7HKLXJ4001584@repo.freebsd.org> <CAJ-Vmonu3ZSKZtZZMa68FAS9nDdiriE3iv81cjoW%2Bk3mHKO2=g@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 20 Aug 2016, at 21:27, Adrian Chadd <adrian@freebsd.org> wrote: > I wonder if the right-er thing to do here is to allow the cpuid to be > whatever it needs to be, but limit the cpuid lookups when it resolves > to a netisr array. >=20 > that'd mean the hybrid model would still return the current CPU up to > the max CPU id, but anything trying to queue into a netisr would not > overflow the netisr queue count. I think some refinement of model is required. The original intent was = that =E2=80=9Cworkstreams=E2=80=9D represented ordered sets of queues = being delivered for deferred dispatch across the set of CPUs (hence = =E2=80=9Cnws=E2=80=9D being allocated using DPCPU). When performing = direct dispatch, as a matter of convenience, we also bill work performed = on a CPU to its workstream: 1109 /* 1110 * If direct dispatch is forced, then unconditionally = dispatch 1111 * without a formal CPU selection. Borrow the current = CPU's stats, 1112 * even if there's no worker on it. In this case we don't = update 1113 * nws_flags because all netisr processing will be source = ordered due 1114 * to always being forced to directly dispatch. 1115 */ In hybrid mode, the world is a little different, because we are willing = to direct dispatch hybrid work *only* if the worker thread isn=E2=80=99t = already processing the workstream: 1147 /*- 1148 * We are willing to direct dispatch only if three = conditions hold: 1149 * 1150 * (1) The netisr worker isn't already running, 1151 * (2) Another thread isn't already directly dispatching, = and 1152 * (3) The netisr hasn't already been woken up. 1153 */ This is important because if the workstream already has a backlog of = work, but the thread isn=E2=80=99t yet running, we must enqueue the work = to ensure it remains ordered with respect to other work in the = workstream. Assuming we wish to retain the current workstream model, then we need to = adapt the code a bit to handle the case where we still have one = workstream per CPU, but where we are only willing to perform deferred = dispatch (or hybrid dispatch) to a CPU that has a worker thread. We = would then bill to any CPU=E2=80=99s workstream for true direct = dispatch, but for hybrid dispatch, continue to always bill a CPU that = has a worker thread to which we were willing to assign the work. I.e., as I think you=E2=80=99re suggesting, we probably need to tweak = the functions slightly to differentiate between =E2=80=9Cyou can run it = here and must use curcpu=E2=80=99s workstream=E2=80=9D and =E2=80=9Cyou = can run it here but must use a specific CPU=E2=80=99s workstream=E2=80=9D = =E2=80=94 the former being true direct dispatch, and the latter being = hybrid dispatch where the netisr in question isn=E2=80=99t already = running, but we bill it to that workstream/netisr even though we run it = on the current CPU. Does this make sense? Robert=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?28FA9F29-FA29-4547-875D-0734DA120636>