Date: Sat, 21 Mar 2015 15:26:25 -0700 From: "K. Macy" <kmacy@freebsd.org> To: John Baldwin <jhb@freebsd.org> Cc: "freebsd-arch@freebsd.org" <freebsd-arch@freebsd.org> Subject: Re: LOCAL_CPUS vs INTR_CPUS Message-ID: <CAHM0Q_O_4VbVxVT5tveTjuXnF2E5CLmutN86GA%2BHf9zJ9gvrww@mail.gmail.com> In-Reply-To: <550D92C6.1080002@FreeBSD.org> References: <CAHM0Q_NRg9b4ZG5cLLO94mzaSWgz0k=TvjdyZffKG6QtNuopLA@mail.gmail.com> <550D92C6.1080002@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
>> In reference to: >> https://github.com/freebsd/freebsd/compare/master...bsdjhb:numa_bus_get_cpus >> >> >> The LOCAL_CPUS option will return the cpuset corresponding to the NUMA >> (_pxm) domain id of a given device. INTR_CPUS can narrow that further. >> However, in this particular branch INTR_CPUS will only ever return the >> value given by LOCAL_CPUS. When and where would this come in to >> effect? Do the new Haswells have cores in the same memory domain but >> with non-uniform access to the PCIe bus? > > On x86 this means that INTR_CPUS only includes one thread per core in a > the NUMA domain a device belongs to whereas LOCAL_CPUS will include all > threads in the domain. So if you have a dual-socket 8-core SB/IB system > for example, a device on the IO hub for the first processor would use > CPUs 0-15 for LOCAL_CPUS and 0,2,4,6,8,10,12,14 for INTR_CPUS if HTT is > enabled. If it is disabled, both requests would return CPUs 0-7. > I was confused by the fact that it was propagating it up the chain only to have the nexus return intr_cpus and was in turn overlooking intr_cpus being initialized in set_interrupt_apic_ids . The appearance being that it was leaving the door open for some intermediate node to narrow the cpuset - and nexus was just the default. As it stands now just directly OR'ing LOCAL_CPUS with intr_cpus now that you've made it a global would make more sense. But perhaps the added flexibility will prove useful in the future. Thanks.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAHM0Q_O_4VbVxVT5tveTjuXnF2E5CLmutN86GA%2BHf9zJ9gvrww>