Date: Tue, 8 Apr 2014 12:03:31 +0500 From: Jordan Hubbard <jkh@ixsystems.com> To: Warner Losh <imp@bsdimp.com> Cc: freebsd-arch <freebsd-arch@freebsd.org> Subject: Re: Compiler toolchain roadmap Message-ID: <482357EF-5596-45FD-8D2C-3742BB4872E8@ixsystems.com> In-Reply-To: <EFEFB531-6279-41C1-B0BF-A0EA1F722E24@bsdimp.com> References: <201404021607.s32G7mhw051355@svn.freebsd.org> <20140404115256.GA85137@ivaldir.etoilebsd.net> <F2A33EA8-14F2-4D62-9021-9023A1751E48@FreeBSD.org> <8D6AF193-A5A3-4A28-A230-97A543395ACA@ixsystems.com> <2E0EC8CB-B3EE-4DB8-A33D-58FD2107F14D@FreeBSD.org> <6A02504F-5543-4F91-92F6-7B4FB9A34DC4@ixsystems.com> <152D73EE-DF9E-4757-B547-F1F22B12C824@FreeBSD.org> <B06E1588-8828-485F-A407-3F19231F8EA5@ixsystems.com> <8E3BD3C1-A441-48C5-97BC-45EF67513096@FreeBSD.org> <6418BE83-BE78-473B-9311-C849507FA885@ixsystems.com> <CAJ-Vmom-19LujsTQ%2Bv4XozE%2BiEH18LMEQitBLC-At=DmsgkB%2BQ@mail.gmail.com> <EB9CE8A8-E897-4DE1-A8BC-80C6CC23E612@ixsystems.com> <9E11A6D4-9D18-422D-9514-4714AADDAEF4@gmail.com> <8E22F8FA-CF71-4A47-BDE8-F3CE6158E1C9@ixsystems.com> <EFEFB531-6279-41C1-B0BF-A0EA1F722E24@bsdimp.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Apr 7, 2014, at 8:03 PM, Warner Losh <imp@bsdimp.com> wrote: > This is a desire, not a hard requirement. As such, there may be = missing bits if you chose to go this route. At least that=92s been the = notion in the past when using llvm features has come up. The notion for = this path has always been =91It is possible, but only a subset of the = functionality may be available.=94 >=20 > So if one were to add blocks, it would need a knob WITHOUT_BLOCKS that = would disable all functionality tied to them. For many applications, = this is a reasonable subset (based on my guesses at how intrusive this = would be to the system). And it might even be automatically selected = based on compiler support, but that=92s another can of worms. I=92m glad to hear that building with foreign compilers is more of a = desire than a hard constraint. I=92m still not sure what the scenario = would look like where libdispatch has been built WITHOUT_BLOCKS and is = now essentially API incompatible with the libdispatch compiled = WITH_BLOCKS (since you wouldn=92t get functions like dispatch_async() = but would get the dispatch_async_f() function which takes a function = pointer/context variable instead), but I guess that=92s a problem for = the hypothetical wacky-compiler user to figure out? You seemed to imply that when you said FreeBSD was already using a = number of custom compiler extensions, I just wasn=92t sure that this = extended as far as =93you can build FreeBSD, for the most part, with = your weird-ass compiler but you can also expect it to be a different = FreeBSD than what=92s in base for x86_64 (for example) because various = things will be switched off, API incompatible, or simply not compiled at = all." If that=92s the case, then woohoo, when can I import libdispatch into = base? :) [I=92m being semi-facetious here, I realize there are a few = more steps to go]. > So to take this a step further=85 There=92s many levels of integration = here=85 First, there=92s the kernel, which is most often the bit of = code people want/need the special compiler for. Next, there=92s having = the feature available in user land. Finally, there=92d be a wide-scale = integration of this feature. I see very few programs in base benefiting = from libdispatch, honestly, but that doesn=92t mean the set is empty. Do = you have a longer write up on what you=92d like to do here? Sure, I can write something up as part of my =93any objections to doing = this?=94 step, but just to summarize briefly for this conversation: 1. Libdispatch does not need to touch the kernel at all, nor does the = kernel require blocks support. The pthread_workqueue() stuff that = Stacey Son did all those years ago would be a nice optimization (this = lets libdispatch balance its thread pool loads across multiple = applications, not just within a single application) but it=92s not = strictly necessary, and it can also be added later without any consumer = of libdispatch (henceforth referred to by its other name of GCD, because = that=92s easier to type!) needing to know or care. 2. Once we have GCD in base, I can then start looking at adding = libnotify and its associated notifyd daemon, which is a generic system = notification mechanism. 3. Once libnotify is in base, I=92ll then add some basic notifications = to Libc, like timezone changes, hostname changes, and so on, = invalidating some of its internal caches in the face of said changes = (there are some fairly expensive stat() calls you can eliminate once you = have the far lighter-weight shared memory flag checking that notifyd = provides, which also helps save power by not walking around the = filesystem so much). The notifyd service can also translate filesystem = notifications into system notifications, which means services like samba = and netatalk can detect (cheaply) when something has changed out from = under them. Beyond that, we probably need to talk about another daemon, which on OS = X is called configd, which deals with larger scope configuration changes = like network devices coming and going, default routes changing, and so = on. This does require some kernel up-call mechanisms, but it will = =93publish=94 its results in terms of notify(3) notifications so = applications can subscribe to those events the same way. If we can hack = usbd into doing the same for USB events, that=92s great, otherwise I = would imagine configd dealing with that sort of hardware notification = service as well. > I see a continuum of answer here: If you want to modify the kernel = extensive to use blocks, then that=92s going to be a much bigger problem = than having a few daemons and a library in the tree that require them = which is a bigger problem than having a few daemons using it, but = optionally, and a library which is a bigger problem than a library in = the tree which is a bigger problem than the status quo. I.. think=85 I follow that sentence enough to hope I=92ve answered it. = No blocks in the kernel. Nothing more than a few daemons and a library = and, at some point, a way of figuring out when the kernel has made = certain changes to the system configuration, if and where said changes = can=92t be intercepted and dealt with at the libc layer or otherwise = noticed by a user-land daemon which can post the notifications for any = subscribers of them. As to the subscribers, they won=92t have to use blocks unless they = choose to consume one or more of notify(3)=92s block APIs. The notify = mechanism was written with both old-style and new-style clients in mind. = If your daemon wants a signal when the notification it=92s subscribing = to changes, fine. If it wants some data on a file descriptor which it=92s= poll(3)/kqueue(3)ing, fine. See = https://developer.apple.com/library/mac/documentation/Darwin/Reference/Man= Pages/man3/notify.3.html and, of course, ignore the mach-specific = delivery methods as N/A. > And 8.1 for libdispatch? Is it really so portable it would work with = our old, crappy gcc pre blocks update? Sorry, misunderstanding. I was referring to the original slashdot / = phoronix article which stated that GCD would be coming to FreeBSD in = 8.1. Apparently, according to FreeBSD=92s own wiki = (https://wiki.freebsd.org/GCD) it works, too. - Jordan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?482357EF-5596-45FD-8D2C-3742BB4872E8>