Date: Wed, 27 Feb 2002 17:17:26 -0500 From: Bosko Milekic <bmilekic@unixdaemons.com> To: Terry Lambert <tlambert2@mindspring.com> Cc: Jeff Roberson <jroberson@chesapeake.net>, arch@FreeBSD.ORG Subject: Re: Slab allocator Message-ID: <20020227171726.A46831@unixdaemons.com> In-Reply-To: <3C7D4401.9E57D6AD@mindspring.com>; from tlambert2@mindspring.com on Wed, Feb 27, 2002 at 12:39:29PM -0800 References: <20020227005915.C17591-100000@mail.chesapeake.net> <3C7D1E31.B13915E7@mindspring.com> <20020227143330.A34054@unixdaemons.com> <3C7D4401.9E57D6AD@mindspring.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Feb 27, 2002 at 12:39:29PM -0800, Terry Lambert wrote: > > > What is the per-CPU lock required for? I think it can be > > > gotten rid of, or at least taken out of the critical path, > > > with more information. > > > > Per-CPU caches. Reduces lock contention and trashes caches less often. > > I thinl you are misunderstanding. If the caches are per-CPU, > then by definition, they will only ever be accessed by a single > CPU, and so contention can be eliminated by ordered atomicity > of operations, unlike where there is inter-CPU contention. > > Per CPU resources are really not something you would expect > to be contended between CPUs, and within the context of a > single CPU, contention is controllable. With all due respect, I think that *you* are mis-understanding. I don't know what book says that only, without exception, CPU A will touch its own cache, without any CPU ever touching it as well. We need those locks for now, if not to deal with preemption then to also enable us to drain all the caches when the time comes to do so. Guys, please do Jeff the favor of reading his code before you go to definition and tell him what do do. > > > > I would eventually like to pull other allocators into uma (The slab > > > > allocator). We could get rid of some of the kernel submaps and provide a > > > > much more dynamic amount of various resources. Something I had in mind > > > > were pbufs and mbufs, which could easily come from uma. This gives us the > > > > ability to redistribute memory to wherever it is needed, and not lock it > > > > in a particular place once it's there. > > > > > > How do you handle interrupt-time allocation of mbufs, in > > > this case? The zalloci() handles this by pre-creation of > > > the PTE's for the page mapping in the KVA, and then only > > > has to deal with grabbing free physical pages to back them, > > > which is a non-blocking operation that can occur at interrupt, > > > and which, if it fails, is not fatal (i.e. it's handled; I've > > > considered doing the same for the page mapping and PTE's, but > > > that would make the time-to-run far less deterministic). > > > > Terry, how long will you keep thinking that mbufs come through the > > zone allocator? :-) For G*d's sake man, we've been over this before! > > Then take that part out, and answer the question about > interrupt time allocations. Whether I'm still substituting > mbufs in there when I shouldn't be or not is irrelevent to > the question. Well, if you look at the code, you'll see that it supports, in the constructor, to setup the routine that will take care of allocating slabs, should the allocator require more. That means that uma allows you to have your slabs allocated from whatever map you want, including one such as, say, mb_map, that has the reserved KVA already. > > > > The other problem is with the per cpu buckets. They are a > > > > fixed size right now. I need to define several zones for > > > > the buckets to come from and a way to manage growing/shrinking > > > > the buckets. > > > > > > I built a "chain" allocator that dealt with this issue, and > > > also the object granularit issue. Basically, it calculated > > > the LCM of the object size rounded to a MAX(sizeof(long),8) > > > boundary for processor alignment sensitivity reasons, and > > > the page size (also for processor sensitivity reasons), and > > > then allocated a contiguous region from which it obtained > > > objects of that type. All in all, it meant zero unnecessary > > > space wastage (for 1,000,000 TCP connections, the savings > > > were 1/4 of a Gigabyte for one zone alone). > > > > That's great, until you run out of pre-allocated contiguous space. > > At which point you've reached the load bearing capacity of the > system, and will have to stop, no matter what. It's not like > you can swap mbufs. Well, quite honestly, I could go on and argue that what you're saying makes no sense. But I am hesitant on wasting my time discussing hypotheticals when I know that no matter what I say you'll be able to nail me to the floor with a counter because I don't have access to the source code of the allocator you're talking about and, supposedly, you do. > -- Terry -- Bosko Milekic bmilekic@unixdaemons.com bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020227171726.A46831>