Date: Fri, 5 Jul 2002 00:20:56 -0400 From: Bosko Milekic <bmilekic@unixdaemons.com> To: Andrew Gallatin <gallatin@cs.duke.edu> Cc: "Kenneth D. Merry" <ken@kdm.org>, current@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: virtually contig jumbo mbufs (was Re: new zero copy sockets snapshot) Message-ID: <20020705002056.A5365@unixdaemons.com> In-Reply-To: <15652.46870.463359.853754@grasshopper.cs.duke.edu>; from gallatin@cs.duke.edu on Thu, Jul 04, 2002 at 04:59:02PM -0400 References: <xzpelf3ida1.fsf@flood.ping.uio.no> <20020619090046.A2063@panzer.kdm.org> <20020619120641.A18434@unixdaemons.com> <15633.17238.109126.952673@grasshopper.cs.duke.edu> <20020619233721.A30669@unixdaemons.com> <15633.62357.79381.405511@grasshopper.cs.duke.edu> <20020620114511.A22413@unixdaemons.com> <15634.534.696063.241224@grasshopper.cs.duke.edu> <20020620134723.A22954@unixdaemons.com> <15652.46870.463359.853754@grasshopper.cs.duke.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jul 04, 2002 at 04:59:02PM -0400, Andrew Gallatin wrote: > > I believe that the Intel chips do "virtual page caching" and that the > > logic that does the virtual -> physical address translation sits between > > the L2 cache and RAM. If that is indeed the case, then your idea of > > testing with virtually contiguous pages is a good one. > > Unfortunately, I don't know if the PIII is doing speculative > > cache-loads, but it could very well be the case. If it is and if in > > fact the chip does caching based on virtual addresses, then providing it > > with virtually contiguous address space may yield better results. If > > you try this, please let me know. I'm extremely interested in seeing > > the results! > > contigmalloc'ed private jumbo mbufs (same as bge, if_ti, etc): > > % iperf -c ugly-my -l 32k -fm > ------------------------------------------------------------ > Client connecting to ugly-my, TCP port 5001 > TCP window size: 0.2 MByte (default) > ------------------------------------------------------------ > [ 3] local 192.168.1.3 port 1031 connected with 192.168.1.4 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.0 sec 2137 MBytes 1792 Mbits/sec > > > > malloc'ed, physically discontigous private jumbo mbufs: > > % iperf -c ugly-my -l 32k -fm > ------------------------------------------------------------ > Client connecting to ugly-my, TCP port 5001 > TCP window size: 0.2 MByte (default) > ------------------------------------------------------------ > [ 3] local 192.168.1.3 port 1029 connected with 192.168.1.4 port 5001 > [ ID] Interval Transfer Bandwidth > [ 3] 0.0-10.0 sec 2131 MBytes 1788 Mbits/sec > > > So I'd be willing to believe that the 4Mb/sec loss was due to > the extra overhead of setting up 2 additional DMAs. > > > So it looks like this idea would work. Yes, it certainly confirms the virtual-based caching assumptions. I would like to provide virtually contiguous large buffers and believe I can do that via mb_alloc... however, they would be several wired-down pages. Would this be in line with the requirements that these buffers would have, in your mind? (wired-down means that your buffers will come out exactly as they would out of malloc(), so if you were using malloc() already, I'm assuming that wired-down is OK). I think I can allocate the jumbo buffers via mb_alloc from the same map as I allocate clusters from - the clust_map - and keep them in buckets/slabs in per-CPU caches, like I do for mbufs and regular clusters right now. Luigi is in the process of doing some optimisation work around mb_alloc and I'll probably be doing the SMP-specific stuff after he's done so once that's taken care of, we can take a stab at this if you think it's worth it. > Drew Regards, -- Bosko Milekic bmilekic@unixdaemons.com bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020705002056.A5365>