From owner-freebsd-hackers Sun Jan 26 5:28:46 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BB8DA37B401 for ; Sun, 26 Jan 2003 05:28:44 -0800 (PST) Received: from mailout.informatik.tu-muenchen.de (mailout.informatik.tu-muenchen.de [131.159.0.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7896A43E4A for ; Sun, 26 Jan 2003 05:28:43 -0800 (PST) (envelope-from langd@informatik.tu-muenchen.de) Received: from mailrelay2.informatik.tu-muenchen.de (mailrelay2.informatik.tu-muenchen.de [131.159.254.8]) by mailout.informatik.tu-muenchen.de (Postfix) with ESMTP id 5BF506371; Sun, 26 Jan 2003 14:28:42 +0100 (MET) Received: from atrbg11.informatik.tu-muenchen.de (atrbg11.informatik.tu-muenchen.de [131.159.42.129]) by mailrelay2.informatik.tu-muenchen.de (Postfix) with ESMTP id 44E9C47444; Sun, 26 Jan 2003 14:28:42 +0100 (MET) Received: by atrbg11.informatik.tu-muenchen.de (Postfix, from userid 20455) id C717D139AA; Sun, 26 Jan 2003 14:28:41 +0100 (CET) Date: Sun, 26 Jan 2003 14:28:41 +0100 From: Daniel Lang To: George Hartzell Cc: freebsd-hackers@freebsd.org Subject: Re: Problem opening /dev/ad0{,s2} O_RDWR (also disklabel, grub) on 5.0. Message-ID: <20030126132841.GA84216@atrbg11.informatik.tu-muenchen.de> References: <15923.18959.507300.156728@rosebud.alerce.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <15923.18959.507300.156728@rosebud.alerce.com> X-Geek: GCS/CC d-- s: a- C++$ UBS++++$ P+++$ L- E-(---) W+++(--) N++ o K w--- O? M? V? PS+(++) PE--(+) Y+ PGP+ t++ 5+++ X R+(-) tv+ b+ DI++ D++ G++ e+++ h---(-) r++>+++ y+ User-Agent: Mutt/1.5.1i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi George, George Hartzell wrote on Sat, Jan 25, 2003 at 06:38:07PM -0800: [..] > open("/dev/ad0", 1)', and 'call open("/dev/ad0", 2)' made it clear > that anything that would write to the disk was failing. [..] > disklabel: /dev/ad0s2: Operation not permitted [..] > So, my questions are: > > 1) does this ring a bell with anyone? > > 2) Is there something in 5.0 that requires special magic to write to > the raw disk devices? You need to run in securelevel < 1. Check "sysctl kern.securelevel", and read init(8). I guess you have some rc.conf entry that raises your securelevel, most probably resulting from the sysinstall. HTH, Daniel -- IRCnet: Mr-Spock - "I hear that, if you play the WindowsXP CD backwards, you get a Satanic message!" - "That's nothing. If you play it forward, it installs WindowsXP!" Daniel Lang * dl@leo.org * +49 89 289 18532 * http://www.leo.org/~dl/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 6:22: 5 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 76AB537B401 for ; Sun, 26 Jan 2003 06:22:04 -0800 (PST) Received: from mta03-svc.ntlworld.com (mta03-svc.ntlworld.com [62.253.162.43]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9788143E4A for ; Sun, 26 Jan 2003 06:21:58 -0800 (PST) (envelope-from jon@witchspace.com) Received: from witchspace.com ([80.3.251.242]) by mta03-svc.ntlworld.com (InterMail vM.4.01.03.27 201-229-121-127-20010626) with SMTP id <20030126142157.YSZO4699.mta03-svc.ntlworld.com@witchspace.com> for ; Sun, 26 Jan 2003 14:21:57 +0000 Received: (qmail 69417 invoked from network); 26 Jan 2003 14:21:56 -0000 Received: from unknown (HELO witchspace.com) (192.168.0.1) by dookie.witchspace.com with SMTP; 26 Jan 2003 14:21:56 -0000 Message-ID: <3E33EF04.9080500@witchspace.com> Date: Sun, 26 Jan 2003 14:21:56 +0000 From: Jonathan Belson User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.2.1) Gecko/20030122 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-hackers@freebsd.org Subject: Tool for creating .fnt files? Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hiya Is there a tool for creating the .fnt files that syscons uses? They appear to be uuencoded binary files but I can't find out any info on the file format. Cheers, --Jon http://www.witchspace.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 6:28:27 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0FFEB37B401 for ; Sun, 26 Jan 2003 06:28:26 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 42EE843EB2 for ; Sun, 26 Jan 2003 06:28:25 -0800 (PST) (envelope-from phk@freebsd.org) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h0QESIQl007308; Sun, 26 Jan 2003 15:28:24 +0100 (CET) (envelope-from phk@freebsd.org) To: Jonathan Belson Cc: freebsd-hackers@freebsd.org Subject: Re: Tool for creating .fnt files? From: phk@freebsd.org In-Reply-To: Your message of "Sun, 26 Jan 2003 14:21:56 GMT." <3E33EF04.9080500@witchspace.com> Date: Sun, 26 Jan 2003 15:28:18 +0100 Message-ID: <7307.1043591298@critter.freebsd.dk> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <3E33EF04.9080500@witchspace.com>, Jonathan Belson writes: >Hiya > > >Is there a tool for creating the .fnt files that syscons uses? They >appear to be uuencoded binary files but I can't find out any info on >the file format. It's a raw bit-map font, this is from iso-8x14: Hex Binary 18 00011000 3c 00111100 3c 00111100 3c 00111100 18 00011000 18 00011000 00 00000000 18 00011000 00 00000000 00 00000000 00 00000000 00 00000000 66 01100110 66 01100110 66 01100110 24 00100100 00 00000000 00 00000000 00 00000000 00 00000000 00 00000000 00 00000000 00 00000000 00 00000000 00 00000000 -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 9:28:49 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C8FC237B401 for ; Sun, 26 Jan 2003 09:28:47 -0800 (PST) Received: from kestrel.alerce.com (kestrel.alerce.com [209.182.219.40]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0B7DF43F18 for ; Sun, 26 Jan 2003 09:28:47 -0800 (PST) (envelope-from hartzell@rosebud.alerce.com) Received: from rosebud.alerce.com (w092.z064001164.sjc-ca.dsl.cnc.net [64.1.164.92]) by kestrel.alerce.com (8.12.4/8.12.4) with ESMTP id h0QHSTk6083314 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=OK); Sun, 26 Jan 2003 09:28:31 -0800 (PST) (envelope-from hartzell@rosebud.alerce.com) X-Authentication-Warning: kestrel.alerce.com: Host w092.z064001164.sjc-ca.dsl.cnc.net [64.1.164.92] claimed to be rosebud.alerce.com Received: from rosebud.alerce.com (rosebud.alerce.com [127.0.0.1]) by rosebud.alerce.com (8.12.7/8.12.7) with ESMTP id h0QHSZN0091047 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Sun, 26 Jan 2003 09:28:36 -0800 (PST) (envelope-from hartzell@rosebud.alerce.com) Received: (from hartzell@localhost) by rosebud.alerce.com (8.12.7/8.12.6/Submit) id h0QHSXBZ091044; Sun, 26 Jan 2003 09:28:33 -0800 (PST) From: George Hartzell MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15924.6849.193023.512666@rosebud.alerce.com> Date: Sun, 26 Jan 2003 09:28:33 -0800 To: Daniel Lang Cc: freebsd-hackers@freebsd.org Subject: Re: Problem opening /dev/ad0{,s2} O_RDWR (also disklabel, grub) on 5.0. In-Reply-To: <20030126132841.GA84216@atrbg11.informatik.tu-muenchen.de> References: <15923.18959.507300.156728@rosebud.alerce.com> <20030126132841.GA84216@atrbg11.informatik.tu-muenchen.de> X-Mailer: VM 7.07 under 21.1 (patch 14) "Cuyahoga Valley" XEmacs Lucid Reply-To: hartzell@kestrel.alerce.com (George Hartzell) Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Daniel Lang writes: > Hi George, > > George Hartzell wrote on Sat, Jan 25, 2003 at 06:38:07PM -0800: > [..] > > open("/dev/ad0", 1)', and 'call open("/dev/ad0", 2)' made it clear > > that anything that would write to the disk was failing. > [..] > > disklabel: /dev/ad0s2: Operation not permitted > [..] > > So, my questions are: > > > > 1) does this ring a bell with anyone? > > > > 2) Is there something in 5.0 that requires special magic to write to > > the raw disk devices? > > > You need to run in securelevel < 1. > > Check "sysctl kern.securelevel", and read init(8). > > I guess you have some rc.conf entry that raises > your securelevel, most probably resulting from the sysinstall. That's not it, kern.securelevel is -1 and this is in /etc/rc.conf kern_securelevel_enable="NO". Any other thoughts? g. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 10:13:30 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8F4FE37B401 for ; Sun, 26 Jan 2003 10:13:28 -0800 (PST) Received: from canning.wemm.org (canning.wemm.org [192.203.228.65]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3C10B43EB2 for ; Sun, 26 Jan 2003 10:13:28 -0800 (PST) (envelope-from peter@wemm.org) Received: from wemm.org (localhost [127.0.0.1]) by canning.wemm.org (Postfix) with ESMTP id 26F172A89E; Sun, 26 Jan 2003 10:13:28 -0800 (PST) (envelope-from peter@wemm.org) X-Mailer: exmh version 2.5 07/13/2001 with nmh-1.0.4 To: hartzell@kestrel.alerce.com (George Hartzell) Cc: Daniel Lang , freebsd-hackers@freebsd.org Subject: Re: Problem opening /dev/ad0{,s2} O_RDWR (also disklabel, grub) on 5.0. In-Reply-To: <15924.6849.193023.512666@rosebud.alerce.com> Date: Sun, 26 Jan 2003 10:13:28 -0800 From: Peter Wemm Message-Id: <20030126181328.26F172A89E@canning.wemm.org> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG George Hartzell wrote: > Daniel Lang writes: > > Hi George, > > > > George Hartzell wrote on Sat, Jan 25, 2003 at 06:38:07PM -0800: > > [..] > > > open("/dev/ad0", 1)', and 'call open("/dev/ad0", 2)' made it clear > > > that anything that would write to the disk was failing. > > [..] > > > disklabel: /dev/ad0s2: Operation not permitted > > [..] > > > So, my questions are: > > > > > > 1) does this ring a bell with anyone? > > > > > > 2) Is there something in 5.0 that requires special magic to write to > > > the raw disk devices? > > > > > You need to run in securelevel < 1. > > > > Check "sysctl kern.securelevel", and read init(8). > > > > I guess you have some rc.conf entry that raises > > your securelevel, most probably resulting from the sysinstall. > > That's not it, > > kern.securelevel is -1 > > and this is in /etc/rc.conf > > kern_securelevel_enable="NO". > > Any other thoughts? > > g. Yes, this is a not-quite-yet resolved side effect of GEOM that is due to be fixed any minute now. Geom is overly protective when partitions are open and mounted. For example, if ad0s1a is open, you cannot open ad0s1 or ad0 for read/write since they overlap the ad0s1a device which has an exclusive lock on it. This is a problem for installing boot code like you were trying to do at the start of the thread. The old partition code had some huge holes in it that meant it was really easy to do bad things, and over time things started taking advantage of it. There are several things you can do. 1) Boot off floppy or fixit cdrom or something and fix it. 2) use the NO_GEOM kernel compile option 3) Or try this hack: ftp://ftp.jurai.net/users/winter/patches/geom-foot.patch This adds a kern.geom.allow_foot_shooting sysctl. #3 is the most useful option at the moment because it enables you to turn on the sysctl and do the update, but still give GEOM a good workout and test in the real world. It is new in 5.0 and it would be nice to get every bit of real-world testing possible. NO_GEOM is fairly likely to not be in 5.1, so we need to get any more suprises found and taken care of. If you are feeling like some excitement, talk with phk@freebsd.org and see if he has patches for adding disklabel bootcode write support in a usable state for testing yet. Cheers, -Peter -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com "All of this is for nothing if we don't go to the stars" - JMS/B5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 11:31:15 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 40E7037B401 for ; Sun, 26 Jan 2003 11:31:14 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id DDD5B43E4A for ; Sun, 26 Jan 2003 11:31:13 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0QJVC0i052102; Sun, 26 Jan 2003 11:31:12 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0QJVCp8052101; Sun, 26 Jan 2003 11:31:12 -0800 (PST) Date: Sun, 26 Jan 2003 11:31:12 -0800 (PST) From: Matthew Dillon Message-Id: <200301261931.h0QJVCp8052101@apollo.backplane.com> To: "Sean Hamilton" Cc: Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Greetings, : :I have a situation where I am reading large quantities of data from disk :sequentially. The problem is that as the data is read, the oldest cached :blocks are thrown away in favor of new ones. When I start re-reading data :from the beginning, it has to read the entire file from disk again. Is there :some sort of sysctl which could be changed to induce a more random expiry of :cached disk blocks? Wouldn't it seem logical to have something like this in :place at all times? : :thanks, : :sh Hi Sean. I've wanted to have a random-disk-cache-expiration feature for a long time. We do not have one now. We do have mechanisms in place to reduce the impact of sequential cycling a large dataset so it does not totally destroy unrelated cached data. Due to the way our page queues work, it's not an easy problem to solve. You might be able to simulate more proactive cache control by using O_DIRECT reads for some of the data, and normal reads for the rest of the data (see the 'fcntl' manual page). But it might be better simply to purchase more memory, purchase a faster hard drive, or stripe two hard drives together. HDs these days can do 25-50MB/s each so striping two together should yield 50-100MB/s of sequential throughput. See the 'STRIPING DISKS' section in 'man tuning'. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 11:39:37 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 125E737B401 for ; Sun, 26 Jan 2003 11:39:36 -0800 (PST) Received: from darkserver.dyndns.org (bzq-198-106.red.bezeqint.net [212.179.198.106]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F29E43ED8 for ; Sun, 26 Jan 2003 11:39:35 -0800 (PST) (envelope-from uri@darkserver.dyndns.org) Received: by darkserver.dyndns.org (Postfix, from userid 1100) id B22231CB46A; Sun, 26 Jan 2003 21:39:33 +0200 (IST) Received: from localhost (localhost [127.0.0.1]) by darkserver.dyndns.org (Postfix) with ESMTP id AA5F31CB422; Sun, 26 Jan 2003 21:39:33 +0200 (IST) Date: Sun, 26 Jan 2003 21:39:33 +0200 (IST) From: Uri Shaked Reply-To: uri@keves.org To: Jonathan Belson Cc: freebsd-hackers@freebsd.org, Subject: Re: Tool for creating .fnt files? (fwd) In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Well Me and my friend have written such a tool, which is available under http://fonteditfs.sourceforge.net/ We've also submitted a port using send-pr, unfortunately, its state is still open (after two months or so)... Let's hope they'll add it soon. Uri. > > - ---------- Forwarded message ---------- > Date: Sun, 26 Jan 2003 14:21:56 +0000 > From: Jonathan Belson > To: freebsd-hackers@freebsd.org > Subject: Tool forcreating .fnt files? > > Hiya > > > Is there a tool for creating the .fnt files that syscons uses?They > appear to be uuencoded binary files but I can't find out any info on > the file format. > > Cheers, > > > - --Jon > > http://www.witchspace.com > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.2.1 (FreeBSD) > > iD8DBQE+NBM/6GJjqfuvOIgRAhRUAJ9KhQT5sIwBeAhNz6njhy+R+DRfWQCfVlNK > egskC9omxoAz+iasY1LS9yM= > =dCQh > -----END PGP SIGNATURE----- > > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 11:57: 5 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3362E37B401 for ; Sun, 26 Jan 2003 11:57:04 -0800 (PST) Received: from willers.employees.org (willers.employees.org [128.107.241.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id B273B43EB2 for ; Sun, 26 Jan 2003 11:57:03 -0800 (PST) (envelope-from stannous@willers.employees.org) Received: from willers.employees.org (localhost [127.0.0.1]) by willers.employees.org (8.12.5/8.12.5) with ESMTP id h0QJuviC015740 for ; Sun, 26 Jan 2003 11:56:57 -0800 (PST) Received: (from stannous@localhost) by willers.employees.org (8.12.5/8.12.5/Submit) id h0QJuv4a015739 for freebsd-hackers@freebsd.org; Sun, 26 Jan 2003 11:56:57 -0800 (PST) Date: Sun, 26 Jan 2003 11:56:57 -0800 From: Sam Tannous To: freebsd-hackers@freebsd.org Subject: max simultaneous TCP connections (32,763)? Message-ID: <20030126195657.GA14704@cisco.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i X-Spam-Status: No, hits=-4.6 required=8.0 tests=SPAM_PHRASE_00_01,USER_AGENT,USER_AGENT_MUTT version=2.41 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I have two freebsd boxes (back to back) and I've been playing with a simple server on one machine and client on the other machine (this was simply an exercise with playing with kqueue). Both the server and the client are single processes and the client seems to stop at 32,763 connections. I've modified the port range, tcp keepalive, kern.ipc.somaxconn, maxfiles, maxsockets, nmbclusters. I even tried net.inet.tcp.tcbhashsize (up to 1024). Is there some other parameter I'm missing? Or is this a known limitation/bug? --Sam To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 12:11:11 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 02DB837B401 for ; Sun, 26 Jan 2003 12:11:10 -0800 (PST) Received: from mailout05.sul.t-online.com (mailout05.sul.t-online.com [194.25.134.82]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33C5343F43 for ; Sun, 26 Jan 2003 12:11:09 -0800 (PST) (envelope-from osgene@web.de) Received: from fwd00.sul.t-online.de by mailout05.sul.t-online.com with smtp id 18ct7U-0004oj-01; Sun, 26 Jan 2003 21:11:08 +0100 Received: from badger.home (520018328195-0001@[80.132.111.4]) by fmrl00.sul.t-online.com with esmtp id 18ct7K-0ibDoOC; Sun, 26 Jan 2003 21:10:58 +0100 Received: (from gene@localhost) by badger.home (8.12.6/8.12.6) id h0QKAYQF000298; Sun, 26 Jan 2003 21:10:34 +0100 (CET) (envelope-from gene) Date: Sun, 26 Jan 2003 21:10:33 +0100 From: Eugene Ossintsev To: uri@keves.org Cc: freebsd-hackers@freebsd.org Subject: Re: Tool for creating .fnt files? (fwd) Message-ID: <20030126201033.GA269@badger.home> Mail-Followup-To: uri@keves.org, freebsd-hackers@freebsd.org References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4i X-Sender: 520018328195-0001@t-dialin.net Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hallo, I've developed another one two or three years ago. It is also BSD licensed, ncurses based, and FreeBSD ported. And I think it has much more functionality. Could you please look at it? http://lrn.ru/~osgene Cheers, Eugene > Me and my friend have written such a tool, which is available under > http://fonteditfs.sourceforge.net/ We've also submitted a port using > send-pr, unfortunately, its state is still open (after two months or > so)... Let's hope they'll add it soon. Uri. -- Eugene Ossintsev To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 12:13:15 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 54F5937B401 for ; Sun, 26 Jan 2003 12:13:14 -0800 (PST) Received: from out3.mx.nwbl.wi.voyager.net (out3.mx.nwbl.wi.voyager.net [169.207.3.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id CFA2043F1E for ; Sun, 26 Jan 2003 12:13:13 -0800 (PST) (envelope-from silby@silby.com) Received: from [10.1.1.6] (d34.as7.nwbl0.wi.voyager.net [169.207.128.162]) by out3.mx.nwbl.wi.voyager.net (Postfix) with ESMTP id 1F9CE771CF; Sun, 26 Jan 2003 14:13:07 -0600 (CST) Date: Sun, 26 Jan 2003 14:20:55 -0600 (CST) From: Mike Silbersack To: Sam Tannous Cc: freebsd-hackers@freebsd.org Subject: Re: max simultaneous TCP connections (32,763)? In-Reply-To: <20030126195657.GA14704@cisco.com> Message-ID: <20030126141644.A607-100000@patrocles.silby.com> References: <20030126195657.GA14704@cisco.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 26 Jan 2003, Sam Tannous wrote: > I have two freebsd boxes (back to back) and I've > been playing with a simple server on one machine > and client on the other machine (this was simply > an exercise with playing with kqueue). Both the > server and the client are single processes and the > client seems to stop at 32,763 connections. > > I've modified the port range, tcp keepalive, > kern.ipc.somaxconn, maxfiles, maxsockets, nmbclusters. > I even tried net.inet.tcp.tcbhashsize (up to 1024). > > Is there some other parameter I'm missing? Or is this > a known limitation/bug? > > --Sam Look more closely at the net.inet.ip.portrange.* sysctl values, you should be able to make the range as large as 1024 to 65535. At present, FreeBSD uses an overly simple hash table which only allows each outgoing port to be used once. As a result, the number of outgoing connections is definitely limited. However, there should be no similar limitation on incoming connections, other than memory size. You might consider setting up more clients and seeing how far you can push the server. :) Also, disabling keepalives shouldn't matter, they don't take up any additional space except for during the period when they are actually sent. (Which would require 2 hours of idle time.) Mike "Silby" Silbersack To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 12:29:42 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0304F37B401 for ; Sun, 26 Jan 2003 12:29:41 -0800 (PST) Received: from kientzle.com (h-66-166-149-50.SNVACAID.covad.net [66.166.149.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4DC1E43F65 for ; Sun, 26 Jan 2003 12:29:37 -0800 (PST) (envelope-from kientzle@acm.org) Received: from acm.org (UGLY.x.kientzle.comg [66.166.149.51] (may be forged)) by kientzle.com (8.11.3/8.11.3) with ESMTP id h0QKTaR16488 for ; Sun, 26 Jan 2003 12:29:36 -0800 (PST) (envelope-from kientzle@acm.org) Message-ID: <3E344532.50007@acm.org> Date: Sun, 26 Jan 2003 12:29:38 -0800 From: Tim Kientzle Reply-To: kientzle@acm.org User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.6) Gecko/20011206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-hackers@FreeBSD.ORG Subject: umount of procfs fails Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Experimenting with 'mount' and stumbled across the following oddity: mount -t procfs proc /mnt umount -t /mnt results in procfs still mounted on /mnt but no longer mounted on /proc. It appears that a umount of procfs is unmounting the most recently mounted instance rather than the instance mounted at the specified location. I haven't checked to see if other filesystem types have this problem. Tim Kientzle To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 12:29:54 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8E6C737B401 for ; Sun, 26 Jan 2003 12:29:53 -0800 (PST) Received: from mastercard.com (MCNSTL41.mastercard.com [12.22.158.102]) by mx1.FreeBSD.org (Postfix) with ESMTP id 36C0043F1E for ; Sun, 26 Jan 2003 12:29:49 -0800 (PST) (envelope-from MCNSTL41%MASTERCARD@mastercard.com) From: "MCNSTL41" X-Priority: 3 (Normal) Date: Sun, 26 Jan 2003 14:29:37 -0600 Subject: Report to Sender To: hackers Message-ID: X-MIMETrack: Serialize by Router on MCNSTL41/MASTERCARD(Release 5.0.11 |July 24, 2002) at 01/26/2003 02:29:53 PM MIME-Version: 1.0 Content-type: text/plain; charset=us-ascii Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Incident Information:- Database: d:/notes/data/mail2.box Originator: hackers Recipients: hostmaster@mastercard.org, hostmaster@mastercard.com Subject: Hi,some questions Date/Time: 01/26/2003 02:29:28 PM The file attachment 1,10121,0-1067-402-0,00[1].exe you sent to the recipients listed above was infected with the W32/Klez.h@MM virus and was not successfully cleaned. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 13:41:27 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DEE3437B401 for ; Sun, 26 Jan 2003 13:41:25 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4C77543EB2 for ; Sun, 26 Jan 2003 13:41:25 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.12.6/8.12.5) with SMTP id h0QLe7P4003907; Sun, 26 Jan 2003 16:40:08 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Sun, 26 Jan 2003 16:40:07 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Sam Tannous Cc: freebsd-hackers@freebsd.org Subject: Re: max simultaneous TCP connections (32,763)? In-Reply-To: <20030126195657.GA14704@cisco.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 26 Jan 2003, Sam Tannous wrote: > I have two freebsd boxes (back to back) and I've been playing with a > simple server on one machine and client on the other machine (this was > simply an exercise with playing with kqueue). Both the server and the > client are single processes and the client seems to stop at 32,763 > connections. > > I've modified the port range, tcp keepalive, kern.ipc.somaxconn, > maxfiles, maxsockets, nmbclusters. I even tried > net.inet.tcp.tcbhashsize (up to 1024). > > Is there some other parameter I'm missing? Or is this a known > limitation/bug? Some of this has to do with limits on the available ancillary ports for out-going connections. Try adding additional IP addresses to the client machine, and forcing your client software to use specific IP addresses. TCP uniquely identifies connections by the pair of port numbers and IP addresses, so assuming unconstrained use of the outgoing port space on a particular IP, that TCP/IP can in theory support up to (approx) 64k outgoing connections. In practice, we only allocate out of specific ranges. By adding additional IP addresses for outgoing connections, you increase the number of potential connections to a particular remote IP/port tuple. However, if you're not specifying a local IP address, the stack will pick "the most appropriate" local address for the route, which is probably the first IP address on the interface associated with the route to the other endpoint. Hard-coding local addreses in your application overrides that. I've never tried this (i.e., using multiple IPs to get around the TCP/IP limit), so if it doesn't work, let me know. In theory, it should. Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 13:55:59 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DA36137B401 for ; Sun, 26 Jan 2003 13:55:57 -0800 (PST) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id EF1FE43EB2 for ; Sun, 26 Jan 2003 13:55:56 -0800 (PST) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.12.6/8.12.5) with SMTP id h0QLZBP4002704; Sun, 26 Jan 2003 16:35:11 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Sun, 26 Jan 2003 16:35:11 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Tim Kientzle Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: umount of procfs fails In-Reply-To: <3E344532.50007@acm.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 26 Jan 2003, Tim Kientzle wrote: > Experimenting with 'mount' and stumbled across the following oddity: > > mount -t procfs proc /mnt > umount -t /mnt You're missing the "proc" after -t here, right? > results in procfs still mounted on /mnt but no longer mounted on /proc. > It appears that a umount of procfs is unmounting the most recently > mounted instance rather than the instance mounted at the specified > location. > > I haven't checked to see if other filesystem types have this problem. I experimented a bit, and found the following: If I unmount /mnt using simply "umount /mnt" on -CURRENT or -STABLE, /mnt is unmounted. If I specify "-t procfs", the same thing happens. If I mis-specify the mount type as "-t proc", then it silently fails, and neither is unmounted. Which isn't to say that it didn't happen for you, but you should probably provide a bit more information. First, could you identify the version of FreeBSD you're running? Second, can you include script output of the shell session in which you mount /proc, /mnt, run mount to confirm they are both mounted, then umount one, run mount to show the wrong one unmounted? Robert N M Watson FreeBSD Core Team, TrustedBSD Projects robert@fledge.watson.org Network Associates Laboratories To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 13:59:22 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B8A9137B401 for ; Sun, 26 Jan 2003 13:59:21 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id E211343E4A for ; Sun, 26 Jan 2003 13:59:20 -0800 (PST) (envelope-from phk@freebsd.org) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h0QLxHQl010924; Sun, 26 Jan 2003 22:59:18 +0100 (CET) (envelope-from phk@freebsd.org) To: Peter Wemm Cc: hartzell@kestrel.alerce.com (George Hartzell), Daniel Lang , freebsd-hackers@freebsd.org Subject: Re: Problem opening /dev/ad0{,s2} O_RDWR (also disklabel, grub) on 5.0. From: phk@freebsd.org In-Reply-To: Your message of "Sun, 26 Jan 2003 10:13:28 PST." <20030126181328.26F172A89E@canning.wemm.org> Date: Sun, 26 Jan 2003 22:59:17 +0100 Message-ID: <10923.1043618357@critter.freebsd.dk> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030126181328.26F172A89E@canning.wemm.org>, Peter Wemm writes: >Yes, this is a not-quite-yet resolved side effect of GEOM that is due to be >fixed any minute now. Geom is overly protective when partitions are open >and mounted. Geom is not overly protective, it only protects what it has to, the problem is that BSD labels have their meta-data in-band, and therefore the necessary protection becomes intrusive. Anyway, I belive I have just committed a working ioctl to the system so that the boot code can be replaced while the disk is active. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 14:44:14 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DC0E037B401; Sun, 26 Jan 2003 14:44:12 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id B80AE43F3F; Sun, 26 Jan 2003 14:44:11 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0QMiA0i069213; Sun, 26 Jan 2003 14:44:10 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0QMiA8s069212; Sun, 26 Jan 2003 14:44:10 -0800 (PST) Date: Sun, 26 Jan 2003 14:44:10 -0800 (PST) From: Matthew Dillon Message-Id: <200301262244.h0QMiA8s069212@apollo.backplane.com> To: Robert Watson Cc: Sam Tannous , freebsd-hackers@FreeBSD.ORG Subject: Re: max simultaneous TCP connections (32,763)? References: Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :> I have two freebsd boxes (back to back) and I've been playing with a :> simple server on one machine and client on the other machine (this was :> simply an exercise with playing with kqueue). Both the server and the :> client are single processes and the client seems to stop at 32,763 :> connections. :> :> I've modified the port range, tcp keepalive, kern.ipc.somaxconn, :> maxfiles, maxsockets, nmbclusters. I even tried :> net.inet.tcp.tcbhashsize (up to 1024). :> :> Is there some other parameter I'm missing? Or is this a known :> limitation/bug? : :Some of this has to do with limits on the available ancillary ports for :out-going connections. Try adding additional IP addresses to the client :... :Robert N M Watson FreeBSD Core Team, TrustedBSD Projects :robert@fledge.watson.org Network Associates Laboratories It could be a file descriptor limit being hit. apollo:/home/dillon> sysctl -a | fgrep maxfi kern.maxfiles: 6120 kern.maxfilesperproc: 5508 test2:/home/dillon> sysctl -a | fgrep maxfi kern.maxfiles: 12328 kern.maxfilesperproc: 11095 -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 15:31: 5 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 27C0A37B401; Sun, 26 Jan 2003 15:31:04 -0800 (PST) Received: from mail.imp.ch (mail.imp.ch [157.161.1.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id E830A43ED8; Sun, 26 Jan 2003 15:31:02 -0800 (PST) (envelope-from mb@imp.ch) Received: from levais.imp.ch (levais.imp.ch [157.161.4.66]) by mail.imp.ch (8.12.6/8.12.3) with ESMTP id h0QNUwvG063960; Mon, 27 Jan 2003 00:30:58 +0100 (CET) (envelope-from Martin.Blapp@imp.ch) Date: Mon, 27 Jan 2003 00:30:56 +0100 (CET) From: Martin Blapp To: Tim Kientzle , "" Cc: freebsd-hackers@freebsd.org Subject: Re: umount of procfs fails Message-ID: <20030127001151.L54133@levais.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi, I just checked this: -su-2.05b# mount -t procfs proc /proc/ -su-2.05b# mount -t procfs proc /mnt -su-2.05b# mount [...] procfs on /proc (procfs, local) procfs on /mnt (procfs, local) -su-2.05b# umount procfs -su-2.05b# mount [...] procfs on /mnt (procfs, local) Looks like the wrong got unmounted. The mountlist should be traversed in reverse order. I fixed exactly this type of bug for NFS mounts and UFS disks. It works there fine. I'll have a look at the code tomorrow. Bye the way, mounting procfs overlay over the same existing mount works, unmounting the first results in a PANIC (TM). I'll try to fix the userland stuff. Martin Martin Blapp, ------------------------------------------------------------------ ImproWare AG, UNIXSP & ISP, Zurlindenstrasse 29, 4133 Pratteln, CH Phone: +41 61 826 93 00 Fax: +41 61 826 93 01 PGP: PGP Fingerprint: B434 53FC C87C FE7B 0A18 B84C 8686 EF22 D300 551E ------------------------------------------------------------------ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 15:50:12 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BE9E837B405; Sun, 26 Jan 2003 15:50:10 -0800 (PST) Received: from mail.imp.ch (mail.imp.ch [157.161.1.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2784343F1E; Sun, 26 Jan 2003 15:50:09 -0800 (PST) (envelope-from mb@imp.ch) Received: from levais.imp.ch (levais.imp.ch [157.161.4.66]) by mail.imp.ch (8.12.6/8.12.3) with ESMTP id h0QNo5vG065601; Mon, 27 Jan 2003 00:50:05 +0100 (CET) (envelope-from Martin.Blapp@imp.ch) Date: Mon, 27 Jan 2003 00:50:03 +0100 (CET) From: Martin Blapp To: Tim Kientzle , "" Cc: freebsd-hackers@freebsd.org Subject: Re: umount of procfs fails In-Reply-To: <20030127001151.L54133@levais.imp.ch> Message-ID: <20030127004626.E54252@levais.imp.ch> References: <20030127001151.L54133@levais.imp.ch> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi all, I just checked the code. Umount(8) is fine. Unmount(2) is buggy. > Looks like the wrong got unmounted. The mountlist should > be traversed in reverse order. umount(8) works as it should: umount -v procfs procfs: unmount from /mnt (but it does unmount /proc) umount(8) hands over the right mountpoint to the kernel, which just unmounts the wrong one :( > Bye the way, mounting procfs overlay over the same existing > mount works, unmounting the first results in a PANIC (TM). There are two bugs in the kernel. First one that the wrong procfs is unmounted inside the kernel if there are two. Second is that we allow overlay mounts with procfs, but panic() on unmounting. Martin To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 16:34:52 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B255637B401; Sun, 26 Jan 2003 16:34:50 -0800 (PST) Received: from kientzle.com (h-66-166-149-50.SNVACAID.covad.net [66.166.149.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id BFB7B43F3F; Sun, 26 Jan 2003 16:34:49 -0800 (PST) (envelope-from kientzle@acm.org) Received: from acm.org (UGLY.x.kientzle.comg [66.166.149.51] (may be forged)) by kientzle.com (8.11.3/8.11.3) with ESMTP id h0R0YnR17319; Sun, 26 Jan 2003 16:34:49 -0800 (PST) (envelope-from kientzle@acm.org) Message-ID: <3E347EAA.8050409@acm.org> Date: Sun, 26 Jan 2003 16:34:50 -0800 From: Tim Kientzle Reply-To: kientzle@acm.org User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.6) Gecko/20011206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Robert Watson Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: umount of procfs fails References: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Robert Watson wrote: > First, could you > identify the version of FreeBSD you're running? This is on -CURRENT as of a few days ago. > Second, can you include > script output of the shell session in which you mount /proc, /mnt, run > mount to confirm they are both mounted, then umount one, run mount to show > the wrong one unmounted? The following demonstrates the problem. I've included a final 'ls' to show that it's not just a reporting problem with 'mount'; the wrong filesystem is really getting unmounted. big# /sbin/mount /dev/ad0s1a on / (ufs, local) devfs on /dev (devfs, local) /dev/ad0s1g on /home (ufs, local, acls) /dev/ad0s1f on /tmp (ufs, local, soft-updates) /dev/ad0s1h on /usr (ufs, local, soft-updates) /dev/ad0s1e on /var (ufs, local, soft-updates) big# /sbin/mount -t procfs proc /proc big# /sbin/mount -t procfs proc /mnt big# mount /dev/ad0s1a on / (ufs, local) devfs on /dev (devfs, local) /dev/ad0s1g on /home (ufs, local, acls) /dev/ad0s1f on /tmp (ufs, local, soft-updates) /dev/ad0s1h on /usr (ufs, local, soft-updates) /dev/ad0s1e on /var (ufs, local, soft-updates) procfs on /proc (procfs, local) procfs on /mnt (procfs, local) big# /sbin/umount /mnt big# mount /dev/ad0s1a on / (ufs, local) devfs on /dev (devfs, local) /dev/ad0s1g on /home (ufs, local, acls) /dev/ad0s1f on /tmp (ufs, local, soft-updates) /dev/ad0s1h on /usr (ufs, local, soft-updates) /dev/ad0s1e on /var (ufs, local, soft-updates) procfs on /mnt (procfs, local) big# ls /proc big# ls /mnt 0 1156 18 24 30 378 436 509 531 9 1 1173 19 25 31 38 44 520 532 curproc 10 12 2 26 32 39 441 521 537 11 13 20 27 33 4 444 522 6 1151 14 21 28 34 40 468 523 624 1153 15 22 280 35 41 470 524 625 1154 16 226 29 36 42 478 525 7 1155 17 23 3 37 43 5 529 8 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 17:30:16 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B4EC937B405 for ; Sun, 26 Jan 2003 17:30:14 -0800 (PST) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id B2AA043ED8 for ; Sun, 26 Jan 2003 17:30:13 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0261.cvx21-bradley.dialup.earthlink.net ([209.179.193.6] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18cy6F-0005ku-00; Sun, 26 Jan 2003 17:30:12 -0800 Message-ID: <3E348B51.6F4D6096@mindspring.com> Date: Sun, 26 Jan 2003 17:28:49 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Matthew Dillon Cc: Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301261931.h0QJVCp8052101@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a43d0b4c5cf2747a5c3ce4a146a14243403ca473d225a0f487350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon wrote: > Hi Sean. I've wanted to have a random-disk-cache-expiration feature > for a long time. We do not have one now. We do have mechanisms in > place to reduce the impact of sequential cycling a large dataset so > it does not totally destroy unrelated cached data. I really dislike the idea of random expiration; I don't understand the point, unless you are trying to get better numbers on some benchmark that intentionally determines the cache size, and then chooses to use a dataset larger than that, to factor out caching and/or make bogus representations about the value of caching in the first place. There are a number of well-known proxy cache benchmarks that do this, because the person who wrote them wanted to advocate against the use of proxy caches entirely. A number of proxy cache vendors use random page replacement to get around the benchmark tuning itself to get the worst possible performance when using a proxy cache; with random replacement, you maintain 25% of your workingselt, instead of 0% with intentional overflow from the benchmark (or 100%, if you are not doing bogus things to try and advocate against cahe vendors). > Due to the way our page queues work, it's not an easy problem to solve. It's actually pretty trivial, if you are willing to do it on a per-vnode basis, rather than a per-open instance basis. This is called a "working set quota" on VMS, and the code to do this has been around there for over 20 years now. The way you do this is to keep a count of pages associated with a given vnode, and if you are over the quota, and want more pages, to steal from the end of the per-vnode LRU, rather than going to a system wide LRU, and competing with other files. This assumes that you are not ignoring the value of locality of reference, and hacking the code to make some bogus benghmark run faster, and are instead replacing pages for performance reasons -- real performance reasons, not fake ones. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 17:37:57 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E511537B401 for ; Sun, 26 Jan 2003 17:37:55 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D97C43EB2 for ; Sun, 26 Jan 2003 17:37:55 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0261.cvx21-bradley.dialup.earthlink.net ([209.179.193.6] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18cyDg-00044t-00; Sun, 26 Jan 2003 17:37:53 -0800 Message-ID: <3E348D1E.DDAF6DF0@mindspring.com> Date: Sun, 26 Jan 2003 17:36:30 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Sam Tannous Cc: freebsd-hackers@freebsd.org Subject: Re: max simultaneous TCP connections (32,763)? References: <20030126195657.GA14704@cisco.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a43d0b4c5cf2747a5cef3ceb1dcc52c4c2548b785378294e88350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Sam Tannous wrote: > I have two freebsd boxes (back to back) and I've > been playing with a simple server on one machine > and client on the other machine (this was simply > an exercise with playing with kqueue). Both the > server and the client are single processes and the > client seems to stop at 32,763 connections. > > I've modified the port range, tcp keepalive, > kern.ipc.somaxconn, maxfiles, maxsockets, nmbclusters. > I even tried net.inet.tcp.tcbhashsize (up to 1024). > > Is there some other parameter I'm missing? Or is this > a known limitation/bug? You must tune up maxfiles at boot time, not afterwards, or the number of tcpcb's and inpcb's will be limited to the value of maxfiles at compile time, no matter what you set it to later with that !#$%@&*! sysctl that pretends to be read/write, but should really be read-only after boot time. The limit on outbound connections is 65535. Technically, this is per IP, but you will have to read the kernel code in some detail to use more than one IP, since FreeBSD has a bug in inpcballoc, in that it treats all outbound sockets as if they are allocated from the INADDR_ANY range, even if you are only using a single IP address (technically, you should be able to bind on the way out before a connect, but there are some hoops ou have to jup through to avoid certain "if" tests that should not be there. Given the 32763 number, it looks like you are just quoting me back to me, pretending the historical answer is the current answer, and that has not been accurate since FreeBSD 4.5. I have *personally* tuned a FreeBSD box with 4G of RAM, and *personally* achieved a reacord number of connections. The current record is 1,603,127 simultaneous connections, in FreeBSD 4.4, without IPSEC. As far as I know, I hold the single machine connection record for an x86 box. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 17:43: 3 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2850B37B401 for ; Sun, 26 Jan 2003 17:42:57 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 433C043EB2 for ; Sun, 26 Jan 2003 17:42:57 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0R1gu0i070183; Sun, 26 Jan 2003 17:42:56 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0R1guR3070182; Sun, 26 Jan 2003 17:42:56 -0800 (PST) Date: Sun, 26 Jan 2003 17:42:56 -0800 (PST) From: Matthew Dillon Message-Id: <200301270142.h0R1guR3070182@apollo.backplane.com> To: Terry Lambert Cc: Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301261931.h0QJVCp8052101@apollo.backplane.com> <3E348B51.6F4D6096@mindspring.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :I really dislike the idea of random expiration; I don't understand :the point, unless you are trying to get better numbers on some :.. Well, the basic scenario is something like this: Lets say you have 512MB of ram and you are reading a 1GB file sequentially, over and over again. The cache scenario that gives you the greatest performance is basically to keep 512MB worth of the file in memory and throw away the rest as you read it. This is effectively what random disk cache expiration gives you (assuming you expire whole tracks worth of data, not just random pages). i.e. it's only useful when you are cycling through a data set that is larger then main memory over and over again. Most people deal with this issue simply by beefing up their disk subsystem. After all, you don't have to stripe very many disks (only 2 or 3) to get 100MB/sec+ in throughput from the disk subsystem. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 17:45:58 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B132737B405 for ; Sun, 26 Jan 2003 17:45:51 -0800 (PST) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 23B8043F1E for ; Sun, 26 Jan 2003 17:45:50 -0800 (PST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.12.6/8.12.6) id h0R1jnae091012; Sun, 26 Jan 2003 19:45:49 -0600 (CST) (envelope-from dan) Date: Sun, 26 Jan 2003 19:45:49 -0600 From: Dan Nelson To: Jonathan Belson Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Tool for creating .fnt files? Message-ID: <20030127014549.GK6137@dan.emsphone.com> References: <3E33EF04.9080500@witchspace.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3E33EF04.9080500@witchspace.com> X-OS: FreeBSD 5.0-CURRENT X-message-flag: Outlook Error User-Agent: Mutt/1.5.3i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In the last episode (Jan 26), Jonathan Belson said: > Is there a tool for creating the .fnt files that syscons uses? They > appear to be uuencoded binary files but I can't find out any info on > the file format. They're only uuencoded for easy storage in CVS. Vidcontrol can take regular raw 8xN font bitmap files as well. I use an old DOS program called Font Mania, and there are hundreds of VGA fonts available for download at Simtel. http://www.simtel.net/pub/msdos/vga/ http://www.simtel.net/pub/msdos/screen/ -- Dan Nelson dnelson@allantgroup.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 17:48: 9 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D879137B401; Sun, 26 Jan 2003 17:48:07 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 63F2E43F13; Sun, 26 Jan 2003 17:48:07 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0261.cvx21-bradley.dialup.earthlink.net ([209.179.193.6] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18cyNa-0005Nx-00; Sun, 26 Jan 2003 17:48:07 -0800 Message-ID: <3E348F84.2546A320@mindspring.com> Date: Sun, 26 Jan 2003 17:46:44 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Robert Watson Cc: Sam Tannous , freebsd-hackers@freebsd.org Subject: Re: max simultaneous TCP connections (32,763)? References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4b604f27772e3c635fefc85f0000696c32601a10902912494350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Robert Watson wrote: > Some of this has to do with limits on the available ancillary ports for > out-going connections. Try adding additional IP addresses to the client > machine, and forcing your client software to use specific IP addresses. [ ... ] > Hard-coding local addreses in your > application overrides that. I've never tried this (i.e., using multiple > IPs to get around the TCP/IP limit), so if it doesn't work, let me know. > In theory, it should. In fact, though, it doesn't, because inpcb_alloc sucks, and no one was willing to check in my proposed fix (see -current list archives). This will not help on FreeBSD, which assumes bound address ports are allocated out of the INADDR_ANY address (unbound) ports list. You have to do extra work. There are 3 "if" tests you have to work around; binding to a local IP before the connect only works around one of these. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 18:10:46 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6E2AA37B401 for ; Sun, 26 Jan 2003 18:10:44 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id BBD9743F43 for ; Sun, 26 Jan 2003 18:10:41 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0261.cvx21-bradley.dialup.earthlink.net ([209.179.193.6] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18cyjO-0007nc-00; Sun, 26 Jan 2003 18:10:39 -0800 Message-ID: <3E3494CC.5895492D@mindspring.com> Date: Sun, 26 Jan 2003 18:09:16 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Matthew Dillon Cc: Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301261931.h0QJVCp8052101@apollo.backplane.com> <3E348B51.6F4D6096@mindspring.com> <200301270142.h0R1guR3070182@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4a930fa71eab4a208d8af91b485eb41eb350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon wrote: > :I really dislike the idea of random expiration; I don't understand > :the point, unless you are trying to get better numbers on some > :.. > > Well, the basic scenario is something like this: Lets say you have > 512MB of ram and you are reading a 1GB file sequentially, over and over > again. The cache scenario that gives you the greatest performance is > basically to keep 512MB worth of the file in memory and throw away the > rest as you read it. You don't need random expiration for this; in fact, random expiration will give you 50% worse performance than you expect. If you are doing this, the proper thing is to map the file window you want to keep seperately from the rest of the file, or, if it's small enough you can mmap the whole thing, then to madvise the part of the file you want to not cache as "sequential", and drop the cache. That this is not well supported is a bug in FreeBSD (IMO). > This is effectively what random disk cache expiration gives you (assuming > you expire whole tracks worth of data, not just random pages). i.e. it's > only useful when you are cycling through a data set that is larger then > main memory over and over again. And not very, there. Because you destroy locality of reference -- or, as in this case, you assume that there is no locality in the first place. I think it works at 50% of the efficiency of the madvise that I suggest, above, which, at least, gets hits on most of the file, without the risk of page replacement of the cached area of the file that you intend to read subsequently (25% hit rate in the hittable area, assuming this, since page replacement is random, if the replacement range is 50% of the size of the data set... i.e.: numbers in real life will vary downward, the larger the data set). > Most people deal with this issue simply by beefing up their disk > subsystem. After all, you don't have to stripe very many disks (only > 2 or 3) to get 100MB/sec+ in throughput from the disk subsystem. I think most people don't have this problem, because this is a very degenerate case, which requires either a serindipitously incredibly bad design, or intential screwing of caching, in order to do something like benchmark the disk I/O subsystem seperately from the case (for legitimate cases) or to "prove" that caching is not useful because you have an axe to grind (e.g. the proxy cache "benchmark" suite that does this because the author is a mathematician that has developed a pessimizing hueristic because he is philosophically opposed to proxy caching, due to his own narrow interpretation of "end to end guarantees"). In general, this is not a useful case to try to optimize, and the people who try to optimize this case are doing it to cheat on a benchmark, not because there's any treal-world application that requires it. 8-(. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 18:46:59 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 746E837B405 for ; Sun, 26 Jan 2003 18:46:55 -0800 (PST) Received: from toscano.org (ip68-100-184-64.nv.nv.cox.net [68.100.184.64]) by mx1.FreeBSD.org (Postfix) with ESMTP id EBAB843F18 for ; Sun, 26 Jan 2003 18:46:47 -0800 (PST) (envelope-from pete@toscano.org) Received: from bubba.toscano.org (localhost.localdomain [127.0.0.1]) by toscano.org (8.12.5/8.12.5) with ESMTP id h0R2kjJC017344 for ; Sun, 26 Jan 2003 21:46:45 -0500 Received: (from pete@localhost) by bubba.toscano.org (8.12.5/8.12.5/Submit) id h0R2kilU017342 for freebsd-hackers@freebsd.org; Sun, 26 Jan 2003 21:46:44 -0500 Date: Sun, 26 Jan 2003 21:46:44 -0500 From: Pete To: freebsd-hackers@freebsd.org Subject: (fwd) frustrating disklabel problem Message-ID: <20030127024644.GA17259@bubba.toscano.org> Mail-Followup-To: freebsd-hackers@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Unexpected: The Spanish Inquisition User-Agent: Mutt/1.5.3i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, I posted this a few weeks ago on freebsd-questions and reposted it a few days ago. I didn't get any responses beyond, "Hey, since the Promise card does RAID, why bother with vinum?" (To which I responded, in a nutshell, I want to learn vinum and the RAID the Promise card does isn't super great anyway.) I'm thinking that either a) I posted in the wrong forum or b) I didn't include enough info. I'm hoping that it was a and that the question was too technical for questions. If this is the wrong forum, please let me know. (And suggestions as to the correct one would be even more greatly appreciated.) If it's b, please let me know what extra info I can provide. Not having vinum set up yet is starting to really worry me. I can do what I need to do with Linux, but I installed FreeBSD on this machine because I wanted to get familiar with FreeBSD, not Linux. I'd like to keep it that way. Thank you..... Now, the original post.... I'm trying to set up a vinum mirrored plex on my 4.7-STABLE (cvsup-ed as of yesterday, 1/3) system. I have 3 30GB drives. Currently, only the first disk is being used. This disk has all the system along with all my digital photos and videos. I'd like to mirror the other two 30GB drives and move the digital photos and videos there, so that if one drive were to die, I'd still have these important pictures and videos. There is one (I hope) bit of strangeness about this system. I was initially using a Promise FastTrack 100 (pdc20267 chip) card. Since I was having problems with this, I recently got a cheap Maxtor adapter card. This Maxtor card looks to be a relabeled Promise card with the pdc20269 chip. The problem below is the same, regardless of which card is used. The system disk is connected to channel one on the card. The other two disks are master and slave on channel two. This is what FreeBSD shows on boot: ============================== atapci1: port 0xe800-0xe80f,0xe400-0xe403,0xe000-0xe007,0xdc00-0xdc03,0xd800-0xd807 mem 0xd5800000-0xd5803fff irq 11 at device 15.0 on pci0 ata2: at 0xd800 on atapci1 ata3: at 0xe000 on atapci1 [snip] ar0: 29314MB [3737/255/63] status: READY subdisks: 0 READY ad4: 29314MB [59560/16/63] at ata2-master UDMA100 ar1: 29314MB [3737/255/63] status: READY subdisks: 0 READY ad6: 29314MB [59560/16/63] at ata3-master UDMA100 ar2: 29314MB [3737/255/63] status: READY subdisks: 0 READY ad7: 29314MB [59560/16/63] at ata3-slave UDMA100 ============================== I used /stand/sysinstall to create a slice on each of the mirror disks and to label each with one huge partition. It newfs-ed them and mounted them. ============================== /dev/ar1s1e on /v1 (ufs, local, soft-updates) /dev/ar2s1e on /v2 (ufs, local, soft-updates) ============================== I have read that the next step involves using disklabel to edit the label of each disk and change the "4.2BSD" fstype to "vinum". I wanted to see the labels first, so I typed "disklabel ar1" and it showed me: ============================== # /dev/ar1c: type: ESDI disk: ar1s1 label: flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 3736 sectors/unit: 60034842 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # milliseconds track-to-track seek: 0 # milliseconds drivedata: 0 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 60034842 0 unused 0 0 # (Cyl. 0 - 3736*) e: 60034842 0 4.2BSD 2048 16384 89 # (Cyl. 0 - 3736*) ============================== The strangeness starts with ar2: ============================== # /dev/ar2: type: unknown disk: amnesiac label: fictitious flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 3737 sectors/unit: 60036417 rpm: 3600 interleave: 1 trackskew: 0 cylinderskew: 0 headswitch: 0 # milliseconds track-to-track seek: 0 # milliseconds drivedata: 0 8 partitions: # size offset fstype [fsize bsize bps/cpg] c: 60036417 0 unused 0 0 # (Cyl. 0 - 3737*) ============================== Even if I power down the machine and change the jumpers on the mirrored drives so that master and slave are swapped, I get the same values for ar1 and ar2. (Also kind of strange is that, while these drives are the same model, whatever one is the slave has 3737 cylinders, while the master always has 3736 cylinders.) If I try to edit the label of ar2 with "disklabel -e" (even if it did show the same label as ar1), I get the following error: "disklabel: Operation not supported by device" Any ideas about what's wrong or (more likely) what I'm doing wrong? Since I can mount the e partition on ar2, I'm guessing that the label info is there, but disklabel cannot see it for some reason. I guess I could just edit the label on ar1, then swap master and slave again, and edit the label on the new ar1 so that both should have fstypes of vinum. I'm not sure if vinum would work after this though. I'm assuming that I should be able to see the labels on both ar1 and ar2, so I want to try to fix this problem first. Thanks, pete To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 19:26:24 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3E56237B401 for ; Sun, 26 Jan 2003 19:26:22 -0800 (PST) Received: from kientzle.com (h-66-166-149-50.SNVACAID.covad.net [66.166.149.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 904F843ED8 for ; Sun, 26 Jan 2003 19:26:21 -0800 (PST) (envelope-from kientzle@acm.org) Received: from acm.org (UGLY.x.kientzle.comg [66.166.149.51] (may be forged)) by kientzle.com (8.11.3/8.11.3) with ESMTP id h0R3PlR17908; Sun, 26 Jan 2003 19:25:47 -0800 (PST) (envelope-from kientzle@acm.org) Message-ID: <3E34A6BB.2090601@acm.org> Date: Sun, 26 Jan 2003 19:25:47 -0800 From: Tim Kientzle Reply-To: kientzle@acm.org User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.6) Gecko/20011206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Terry Lambert Cc: Matthew Dillon , Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301261931.h0QJVCp8052101@apollo.backplane.com> <3E348B51.6F4D6096@mindspring.com> <200301270142.h0R1guR3070182@apollo.backplane.com> <3E3494CC.5895492D@mindspring.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Sean Hamilton proposes: > Wouldn't it seem logical to have [randomized disk cache expiration] in > place at all times? Terry Lambert responds: >>:I really dislike the idea of random expiration; I don't understand >>:the point, unless you are trying to get better numbers on some >>:benchmark. Matt Dillon concedes: >> ... it's only useful when you are cycling through a [large] data set ... Cycling through large data sets is not really that uncommon. I do something like the following pretty regularly: find /usr/src -type f | xargs grep function_name Even scanning through a large dataset once can really hurt competing applications on the same machine by flushing their data from the cache for no gain. I think this is where randomized expiration might really win, by reducing the penalty for disk-cache-friendly applications who are competing with disk-cache-unfriendly applications. There's an extensive literature on randomized algorithms. Although I'm certainly no expert, I understand that such algorithms work very well in exactly this sort of application, since they "usually" avoid worst-case behavior under a broad variety of inputs. The current cache is, in essence, tuned specifically to work badly on a system where applications are scanning through large amounts of data. No matter what deterministic caching algorithm you use, you're choosing to behave badly under some situation. Personally, I think there's a lot of merit to _trying_ randomized disk cache expiry and seeing how it works in practice. (I would also observe here that 5.0 now has a fast, high-quality source of randomness that seems ideal for exactly such applications.) I don't believe that it would _prevent_ applications from using optimizations such as those that Terry suggests, while possibly providing reasonable performance under a broader range of scenarios than are currently supported. Sounds like a good idea to me. Tim Kientzle To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 20:55:42 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AE8F537B401 for ; Sun, 26 Jan 2003 20:55:40 -0800 (PST) Received: from priv-edtnes62.telusplanet.net (outbound01.telus.net [199.185.220.220]) by mx1.FreeBSD.org (Postfix) with ESMTP id BA96843E4A for ; Sun, 26 Jan 2003 20:55:39 -0800 (PST) (envelope-from sh@bel.bc.ca) Received: from reason ([207.6.227.22]) by priv-edtnes62.telusplanet.net (InterMail vM.5.01.05.17 201-253-122-126-117-20021021) with SMTP id <20030127045539.UGBL1598.priv-edtnes62.telusplanet.net@reason> for ; Sun, 26 Jan 2003 21:55:39 -0700 Message-ID: <001801c2c5c0$5666de10$16e306cf@slugabed.org> From: "Sean Hamilton" To: References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301261931.h0QJVCp8052101@apollo.backplane.com> <3E348B51.6F4D6096@mindspring.com> <200301270142.h0R1guR3070182@apollo.backplane.com> <3E3494CC.5895492D@mindspring.com> <3E34A6BB.2090601@acm.org> Subject: Re: Random disk cache expiry Date: Sun, 26 Jan 2003 20:55:39 -0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG ----- Original Message ----- From: "Tim Kientzle" | Cycling through large data sets is not really that uncommon. | I do something like the following pretty regularly: | find /usr/src -type f | xargs grep function_name | | Even scanning through a large dataset once can really hurt | competing applications on the same machine by flushing | their data from the cache for no gain. I think this | is where randomized expiration might really win, by reducing the | penalty for disk-cache-friendly applications who are competing | with disk-cache-unfriendly applications. In my case I have a webserver serving up a few dozen files of about 10 MB each. While yes it is true that I could purchase more memory, and I could purchase more drives and stripe them, I am more interested in the fact that this server is constantly grinding away because it has found a weakness in the caching algorithm. After further thought, I propose something much simpler: when the kernel is hinted that access will be sequential, it should stop caching when there is little cache space available, instead of throwing away old blocks, or be much more hesitant to throw away old blocks. Consider that in almost all cases where access is sequential, as reading continues, the chances of the read being aborted increase: ie, users downloading files, directory tree traversal, etc. Since the likelihood of the first byte reading the first byte is very high, and the next one less high, and the next less yet, etc, it seems to make sense to tune the caching algorithm to accomodate this. While discussing disks, I have a minor complaint: at least on IDE systems, when doing something like an untar, the entire system is painfully unresponsive, even though CPU load is low. I presume this is because when an executable is run, it needs to sit and wait for the disk. Wouldn't it make sense to give very high disk priority to executables? Isn't that worth the extra seeks? sh To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 21:17:32 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 76DBE37B401 for ; Sun, 26 Jan 2003 21:17:31 -0800 (PST) Received: from tomts14-srv.bellnexxia.net (tomts14.bellnexxia.net [209.226.175.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id 983A043F18 for ; Sun, 26 Jan 2003 21:17:30 -0800 (PST) (envelope-from asr@softhome.net) Received: from Toronto-HSE-ppp3717089.sympatico.ca ([65.95.43.90]) by tomts14-srv.bellnexxia.net (InterMail vM.5.01.04.19 201-253-122-122-119-20020516) with ESMTP id <20030127051730.LPHF23890.tomts14-srv.bellnexxia.net@Toronto-HSE-ppp3717089.sympatico.ca>; Mon, 27 Jan 2003 00:17:30 -0500 Date: Mon, 27 Jan 2003 00:17:24 -0500 (EST) From: "Ashutosh S. Rajekar" To: Sean Hamilton Cc: Subject: Re: Random disk cache expiry In-Reply-To: <001801c2c5c0$5666de10$16e306cf@slugabed.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 26 Jan 2003, Sean Hamilton wrote: > > In my case I have a webserver serving up a few dozen files of about 10 MB > each. While yes it is true that I could purchase more memory, and I could > purchase more drives and stripe them, I am more interested in the fact that > this server is constantly grinding away because it has found a weakness in > the caching algorithm. > > After further thought, I propose something much simpler: when the kernel is > hinted that access will be sequential, it should stop caching when there is > little cache space available, instead of throwing away old blocks, or be > much more hesitant to throw away old blocks. Consider that in almost all > cases where access is sequential, as reading continues, the chances of the > read being aborted increase: ie, users downloading files, directory tree > traversal, etc. Since the likelihood of the first byte reading the first > byte is very high, and the next one less high, and the next less yet, etc, > it seems to make sense to tune the caching algorithm to accomodate this. Your case seems to be a highly specific one, and would require very specific tuning. And then one will be able to find some other "unwanted behaviour" once you tune your system for particular behaviour. -ASR http://www.crosswinds.net/~rajekar MERYL STREEP is my obstetrician! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 21:20:40 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3F23837B401 for ; Sun, 26 Jan 2003 21:20:38 -0800 (PST) Received: from ms-smtp-02.southeast.rr.com (ms-smtp-02.southeast.rr.com [24.93.67.83]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2D05C43F13 for ; Sun, 26 Jan 2003 21:20:32 -0800 (PST) (envelope-from bts@fake.com) Received: from mail3.nc.rr.com (fe3 [24.93.67.50]) by ms-smtp-02.southeast.rr.com (8.12.5/8.12.2) with ESMTP id h0R5JMo5022354; Mon, 27 Jan 2003 00:19:25 -0500 (EST) Received: from this.is.fake.com ([24.74.172.220]) by mail3.nc.rr.com with Microsoft SMTPSVC(5.5.1877.757.75); Mon, 27 Jan 2003 00:18:55 -0500 Received: by this.is.fake.com (Postfix, from userid 111) id 9BD38BBC8; Mon, 27 Jan 2003 00:19:48 -0500 (EST) Content-Type: text/plain; charset="iso-8859-1" From: "Brian T. Schellenberger" To: "Sean Hamilton" , Subject: Re: Random disk cache expiry Date: Mon, 27 Jan 2003 00:19:41 -0500 User-Agent: KMail/1.4.2 References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <3E34A6BB.2090601@acm.org> <001801c2c5c0$5666de10$16e306cf@slugabed.org> In-Reply-To: <001801c2c5c0$5666de10$16e306cf@slugabed.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200301270019.44066.bschellenberger@nc.rr.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sunday 26 January 2003 11:55 pm, Sean Hamilton wrote: | ----- Original Message ----- | From: "Tim Kientzle" | | | Cycling through large data sets is not really that uncommon. | | I do something like the following pretty regularly: | | find /usr/src -type f | xargs grep function_name | | | | Even scanning through a large dataset once can really hurt | | competing applications on the same machine by flushing | | their data from the cache for no gain. I think this | | is where randomized expiration might really win, by reducing the | | penalty for disk-cache-friendly applications who are competing | | with disk-cache-unfriendly applications. | | | After further thought, I propose something much simpler: when the | kernel is hinted that access will be sequential, it should stop | caching when there is little cache space available, instead of | throwing away old blocks, or be much more hesitant to throw away old | blocks. This to me is imminently sensible. In fact there seem like two rules that have come up in this discussion: 1. For sequential access, you should be very hesitant to throw away *another* processes blocks, at least once you have used more than, say, 25% of the cache or potential cache. 2. For sequential access, you should stop caching before you throw away your own blocks. If it's sequential it is, it seems to me, always a lose to throw away your *own* processes older bllocks on thee same file. These algorithmic changes seem to me to be more likely to be optimal most of the time. A random approach *does* reduce the penalty for worst-case scenarios but at the cost of reducing the benefit of both "normal" and "best-case" scenarios, it seems to me, even more. | Consider that in almost all cases where access is sequential, | as reading continues, the chances of the read being aborted increase: | ie, users downloading files, directory tree traversal, etc. Since the | likelihood of the first byte reading the first byte is very high, and | the next one less high, and the next less yet, etc, it seems to make | sense to tune the caching algorithm to accomodate this. | | While discussing disks, I have a minor complaint: at least on IDE | systems, when doing something like an untar, the entire system is | painfully unresponsive, even though CPU load is low. I presume this | is because when an executable is run, it needs to sit and wait for | the disk. Wouldn't it make sense to give very high disk priority to | executables? Isn't that worth the extra seeks? | | sh | | | To Unsubscribe: send mail to majordomo@FreeBSD.org | with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 21:44:51 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E2E5C37B401 for ; Sun, 26 Jan 2003 21:44:49 -0800 (PST) Received: from kientzle.com (h-66-166-149-50.SNVACAID.covad.net [66.166.149.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id A163543F18 for ; Sun, 26 Jan 2003 21:44:48 -0800 (PST) (envelope-from kientzle@acm.org) Received: from acm.org (UGLY.x.kientzle.comg [66.166.149.51] (may be forged)) by kientzle.com (8.11.3/8.11.3) with ESMTP id h0R5iKR18377; Sun, 26 Jan 2003 21:44:20 -0800 (PST) (envelope-from kientzle@acm.org) Message-ID: <3E34C734.8010801@acm.org> Date: Sun, 26 Jan 2003 21:44:20 -0800 From: Tim Kientzle Reply-To: kientzle@acm.org User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.6) Gecko/20011206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: "Brian T. Schellenberger" Cc: Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <3E34A6BB.2090601@acm.org> <001801c2c5c0$5666de10$16e306cf@slugabed.org> <200301270019.44066.bschellenberger@nc.rr.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Brian T. Schellenberger wrote: > This to me is imminently sensible. > In fact there seem like two rules that have come up in this discussion: > > 1. For sequential access, you should be very hesitant to throw away > *another* processes blocks, at least once you have used more than, say, > 25% of the cache or potential cache. > > 2. For sequential access, you should stop caching before you throw away > your own blocks. If it's sequential it is, it seems to me, always a > lose to throw away your *own* processes older bllocks on thee same > file. The question isn't "should I throw away blocks or not?". Obviously, the ideal is to keep everything in the cache, but that's not possible. The question is "what blocks should be discarded?" You've ruled out quite a few possibilities here. What blocks _should_ be discarded? Tim Kientzle To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 22: 6:18 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EA61737B401 for ; Sun, 26 Jan 2003 22:06:14 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 41B8F43F13 for ; Sun, 26 Jan 2003 22:06:14 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0003.cvx22-bradley.dialup.earthlink.net ([209.179.198.3] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18d2PC-0005Ur-00; Sun, 26 Jan 2003 22:06:03 -0800 Message-ID: <3E34CA7F.D0E0A260@mindspring.com> Date: Sun, 26 Jan 2003 21:58:23 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: kientzle@acm.org Cc: Matthew Dillon , Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301261931.h0QJVCp8052101@apollo.backplane.com> <3E348B51.6F4D6096@mindspring.com> <200301270142.h0R1guR3070182@apollo.backplane.com> <3E3494CC.5895492D@mindspring.com> <3E34A6BB.2090601@acm.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4508c9def3780424d915691749bd60981667c3043c0873f7e350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Tim Kientzle wrote: > Cycling through large data sets is not really that uncommon. > I do something like the following pretty regularly: > find /usr/src -type f | xargs grep function_name > > Even scanning through a large dataset once can really hurt > competing applications on the same machine by flushing > their data from the cache for no gain. I think this > is where randomized expiration might really win, by reducing the > penalty for disk-cache-friendly applications who are competing > with disk-cache-unfriendly applications. Let's set aside for a minute that this discussion does not -- can not -- apply to directory metadata, and that the limiting factor in your example is going to be the divorce of inode data from vnodes in the UFS/UFS2/EXT2 ihash code, or, if there is a low maximum vnode count, the vnodes themselves, and in any divorce case or vnode LRU, the association between the cached data and the FS object will be lost -- effectively losing the cached data, no matter what, since it can't be recovered, even if it's in memory, because nothing references it or remembers which pages it's in. Let's say, instead, that we are only talking about the data being grep'ed. There are a million ways to "win" against such applications; one obvious one is to establish a "cache time to live" for any cached data, and not evict if it has not passed its time to live (effectively establishing two zones, one for cached data, and one for transient data which would be cached, if you had the RAM for it). Doing this randomized drop approach is bogus. What you are effectively hoping for is that the pages will not be evicted before they are hit again, because your shotgun pellets will not shoot them down. I can prove mathematically (or you can read it out of Knuth's Seminumerical Algorithms: Sorting and Searching) that even with a "perfect" random number generator, as soon as the cache size is 115% or less of the data set size, you are almost certainly guaranteed that the pages will be evicted before they are rereferenced, given sequential access. > There's an extensive literature on randomized algorithms. > Although I'm certainly no expert, I understand that such > algorithms work very well in exactly this sort of application, > since they "usually" avoid worst-case behavior under a broad > variety of inputs. The current cache is, in essence, > tuned specifically to work badly on a system where applications > are scanning through large amounts of data. No matter what > deterministic caching algorithm you use, you're choosing > to behave badly under some situation. As I stated before, this is frequently used to "cheat" benchmarks that are intended to evict cache content to defeat performance improvements from caching (bechmarks intended to "cheat" caching). The "magic" part here, though is that the working set size ends up being only fractionally larger than the cache size. For example, the "Polygraph" benchmark intentionally "busts" the cache on intermediate servers, to make a political statement: http://www.web-polygraph.org/docs/workloads/ Network Appliance and other vendors have a random page replacement policy that scores them much better numbers on this test, simply by doing random replacement instead of LRU or some other weight based replacement which the test software is capable of detecting during the "run up" phase of the test, where th software attempts to determine the "peak rate" following cache overflow. While this gives "good numbers" for the benchmark, in a real workload, this is unlikely to be representative of real performance. Likewuse, since their intent is to find the "knee" in the performance curve, and populate the cache beyond that, then do testing there (thus "proving" that all connections "should be" end-to-end, rather than permitting the existance of proxies), this is unlikely to be representative of a real-world cache overflow condition, since in the real world, the overflow is unlikely to be limited to "101% of cache capacity", but will instead not stop there; going above 15% of planned capacity will result in significant failure of random page replacement to result in cache hits; likewise, going to 85% overage will practically guarantee an almost 100% failure rate, as cyclical access with random replacement is statistically more likely, in aggregate, to replace the pages which are there longer (the probability is iterative and additive: it's effectively a permutation). > Personally, I think there's a lot of merit to _trying_ I think that if this is implemented by someone who doesn't want to believe Donald Knuth (8-)), they should at least be willing to test with a data set of 3 times the available cache size, so that they can see that the algorithm does not result in any performance improvement. > randomized disk cache expiry and seeing how it works in practice. It should work very poorly, according to the math. Another approach to "get what you want" would be preferential caching of metadata vis-a-vis user data, for your specific example: by caching metadata preferentially, you avoid a significant number of data waits on directory and inode data, each of which represents a system call and a user space program stall, as a result. > (I would also observe here that 5.0 now has a fast, high-quality > source of randomness that seems ideal for exactly such > applications.) I don't believe that it would _prevent_ applications > from using optimizations such as those that Terry suggests, > while possibly providing reasonable performance under a > broader range of scenarios than are currently supported. > > Sounds like a good idea to me. As I said above: if you want to do it, do it. Then benchmark it under load conditions that aren't fictitiously close to "just above 100% cache fill". If you do this, it would also be useful to benchmark the "hit rate". On the other hand, if the intent of this is manufacture benchmarks on which FreeBSD can look much better than other OS's, there's a lot of space for that already, making it very easy to do, if you were inspired to do that. Personally, my take on that is that it's tantamount to lying; it's one of the reasons most of us, when referring to these things, call them "microbenchmarks" instead of just "benchmarks": they measure things, and such measurements can be made out of context -- if they are not inherently out of context (i.e. you could use "getpid" as you "null system call", and then modify the libc stub to cache the result, and zap it to -1 after fork, so that it would be recached, and the you would "get good numbers" for the "null system call" -- which were not really measuring what they were supposed to be). "There are lies, damn lies, and statistics" ... and benchmarks. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 22:26:35 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D5A3437B401 for ; Sun, 26 Jan 2003 22:26:31 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7AFE643F13 for ; Sun, 26 Jan 2003 22:26:30 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0003.cvx22-bradley.dialup.earthlink.net ([209.179.198.3] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18d2iw-00076y-00; Sun, 26 Jan 2003 22:26:27 -0800 Message-ID: <3E34CF3D.A09B3C35@mindspring.com> Date: Sun, 26 Jan 2003 22:18:37 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Sean Hamilton Cc: hackers@freebsd.org Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301261931.h0QJVCp8052101@apollo.backplane.com> <3E348B51.6F4D6096@mindspring.com> <200301270142.h0R1guR3070182@apollo.backplane.com> <3E3494CC.5895492D@mindspring.com> <3E34A6BB.2090601@acm.org> <001801c2c5c0$5666de10$16e306cf@slugabed.org> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a475b0a84d64a3f9ad554c3dc88b623b8f93caf27dac41a8fd350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Sean Hamilton wrote: > In my case I have a webserver serving up a few dozen files of about 10 MB > each. While yes it is true that I could purchase more memory, and I could > purchase more drives and stripe them, I am more interested in the fact that > this server is constantly grinding away because it has found a weakness in > the caching algorithm. This is a problem in Linux, nd in SVR4, as well. THis is the main reason that scheduler classes were implemented for SVR4, and the main reason the "fixed" scheduler class was implemented in SVR4.0.2/SVR4.2, as of UnixWare. The system guarantees time to the programs running under this scheduling class, and what other software pages out, the program then has the CPU time to page back in. This was UnixWare's response to "move mouse, wiggle cursor", when the X server performance went in the toilet. The reason the UnixWare X server performance was in the toilet is that the UnixWare "ld" program mmap's the .o files, and then seeks around in them to resolve symbols, rather than reading the parts in and building in-core data structures. The result is predictable: the linker, when linking any sufficiently large set of object files, thrashes all other pages out of core, as it moves around accessing pages in these files. The obvious answer to this problem -- and you problem -- is to implement a working set size quota on a per-vnode basis, as I previously suggested. By doing this, you cause the pages in the large objects to replace the old pages in the large objects, and your other data does not get thrashed out of core. THis works great on UnixWare, but requires modification of the VM system in order to provide counting for the number of buffers that are hubg off a given file object, with the result being that the code is not binary compatible (this is why the modification never made it into UnixWare, even though it was tested, and found to solve the "ld/X server problem" without leading to more thrashing, and do it without needing to introduce a new scheduling class, and make less CPU time available to other applications on the system to let the X server have sufficient time to thrash its pages back in). This approach will also work for your problem, which is that your several 10M files thrash everything else out of the cache, including the executable pages. Note that this approach need not have any effect under normal conditionas, until available memory hits some low watermark, causing it to trigger, and therefore we are talking a single "if" test in the page replacement path, in the normal non-trigger case, to implement it. > After further thought, I propose something much simpler: when the kernel is > hinted that access will be sequential, it should stop caching when there is > little cache space available, instead of throwing away old blocks, or be > much more hesitant to throw away old blocks. Consider that in almost all > cases where access is sequential, as reading continues, the chances of the > read being aborted increase: ie, users downloading files, directory tree > traversal, etc. Since the likelihood of the first byte reading the first > byte is very high, and the next one less high, and the next less yet, etc, > it seems to make sense to tune the caching algorithm to accomodate this. This is much harder to implement. Specifically, the sequential nature is hueristically detected, and it is this hueristic, not the madvise, which is at issue. If this hueristic did *not* get triggered, then you would lose your read-ahead. Therefore it's not something that can be easily turned off. Second, the VM and buffer cache are unified in FreeBSD. THis means that you can not "reserve" buffer cache entries that are then not cached in VM objects, in order to cause the entries to turn over. Even if you were able to do this, through some serious kludge, you would not be able to differentiate the things that needed to be thrown out to make room for the transient pages, which leaves you in the sme boat you were in before. > While discussing disks, I have a minor complaint: at least on IDE systems, > when doing something like an untar, the entire system is painfully > unresponsive, even though CPU load is low. I presume this is because when an > executable is run, it needs to sit and wait for the disk. Wouldn't it make > sense to give very high disk priority to executables? Isn't that worth the > extra seeks? Actually, there reason for this is that the data portion of tagged writes can not disconnect from the drive in ATA, and therefore the tagged command queueing on reads does not help you for writes. The ATA protocol is broken. If you switched to a SCSI disk, you would see this problem go away. This was recently discussed in detail in the context of a discussion with a MAXTOR drive engineer in a thread on the FreeBSD-FS mailing list. In the limit, though, all disk requests come about as a result of executables making demands. Another serious issue that can cause this depends on it being a particular case, specifically, the untarring of the "ports" tree. If this is what you are talking about, the prblem has to do with bredth-first vs. depth-first storage, and therefore caching, on packing vs. unpacking of the tar archive. One approach to making this less of a problem is to reorder the archive index (and it's contents, to match, if you are using serial media, like tape, to access the archive). This has been discussed before in great detail, only the peple who care about it taking a long time have (so far) been unwilling to write the "archive optmizer". 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 22:35:32 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 884F837B405 for ; Sun, 26 Jan 2003 22:35:30 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id C10E343F1E for ; Sun, 26 Jan 2003 22:35:29 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0003.cvx22-bradley.dialup.earthlink.net ([209.179.198.3] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18d2rf-0007df-00; Sun, 26 Jan 2003 22:35:28 -0800 Message-ID: <3E34D152.56ADDE51@mindspring.com> Date: Sun, 26 Jan 2003 22:27:30 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Brian T. Schellenberger" Cc: Sean Hamilton , hackers@freebsd.org Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <3E34A6BB.2090601@acm.org> <001801c2c5c0$5666de10$16e306cf@slugabed.org> <200301270019.44066.bschellenberger@nc.rr.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4098b23bf375b56ba146d0c3ee70ddd0e2601a10902912494350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Brian T. Schellenberger" wrote: > 2. For sequential access, you should stop caching before you throw away > your own blocks. If it's sequential it is, it seems to me, always a > lose to throw away your *own* processes older bllocks on thee same > file. You can not have a block in a vm object which is not a cached copy of a data block in the backing store object. You can not access data blocks from the disk, except in the context of a backing store object. FreeBSD has a unified VM and buffer cache. FreeBSD does not copy data off the disk into a buffer, and then decide whether to copy the buffer/reference it from a cache: if it is in a buffer, it's in the cache, period. You are talking about avoiding a copy that doesn't happen. The real issue is whether, after notification of completion of the transfer (via wakeup of the blocked "read" system call), the VM buffer in question is relinked to the other end of the LRU list, artificially "aging" it. The answer to this is that you can't do it, because you do not know that the file system access itself is sequential for all people who have the file open at the time you make the request, and you can't know to which buffers you are actually referring, because this is handled via demand paging, "under the covers", and therefore not visible to the "read" call at that level. > These algorithmic changes seem to me to be more likely to be optimal > most of the time. No. All openers get different struct file's, but they get the same vp, which means they get the same vm_object_t. > A random approach *does* reduce the penalty for worst-case scenarios but > at the cost of reducing the benefit of both "normal" and "best-case" > scenarios, it seems to me, even more. I believe the "random page replacement" alrgotihm that was being referred to here is actually only supposed to be triggered if the file access has been advised tbe sequential; even so, there is no win in doing this, as you say (see other posting). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 22:49:31 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A1E8937B405 for ; Sun, 26 Jan 2003 22:49:28 -0800 (PST) Received: from HAL9000.homeunix.com (12-233-57-224.client.attbi.com [12.233.57.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id B5C9243F5B for ; Sun, 26 Jan 2003 22:49:27 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.6/8.12.5) with ESMTP id h0R6nO1Q001223; Sun, 26 Jan 2003 22:49:24 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.6/8.12.5/Submit) id h0R6nKww001222; Sun, 26 Jan 2003 22:49:20 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Date: Sun, 26 Jan 2003 22:49:20 -0800 From: David Schultz To: Tim Kientzle Cc: Terry Lambert , Matthew Dillon , Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry Message-ID: <20030127064920.GB1018@HAL9000.homeunix.com> Mail-Followup-To: Tim Kientzle , Terry Lambert , Matthew Dillon , Sean Hamilton , hackers@FreeBSD.ORG References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301261931.h0QJVCp8052101@apollo.backplane.com> <3E348B51.6F4D6096@mindspring.com> <200301270142.h0R1guR3070182@apollo.backplane.com> <3E3494CC.5895492D@mindspring.com> <3E34A6BB.2090601@acm.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3E34A6BB.2090601@acm.org> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thus spake Tim Kientzle : > Sean Hamilton proposes: > > >Wouldn't it seem logical to have [randomized disk cache expiration] in > >place at all times? > > Terry Lambert responds: > > >>:I really dislike the idea of random expiration; I don't understand > >>:the point, unless you are trying to get better numbers on some > > >>:benchmark. > > Matt Dillon concedes: > > >>... it's only useful when you are cycling through a [large] data set ... > > Cycling through large data sets is not really that uncommon. > I do something like the following pretty regularly: > find /usr/src -type f | xargs grep function_name > > Even scanning through a large dataset once can really hurt > competing applications on the same machine by flushing > their data from the cache for no gain. I think this > is where randomized expiration might really win, by reducing the > penalty for disk-cache-friendly applications who are competing > with disk-cache-unfriendly applications. > > There's an extensive literature on randomized algorithms. > Although I'm certainly no expert, I understand that such > algorithms work very well in exactly this sort of application, > since they "usually" avoid worst-case behavior under a broad > variety of inputs. The current cache is, in essence, > tuned specifically to work badly on a system where applications > are scanning through large amounts of data. No matter what > deterministic caching algorithm you use, you're choosing > to behave badly under some situation. Yes, if you randomly vary the behavior of the algorithm, you can guarantee that on average, performance will never be too bad for any particular input, but it will never be very good, on average, either. You can't mathematically prove everything about the memory access patterns of real-world programs, but LRU seems to do pretty well in a variety of situations. It does, however, have its worst cases. A random replacement algorithm is very unlikely to do *as* badly as LRU in LRU's worst case; its performance is consistent and relatively poor. Keep in mind that there's a bias in most real-world programs that favors LRU more than randomness. So should FreeBSD make it possible to ask for random replacement? Probably, since it would be helpful for those times when you *know* that LRU isn't going to do the right thing. (In the sequential read special case the OP mentions, random replacement is better than LRU, but still worse than a deterministic algorithm that just caches the prefix of the file that will fit in memory. So in this situation we could do even better in theory.) Should randomness be part of the default replacement algorithm, as the OP suggests? Probably not, since that would be pessimizing performance in the common case for the sake of improving it in an uncommon case. Should the system be able to detect cases where the default replacement algorithm is failing and dynamically modify its behavior? I think that would be really cool, if it is possible... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sun Jan 26 22:57: 8 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 18CC837B401 for ; Sun, 26 Jan 2003 22:57:06 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7D7C843F3F for ; Sun, 26 Jan 2003 22:57:05 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0R6v20i071775; Sun, 26 Jan 2003 22:57:02 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0R6v2qH071774; Sun, 26 Jan 2003 22:57:02 -0800 (PST) Date: Sun, 26 Jan 2003 22:57:02 -0800 (PST) From: Matthew Dillon Message-Id: <200301270657.h0R6v2qH071774@apollo.backplane.com> To: Terry Lambert Cc: kientzle@acm.org, Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301261931.h0QJVCp8052101@apollo.backplane.com> <3E348B51.6F4D6096@mindspring.com> <200301270142.h0R1guR3070182@apollo.backplane.com> <3E3494CC.5895492D@mindspring.com> <3E34A6BB.2090601@acm.org> <3E34CA7F.D0E0A260@mindspring.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Mmmmm. Basically what it comes down to is that without foreknowledge of the data locations being accessed, it is not possible for any cache algorithm to adapt to all the myrid ways data might be accessed. If you focus the cache on one methodology it will probably perform terribly when presented with some other methodology. What this means is that for the cases where you *KNOW* how a program intends to access a dataset larger then main memory, it is better to have the program explicitly cache/not-cache the data under program control rather then trying to force the system cache to adapt. I'll also be nice and decode some of Terry's Jargon for the rest of the readers. :will result in significant failure of random page replacement to :result in cache hits; likewise, going to 85% overage will practically :guarantee an almost 100% failure rate, as cyclical access with random :replacement is statistically more likely, in aggregate, to replace :the pages which are there longer (the probability is iterative and :additive: it's effectively a permutation). What Terry is saying is that if you have a dataset that is 2x the size of your cache, the cache hit rate on that data with random page replacement is NOT going to be 50%. This is because with random page replacement the likelihood of a piece of data being found in the cache depends on how long the data has been sitting in the cache. The longer the data has been sitting in the cache, the less likely you will find it when you need it (because it is more likely to have been replaced by the random replacement algorithm over time). So, again, the best caching methodology to use in the case where you *know* how much data you will be accessing and how you will access it is to build the caching directly into your program and not depend on system caching at all (for datasets which are far larger then main memory). This is true of all systems, not just BSD. This is one reason why databases do their own caching (another is so databases know when an access will require I/O for scheduling reasons, but that's a different story). The FreeBSD VM caching system does prevent one process from exhausting another process's cached data due to a sequential access, but the FreeBSD VM cache does not try to outsmart sequential data accesses to datasets which are larger then available cache space because it's an insanely difficult (impossible) problem to solve properly without foreknowledge of what data elements will be accessed when. This isn't to say that we can't improve on what we have now. I certainly think we can. But there is no magic bullet that will deal with every situation. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Jan 27 6: 5:24 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2429D37B401 for ; Mon, 27 Jan 2003 06:05:22 -0800 (PST) Received: from ms-smtp-02.southeast.rr.com (ms-smtp-02.southeast.rr.com [24.93.67.83]) by mx1.FreeBSD.org (Postfix) with ESMTP id 13A9A43F13 for ; Mon, 27 Jan 2003 06:05:14 -0800 (PST) (envelope-from bts@fake.com) Received: from mail3.nc.rr.com (fe3 [24.93.67.50]) by ms-smtp-02.southeast.rr.com (8.12.5/8.12.2) with ESMTP id h0RE44oB025537; Mon, 27 Jan 2003 09:04:07 -0500 (EST) Received: from this.is.fake.com ([24.74.172.220]) by mail3.nc.rr.com with Microsoft SMTPSVC(5.5.1877.757.75); Mon, 27 Jan 2003 09:03:37 -0500 Received: by this.is.fake.com (Postfix, from userid 111) id 206AABBC8; Mon, 27 Jan 2003 09:04:44 -0500 (EST) Content-Type: text/plain; charset="iso-8859-1" From: "Brian T. Schellenberger" To: kientzle@acm.org Subject: Re: Random disk cache expiry Date: Mon, 27 Jan 2003 09:04:43 -0500 User-Agent: KMail/1.4.2 Cc: Sean Hamilton , hackers@freebsd.org References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301270019.44066.bschellenberger@nc.rr.com> <3E34C734.8010801@acm.org> In-Reply-To: <3E34C734.8010801@acm.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200301270904.43899.bschellenberger@nc.rr.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG The suggestion here basically boils down to this: if the system could act on hints that somebody will be doing sequential access, then it should be more timid about caching for that file access. That is to say, it should allow that file to "use up" a smaller number of blocks from the cache (yes, the VM) at a time, and it should favor, if anything, a LIFO scheme instead of the usual FIFO (LRU) scheme. (That is to say, for the special case of *sequential* access, LRU == FIFO, and yet LIFO is probably more optimal for this case, at least if the file will be re-read later.) Caching will do more good on files that that will be randomly accessed; an intermediate amount of good on files sequentially accessed but rewound and/or accessed over and over, and if the file system could somehow know (or be hinted) that the file is being sequentially accessed and is unlikely to be accessed again for a good long while it would clearly be better off not caching it at all. Of course the trick here is waving my hands and saying "assume that you know how the file will be accessed in the future." You ought to pillory me for *that* bit. Even with hinting there are problems with this whole idea. Still with some hinting the algorithm could probably be a little more clever. (Actually, Terry Lambert *did* pillory me for that bit, just a bit, when he pointed out the impossibility of knowing whether the file is being used in the same way by other processes.) And . . . also to Terry, yes, I know that my proposal about over-simplifies, but the point is that for sequential access you want to go "gentle" on making the cache of other process' and earlier reads go away. On Monday 27 January 2003 12:44 am, Tim Kientzle wrote: | Brian T. Schellenberger wrote: | > This to me is imminently sensible. | > In fact there seem like two rules that have come up in this | > discussion: | > | > 1. For sequential access, you should be very hesitant to throw away | > *another* processes blocks, at least once you have used more than, | > say, 25% of the cache or potential cache. | > | > 2. For sequential access, you should stop caching before you throw | > away your own blocks. If it's sequential it is, it seems to me, | > always a lose to throw away your *own* processes older bllocks on | > thee same file. | | The question isn't "should I throw away blocks or not?". | Obviously, the ideal is to keep everything in the cache, but | that's not possible. | | The question is "what blocks should be discarded?" You've | ruled out quite a few possibilities here. What blocks _should_ | be discarded? | | Tim Kientzle To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Jan 27 9:38:15 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DF28937B401 for ; Mon, 27 Jan 2003 09:38:14 -0800 (PST) Received: from PIKES.panasas.com (gw2.panasas.com [65.194.124.178]) by mx1.FreeBSD.org (Postfix) with ESMTP id 220C643E4A for ; Mon, 27 Jan 2003 09:38:14 -0800 (PST) (envelope-from rgrover@panasas.com) Received: from tiltill ([172.17.132.191]) by PIKES.panasas.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2653.13) id XA314S1Z; Mon, 27 Jan 2003 12:38:07 -0500 Content-Type: text/plain; charset="us-ascii" From: Rohit Grover Organization: Panasas Inc. To: freebsd-hackers@freebsd.org Subject: having trouble understanding something about ccd Date: Mon, 27 Jan 2003 09:38:06 -0800 X-Mailer: KMail [version 1.4] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200301270938.06988.rgrover@panasas.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello, This question is for the current maintainer(s) of ccd. The email listed in ccd.c is invalid. In ccdioctl, for the command CCDIOCSET, ccdgetdisklabel() is called after ccdinit(). In ccdgetdisklabel(), the raw partition's size is initialized (to a value computed by ccdinit()) before calling readdisklabel() which overwrites it with stuff read off the media. This results in the loss of the computed value for size. Could you please shed some light on the merits of this sequence of actions. rohit. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Jan 27 9:50: 0 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E5B7C37B401 for ; Mon, 27 Jan 2003 09:49:58 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id DBE5243EB2 for ; Mon, 27 Jan 2003 09:49:57 -0800 (PST) (envelope-from phk@freebsd.org) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h0RHnuQl024171; Mon, 27 Jan 2003 18:49:56 +0100 (CET) (envelope-from phk@freebsd.org) To: Rohit Grover Cc: freebsd-hackers@freebsd.org Subject: Re: having trouble understanding something about ccd From: phk@freebsd.org In-Reply-To: Your message of "Mon, 27 Jan 2003 09:38:06 PST." <200301270938.06988.rgrover@panasas.com> Date: Mon, 27 Jan 2003 18:49:56 +0100 Message-ID: <24170.1043689796@critter.freebsd.dk> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <200301270938.06988.rgrover@panasas.com>, Rohit Grover writes: >Hello, > >This question is for the current maintainer(s) of ccd. The email >listed in ccd.c is invalid. > >In ccdioctl, for the command CCDIOCSET, ccdgetdisklabel() is called >after ccdinit(). In ccdgetdisklabel(), the raw partition's size is >initialized (to a value computed by ccdinit()) before calling >readdisklabel() which overwrites it with stuff read off the >media. This results in the loss of the computed value for size. Could >you please shed some light on the merits of this sequence of actions. I think this code is largely overtaken by events, as recently as a few days ago I ripped out the entire disklabel-fiddling code from CCD and set it to use the systems default code instead. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Jan 27 19:41:59 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B02F537B401 for ; Mon, 27 Jan 2003 19:41:58 -0800 (PST) Received: from smtp-relay.omnis.com (smtp-relay.omnis.com [216.239.128.27]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1779143F85 for ; Mon, 27 Jan 2003 19:41:58 -0800 (PST) (envelope-from shane@clatterbone.com) Received: from clatterbone.com (5-213-237-24-cable.anchorageak.net [24.237.213.5]) by smtp-relay.omnis.com (Postfix) with ESMTP id 15D9543800 for ; Mon, 27 Jan 2003 19:41:57 -0800 (PST) Date: Mon, 27 Jan 2003 18:41:56 -0900 Mime-Version: 1.0 (Apple Message framework v551) Content-Type: text/plain; charset=US-ASCII; format=flowed Subject: subscribe From: Shane Scot To: freebsd-hackers@FreeBSD.org Content-Transfer-Encoding: 7bit Message-Id: <7305FEE8-3272-11D7-9269-0050E47AA862@clatterbone.com> X-Mailer: Apple Mail (2.551) Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Please subscribe me to the FreeBSD mailing list. thank you, Shane To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Mon Jan 27 20: 5:46 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0E26537B401; Mon, 27 Jan 2003 20:05:46 -0800 (PST) Received: from ns.utk.ru (P91-36-5.utk.ru [81.91.36.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5551043F9B; Mon, 27 Jan 2003 20:05:44 -0800 (PST) (envelope-from none@none.ru) Received: from Sender (vc105.utk.ru [81.91.32.105]) by ns.utk.ru (8.9.3/8.9.3) with SMTP id JAA87295; Tue, 28 Jan 2003 09:05:34 +0500 (ESK) Message-Id: <200301280405.JAA87295@ns.utk.ru> From: Àíäðåé To: "" <> Subject: Ïðåäëîæåíèå Reply-To: none@none.ru X-Mailer: Microsoft Outlook Express 4.72.3110.5 X-Priority: 3 X-MSMail-Priority: Normal Mime-Version: 1.0 Content-Type: text/plain; charset="Windows-1251" Date: Tue, 28 Jan 2003 09:06:14 +0500 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Ïðåäëîãàÿ Âàì èçãîòîâëåíèå ïëàñòèêîâûõ êàðò. Áûñòðî è íåäîðîãî. ò. 8-902-8868204 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Jan 28 7:20: 5 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ED27B37B401 for ; Tue, 28 Jan 2003 07:20:03 -0800 (PST) Received: from web41310.mail.yahoo.com (web41310.mail.yahoo.com [66.218.93.59]) by mx1.FreeBSD.org (Postfix) with SMTP id 8F2D943F75 for ; Tue, 28 Jan 2003 07:20:03 -0800 (PST) (envelope-from ron_chen_123@yahoo.com) Message-ID: <20030128152003.22190.qmail@web41310.mail.yahoo.com> Received: from [199.246.40.54] by web41310.mail.yahoo.com via HTTP; Tue, 28 Jan 2003 07:20:03 PST Date: Tue, 28 Jan 2003 07:20:03 -0800 (PST) From: Ron Chen Subject: Fwd: [GE dev] FreeBSD port To: freebsd-hackers@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG FYI, newest FreeBSD port of GridEngine. For more details about GridEngine: http://gridengine.sunsource.net/ -Ron --- Brooks Davis wrote: > I've updated my port of SGE to FreeBSD (based on Ron > Chen's patch). > It's available online at: > > http://people.freebsd.org/~brooks/sge/ > > You need to extract sge-freebsd-files.tar.gz in > gridengine/source and > use patch to apply sge-freebsd.diff there as well. > BUILD shows how I > build my copy. > > At this point job submission and deletion (both > batch and parallel) > works and jobs run. Running qmake doesn't work, it > hangs. > > Please test it out. > > I'm intested in seeing FreeBSD become a fully > supported platform at some > time. I saw one message indicating that one issue > was the lack of a > maintainer was part of the problem. I'm willing to > take on the role if > needed. > > -- Brooks > > --------------------------------------------------------------------- > To unsubscribe, e-mail: > dev-unsubscribe@gridengine.sunsource.net > For additional commands, e-mail: > dev-help@gridengine.sunsource.net > __________________________________________________ Do you Yahoo!? Yahoo! Mail Plus - Powerful. Affordable. Sign up now. http://mailplus.yahoo.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Jan 28 12:43:58 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A10FE37B401 for ; Tue, 28 Jan 2003 12:43:57 -0800 (PST) Received: from host.dedicatedsrvr.net (host.dedicatedsrvr.net [66.96.230.225]) by mx1.FreeBSD.org (Postfix) with ESMTP id 21CF243F43 for ; Tue, 28 Jan 2003 12:43:57 -0800 (PST) (envelope-from cpanel@host.dedicatedsrvr.net) Received: from cpanel by host.dedicatedsrvr.net with local (Exim 3.36 #1) id 18dcaE-0007Yj-00 for freebsd-hackers@freebsd.org; Tue, 28 Jan 2003 12:43:50 -0800 Received: from 212.138.47.27 ( [212.138.47.27]) as user samy@kerneled.com@localhost by www.kerneled.com with HTTP; Tue, 28 Jan 2003 12:43:50 -0800 Message-ID: <1043786630.3e36eb862a3a8@www.kerneled.com> Date: Tue, 28 Jan 2003 12:43:50 -0800 From: samy@kerneled.com To: freebsd-hackers@freebsd.org Subject: rfork behaviour MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: Internet Messaging Program (IMP) 3.1 X-Originating-IP: 212.138.47.27 X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - host.dedicatedsrvr.net X-AntiAbuse: Original Domain - freebsd.org X-AntiAbuse: Originator/Caller UID/GID - [32001 502] / [32001 502] X-AntiAbuse: Sender Address Domain - host.dedicatedsrvr.net Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG The rfork() man page states that RFPROC should be set as the current implementation 'requires' this flag to always be set. Well from usage and kern_fork.c (/usr/src/sys/kern/kern_fork.c) and vm_glue.c (/usr/src/sys/vm/vm_glue.c) it seems that the current implementation does not 'require' this at all and it behaves as it should behave. Should I just forward a patch of some sort? or is there another reason for leaving that note in there... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Jan 28 23:28:53 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C7FD737B401 for ; Tue, 28 Jan 2003 23:28:52 -0800 (PST) Received: from istanbul.enderunix.org (istanbul.enderunix.org [212.65.128.82]) by mx1.FreeBSD.org (Postfix) with SMTP id 3139A43F43 for ; Tue, 28 Jan 2003 23:28:51 -0800 (PST) (envelope-from murat@enderunix.org) Received: (qmail 86318 invoked by uid 1001); 29 Jan 2003 07:27:14 -0000 Date: Wed, 29 Jan 2003 09:27:14 +0200 From: Murat Balaban To: freebsd-hackers@freebsd.org Subject: memcpy and multithreading Message-ID: <20030129092714.A86300@enderunix.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi, Are there any issues that i should now about memcpy and multithreading in FreeBSD? I've got a code which reads: memcpy(&(h.req), req, sizeof(struct request)); and I keep getting SIGSEGV there in FreeBSD but not in Linux. PS: Of course, I protect the write operation with a mutex. PS2: typedef struct hist_data { struct request req; int nthreads; int bwritten; struct thread_data wthread[MAXTHREADS]; } hist_data; - Murat Balaban To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Tue Jan 28 23:51:39 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 823E537B401 for ; Tue, 28 Jan 2003 23:51:38 -0800 (PST) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id EE9B843F3F for ; Tue, 28 Jan 2003 23:51:37 -0800 (PST) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.12.6/8.12.6) id h0T7pTNR081673; Wed, 29 Jan 2003 01:51:29 -0600 (CST) (envelope-from dan) Date: Wed, 29 Jan 2003 01:51:29 -0600 From: Dan Nelson To: Murat Balaban Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: memcpy and multithreading Message-ID: <20030129075129.GK17299@dan.emsphone.com> References: <20030129092714.A86300@enderunix.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030129092714.A86300@enderunix.org> X-OS: FreeBSD 5.0-CURRENT X-message-flag: Outlook Error User-Agent: Mutt/1.5.3i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In the last episode (Jan 29), Murat Balaban said: > Are there any issues that i should now about memcpy and > multithreading in FreeBSD? I've got a code which reads: > > memcpy(&(h.req), req, sizeof(struct request)); What are the values of h and req at the time of segfault? You might want to try running with MALLOC_OPTIONS=J, which makes sure you don't accidentally use free'd memory, or assume that malloced memory is zeroed. If you see pointers with values near 0xD0D0D0D0, you have found a bug in your code... > and I keep getting SIGSEGV there in FreeBSD but not in Linux. > > PS: Of course, I protect the write operation with a mutex. Mutexes will protect you from someone stepping on your memory while you're copying. They don't protect you from seg faults if you have the wrong pointers in the first place. -- Dan Nelson dnelson@allantgroup.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 0:40:13 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B516B37B401 for ; Wed, 29 Jan 2003 00:40:11 -0800 (PST) Received: from cache1.telkomsel.co.id (cache1.telkomsel.co.id [202.155.14.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id DBFF643F93 for ; Wed, 29 Jan 2003 00:40:07 -0800 (PST) (envelope-from arief@bna.telkomsel.co.id) Received: from gsi-wall1 (252.83.1.10.in-addr.arpa [10.1.83.252]) by cache1.telkomsel.co.id (8.12.6/8.12.6) with SMTP id h0T8dlIe074650; Wed, 29 Jan 2003 15:39:49 +0700 (JAVT) (envelope-from arief@bna.telkomsel.co.id) Received: from ([10.1.92.57]) by gsi-wall1; Wed, 29 Jan 2003 15:36:31 +0700 (WIT) Message-ID: <3E378EAE.9050209@bna.telkomsel.co.id> Date: Wed, 29 Jan 2003 15:19:58 +0700 From: arief_mulya Organization: damai itu indah User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.0.0) Gecko/20020623 Debian/1.0.0-0.woody.1 MIME-Version: 1.0 To: linux-kernel , tech-kern , tech , freebsd-hackers Subject: Thanks (was: Linux - *BSD diff) Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dear all, Well, I've been receiving quite lots of nice and *warm* and joyful email for the thread ;-) Some quite of enlightnment (including your email, Theo, it gives me lots of enlightment, you surely is a funny guy ;-) After also lots of google clickings, I think I get the big picture of it now. And I've also made my decision. For now, I'm going to stick w/ Linux Kernel. And see how much can I give in till I give up ;-) Thanks for all of the answers. Special thanks for Rik van Riel and Bill Studenmund for your kind insights and supports :-) There's a saying in Java-nese (not *that* Java) language: Mangan ora mangan kumpul. Which means, Have something to eat or not Have something to eat, the important thing is to stick together. Forever. And that's what I felt about Linux Developers... if you know what I mean :-) See you. Best Regards, arief_mulya -- To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 1:25:18 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AD7B837B401 for ; Wed, 29 Jan 2003 01:25:16 -0800 (PST) Received: from ns1.ovis.net (ns1.ovis.net [207.0.147.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3081443F75 for ; Wed, 29 Jan 2003 01:25:16 -0800 (PST) (envelope-from chromexa@ovis.net) Received: from ovis.net (s27.pm5.ovis.net [207.0.147.93]) by ns1.ovis.net (Postfix) with ESMTP id 43D833AEE; Wed, 29 Jan 2003 04:25:08 -0500 (EST) Message-ID: <3E379E7D.FF8287C7@ovis.net> Date: Wed, 29 Jan 2003 09:27:25 +0000 From: Steve Kudlak Reply-To: chromexa@ovis.net X-Mailer: Mozilla 4.5 [en]C-CCK-MCD ezn/58/n (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: arief_mulya , "'hackers@freebsd.org'" Subject: Re: Thanks (was: Linux - *BSD diff) References: <3E378EAE.9050209@bna.telkomsel.co.id> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG arief_mulya wrote: > Dear all, > > Well, > I've been receiving quite lots of nice and *warm* and joyful > email for the thread ;-) > > Some quite of enlightnment (including your email, Theo, it > gives me lots of enlightment, you surely is a funny guy ;-) > > After also lots of google clickings, I think I get the big > picture of it now. And I've also made my decision. For now, > I'm going to stick w/ Linux Kernel. And see how much can I > give in till I give up ;-) > > Thanks for all of the answers. > Special thanks for Rik van Riel and Bill Studenmund for your > kind insights and supports :-) > > There's a saying in Java-nese (not *that* Java) language: > > Mangan ora mangan kumpul. > > Which means, Have something to eat or not Have something to > eat, the important thing is to stick together. Forever. > > And that's what I felt about Linux Developers... if you know > what I mean :-) > > See you. > > Best Regards, > arief_mulya > -- > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message Steve emerges from lurking and declaims. Well it seems that "hackers" is the real alive BSD mailing list. If you ask a question it generally gets reasonably answered with a only an occasional snarl. I know it isn't a *nix internals list, but that information is helpful. It is also helpful when someone explains what they have actually done with something down to the details. The types I deal with are printer, publishers, artists and animators and so I have to have something of the "what will it do for me", when I suggest it. So knowing people have set up Ray Tracers and the like is helpful. Thanks and Have Fun, Sends Steve To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 2:24:56 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 16C7937B401; Wed, 29 Jan 2003 02:24:55 -0800 (PST) Received: from angelo.kcl.ac.uk (angelo.kcl.ac.uk [137.73.66.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id E456F43F3F; Wed, 29 Jan 2003 02:24:49 -0800 (PST) (envelope-from dev.dhas@kcl.ac.uk) Received: from ctr-Dev.kcl.ac.uk (EE077.eee.kcl.ac.uk [137.73.10.124]) by angelo.kcl.ac.uk with ESMTP id h0TAFue2027967; Wed, 29 Jan 2003 10:15:59 GMT Message-Id: <5.2.0.9.0.20030129102140.00ae60a0@pop2.kcl.ac.uk> X-Sender: kkqd2740@pop2.kcl.ac.uk X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Date: Wed, 29 Jan 2003 10:23:22 +0000 To: freebsd-hackers@freebsd.org From: Audsin Subject: Changing the Maximum Segment Size (MSS) of Kame MIP6 Free BSD4.4 Cc: freebsd-net@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-MailScanner: Found to be clean Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Respected Sir/ Madam I am Dev, doing my research in Centre for Telecommunications Research, King's college London. My research project involves evaluating the performance of MIP6 TCP in the presence of fragmentation and without fragmentation. I am using Kame MIP6 for Free BSD 4.4 and have configured gif0 interface for ipv6ip tunnel. I wish to change the Maximum segment size of the TCP. Can you please help me , where i should change the MSS of the TCP. Can you tell me where the default size of the MSS mentioned? Eagerly waiting for the reply Best Regards Dev A. Dev pramil Research Student Center for Telecommunications Research University of London, Kings College Strand WC2R 2LS To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 3: 1:30 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7D9E837B401 for ; Wed, 29 Jan 2003 03:01:29 -0800 (PST) Received: from 12-234-22-23.client.attbi.com (12-234-22-23.client.attbi.com [12.234.22.23]) by mx1.FreeBSD.org (Postfix) with ESMTP id 37DD443F43 for ; Wed, 29 Jan 2003 03:01:18 -0800 (PST) (envelope-from DougB@FreeBSD.org) Received: from slave.gorean.org (n3i97n92b8ixb9f0@slave.gorean.org [10.0.0.1]) by 12-234-22-23.client.attbi.com (8.12.6/8.12.6) with ESMTP id h0TB1DpP002835; Wed, 29 Jan 2003 03:01:18 -0800 (PST) (envelope-from DougB@FreeBSD.org) Date: Wed, 29 Jan 2003 03:01:13 -0800 (PST) From: Doug Barton To: Audsin Cc: freebsd-hackers@FreeBSD.org Subject: Re: Changing the Maximum Segment Size (MSS) of Kame MIP6 Free BSD4.4 In-Reply-To: <5.2.0.9.0.20030129102140.00ae60a0@pop2.kcl.ac.uk> Message-ID: <20030129030042.J1559@12-234-22-23.pyvrag.nggov.pbz> References: <5.2.0.9.0.20030129102140.00ae60a0@pop2.kcl.ac.uk> Organization: http://www.FreeBSD.org/ X-message-flag: Outlook -- Not just for spreading viruses anymore! MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I think that you just burned all possible bridges with your rampant cross posting. At least I hope so. -- If it's moving, encrypt it. If it's not moving, encrypt it till it moves, then encrypt it some more. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 10:18:42 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CDDFE37B401 for ; Wed, 29 Jan 2003 10:18:41 -0800 (PST) Received: from sabre.velocet.net (sabre.velocet.net [216.138.209.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id 50F0043F3F for ; Wed, 29 Jan 2003 10:18:36 -0800 (PST) (envelope-from dgilbert@velocet.ca) Received: from trooper.velocet.ca (trooper.velocet.net [216.138.242.2]) by sabre.velocet.net (Postfix) with ESMTP id D0F51137F49 for ; Wed, 29 Jan 2003 13:18:31 -0500 (EST) Received: by trooper.velocet.ca (Postfix, from userid 66) id A5F5274629; Wed, 29 Jan 2003 13:18:31 -0500 (EST) Received: by canoe.velocet.net (Postfix, from userid 101) id 1C846567628; Wed, 29 Jan 2003 13:18:29 -0500 (EST) From: David Gilbert MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15928.6900.948346.474717@canoe.velocet.net> Date: Wed, 29 Jan 2003 13:18:28 -0500 To: freebsd-hackers@freebsd.org Subject: Network block device. X-Mailer: VM 7.07 under 21.1 (patch 14) "Cuyahoga Valley" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG While I'm 100% aware of the pitfalls of such a setup, I find myself implementing linux in a cluster because it can export 5G-ish of a disk on each node to one machine that generates a gigantic filesystem. This is done with linux's network-block-device (NBD). I'd like to know if someone has generated a similar FreeBSD facility. Dave. -- ============================================================================ |David Gilbert, Velocet Communications. | Two things can only be | |Mail: dgilbert@velocet.net | equal if and only if they | |http://daveg.ca | are precisely opposite. | =========================================================GLO================ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 10:24: 7 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D59AF37B401 for ; Wed, 29 Jan 2003 10:24:05 -0800 (PST) Received: from silmaril.rt.ru (silmaril.rt.ru [217.107.221.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id 75B2343F3F for ; Wed, 29 Jan 2003 10:24:04 -0800 (PST) (envelope-from dima@rt.ru) Received: from rt.ru (localhost [127.0.0.1]) by silmaril.rt.ru (8.12.6/8.12.6) with ESMTP id h0TIINik032251 for ; Wed, 29 Jan 2003 21:18:24 +0300 (MSK) (envelope-from dima@rt.ru) Message-ID: <3E381AEF.C5E3CE4D@rt.ru> Date: Wed, 29 Jan 2003 21:18:23 +0300 From: "Dmitry S. Rzhavin" Reply-To: dima@rt.ru Organization: RTComm.RU X-Mailer: Mozilla 4.8 [en] (X11; U; Linux 2.4.2 i386) X-Accept-Language: ru, en MIME-Version: 1.0 To: hackers@freebsd.org Subject: problems with Logitech iFeel MouseMan mouce Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello! I have problems with an usb Logitech iFeel MouseMan mouce. When I connect it, boot log looks like: === uhci0: port 0xe400-0xe41f irq 9 at device 7.2 on pci0 usb0: on uhci0 usb0: USB revision 1.0 uhub0: Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 2 ports with 2 removable, self powered < long delay > ums0: Logitech product 0xc032, rev 1.00/1.01, addr 2, iclass 3/1 < long delay > device_probe_and_attach: ums0 attach returned 6 < long delay > ugen0: Logitech product 0xc032, rev 1.00/1.01, addr 2 < long delay > ugen0: setting configuration index 0 failed < long delay > device_probe_and_attach: ugen0 attach returned 6 uhub1: Texas Instruments TUSB2046 hub, class 9/0, rev 1.10/1.25, addr 3 uhub1: 4 ports with 4 removable, self powered === and FreeBSD can not use it: # usbdevs -dv Controller /dev/usb0: addr 1: self powered, config 1, UHCI root hub(0x0000), Intel(0x0000), rev 1.00 uhub0 port 1 addr 2: low speed, power 500 mA, unconfigured, product 0xc032(0xc032), Logitech(0x046d), rev 1.01 port 2 addr 3: self powered, config 1, TUSB2046 hub(0x2046), Texas Instruments(0x0451), rev 1.25 How can I use my mouse? # uname -r 4.7-STABLE # strings -n 3 /kernel | sed -n 's/^___//p' | g -i '\(usb\|ums\)' device uhci # UHCI PCI->USB interface device ohci # OHCI PCI->USB interface device usb # USB Bus (required) device ums # Mouse To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 11:40:41 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4DF7B37B401 for ; Wed, 29 Jan 2003 11:40:40 -0800 (PST) Received: from gatekeeper.oremut01.us.wh.verio.net (gatekeeper.oremut01.us.wh.verio.net [198.65.168.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9CF0F43E4A for ; Wed, 29 Jan 2003 11:40:39 -0800 (PST) (envelope-from gritton@iserver.com) Received: from guppy.dmz.orem.verio.net (guppy.dmz.orem.verio.net [10.1.1.55]) by gatekeeper.oremut01.us.wh.verio.net (Postfix) with ESMTP id 28AC53BF19A for ; Wed, 29 Jan 2003 12:40:34 -0700 (MST) Received: from guppy.dmz.orem.verio.net by guppy.dmz.orem.verio.net; Wed, 29 Jan 2003 12:40:33 -0700 (MST) Received: (from gritton@localhost) by guppy.dmz.orem.verio.net (8.12.3/8.12.3/Submit) id h0TJeXm1022866; Wed, 29 Jan 2003 12:40:33 -0700 (MST) Date: Wed, 29 Jan 2003 12:40:33 -0700 (MST) Message-Id: <200301291940.h0TJeXm1022866@guppy.dmz.orem.verio.net> X-Authentication-Warning: guppy.dmz.orem.verio.net: gritton set sender to gritton@iserver.com using -f From: James Gritton To: freebsd-hackers@FreeBSD.ORG Subject: What's the memory footprint of a set of processes? Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG How do I find how much memory (real and/or virtual) is being used by a set of processes, taking shared pages into account? I see per-process numbers I can use (vmspace_resident_count and vmspace_swap_count), and overall usage numbers exist, but I can't find a better way of measuring multiple processes than adding their individual totals together. This can lead to wild inaccuracies if a few processes share a lot of pages (Java comes to mind, as do database servers). Is there some kind of reference count I can access per page, or some not-too-expensive way to see what processes are using a page? I see some things in the VM code that look like recerence counts (such as act_count in struct vm_page), but they don't seem to really be such, or at least they don't count what I'm expecting them to. This is on 4.7. I haven't really looked at 5, so I don't know how different it is. - James Gritton gritton@iserver.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 11:45:32 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6E87037B405 for ; Wed, 29 Jan 2003 11:45:31 -0800 (PST) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id A5E8543F43 for ; Wed, 29 Jan 2003 11:45:30 -0800 (PST) (envelope-from julian@elischer.org) Received: from InterJet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc52.attbi.com (rwcrmhc52) with ESMTP id <2003012919453005200dtskae>; Wed, 29 Jan 2003 19:45:30 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id LAA26201; Wed, 29 Jan 2003 11:45:29 -0800 (PST) Date: Wed, 29 Jan 2003 11:45:28 -0800 (PST) From: Julian Elischer To: James Gritton Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? In-Reply-To: <200301291940.h0TJeXm1022866@guppy.dmz.orem.verio.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG check out /proc//map for a really detailed map of the process. On Wed, 29 Jan 2003, James Gritton wrote: > How do I find how much memory (real and/or virtual) is being used by a set > of processes, taking shared pages into account? I see per-process numbers I > can use (vmspace_resident_count and vmspace_swap_count), and overall usage > numbers exist, but I can't find a better way of measuring multiple processes > than adding their individual totals together. This can lead to wild > inaccuracies if a few processes share a lot of pages (Java comes to mind, as > do database servers). Is there some kind of reference count I can access > per page, or some not-too-expensive way to see what processes are using a > page? I see some things in the VM code that look like recerence counts > (such as act_count in struct vm_page), but they don't seem to really be > such, or at least they don't count what I'm expecting them to. > > This is on 4.7. I haven't really looked at 5, so I don't know how different > it is. > > - James Gritton > gritton@iserver.com > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 12:28:18 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C1FFB37B401 for ; Wed, 29 Jan 2003 12:28:16 -0800 (PST) Received: from gatekeeper.oremut01.us.wh.verio.net (gatekeeper.oremut01.us.wh.verio.net [198.65.168.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id AFC7543F3F for ; Wed, 29 Jan 2003 12:28:15 -0800 (PST) (envelope-from gritton@iserver.com) Received: from guppy.dmz.orem.verio.net (guppy.dmz.orem.verio.net [10.1.1.55]) by gatekeeper.oremut01.us.wh.verio.net (Postfix) with ESMTP id 24B873BF121 for ; Wed, 29 Jan 2003 13:28:10 -0700 (MST) Received: from guppy.dmz.orem.verio.net by guppy.dmz.orem.verio.net; Wed, 29 Jan 2003 13:28:09 -0700 (MST) Received: (from gritton@localhost) by guppy.dmz.orem.verio.net (8.12.3/8.12.3/Submit) id h0TKS9Vu023133; Wed, 29 Jan 2003 13:28:09 -0700 (MST) To: freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? References: From: James Gritton Date: 29 Jan 2003 13:28:09 -0700 In-Reply-To: Julian Elischer's message of "Wed, 29 Jan 2003 11:45:28 -0800 (PST)" Message-ID: Lines: 46 X-Mailer: Gnus v5.7/Emacs 20.7 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer writes: > check out /proc//map for a really detailed map of the process. That looks good for a single process, suffers from the problem I'm having. For example, if I run a program that simply mallocs a chumk of memory and reads through it (to map it all in), it's pretty easy to identify the affected object: guppy% cat /proc/23072/map 0x8048000 0x8049000 1 0 0xcdd610c0 r-x 1 0 0x0 COW NC vnode 0x8049000 0x804a000 1 0 0xce619060 rw- 2 0 0x2180 NCOW NNC default 0x804a000 0xc04b000 16385 0 0xce619060 rwx 2 0 0x2180 NCOW NNC default ... But after sleeping for a bit, the program then forks, and I have two processes sharing the same memory. I hope to see something that reflects this, but: guppy% cat /proc/23072/map 0x8048000 0x8049000 1 0 0xcdd610c0 r-x 2 0 0x0 COW NC vnode 0x8049000 0x804a000 1 0 0xcdc84360 rw- 2 0 0x180 COW NC default 0x804a000 0xc04b000 16385 0 0xce619060 rwx 2 0 0x180 COW NC default ... guppy% cat /proc/23074/map 0x8048000 0x8049000 1 0 0xcdd610c0 r-x 2 0 0x0 COW NC vnode 0x8049000 0x804a000 1 0 0xcdc84360 rw- 2 0 0x180 COW NC default 0x804a000 0xc04b000 16385 0 0xce619060 rwx 2 0 0x180 COW NC default ... The object's ref_count hasn't changed, which is what I meant about seeing reference counts in the kernel that were apparently not counting what I'm looking for. I did see a ref_count increase on the first object (presumably the text image), but nothing on the allocated memory. It seems the object level isn't fine enough, but the deeper I go into the VM code, the more confused I become. In this forked process example, what happens when I alter a few COW pages in the currently-shared object? Apparently a shadow object is created, but it claims to be the same size as the original object. True, but I know it's not actually using that many pages, since most of them are still validly shared. System usage numbers tell me this is true, but I can't find what in the process or object data structures reflect this fact. - Jamie To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 14: 5:57 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6F49937B401 for ; Wed, 29 Jan 2003 14:05:56 -0800 (PST) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by mx1.FreeBSD.org (Postfix) with ESMTP id 872E843F79 for ; Wed, 29 Jan 2003 14:05:55 -0800 (PST) (envelope-from winter@jurai.net) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by sasami.jurai.net (8.12.6/8.12.5) with ESMTP id h0TM5svA031188; Wed, 29 Jan 2003 17:05:54 -0500 (EST) (envelope-from winter@jurai.net) Date: Wed, 29 Jan 2003 17:05:54 -0500 (EST) From: "Matthew N. Dodd" To: David Gilbert Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. In-Reply-To: <15928.6900.948346.474717@canoe.velocet.net> Message-ID: <20030129170512.Y8642@sasami.jurai.net> References: <15928.6900.948346.474717@canoe.velocet.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 29 Jan 2003, David Gilbert wrote: > While I'm 100% aware of the pitfalls of such a setup, I find myself > implementing linux in a cluster because it can export 5G-ish of a disk > on each node to one machine that generates a gigantic filesystem. This > is done with linux's network-block-device (NBD). I'd like to know if > someone has generated a similar FreeBSD facility. You could use NFS and 'mdconfig/vnconfig'. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | For Great Justice! | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 14:12:55 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D6B3E37B401; Wed, 29 Jan 2003 14:12:53 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 739C543FC3; Wed, 29 Jan 2003 14:12:51 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0131.cvx22-bradley.dialup.earthlink.net ([209.179.198.131] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18e0Rs-0007ad-00; Wed, 29 Jan 2003 14:12:49 -0800 Message-ID: <3E385188.F3404260@mindspring.com> Date: Wed, 29 Jan 2003 14:11:20 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Audsin Cc: freebsd-hackers@freebsd.org, freebsd-net@freebsd.org Subject: Re: Changing the Maximum Segment Size (MSS) of Kame MIP6 FreeBSD4.4 References: <5.2.0.9.0.20030129102140.00ae60a0@pop2.kcl.ac.uk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a47e512cc7f6a22f6bd971795d613311832601a10902912494350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Audsin wrote: > I am Dev, doing my research in Centre for Telecommunications Research, > King's college London. My research project involves evaluating the > performance of MIP6 TCP in the presence of fragmentation and without > fragmentation. I am using Kame MIP6 for Free BSD 4.4 and have configured > gif0 interface for ipv6ip tunnel. I wish to change the Maximum segment size > of the TCP. Can you please help me , where i should change the MSS of the > TCP. Can you tell me where the default size of the MSS mentioned? man ifconfig /mtu It sounds like you are trying to do something you don't have an idea of how to do. In general, this isn't a bad thing for a student to do, but you should probably read the source code and the RFC's and understand them well enough that you understand the effect of an intermediate MTU on the MSS, and what the MSS is vs. your trying to throttle it, and how one effects the other. My hunch is that you are wanting to have an intermediate link that is a fragmentation bottleneck, given what you say you are studying. Don't expect a lot of help from these lists until after you are asking questions that indicate you've read the standards and the current code. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 14:17:43 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8CC5237B401 for ; Wed, 29 Jan 2003 14:17:42 -0800 (PST) Received: from sabre.velocet.net (sabre.velocet.net [216.138.209.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id B3C7643F3F for ; Wed, 29 Jan 2003 14:17:41 -0800 (PST) (envelope-from dgilbert@velocet.ca) Received: from trooper.velocet.ca (trooper.velocet.net [216.138.242.2]) by sabre.velocet.net (Postfix) with ESMTP id 947001382C2; Wed, 29 Jan 2003 17:17:40 -0500 (EST) Received: by trooper.velocet.ca (Postfix, from userid 66) id 4D8A17462B; Wed, 29 Jan 2003 17:17:40 -0500 (EST) Received: by canoe.velocet.net (Postfix, from userid 101) id 4A558567628; Wed, 29 Jan 2003 17:17:37 -0500 (EST) From: David Gilbert MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15928.21248.483298.203713@canoe.velocet.net> Date: Wed, 29 Jan 2003 17:17:36 -0500 To: "Matthew N. Dodd" Cc: David Gilbert , freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. In-Reply-To: <20030129170512.Y8642@sasami.jurai.net> References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> X-Mailer: VM 7.07 under 21.1 (patch 14) "Cuyahoga Valley" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> "Matthew" == Matthew N Dodd writes: Matthew> On Wed, 29 Jan 2003, David Gilbert wrote: >> While I'm 100% aware of the pitfalls of such a setup, I find myself >> implementing linux in a cluster because it can export 5G-ish of a >> disk on each node to one machine that generates a gigantic >> filesystem. This is done with linux's network-block-device (NBD). >> I'd like to know if someone has generated a similar FreeBSD >> facility. Matthew> You could use NFS and 'mdconfig/vnconfig'. but that would be no different than using the nfs directly. mdconfig won't aggregate several chunks of files ... and last I checked md wasn't entirely happy with nfs (some form of chicken-and-egg problem) Dave. -- ============================================================================ |David Gilbert, Velocet Communications. | Two things can only be | |Mail: dgilbert@velocet.net | equal if and only if they | |http://daveg.ca | are precisely opposite. | =========================================================GLO================ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 14:21:20 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4BB7A37B401 for ; Wed, 29 Jan 2003 14:21:19 -0800 (PST) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by mx1.FreeBSD.org (Postfix) with ESMTP id 691FE43F3F for ; Wed, 29 Jan 2003 14:21:18 -0800 (PST) (envelope-from winter@jurai.net) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by sasami.jurai.net (8.12.6/8.12.5) with ESMTP id h0TMLHvA031522; Wed, 29 Jan 2003 17:21:17 -0500 (EST) (envelope-from winter@jurai.net) Date: Wed, 29 Jan 2003 17:21:17 -0500 (EST) From: "Matthew N. Dodd" To: David Gilbert Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. In-Reply-To: <15928.21248.483298.203713@canoe.velocet.net> Message-ID: <20030129171908.G8642@sasami.jurai.net> References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> <15928.21248.483298.203713@canoe.velocet.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 29 Jan 2003, David Gilbert wrote: > but that would be no different than using the nfs directly. mdconfig > won't aggregate several chunks of files ... and last I checked md wasn't > entirely happy with nfs (some form of chicken-and-egg problem) So use vinum, CCD or add the files as swap and make a swap-backed filesystem. No reason to invent a totally new low level filesystem here. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | For Great Justice! | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 14:30: 7 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F3B8437B401 for ; Wed, 29 Jan 2003 14:30:05 -0800 (PST) Received: from sabre.velocet.net (sabre.velocet.net [216.138.209.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2C5F243FA7 for ; Wed, 29 Jan 2003 14:30:05 -0800 (PST) (envelope-from dgilbert@velocet.ca) Received: from trooper.velocet.ca (trooper.velocet.net [216.138.242.2]) by sabre.velocet.net (Postfix) with ESMTP id 48FE81382C9; Wed, 29 Jan 2003 17:30:04 -0500 (EST) Received: by trooper.velocet.ca (Postfix, from userid 66) id 737A77462B; Wed, 29 Jan 2003 17:30:03 -0500 (EST) Received: by canoe.velocet.net (Postfix, from userid 101) id AAD58567628; Wed, 29 Jan 2003 17:30:00 -0500 (EST) From: David Gilbert MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15928.21992.586804.141143@canoe.velocet.net> Date: Wed, 29 Jan 2003 17:30:00 -0500 To: "Matthew N. Dodd" Cc: David Gilbert , freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. In-Reply-To: <20030129171908.G8642@sasami.jurai.net> References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> <15928.21248.483298.203713@canoe.velocet.net> <20030129171908.G8642@sasami.jurai.net> X-Mailer: VM 7.07 under 21.1 (patch 14) "Cuyahoga Valley" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> "Matthew" == Matthew N Dodd writes: Matthew> So use vinum, CCD or add the files as swap and make a Matthew> swap-backed filesystem. Matthew> No reason to invent a totally new low level filesystem here. Actually, I can see that working ... but it's going to be a whole lot less efficient than NBD. You're doing block io that gets replicated (say ... raid 1) by vinum and then then turned back into a block transaction by md and then into a network transaction through nfs back to a filesystem transaction on the remote machine (remember md is working on the file) which is then blocked by the remote filesystem. Did I miss anything? As I understand, NBD is just a little driver that lets you mount foo:/dev/ad0s1g over the network and proxies the block transactions across. Dave. -- ============================================================================ |David Gilbert, Velocet Communications. | Two things can only be | |Mail: dgilbert@velocet.net | equal if and only if they | |http://daveg.ca | are precisely opposite. | =========================================================GLO================ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 14:36:23 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 08D0437B401 for ; Wed, 29 Jan 2003 14:36:23 -0800 (PST) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by mx1.FreeBSD.org (Postfix) with ESMTP id 45BCD43F3F for ; Wed, 29 Jan 2003 14:36:22 -0800 (PST) (envelope-from winter@jurai.net) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by sasami.jurai.net (8.12.6/8.12.5) with ESMTP id h0TMaLvA031846; Wed, 29 Jan 2003 17:36:21 -0500 (EST) (envelope-from winter@jurai.net) Date: Wed, 29 Jan 2003 17:36:21 -0500 (EST) From: "Matthew N. Dodd" To: David Gilbert Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. In-Reply-To: <15928.21992.586804.141143@canoe.velocet.net> Message-ID: <20030129173416.U8642@sasami.jurai.net> References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> <15928.21248.483298.203713@canoe.velocet.net> <20030129171908.G8642@sasami.jurai.net> <15928.21992.586804.141143@canoe.velocet.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 29 Jan 2003, David Gilbert wrote: > As I understand, NBD is just a little driver that lets you mount > foo:/dev/ad0s1g over the network and proxies the block transactions > across. Right, you still have to stripe/mirror on the client side though. I don't think it will be all that bad. Any chance of you testing Linux NBD and FreeBSD NFS/vnconfig/CCD? -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | For Great Justice! | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 14:59: 2 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4525A37B401 for ; Wed, 29 Jan 2003 14:59:01 -0800 (PST) Received: from sabre.velocet.net (sabre.velocet.net [216.138.209.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8511243F75 for ; Wed, 29 Jan 2003 14:59:00 -0800 (PST) (envelope-from dgilbert@velocet.ca) Received: from trooper.velocet.ca (trooper.velocet.net [216.138.242.2]) by sabre.velocet.net (Postfix) with ESMTP id DAB7B137FCD; Wed, 29 Jan 2003 17:58:59 -0500 (EST) Received: by trooper.velocet.ca (Postfix, from userid 66) id 617957462A; Wed, 29 Jan 2003 17:58:59 -0500 (EST) Received: by canoe.velocet.net (Postfix, from userid 101) id A0217567628; Wed, 29 Jan 2003 17:58:56 -0500 (EST) From: David Gilbert MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15928.23728.549120.559276@canoe.velocet.net> Date: Wed, 29 Jan 2003 17:58:56 -0500 To: "Matthew N. Dodd" Cc: David Gilbert , freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. In-Reply-To: <20030129173416.U8642@sasami.jurai.net> References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> <15928.21248.483298.203713@canoe.velocet.net> <20030129171908.G8642@sasami.jurai.net> <15928.21992.586804.141143@canoe.velocet.net> <20030129173416.U8642@sasami.jurai.net> X-Mailer: VM 7.07 under 21.1 (patch 14) "Cuyahoga Valley" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> "Matthew" == Matthew N Dodd writes: Matthew> On Wed, 29 Jan 2003, David Gilbert wrote: >> As I understand, NBD is just a little driver that lets you mount >> foo:/dev/ad0s1g over the network and proxies the block transactions >> across. Matthew> Right, you still have to stripe/mirror on the client side Matthew> though. I don't think it will be all that bad. Matthew> Any chance of you testing Linux NBD and FreeBSD Matthew> NFS/vnconfig/CCD? it doesn't work that way. the result of NBD is a /dev/nbd0 not a filesystem. Block 0 of /dev/nbd0 is block 0 of /dev/hda1 (say). nbd runs as a server on the node with the disk and as a client on the node using the disk. Yes, you still stripe on the client side... but you stripe across directly mapped block devices (no NFS involved). Dave. -- ============================================================================ |David Gilbert, Velocet Communications. | Two things can only be | |Mail: dgilbert@velocet.net | equal if and only if they | |http://daveg.ca | are precisely opposite. | =========================================================GLO================ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 15: 6:25 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B7B3137B401 for ; Wed, 29 Jan 2003 15:06:24 -0800 (PST) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by mx1.FreeBSD.org (Postfix) with ESMTP id CFA1B43FA3 for ; Wed, 29 Jan 2003 15:06:23 -0800 (PST) (envelope-from winter@jurai.net) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by sasami.jurai.net (8.12.6/8.12.5) with ESMTP id h0TN6KvA032442; Wed, 29 Jan 2003 18:06:21 -0500 (EST) (envelope-from winter@jurai.net) Date: Wed, 29 Jan 2003 18:06:20 -0500 (EST) From: "Matthew N. Dodd" To: David Gilbert Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. In-Reply-To: <15928.23728.549120.559276@canoe.velocet.net> Message-ID: <20030129180043.S8642@sasami.jurai.net> References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> <15928.21248.483298.203713@canoe.velocet.net> <20030129171908.G8642@sasami.jurai.net> <15928.21992.586804.141143@canoe.velocet.net> <20030129173416.U8642@sasami.jurai.net> <15928.23728.549120.559276@canoe.velocet.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 29 Jan 2003, David Gilbert wrote: > it doesn't work that way. the result of NBD is a /dev/nbd0 not a > filesystem. Block 0 of /dev/nbd0 is block 0 of /dev/hda1 (say). nbd > runs as a server on the node with the disk and as a client on the node > using the disk. Yes, you still stripe on the client side... but you > stripe across directly mapped block devices (no NFS involved). So involving NFS isn't really going to make that much of a difference. Likely the NBD will have lower overhead but probably not much. What you really want is SCSI over IP. Anything else is just a hack and not to be trusted. I think that NFS is less of a hack than NBD though. Of course if Linux still suffers from poor NFS performance that might explain why they came up with NBD in the first place. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | For Great Justice! | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 15:13:48 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0223F37B401 for ; Wed, 29 Jan 2003 15:13:47 -0800 (PST) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2CC2443F75 for ; Wed, 29 Jan 2003 15:13:46 -0800 (PST) (envelope-from phk@freebsd.org) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h0TNDQZE016444; Thu, 30 Jan 2003 00:13:27 +0100 (CET) (envelope-from phk@freebsd.org) To: "Matthew N. Dodd" Cc: David Gilbert , freebsd-hackers@freebsd.org Subject: Re: Network block device. From: phk@freebsd.org In-Reply-To: Your message of "Wed, 29 Jan 2003 18:06:20 EST." <20030129180043.S8642@sasami.jurai.net> Date: Thu, 30 Jan 2003 00:13:26 +0100 Message-ID: <16443.1043882006@critter.freebsd.dk> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <20030129180043.S8642@sasami.jurai.net>, "Matthew N. Dodd" writes: >On Wed, 29 Jan 2003, David Gilbert wrote: >> it doesn't work that way. the result of NBD is a /dev/nbd0 not a >> filesystem. Block 0 of /dev/nbd0 is block 0 of /dev/hda1 (say). nbd >> runs as a server on the node with the disk and as a client on the node >> using the disk. Yes, you still stripe on the client side... but you >> stripe across directly mapped block devices (no NFS involved). > >So involving NFS isn't really going to make that much of a difference. Yes, it sure would. NBD wouldn't be hard to implement on FreeBSD, the easiest way would be to write two GEOM modules to do it: a client and a server. No, I don't have time to do that right now, but I will happily guide anybody who wants to try. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 15:28:12 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7BE4F37B401; Wed, 29 Jan 2003 15:28:11 -0800 (PST) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by mx1.FreeBSD.org (Postfix) with ESMTP id AEC0743F43; Wed, 29 Jan 2003 15:28:10 -0800 (PST) (envelope-from winter@jurai.net) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by sasami.jurai.net (8.12.6/8.12.5) with ESMTP id h0TNS9vA032891; Wed, 29 Jan 2003 18:28:09 -0500 (EST) (envelope-from winter@jurai.net) Date: Wed, 29 Jan 2003 18:28:09 -0500 (EST) From: "Matthew N. Dodd" To: phk@freebsd.org Cc: David Gilbert , freebsd-hackers@freebsd.org Subject: Re: Network block device. In-Reply-To: <16443.1043882006@critter.freebsd.dk> Message-ID: <20030129181859.X8642@sasami.jurai.net> References: <16443.1043882006@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 30 Jan 2003 phk@freebsd.org wrote: > In message <20030129180043.S8642@sasami.jurai.net>, "Matthew N. Dodd" writes: > >On Wed, 29 Jan 2003, David Gilbert wrote: > >So involving NFS isn't really going to make that much of a difference. > > Yes, it sure would. nfs1:/foo/foo1 -> md1 nfs2:/foo/foo2 -> md2 ccd0 64 none md1 md2 # bonnie -s 16m ... -------Sequential Output-------- ---Sequential Input-- --Random-- -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--- Machine MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU 16 783 7.0 629 0.6 862 0.7 10813 95.1 210575 93.2 21433.1 98.8 Ignore the reads here. Writes seem good for a half-duplex 10baseT network. The NFS servers aren't very fast either. I'd say 600kb/sec+ is pretty reasonable. # rawio -a /dev/ccd0 -s 16777216 Random read Sequential read Random write Sequential write ID K/sec /sec K/sec /sec K/sec /sec K/sec /sec ccd0 13192.4 789 0.0 0 13200.1 790 0.0 0 I'm not sure what to make of this. I'll try with larger files when I get home and add a 3rd server. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | For Great Justice! | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 15:54:46 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9423B37B401 for ; Wed, 29 Jan 2003 15:54:45 -0800 (PST) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9817A43F3F for ; Wed, 29 Jan 2003 15:54:44 -0800 (PST) (envelope-from winter@jurai.net) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by sasami.jurai.net (8.12.6/8.12.5) with ESMTP id h0TNsevA033447; Wed, 29 Jan 2003 18:54:40 -0500 (EST) (envelope-from winter@jurai.net) Date: Wed, 29 Jan 2003 18:54:40 -0500 (EST) From: "Matthew N. Dodd" To: "Brandon D. Valentine" Cc: David Gilbert , freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. In-Reply-To: <20030129235040.GY16038@geekpunk.net> Message-ID: <20030129185154.U8642@sasami.jurai.net> References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> <15928.21248.483298.203713@canoe.velocet.net> <20030129171908.G8642@sasami.jurai.net> <15928.21992.586804.141143@canoe.velocet.net> <20030129173416.U8642@sasami.jurai.net> <15928.23728.549120.559276@canoe.velocet.net> <20030129180043.S8642@sasami.jurai.net> <20030129235040.GY16038@geekpunk.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 29 Jan 2003, Brandon D. Valentine wrote: > IMO NBD is less of a hack than you think it is. It is one of the > necessary components for creating a single system image from a cluster > of commodity hardware and this is something Linux developers are working > earnestly on. They're targeting a poor man's NUMA. Sorry, it still sounds dumb. They should really look at Sprite. (And anyone thats doing clustering and not looking at VMS deserves what they get.) On a real cluster running a single image all all the drives would just show up. There wouldn't be any hacking going on. Stuff like this kind of requires 64 bit machines to be at all useful. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | For Great Justice! | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 15:57:53 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 53F4137B401; Wed, 29 Jan 2003 15:57:52 -0800 (PST) Received: from sccrmhc02.attbi.com (sccrmhc02.attbi.com [204.127.202.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 162ED43F43; Wed, 29 Jan 2003 15:57:50 -0800 (PST) (envelope-from julian@elischer.org) Received: from InterJet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc02.attbi.com (sccrmhc02) with ESMTP id <2003012923574800200i7kdqe>; Wed, 29 Jan 2003 23:57:49 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA28005; Wed, 29 Jan 2003 15:57:47 -0800 (PST) Date: Wed, 29 Jan 2003 15:57:46 -0800 (PST) From: Julian Elischer To: phk@freebsd.org Cc: "Matthew N. Dodd" , David Gilbert , freebsd-hackers@freebsd.org Subject: Re: Network block device. In-Reply-To: <16443.1043882006@critter.freebsd.dk> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG geom meets netgraph.. :-) You could possibly do something with the ng_device node that exports a device into teh dev namesapce from netgraph. (the version in the tree is curently broken, the author is rewrituing it..) Adding a geom top-end to it might give you something quite cute.. On Thu, 30 Jan 2003 phk@freebsd.org wrote: > In message <20030129180043.S8642@sasami.jurai.net>, "Matthew N. Dodd" writes: > >On Wed, 29 Jan 2003, David Gilbert wrote: > >> it doesn't work that way. the result of NBD is a /dev/nbd0 not a > >> filesystem. Block 0 of /dev/nbd0 is block 0 of /dev/hda1 (say). nbd > >> runs as a server on the node with the disk and as a client on the node > >> using the disk. Yes, you still stripe on the client side... but you > >> stripe across directly mapped block devices (no NFS involved). > > > >So involving NFS isn't really going to make that much of a difference. > > Yes, it sure would. > > NBD wouldn't be hard to implement on FreeBSD, the easiest way would > be to write two GEOM modules to do it: a client and a server. > > No, I don't have time to do that right now, but I will happily > guide anybody who wants to try. > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 17: 3:16 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 017DE37B401; Wed, 29 Jan 2003 17:03:15 -0800 (PST) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by mx1.FreeBSD.org (Postfix) with ESMTP id 49C2743F93; Wed, 29 Jan 2003 17:03:14 -0800 (PST) (envelope-from winter@jurai.net) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by sasami.jurai.net (8.12.6/8.12.5) with ESMTP id h0U139vA034577; Wed, 29 Jan 2003 20:03:09 -0500 (EST) (envelope-from winter@jurai.net) Date: Wed, 29 Jan 2003 20:03:08 -0500 (EST) From: "Matthew N. Dodd" To: Julian Elischer Cc: phk@freebsd.org, David Gilbert , freebsd-hackers@freebsd.org Subject: Re: Network block device. In-Reply-To: Message-ID: <20030129195434.L8642@sasami.jurai.net> References: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 29 Jan 2003, Julian Elischer wrote: > geom meets netgraph.. :-) > > You could possibly do something with the ng_device node that exports a > device into teh dev namesapce from netgraph. (the version in the tree is > curently broken, the author is rewrituing it..) Adding a geom top-end > to it might give you something quite cute.. Slightly modify NFS to allow you to "export" a device. Since the NFS code already allows you to mount remote 'files' you're good to go. Congradulations you've got your NBD and you only spent 2 hours (if that) on it. Jesus people... If you've got this much free time, implement something that will actually be USEFUL over a wide variety of problem sets like SCSI over IP. Hacking around implementing anything else from scratch is a waste of time. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | For Great Justice! | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 18:30:51 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BC22037B401 for ; Wed, 29 Jan 2003 18:30:49 -0800 (PST) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 27C0243F75 for ; Wed, 29 Jan 2003 18:30:44 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0224.cvx22-bradley.dialup.earthlink.net ([209.179.198.224] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18e4TP-0006Sj-00; Wed, 29 Jan 2003 18:30:40 -0800 Message-ID: <3E388DF8.52C6235B@mindspring.com> Date: Wed, 29 Jan 2003 18:29:12 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: "Matthew N. Dodd" Cc: "Brandon D. Valentine" , David Gilbert , freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> <15928.21248.483298.203713@canoe.velocet.net> <20030129171908.G8642@sasami.jurai.net> <15928.21992.586804.141143@canoe.velocet.net> <20030129173416.U8642@sasami.jurai.net> <15928.23728.549120.559276@canoe.velocet.net> <20030129180043.S8642@sasami.jurai.net> <20030129235040.GY16038@geekpunk.net> <20030129185154.U8642@sasami.jurai.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a49ade9284a5d4077272d3a14f3e25ee842601a10902912494350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG "Matthew N. Dodd" wrote: > They should really look at Sprite. (And anyone thats doing clustering and > not looking at VMS deserves what they get.) > > On a real cluster running a single image all all the drives would just > show up. There wouldn't be any hacking going on. Stuff like this kind of > requires 64 bit machines to be at all useful. And anyone that's doing clustering and things that it can't be done on a 32 bit machine, and not looking at the VAX, which runs VMS, deserves what they get. ...Sorry, had to be said... 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 18:45: 4 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C937937B401 for ; Wed, 29 Jan 2003 18:45:03 -0800 (PST) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by mx1.FreeBSD.org (Postfix) with ESMTP id D214A43F79 for ; Wed, 29 Jan 2003 18:45:02 -0800 (PST) (envelope-from winter@jurai.net) Received: from sasami.jurai.net (sasami.jurai.net [66.92.160.223]) by sasami.jurai.net (8.12.6/8.12.5) with ESMTP id h0U2ixvA036326; Wed, 29 Jan 2003 21:44:59 -0500 (EST) (envelope-from winter@jurai.net) Date: Wed, 29 Jan 2003 21:44:59 -0500 (EST) From: "Matthew N. Dodd" To: Terry Lambert Cc: "Brandon D. Valentine" , David Gilbert , freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. In-Reply-To: <3E388DF8.52C6235B@mindspring.com> Message-ID: <20030129214141.R8642@sasami.jurai.net> References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> <15928.21248.483298.203713@canoe.velocet.net> <20030129171908.G8642@sasami.jurai.net> <15928.21992.586804.141143@canoe.velocet.net> <20030129173416.U8642@sasami.jurai.net> <15928.23728.549120.559276@canoe.velocet.net> <20030129180043.S8642@sasami.jurai.net> <20030129235040.GY16038@geekpunk.net> <20030129185154.U8642@sasami.jurai.net> <3E388DF8.52C6235B@mindspring.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 29 Jan 2003, Terry Lambert wrote: > And anyone that's doing clustering and things that it can't be done on a > 32 bit machine, and not looking at the VAX, which runs VMS, deserves > what they get. > > ...Sorry, had to be said... 8-). If we were talking about clustering 32 bit machines with less than 128mb of memory each that would be true. I suppose you could use some sort of PAE to allow every cluster member's address space to be mapped but a 64 bit address space means that you don't have to worry as much about this. -- | Matthew N. Dodd | '78 Datsun 280Z | '75 Volvo 164E | FreeBSD/NetBSD | | winter@jurai.net | 2 x '84 Volvo 245DL | ix86,sparc,pmax | | http://www.jurai.net/~winter | For Great Justice! | ISO8802.5 4ever | To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 21:30:28 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6500E37B405 for ; Wed, 29 Jan 2003 21:30:26 -0800 (PST) Received: from geekpunk.net (adsl-154-186-63.bna.bellsouth.net [68.154.186.63]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9137D43F93 for ; Wed, 29 Jan 2003 21:30:18 -0800 (PST) (envelope-from bandix@geekpunk.net) Received: from localhost.my.domain (taran [127.0.0.1]) by geekpunk.net (8.12.6/8.12.6) with ESMTP id h0TNojHa008646; Wed, 29 Jan 2003 17:50:45 -0600 (CST) (envelope-from bandix@geekpunk.net) Received: (from bandix@localhost) by localhost.my.domain (8.12.6/8.12.6/Submit) id h0TNoeLH008645; Wed, 29 Jan 2003 17:50:40 -0600 (CST) (envelope-from bandix) Date: Wed, 29 Jan 2003 17:50:40 -0600 From: "Brandon D. Valentine" To: "Matthew N. Dodd" Cc: David Gilbert , freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. Message-ID: <20030129235040.GY16038@geekpunk.net> References: <15928.6900.948346.474717@canoe.velocet.net> <20030129170512.Y8642@sasami.jurai.net> <15928.21248.483298.203713@canoe.velocet.net> <20030129171908.G8642@sasami.jurai.net> <15928.21992.586804.141143@canoe.velocet.net> <20030129173416.U8642@sasami.jurai.net> <15928.23728.549120.559276@canoe.velocet.net> <20030129180043.S8642@sasami.jurai.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030129180043.S8642@sasami.jurai.net> User-Agent: Mutt/1.4i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 29, 2003 at 06:06:20PM -0500, Matthew N. Dodd wrote: > > What you really want is SCSI over IP. Anything else is just a hack and > not to be trusted. I think that NFS is less of a hack than NBD though. IMO NBD is less of a hack than you think it is. It is one of the necessary components for creating a single system image from a cluster of commodity hardware and this is something Linux developers are working earnestly on. They're targeting a poor man's NUMA. In this case the idea is that each node [network] boots a very minimal operating system image which acts as a slave to the master node, making its hardware (disks in this case) available via some interconnect using network block devices. Then the operating system on the master node can mount the devices under one unified /. OpenMosix is working on solving the other parts of the puzzle, which are scheduling and cache/memory coherency. Myrinet is rapidly converging on SGI's CrayLink interconnect as a low-latency memory bus, making this possible in the next few years on PC hardware. Of course I think they're using an inferior operating system as the basis for all this, but since I'm not willing to put code where my mouth is in this instance, I'm not going to go around telling them which OS to use. ;-) > Of course if Linux still suffers from poor NFS performance that might > explain why they came up with NBD in the first place. As someone who currently has several terabytes served off of Linux 2.4/XFS servers let me say that Linux *definitely* still suffers from poor NFS performance. Not to mention fragility. The Linux in-kernel NFS client and server are notorious for locking up machines around here when remote NFS mounts go away unexpectedly. There provide little fault tolerance at all, with both soft and hard NFS. Brandon D. Valentine -- brandon@dvalentine.com http://www.geekpunk.net "We've been raised on replicas of fake and winding roads, and day after day up on this beautiful stage we've been playing tambourine for minimum wage, but we are real; I know we are real." -- David Berman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 21:30:30 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8709C37B401; Wed, 29 Jan 2003 21:30:28 -0800 (PST) Received: from geekpunk.net (adsl-154-186-63.bna.bellsouth.net [68.154.186.63]) by mx1.FreeBSD.org (Postfix) with ESMTP id A0E7D43F43; Wed, 29 Jan 2003 21:30:26 -0800 (PST) (envelope-from bandix@geekpunk.net) Received: from localhost.my.domain (taran [127.0.0.1]) by geekpunk.net (8.12.6/8.12.6) with ESMTP id h0TNtfHa008669; Wed, 29 Jan 2003 17:55:41 -0600 (CST) (envelope-from bandix@geekpunk.net) Received: (from bandix@localhost) by localhost.my.domain (8.12.6/8.12.6/Submit) id h0TNteSw008668; Wed, 29 Jan 2003 17:55:40 -0600 (CST) (envelope-from bandix) Date: Wed, 29 Jan 2003 17:55:39 -0600 From: "Brandon D. Valentine" To: phk@freebsd.org Cc: "Matthew N. Dodd" , David Gilbert , freebsd-hackers@freebsd.org Subject: Re: Network block device. Message-ID: <20030129235539.GZ16038@geekpunk.net> References: <20030129180043.S8642@sasami.jurai.net> <16443.1043882006@critter.freebsd.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <16443.1043882006@critter.freebsd.dk> User-Agent: Mutt/1.4i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, Jan 30, 2003 at 12:13:26AM +0100, phk@freebsd.org wrote: > > NBD wouldn't be hard to implement on FreeBSD, the easiest way would > be to write two GEOM modules to do it: a client and a server. > > No, I don't have time to do that right now, but I will happily > guide anybody who wants to try. This would be extremely useful, if there is someone out there with enough Copious Free Time to tackle it. With UFS2 now in the tree it is possible to create filesystems much larger than anyone is likely to connect to a single physical machine. The ability to build storage clusters using FreeBSD would be quite cool. Brandon D. Valentine -- brandon@dvalentine.com http://www.geekpunk.net "We've been raised on replicas of fake and winding roads, and day after day up on this beautiful stage we've been playing tambourine for minimum wage, but we are real; I know we are real." -- David Berman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 22:18:29 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DECFF37B401 for ; Wed, 29 Jan 2003 22:18:27 -0800 (PST) Received: from geekpunk.net (adsl-154-186-63.bna.bellsouth.net [68.154.186.63]) by mx1.FreeBSD.org (Postfix) with ESMTP id CAFD643F85 for ; Wed, 29 Jan 2003 22:18:26 -0800 (PST) (envelope-from bandix@geekpunk.net) Received: from localhost.my.domain (taran [127.0.0.1]) by geekpunk.net (8.12.6/8.12.6) with ESMTP id h0U6IY3r009023; Thu, 30 Jan 2003 00:18:34 -0600 (CST) (envelope-from bandix@geekpunk.net) Received: (from bandix@localhost) by localhost.my.domain (8.12.6/8.12.6/Submit) id h0U6ITdP009022; Thu, 30 Jan 2003 00:18:29 -0600 (CST) (envelope-from bandix) Date: Thu, 30 Jan 2003 00:18:29 -0600 From: "Brandon D. Valentine" To: "Matthew N. Dodd" Cc: Terry Lambert , David Gilbert , freebsd-hackers@FreeBSD.ORG Subject: Re: Network block device. Message-ID: <20030130061829.GC16038@geekpunk.net> References: <15928.21248.483298.203713@canoe.velocet.net> <20030129171908.G8642@sasami.jurai.net> <15928.21992.586804.141143@canoe.velocet.net> <20030129173416.U8642@sasami.jurai.net> <15928.23728.549120.559276@canoe.velocet.net> <20030129180043.S8642@sasami.jurai.net> <20030129235040.GY16038@geekpunk.net> <20030129185154.U8642@sasami.jurai.net> <3E388DF8.52C6235B@mindspring.com> <20030129214141.R8642@sasami.jurai.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030129214141.R8642@sasami.jurai.net> User-Agent: Mutt/1.4i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, Jan 29, 2003 at 09:44:59PM -0500, Matthew N. Dodd wrote: > > If we were talking about clustering 32 bit machines with less than 128mb > of memory each that would be true. > > I suppose you could use some sort of PAE to allow every cluster member's > address space to be mapped but a 64 bit address space means that you don't > have to worry as much about this. Admittedly nobody [that I know of] has a working single system image shared memory Linux cluster outside of SGI[0], and they're using 64-bit hardware. I fully expect that those working on single system image clustering for Linux expect to use it primarily on 64-bit PC hardware once it becomes available. The OpenMosix folks are actually not using a unified address space in thier SSI model. Each node in an OpenMosix cluster runs its own mosix kernel which manages the memory for that node exclusively. When a process is scheduled by the master to run on a particular node then the process' page tables and dirty pages are transferred into memory on the remote node. This is obviously suboptimal and does not support applications which use shared memory. The OpenMosix roadmap does indicate that at some point they plan to add shared memory support. I have not seen any discussion on how they plan to solve the address space limitation. For those who might not be familiar: OpenMosix is a recent fork of the original mosix codebase from Prof. Amnon Barak of the Hebrew University of Jerusalem and his students. At some point Prof. Barak decide to close his source and so a GPL'd replacement sprung up in its place. IIRC mosix was originally written for the US Air Force in the 70s to run on their PDP-11s running Bell Labs' UNIX. It was later ported to BSD/OS before being ported to Linux. I am sure the code to the BSD/OS port is still lurking around somewhere behind an NDA. If someone were interested in bringing this sort of clustering functionality to BSD it might be worth trying to track that code down. [0] - http://www.linuxjournal.com/article.php?sid=6440 Brandon D. Valentine -- brandon@dvalentine.com http://www.geekpunk.net "We've been raised on replicas of fake and winding roads, and day after day up on this beautiful stage we've been playing tambourine for minimum wage, but we are real; I know we are real." -- David Berman To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 22:44:51 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A1F9237B401 for ; Wed, 29 Jan 2003 22:44:50 -0800 (PST) Received: from HAL9000.homeunix.com (12-233-57-224.client.attbi.com [12.233.57.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1290843F3F for ; Wed, 29 Jan 2003 22:44:50 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.6/8.12.5) with ESMTP id h0U6imNt007470; Wed, 29 Jan 2003 22:44:48 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.6/8.12.5/Submit) id h0U6im8b007469; Wed, 29 Jan 2003 22:44:48 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Date: Wed, 29 Jan 2003 22:44:48 -0800 From: David Schultz To: James Gritton Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? Message-ID: <20030130064448.GA7258@HAL9000.homeunix.com> Mail-Followup-To: James Gritton , freebsd-hackers@FreeBSD.ORG References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thus spake James Gritton : > The object's ref_count hasn't changed, which is what I meant about seeing > reference counts in the kernel that were apparently not counting what I'm > looking for. I did see a ref_count increase on the first object > (presumably the text image), but nothing on the allocated memory. > > It seems the object level isn't fine enough, but the deeper I go into the > VM code, the more confused I become. In this forked process example, what > happens when I alter a few COW pages in the currently-shared object? > Apparently a shadow object is created, but it claims to be the same size as > the original object. True, but I know it's not actually using that many > pages, since most of them are still validly shared. System usage numbers > tell me this is true, but I can't find what in the process or object data > structures reflect this fact. No, you don't have enough information. Even if you knew which objects shadowed which, I still don't think you would have enough information. You want to account for physical pages, so you should be looking at vm_page structures. AFAIK, there isn't an interface to do that, but one shouldn't be too hard to implement. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Wed Jan 29 23:19:28 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3202E37B401 for ; Wed, 29 Jan 2003 23:19:25 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id A1F3143F75 for ; Wed, 29 Jan 2003 23:19:24 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0U7JO0i086055; Wed, 29 Jan 2003 23:19:24 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0U7JOfI086054; Wed, 29 Jan 2003 23:19:24 -0800 (PST) Date: Wed, 29 Jan 2003 23:19:24 -0800 (PST) From: Matthew Dillon Message-Id: <200301300719.h0U7JOfI086054@apollo.backplane.com> To: David Schultz Cc: James Gritton , freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? References: <20030130064448.GA7258@HAL9000.homeunix.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Thus spake James Gritton : :> The object's ref_count hasn't changed, which is what I meant about seeing :> reference counts in the kernel that were apparently not counting what I'm :> looking for. I did see a ref_count increase on the first object :> (presumably the text image), but nothing on the allocated memory. :> :> It seems the object level isn't fine enough, but the deeper I go into the :> VM code, the more confused I become. In this forked process example, what :> happens when I alter a few COW pages in the currently-shared object? :> Apparently a shadow object is created, but it claims to be the same size as :> the original object. True, but I know it's not actually using that many :> pages, since most of them are still validly shared. System usage numbers :> tell me this is true, but I can't find what in the process or object data :> structures reflect this fact. : :No, you don't have enough information. Even if you knew which :objects shadowed which, I still don't think you would have enough :information. You want to account for physical pages, so you :should be looking at vm_page structures. AFAIK, there isn't an :interface to do that, but one shouldn't be too hard to implement. Well, first he should read my DaemonNews article. http://www.daemonnews.org/200001/freebsd_vm.html Now, in regards to COW faults and shadow pages, it basically comes down to the process's VM map (struct vm_map) which contains a list of vm_map_entry structures which tell the VM system what VM address ranges correspond to what VM objects. For all intents and purposes, the size of a VM object (struct vm_object) which represents anonymous memory is irrelevant. What is relevant is the size of the 'window' into the VM object defined by the vm_map_entry. You can get a list of vm_map_entry's associated with a process using the /proc filesystem. e.g. dd if=/proc/PID/map bs=256k dd if=/proc/86006/map bs=256k Shadow objects are basically just a way for the VM system to be able to mix pages that are COW faulted and pages that have not yet been COW faulted together without having to create independant little VM objects for each faulted page. The shadowing is basically just layering two or more VM objects on top of each other. If the top VM object doesn't have the page the system recurses into deeper VM objects to find the page. If the access is a read the deeper page is simply used straight out. If the access is a write the deeper page is COW copied into the top level VM object. Generally speaking I don't think you have to go into this level of detail to figure out approximate real memory use. Just look at the output of the /proc/PID/map and pull out the 'default' or 'swap' lines. Ignore the 'vnode' lines. apollo:/home/dillon# dd if=/proc/85965/map bs=256k 0x8048000 0x805b000 14 15 0xd2482600 r-x 2 1 0x0 COW NC vnode 0x805b000 0x805c000 1 0 0xd2830960 rw- 1 0 0x2180 COW NNC vnode 0x805c000 0x8061000 5 0 0xd29ea360 rw- 2 0 0x2180 NCOW NNC default <<<<< 0x8061000 0x8156000 237 0 0xd29ea360 rwx 2 0 0x2180 NCOW NNC default <<<<< 0x2805b000 0x2806d000 17 0 0xc031df60 r-x 77 34 0x4 COW NC vnode 0x2806d000 0x2806e000 1 0 0xd2c12d80 rw- 1 0 0x2180 COW NNC vnode 0x2806e000 0x28070000 2 0 0xd2c84c60 rw- 2 0 0x2180 NCOW NNC default <<<<< 0x28070000 0x28078000 6 0 0xd2c84c60 rwx 2 0 0x2180 NCOW NNC default <<<<< 0x28078000 0x280f8000 90 0 0xc031e2c0 r-x 100 58 0x4 COW NC vnode 0x280f8000 0x280f9000 1 0 0xd2d5ad20 r-x 1 0 0x2180 COW NNC vnode 0x280f9000 0x280fe000 5 0 0xd34e2f00 rwx 1 0 0x2180 COW NNC vnode 0x280fe000 0x28112000 7 0 0xd34100c0 rwx 1 0 0x2180 NCOW NNC default <<<<< 0x2a07b000 0x2bfe4000 8041 0 0xd342e4e0 r-x 1 0 0x0 NCOW NNC vnode 0xbfbe0000 0xbfc00000 2 0 0xd2730300 rwx 1 0 0x2180 NCOW NNC default <<<<< Now you have two metrics. You can calculate the size based on the range (the first two arguments). (addressB - addressA), and you can get the total number of resident pages from the third argument. What you can't do easily is figure out the total allocation because that is actually going to be resident pages + swapped pages. swapped pages is not included in the proc output. The fifth argument is the VM object. You could garner more information from the VM object by accessing it via /dev/kvm. You can get an approximate number of swapped out pages using: (vm_object->un_pager.swp.swp_bcount * 16) (only from VM objects of type OBJT_SWAP). You can get an exact count by delving into the swap array and locating all the swblock meta structures associated with the object used to store swap information for the object. That gets rather involved, I wouldn't bother. You can also theoretically push into shadow VM objects to locate pages from the parent process that have not yet been COW'd into the child (in the case of a fork()), noting also that these shadow objects might be shared with other children of the parent and by the parent process itself, but under most conditions this information will not be significant and can be ignored. Any vnode object is always shared with other processes mapping the same vnode. Since this information is backed by a file, figuring out how much 'memory' it represents by any reasonable definition is guesswork. The resident page count will represent how much of the vnode is cached, but not how much of the vnode is actually being accessed by the process. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 1:14:25 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 769AF37B401 for ; Thu, 30 Jan 2003 01:14:24 -0800 (PST) Received: from HAL9000.homeunix.com (12-233-57-224.client.attbi.com [12.233.57.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id 454D243F93 for ; Thu, 30 Jan 2003 01:14:23 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.6/8.12.5) with ESMTP id h0U9EKNt007995; Thu, 30 Jan 2003 01:14:20 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.6/8.12.5/Submit) id h0U9EJBR007994; Thu, 30 Jan 2003 01:14:19 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Date: Thu, 30 Jan 2003 01:14:19 -0800 From: David Schultz To: Matthew Dillon Cc: James Gritton , freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? Message-ID: <20030130091419.GA7776@HAL9000.homeunix.com> Mail-Followup-To: Matthew Dillon , James Gritton , freebsd-hackers@FreeBSD.ORG References: <20030130064448.GA7258@HAL9000.homeunix.com> <200301300719.h0U7JOfI086054@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200301300719.h0U7JOfI086054@apollo.backplane.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thus spake Matthew Dillon : > Generally speaking I don't think you have to go into this level of > detail to figure out approximate real memory use. Just look at the > output of the /proc/PID/map and pull out the 'default' or 'swap' > lines. Ignore the 'vnode' lines. ... > Now you have two metrics. You can calculate the size based on the > range (the first two arguments). (addressB - addressA), and you can get > the total number of resident pages from the third argument. What you > can't do easily is figure out the total allocation because that is > actually going to be resident pages + swapped pages. swapped pages is > not included in the proc output. I think the original poster wanted to know the real memory use of a set of processes, taking sharing into account. I don't see how your approach could do that. Even if you knew the structure of the shadow chain, you would have to know specifically which pages had been COWed, no? I thought counting vm_page structures would be the way to go... I really want to find the time to learn all this stuff better. I still don't know the difference between COW and NEEDS_COPY, or why the pageout daemon seems to be biased against shared pages, or a lot of things about pv_entry structures. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 1:32:47 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1CBB337B401 for ; Thu, 30 Jan 2003 01:32:46 -0800 (PST) Received: from rootshell.be (phenix.rootshell.be [195.74.192.154]) by mx1.FreeBSD.org (Postfix) with ESMTP id D51A443F43 for ; Thu, 30 Jan 2003 01:32:44 -0800 (PST) (envelope-from nmlabs@phenix.rootshell.be) Received: from localhost (nmlabs@localhost) by rootshell.be (8.11.5/8.11.5) with ESMTP id h0U9WWo04822 for ; Thu, 30 Jan 2003 10:32:32 +0100 Date: Thu, 30 Jan 2003 10:32:32 +0100 (CET) From: Nicolas Mallet To: freebsd-hackers@FreeBSD.ORG Subject: Re: Changing the Maximum Segment Size (MSS) of Kame MIP6 Free BSD4.4 Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Audsin wrote: >I wish to change the Maximum segment size >of the TCP. Can you please help me , where i should change the MSS of the >TCP. Can you tell me where the default size of the MSS mentioned? $ sysctl -a | grep mss net.inet.tcp.mssdflt: 512 net.inet.tcp.v6mssdflt: 1024 Ex: $ sysctl -w net.inet.tcp.v6mssdflt=new_value To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 1:44:20 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 42C4E37B401; Thu, 30 Jan 2003 01:44:19 -0800 (PST) Received: from sun-fish.com (border.sun-fish.com [62.176.74.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9E94543F3F; Thu, 30 Jan 2003 01:44:17 -0800 (PST) (envelope-from vladimir.terziev@sun-fish.com) Received: from 127.0.0.1 (localhost [127.0.0.1]) by antivirus.software (Postfix) with SMTP id 4968E14A09; Thu, 30 Jan 2003 10:44:05 +0100 (CET) Received: from daemon.cmotd.com (daemon.cmotd.com [192.168.33.170]) by sun-fish.com (Postfix) with SMTP id 4946114A08; Thu, 30 Jan 2003 10:44:04 +0100 (CET) Date: Thu, 30 Jan 2003 11:44:01 +0200 From: Vladimir Terziev To: hackers@FreeBSD.ORG, security@freebsd.org Subject: Kerberos & OpenSSH+GSSAPI problem Message-Id: <20030130114401.38eeffa2.vlady@sun-fish.com> Organization: SunFish Ltd. X-Mailer: Sylpheed version 0.8.6claws (GTK+ 1.2.10; ) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi hackers, I implement a Kerberos in my company. For the purpose I use MIT Kerberos v5, OpenSSH v3.4p1 and approriate GSSAPI patches for OpenSSH from http://www.sxw.org.uk/computing/patches/openssh.html . Kerbelized sshd works fine and uses Kerberos tickets for authentication when the machine have single interface. But I have some multihomed machines which participate in different domains (respectively in different Kerberos realms). Sshd on these machines refuses to use my Kerberos tickes for authentication. I think this is because GSSAPI patches for OpenSSH use hostname for forming of Kerberos principals. I my case, with mulultihomed machines, hostname is different from the one or more of the interface names of the machine. Does anybody have any idea how I can solve that nasty problem? Regards, Vladimir To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 3:50:50 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2B72537B401 for ; Thu, 30 Jan 2003 03:50:48 -0800 (PST) Received: from mgr2.xmission.com (mgr2.xmission.com [198.60.22.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9C45943E4A for ; Thu, 30 Jan 2003 03:50:47 -0800 (PST) (envelope-from glewis@eyesbeyond.com) Received: from mail by mgr2.xmission.com with spam-scanned (Exim 3.35 #1) id 18eDCn-0000DL-02 for freebsd-hackers@freebsd.org; Thu, 30 Jan 2003 04:50:05 -0700 Received: from [207.135.128.145] (helo=misty.eyesbeyond.com) by mgr2.xmission.com with esmtp (Exim 3.35 #1) id 18eD9I-0007MM-02 for freebsd-hackers@freebsd.org; Thu, 30 Jan 2003 04:46:28 -0700 Received: (from glewis@localhost) by misty.eyesbeyond.com (8.11.6/8.11.6) id h0UBkO021632 for freebsd-hackers@freebsd.org; Thu, 30 Jan 2003 22:16:25 +1030 (CST) (envelope-from glewis@eyesbeyond.com) X-Authentication-Warning: misty.eyesbeyond.com: glewis set sender to glewis@eyesbeyond.com using -f Date: Thu, 30 Jan 2003 22:16:24 +1030 From: Greg Lewis To: freebsd-hackers@freebsd.org Subject: VIA Rhine III (MII without any phy!) Message-ID: <20030130221623.A21565@misty.eyesbeyond.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i X-Spam-Status: No, hits=-1.7 required=8.0 tests=SIGNATURE_SHORT_DENSE,SPAM_PHRASE_01_02,USER_AGENT, USER_AGENT_MUTT,X_AUTH_WARNING version=2.43 X-Spam-Level: Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi all, I've recently acquired a motherboard with a VIA Rhine III onboard NIC. I couldn't see support for this in the current vr(4) driver in either -current or -stable and am trying to add support for it. Having not done any kernel hacking before I'd appreciate some pointers if someone would be so kind. Finding patches like that in http://www.uwsg.iu.edu/hypermail/linux/kernel/0204.1/0887.html which added the driver to the Linux kernel simply by adding the chip id's to their driver, I thought I'd give the same thing a try. I added the appropriate code to if_vrreg.h and if_vr.c and the chip now seems to detect at least partially: vr0: port 0xd400-0xd4ff mem 0xea001000-0xea0010ff irq 5 at device 14.0 on pci0 vr0: Ethernet address: 00:40:63:c2:49:3a However, I then get the dreaded: vr0: MII without any phy! device_probe_and_attach: vr0 attach returned 6 This seems to come up a number of times in the archives, but I don't see an explanation or solution to it. ENXIO implies the device isn't configured, yet I'm using GENERIC as my config at the moment and that has both vr and miibus configured into it. I would appreciate any pointers anyone can give. I'm quite prepared to do more research/hacking, I'd just like a push in the right direction :). Apologies for not providing a diff for the two files, but all that I've done is add these two lines to if_vrreg.h: #define VIA_DEVICEID_RHINE_III 0x3106 #define VIA_DEVICEID_RHINE_III_M 0x3053 (in the VIA Rhine device IDs section) and added these lines to the vr_devs[] struct in if_vr.c { VIA_VENDORID, VIA_DEVICEID_RHINE_III, "VIA VT6105 Rhine III 10/100BaseTX" }, { VIA_VENDORID, VIA_DEVICEID_RHINE_III_M, "VIA VT6105M Rhine III 10/100BaseTX" }, I've also found what looks to be good documentation for the chip on the VIA web site, so I should be able to track down appropriate information as necessary. Thanks! -- Greg Lewis Email : glewis@eyesbeyond.com Eyes Beyond Web : http://www.eyesbeyond.com Information Technology FreeBSD : glewis@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 4:50: 6 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A4E8637B405 for ; Thu, 30 Jan 2003 04:50:05 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 27A0543F85 for ; Thu, 30 Jan 2003 04:50:00 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0033.cvx21-bradley.dialup.earthlink.net ([209.179.192.33] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18eE8Y-0004x5-00; Thu, 30 Jan 2003 04:49:47 -0800 Message-ID: <3E391F12.D933AC3E@mindspring.com> Date: Thu, 30 Jan 2003 04:48:18 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Nicolas Mallet Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Changing the Maximum Segment Size (MSS) of Kame MIP6 Free BSD4.4 References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4128897a3b2e2234811bd07861ab4b735387f7b89c61deb1d350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Nicolas Mallet wrote: > Audsin wrote: > >I wish to change the Maximum segment size > >of the TCP. Can you please help me , where i should change the MSS of the > >TCP. Can you tell me where the default size of the MSS mentioned? > > $ sysctl -a | grep mss > net.inet.tcp.mssdflt: 512 > net.inet.tcp.v6mssdflt: 1024 > > Ex: > $ sysctl -w net.inet.tcp.v6mssdflt=new_value These don't work the way you'd think they work from their names. They are just initial (default) values, not bounds or limits. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 6:11:21 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1479937B401; Thu, 30 Jan 2003 06:11:20 -0800 (PST) Received: from sabre.velocet.net (sabre.velocet.net [216.138.209.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9063743F93; Thu, 30 Jan 2003 06:11:09 -0800 (PST) (envelope-from dgilbert@velocet.ca) Received: from trooper.velocet.ca (trooper.velocet.net [216.138.242.2]) by sabre.velocet.net (Postfix) with ESMTP id 799F513831D; Thu, 30 Jan 2003 09:11:03 -0500 (EST) Received: by trooper.velocet.ca (Postfix, from userid 66) id 1C32F7462A; Thu, 30 Jan 2003 09:11:03 -0500 (EST) Received: by canoe.velocet.net (Postfix, from userid 101) id DD523567628; Thu, 30 Jan 2003 06:48:14 -0500 (EST) From: David Gilbert MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15929.4350.816918.384371@canoe.velocet.net> Date: Thu, 30 Jan 2003 06:48:14 -0500 To: phk@freebsd.org Cc: "Matthew N. Dodd" , David Gilbert , freebsd-hackers@freebsd.org Subject: Re: Network block device. In-Reply-To: <16443.1043882006@critter.freebsd.dk> References: <20030129180043.S8642@sasami.jurai.net> <16443.1043882006@critter.freebsd.dk> X-Mailer: VM 7.07 under 21.1 (patch 14) "Cuyahoga Valley" XEmacs Lucid Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >>>>> "phk" == phk writes: phk> NBD wouldn't be hard to implement on FreeBSD, the easiest way phk> would be to write two GEOM modules to do it: a client and a phk> server. phk> No, I don't have time to do that right now, but I will happily phk> guide anybody who wants to try. I would be interested in knowing what you think would be required ... and some pointers. This sounds like a task I could bite off. Dave. -- ============================================================================ |David Gilbert, Velocet Communications. | Two things can only be | |Mail: dgilbert@velocet.net | equal if and only if they | |http://daveg.ca | are precisely opposite. | =========================================================GLO================ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 6:24:31 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5EB9337B401 for ; Thu, 30 Jan 2003 06:24:30 -0800 (PST) Received: from critter.freebsd.dk (esplanaden.cybercity.dk [212.242.40.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id 887E243F3F for ; Thu, 30 Jan 2003 06:24:29 -0800 (PST) (envelope-from phk@freebsd.org) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h0UEOAj1018288; Thu, 30 Jan 2003 15:24:28 +0100 (CET) (envelope-from phk@freebsd.org) To: David Gilbert Cc: "Matthew N. Dodd" , freebsd-hackers@freebsd.org Subject: Re: Network block device. From: phk@freebsd.org In-Reply-To: Your message of "Thu, 30 Jan 2003 06:48:14 EST." <15929.4350.816918.384371@canoe.velocet.net> Date: Thu, 30 Jan 2003 15:24:10 +0100 Message-ID: <18287.1043936650@critter.freebsd.dk> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <15929.4350.816918.384371@canoe.velocet.net>, David Gilbert writes: >>>>>> "phk" == phk writes: > >phk> NBD wouldn't be hard to implement on FreeBSD, the easiest way >phk> would be to write two GEOM modules to do it: a client and a >phk> server. > >phk> No, I don't have time to do that right now, but I will happily >phk> guide anybody who wants to try. > >I would be interested in knowing what you think would be required >... and some pointers. This sounds like a task I could bite off. First, find out what protocol you will use. If nothing else stay compatible with Linux (at least as an option if what they use suck too much). Then write a userland server and client so you have something to test against. Then write the kernel-client. There are two bits of this, one is the network part, one is the GEOM part. Julian proposed using netgraph for the network part, I'm not sure I agree with that idea, I think it would be more efficient to just make a kthread and go directly on the socket. I can't see what netgraph would add in this case, except a lot of code doing nothing but getting in the way. Once your kernel client can actually send a request and receive the reply, we will tack it onto GEOM. From there I expect doing the kernel server is pretty much the same, just the other way around :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 7: 3:27 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 98ABE37B401 for ; Thu, 30 Jan 2003 07:03:25 -0800 (PST) Received: from webmail1.isg.siue.edu (webmail1.isg.siue.edu [146.163.5.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7CB3043E4A for ; Thu, 30 Jan 2003 07:03:24 -0800 (PST) (envelope-from wgrim@siue.edu) Received: (from nobody@localhost) by webmail1.isg.siue.edu (8.11.6/8.11.6) id h0UF3Mx31155 for freebsd-hackers@freebsd.org; Thu, 30 Jan 2003 09:03:22 -0600 X-Authentication-Warning: webmail1.isg.siue.edu: nobody set sender to wgrim@siue.edu using -f Received: from 146.163.179.122 ( [146.163.179.122]) as user wgrim@146.163.5.4 by webmail1.isg.siue.edu with HTTP; Thu, 30 Jan 2003 09:03:22 -0600 Message-ID: <1043939002.3e393ebaaf94a@webmail1.isg.siue.edu> Date: Thu, 30 Jan 2003 09:03:22 -0600 From: wgrim@siue.edu To: freebsd-hackers@freebsd.org Subject: Re: Network block device. References: <18287.1043936650@critter.freebsd.dk> In-Reply-To: <18287.1043936650@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: Internet Messaging Program (IMP) 3.1 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I haven't been following this thread too closely, but I was hoping you could clarify something for me. For what does GEOM mean/stand? Thanks for the clarification, Mike Quoting phk@FreeBSD.ORG: > In message <15929.4350.816918.384371@canoe.velocet.net>, David Gilbert > writes: > >>>>>> "phk" == phk writes: > > > >phk> NBD wouldn't be hard to implement on FreeBSD, the easiest way > >phk> would be to write two GEOM modules to do it: a client and a > >phk> server. > > > >phk> No, I don't have time to do that right now, but I will happily > >phk> guide anybody who wants to try. > > > >I would be interested in knowing what you think would be required > >... and some pointers. This sounds like a task I could bite off. > > First, find out what protocol you will use. If nothing else stay > compatible with Linux (at least as an option if what they use suck > too much). > > Then write a userland server and client so you have something to > test against. > > Then write the kernel-client. There are two bits of this, one is > the network part, one is the GEOM part. > > Julian proposed using netgraph for the network part, I'm not sure > I agree with that idea, I think it would be more efficient to just > make a kthread and go directly on the socket. I can't see what > netgraph would add in this case, except a lot of code doing nothing > but getting in the way. > > Once your kernel client can actually send a request and receive > the reply, we will tack it onto GEOM. > > >From there I expect doing the kernel server is pretty much the > same, just the other way around :-) > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetence. > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > ------------------------------------------------- SIUE Web Mail To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 7: 4:14 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8DA8437B401 for ; Thu, 30 Jan 2003 07:04:12 -0800 (PST) Received: from mail.eecs.harvard.edu (bowser.eecs.harvard.edu [140.247.60.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2D60443F85 for ; Thu, 30 Jan 2003 07:04:08 -0800 (PST) (envelope-from ellard@eecs.harvard.edu) Received: by mail.eecs.harvard.edu (Postfix, from userid 465) id 7E2E654C446; Thu, 30 Jan 2003 10:03:55 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mail.eecs.harvard.edu (Postfix) with ESMTP id 7B89654C43F; Thu, 30 Jan 2003 10:03:55 -0500 (EST) Date: Thu, 30 Jan 2003 10:03:55 -0500 (EST) From: Daniel Ellard To: freebsd-hackers@freebsd.org Cc: Audsin Subject: Re: Changing the Maximum Segment Size (MSS) of Kame MIP6 Free BSD4.4 Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Terry Lambert wrote: > Audsin wrote: > > I am Dev, doing my research in Centre for Telecommunications Research, > > King's college London. My research project involves evaluating the > > performance of MIP6 TCP in the presence of fragmentation and without > > fragmentation. I am using Kame MIP6 for Free BSD 4.4 and have configured > > gif0 interface for ipv6ip tunnel. I wish to change the Maximum segment size > > of the TCP. Can you please help me , where i should change the MSS of the > > TCP. Can you tell me where the default size of the MSS mentioned? > > man ifconfig > /mtu The original question asked about the TCP MSS, not the MTU. Looking at ifconfig isn't going to help. I don't have the 4.4 source sitting around, but assuming that it hasn't changed much, in 4.6.2 you can find (or change) the defaults in /usr/src/sys/netinet/tcp.h. The default for TCP4 is 512, and 1024 for TCP6. You can get at them via sysctl (along with a lot of other TCP-related things): net.inet.tcp.mssdflt: 512 net.inet.tcp.v6mssdflt: 1024 Note that the actual MSS is negotiated; if both ends can't support the same value, the smaller is chosen. For MTUs, in case that's really what you meant, it's even more strict and depending on how the transport layer is implemented, it may be impossible (or reckless) to increase the MTU. For example, on ethernet many switches simply do not support an MTU larger than 1500. -Dan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 7: 7:40 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB44F37B401 for ; Thu, 30 Jan 2003 07:07:39 -0800 (PST) Received: from critter.freebsd.dk (esplanaden.cybercity.dk [212.242.40.114]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1700A43F43 for ; Thu, 30 Jan 2003 07:07:39 -0800 (PST) (envelope-from phk@freebsd.org) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.12.6/8.12.6) with ESMTP id h0UF7Xj1018888; Thu, 30 Jan 2003 16:07:33 +0100 (CET) (envelope-from phk@freebsd.org) To: wgrim@siue.edu Cc: freebsd-hackers@freebsd.org Subject: Re: Network block device. From: phk@freebsd.org In-Reply-To: Your message of "Thu, 30 Jan 2003 09:03:22 CST." <1043939002.3e393ebaaf94a@webmail1.isg.siue.edu> Date: Thu, 30 Jan 2003 16:07:33 +0100 Message-ID: <18887.1043939253@critter.freebsd.dk> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <1043939002.3e393ebaaf94a@webmail1.isg.siue.edu>, wgrim@siue.edu wri tes: >I haven't been following this thread too closely, but I was hoping you could >clarify something for me. For what does GEOM mean/stand? GEOM is basically our disk-I/O subsystem at this point. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 7:22:19 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 85E5537B401; Thu, 30 Jan 2003 07:22:18 -0800 (PST) Received: from gw.nectar.cc (gw.nectar.cc [208.42.49.153]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1372943F79; Thu, 30 Jan 2003 07:22:18 -0800 (PST) (envelope-from nectar@celabo.org) Received: from opus.celabo.org (opus.celabo.org [10.0.1.111]) by gw.nectar.cc (Postfix) with ESMTP id 86F824; Thu, 30 Jan 2003 09:22:17 -0600 (CST) Received: by opus.celabo.org (Postfix, from userid 1001) id 1EA6C59AC; Thu, 30 Jan 2003 09:20:26 -0600 (CST) Date: Thu, 30 Jan 2003 09:20:26 -0600 From: "Jacques A. Vidrine" To: Vladimir Terziev Cc: hackers@FreeBSD.ORG, security@freebsd.org Subject: Re: Kerberos & OpenSSH+GSSAPI problem Message-ID: <20030130152025.GB73428@opus.celabo.org> References: <20030130114401.38eeffa2.vlady@sun-fish.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030130114401.38eeffa2.vlady@sun-fish.com> X-Url: http://www.celabo.org/ User-Agent: Mutt/1.5.1i-ja.1 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG You mailed me personally; you mailed the MIT Kerberos list; and you cross-posted on (at least) two FreeBSD mailing lists, all at approximately the same time. Please don't do that: it is rude. At least wait for replies in one area before launching into another. Cheers, -- Jacques A. Vidrine http://www.celabo.org/ NTT/Verio SME . FreeBSD UNIX . Heimdal Kerberos jvidrine@verio.net . nectar@FreeBSD.org . nectar@kth.se To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 8: 2:20 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 524CB37B401 for ; Thu, 30 Jan 2003 08:02:19 -0800 (PST) Received: from out6.mx.nwbl.wi.voyager.net (out6.mx.nwbl.wi.voyager.net [169.207.3.77]) by mx1.FreeBSD.org (Postfix) with ESMTP id 87B5D43E4A for ; Thu, 30 Jan 2003 08:02:18 -0800 (PST) (envelope-from silby@silby.com) Received: from [10.1.1.6] (d100.as2.nwbl0.wi.voyager.net [169.207.92.100]) by out6.mx.nwbl.wi.voyager.net (Postfix) with ESMTP id CDC9BDBEDE; Thu, 30 Jan 2003 09:59:22 -0600 (CST) Date: Thu, 30 Jan 2003 10:07:23 -0600 (CST) From: Mike Silbersack To: Greg Lewis Cc: freebsd-hackers@freebsd.org Subject: Re: VIA Rhine III (MII without any phy!) In-Reply-To: <20030130221623.A21565@misty.eyesbeyond.com> Message-ID: <20030130100410.A10212-100000@patrocles.silby.com> References: <20030130221623.A21565@misty.eyesbeyond.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thu, 30 Jan 2003, Greg Lewis wrote: > Hi all, > > I've recently acquired a motherboard with a VIA Rhine III onboard NIC. > I couldn't see support for this in the current vr(4) driver in either > -current or -stable and am trying to add support for it. Having not > done any kernel hacking before I'd appreciate some pointers if someone > would be so kind. Thomas Nystrom has found the necessary fixes to our driver, I should be committing them this weekend (and MFCing them to 4.x soon after that.) Soo... wait a few days, and all will be good. :) > I've also found what looks to be good documentation for the chip on the > VIA web site, so I should be able to track down appropriate information > as necessary. Sorry to be pedantic, but there's a problem with that statement. Via's documentation isn't "good", there are many tiny glitches they don't list. Nonetheless, they are far better than Nvidia, who isn't releasing any specs for their NIC chipset... Mike "Silby" Silbersack To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 8:55:47 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DD04137B401 for ; Thu, 30 Jan 2003 08:55:45 -0800 (PST) Received: from gatekeeper.oremut01.us.wh.verio.net (gatekeeper.oremut01.us.wh.verio.net [198.65.168.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2A93A43F75 for ; Thu, 30 Jan 2003 08:55:45 -0800 (PST) (envelope-from gritton@iserver.com) Received: from guppy.dmz.orem.verio.net (guppy.dmz.orem.verio.net [10.1.1.55]) by gatekeeper.oremut01.us.wh.verio.net (Postfix) with ESMTP id E55193BF11C for ; Thu, 30 Jan 2003 09:55:43 -0700 (MST) Received: from guppy.dmz.orem.verio.net by guppy.dmz.orem.verio.net; Thu, 30 Jan 2003 09:55:43 -0700 (MST) Received: (from gritton@localhost) by guppy.dmz.orem.verio.net (8.12.3/8.12.3/Submit) id h0UGthYh027119; Thu, 30 Jan 2003 09:55:43 -0700 (MST) To: freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? References: <20030130064448.GA7258@HAL9000.homeunix.com> <200301300719.h0U7JOfI086054@apollo.backplane.com> From: James Gritton Date: 30 Jan 2003 09:55:42 -0700 In-Reply-To: Matthew Dillon's message of "Wed, 29 Jan 2003 23:19:24 -0800 (PST)" Message-ID: Lines: 27 X-Mailer: Gnus v5.7/Emacs 20.7 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon writes: > You can also theoretically push into shadow VM objects to locate > pages from the parent process that have not yet been COW'd into the > child (in the case of a fork()), noting also that these shadow objects > might be shared with other children of the parent and by the parent > process itself, but under most conditions this information will not > be significant and can be ignored. Actually, that's just the sort of thing I'm looking for. The shared case may be relatively rare, but it tends to be extreme: the processes that use the most memory seem to be the ones that fork the most - database servers, java, etc. The point of this whole thing is an attempt to limit the memory use of a user (instead of a process), but I don't want to penalize such sharing. > Any vnode object is always shared with other processes mapping the > same vnode. Since this information is backed by a file, figuring out > how much 'memory' it represents by any reasonable definition is > guesswork. The resident page count will represent how much of the > vnode is cached, but not how much of the vnode is actually being accessed > by the process. That's OK - resident count is what I'm interested in. That, and the swap approximation (which should suffice) you mentioned for non-vnode objects. - Jamie To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 11:23:33 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B75E437B405 for ; Thu, 30 Jan 2003 11:23:31 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 78BBB43E4A for ; Thu, 30 Jan 2003 11:23:30 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0UJNT0i089038; Thu, 30 Jan 2003 11:23:30 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0UJNT0l089037; Thu, 30 Jan 2003 11:23:29 -0800 (PST) Date: Thu, 30 Jan 2003 11:23:29 -0800 (PST) From: Matthew Dillon Message-Id: <200301301923.h0UJNT0l089037@apollo.backplane.com> To: David Schultz Cc: James Gritton , freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? References: <20030130064448.GA7258@HAL9000.homeunix.com> <200301300719.h0U7JOfI086054@apollo.backplane.com> <20030130091419.GA7776@HAL9000.homeunix.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :I think the original poster wanted to know the real memory use of :a set of processes, taking sharing into account. I don't see how :your approach could do that. Even if you knew the structure of :the shadow chain, you would have to know specifically which pages :had been COWed, no? : :I thought counting vm_page structures would be the way to go... :I really want to find the time to learn all this stuff better. :I still don't know the difference between COW and NEEDS_COPY, :or why the pageout daemon seems to be biased against shared :pages, or a lot of things about pv_entry structures. It's not possible to get a wholely accurate count no matter what you do. For example, to truely know whether a process is using a page you have to run through the process's page table (PMAP), access the vm_page, then locate where in the shadow chain the VM object the vm_page belongs to resides. But since hardware page tables are throw-away, the system could very well have thrown away whole page tables so this method is no more accurate then any other. Shadow VM objects are optimized on the fly. When a shadow object representing data shared by multiple processes is completely covered (due to COW faults) the system will delete the shadow object. So you can get a pretty good idea in regards to shared pages simply by noting the existance of the shadow object and the number of resident or swap pages residing in that object. MAP_ENTRY_NEEDS_COPY is a vm_map_entry optimization that allows two vm_map_entry's (for two different processes) to share the same vm_object even though they are copy-on-write. This allows us to defer allocating new VM objects (defer creating the shadow structures) for anonymous area of memory when a process fork()s, until an actual copy-on-write occurs. Being able to defer this allocation greatly reduces the time and memory required to fork() and gives us a flatter, more easily optimized VM object structure. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 11:41:39 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DFC0737B401 for ; Thu, 30 Jan 2003 11:41:37 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 70D4F43F43 for ; Thu, 30 Jan 2003 11:41:37 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0UJfb0i089231; Thu, 30 Jan 2003 11:41:37 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0UJfa0e089230; Thu, 30 Jan 2003 11:41:36 -0800 (PST) Date: Thu, 30 Jan 2003 11:41:36 -0800 (PST) From: Matthew Dillon Message-Id: <200301301941.h0UJfa0e089230@apollo.backplane.com> To: James Gritton Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? References: <20030130064448.GA7258@HAL9000.homeunix.com> <200301300719.h0U7JOfI086054@apollo.backplane.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I just had another idea on how you could do this. It's actually not all that complex. All you do is collect statistics on each VM object individually (whether shadow or top level or whatever). e.g. Resident page count, swap usage. Then you collect a list of VM objects associated with a process, regardless of HOW they are associated. Recurse through the vm_map_entry's and the shadow chains to get the list. Then you simply group all the processes which share VM objects together and report statistics on a group-by-group basis rather then on a process-by-process basis. You won't know what an individual process uses but you know exactly what the group of processes use in aggregate. Groups of processes will tend to form due to fork()ing, processes sharing memory via SysV shared memory, and processes that happen to be mapping the same file (vnode). You will get accurate memory use for each group. -Matt : Actually, that's just the sort of thing I'm looking for. The shared case :may be relatively rare, but it tends to be extreme: the processes that use :the most memory seem to be the ones that fork the most - database servers, :java, etc. The point of this whole thing is an attempt to limit the memory :use of a user (instead of a process), but I don't want to penalize such :sharing. : :.. : That's OK - resident count is what I'm interested in. That, and the swap :approximation (which should suffice) you mentioned for non-vnode objects. : :- Jamie To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 11:51:47 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 949A337B401 for ; Thu, 30 Jan 2003 11:51:46 -0800 (PST) Received: from gatekeeper.oremut01.us.wh.verio.net (gatekeeper.oremut01.us.wh.verio.net [198.65.168.16]) by mx1.FreeBSD.org (Postfix) with ESMTP id 000C743F75 for ; Thu, 30 Jan 2003 11:51:45 -0800 (PST) (envelope-from gritton@iserver.com) Received: from guppy.dmz.orem.verio.net (guppy.dmz.orem.verio.net [10.1.1.55]) by gatekeeper.oremut01.us.wh.verio.net (Postfix) with ESMTP id D3D513BF292 for ; Thu, 30 Jan 2003 12:51:45 -0700 (MST) Received: from guppy.dmz.orem.verio.net by guppy.dmz.orem.verio.net; Thu, 30 Jan 2003 12:51:45 -0700 (MST) Received: (from gritton@localhost) by guppy.dmz.orem.verio.net (8.12.3/8.12.3/Submit) id h0UJpjI8027789; Thu, 30 Jan 2003 12:51:45 -0700 (MST) To: freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? References: <20030130064448.GA7258@HAL9000.homeunix.com> <200301300719.h0U7JOfI086054@apollo.backplane.com> <200301301941.h0UJfa0e089230@apollo.backplane.com> From: James Gritton Date: 30 Jan 2003 12:51:45 -0700 In-Reply-To: Matthew Dillon's message of "Thu, 30 Jan 2003 11:41:36 -0800 (PST)" Message-ID: Lines: 15 X-Mailer: Gnus v5.7/Emacs 20.7 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon writes: > Then you simply group all the processes which share VM > objects together and report statistics on a group-by-group basis > rather then on a process-by-process basis. You won't know what an > individual process uses but you know exactly what the group of > processes use in aggregate. Yes, that sounds good. I've got a jail-like setup, so the process groups I'm interested in should be nearly identical to those sharing objects. I don't need to know individual process usage, so this seems to be just what I'm looking for. Now on to some actual coding :-). - Jamie To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 13:18:58 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 226B937B401 for ; Thu, 30 Jan 2003 13:18:57 -0800 (PST) Received: from warez.scriptkiddie.org (uswest-dsl-142-38.cortland.com [209.162.142.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id F187243FAF for ; Thu, 30 Jan 2003 13:18:49 -0800 (PST) (envelope-from lamont@scriptkiddie.org) Received: from [192.168.69.11] (unknown [192.168.69.11]) by warez.scriptkiddie.org (Postfix) with ESMTP id 3FEEB62D1A; Thu, 30 Jan 2003 13:18:44 -0800 (PST) Date: Thu, 30 Jan 2003 13:18:44 -0800 (PST) From: Lamont Granquist To: "Matthew N. Dodd" Cc: David Gilbert , Subject: Re: Network block device. In-Reply-To: <20030129180043.S8642@sasami.jurai.net> Message-ID: <20030130131600.Q823-100000@coredump.scriptkiddie.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 29 Jan 2003, Matthew N. Dodd wrote: > What you really want is SCSI over IP. Anything else is just a hack and > not to be trusted. And iSCSI isn't? > I think that NFS is less of a hack than NBD though. > Of course if Linux still suffers from poor NFS performance that might > explain why they came up with NBD in the first place. And Linux still suffers from poor NFS stability. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 14:22:48 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 564A637B401 for ; Thu, 30 Jan 2003 14:22:45 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id D479543F79 for ; Thu, 30 Jan 2003 14:22:44 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0UMMg0i090350; Thu, 30 Jan 2003 14:22:42 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0UMMfFI090349; Thu, 30 Jan 2003 14:22:41 -0800 (PST) Date: Thu, 30 Jan 2003 14:22:41 -0800 (PST) From: Matthew Dillon Message-Id: <200301302222.h0UMMfFI090349@apollo.backplane.com> To: "Brian T. Schellenberger" Cc: kientzle@acm.org, Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301270019.44066.bschellenberger@nc.rr.com> <3E34C734.8010801@acm.org> <200301270904.43899.bschellenberger@nc.rr.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Well, here's a counterpoint. Lets say you have an FTP server with 1G of ram full of, say, pirated CDs at 600MB a pop. Now lets say someone puts up a new madonna CD and suddenly you have thousands of people from all over the world trying to download a single 600MB file. Lets try another one. Lets say you have an FTP server with 1G of ram full of hundreds of MPEG encoded pirated CDs at 50MB a pop and you have thousands of people from all over the world trying to download a core set of 25 CDs, which exceeds the available ram you have to cache all of them. What I'm trying to illustrate here is the impossibility of what you are asking. Your idea of 'sequential' access cache restriction only works if there is just one process doing the accessing. But if you have 25 processes accessing 25 different files sequentially it doesn't work, and how is the system supposed to detect the difference between 25 processes accessing 25 50MB files on a 1G machine (which doesn't fit in the cache) verses 300 processes accessing 15 50MB files on a 1G machine (which does fit). Furthermore, how do you differentiate between 30 processes all downloading the same 600MB CD verses 30 processes downloading two different 600MB CD's, on a machine with 1G of cache? You can't. That's the problem. There is no magic number between 0 and the amount of memory you have where you can say "I am going to stop caching this sequential file" that covers even the more common situations that come up. There is no algorithm that can detect the above situations before the fact or on the fly. You can analyize the situation after the fact, but by then it is too late, and the situation may change from minute to minute. One minute you have 300 people trying to download one CD, the next minute you have 20 people trying to download 10 different CD's. -Matt :The suggestion here basically boils down to this: if the system could :act on hints that somebody will be doing sequential access, then it :should be more timid about caching for that file access. That is to :say, it should allow that file to "use up" a smaller number of blocks :from the cache (yes, the VM) at a time, and it should favor, if :anything, a LIFO scheme instead of the usual FIFO (LRU) scheme. (That :is to say, for the special case of *sequential* access, LRU == FIFO, :and yet LIFO is probably more optimal for this case, at least if the :file will be re-read later.) : :Caching will do more good on files that that will be randomly accessed; :an intermediate amount of good on files sequentially accessed but :rewound and/or accessed over and over, and if the file system could :somehow know (or be hinted) that the file is being sequentially :accessed and is unlikely to be accessed again for a good long while it :would clearly be better off not caching it at all. : :Of course the trick here is waving my hands and saying "assume that you :know how the file will be accessed in the future." You ought to :pillory me for *that* bit. Even with hinting there are problems with :this whole idea. Still with some hinting the algorithm could probably :be a little more clever. : :(Actually, Terry Lambert *did* pillory me for that bit, just a bit, when :he pointed out the impossibility of knowing whether the file is being :used in the same way by other processes.) : :And . . . also to Terry, yes, I know that my proposal about :over-simplifies, but the point is that for sequential access you want :to go "gentle" on making the cache of other process' and earlier reads :go away. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 14:43:29 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BACE637B401 for ; Thu, 30 Jan 2003 14:43:27 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id EEF0243F9B for ; Thu, 30 Jan 2003 14:43:26 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0073.cvx22-bradley.dialup.earthlink.net ([209.179.198.73] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18eNOv-0000qC-00; Thu, 30 Jan 2003 14:43:18 -0800 Message-ID: <3E39AA2D.FCD12EC6@mindspring.com> Date: Thu, 30 Jan 2003 14:41:49 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Daniel Ellard Cc: freebsd-hackers@freebsd.org, Audsin Subject: Re: Changing the Maximum Segment Size (MSS) of Kame MIP6 Free BSD4.4 References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4ad621738ff9386f286c9c773bfa224c7350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Daniel Ellard wrote: > > man ifconfig > > /mtu > > The original question asked about the TCP MSS, not the MTU. Looking > at ifconfig isn't going to help. Actually, the original question is about how to cause the creation of fragments, for the purposes of testing. The MSS question is kind of based on a loose assumption about where baby frags come from, which isn't quite correct. > Note that the actual MSS is negotiated; if both ends can't support the > same value, the smaller is chosen. This is the problem. The MSS can be negotiated up to the MTU size from the default, or down, to match an intermediate hop. The only thing changing the default will do for you is increase the negotiation time. > For MTUs, in case that's really what you meant, it's even more strict > and depending on how the transport layer is implemented, it may be > impossible (or reckless) to increase the MTU. For example, on > ethernet many switches simply do not support an MTU larger than 1500. This is the one thing he *can* hard-code, via ifconfig. MSS is not hard-codeable. There's an outside chance that he wanted the MTU larger (e.g. 9k for "jumbograms" on gigabit hardware). Actually, hardware MSS negotiation doesn't work between some gigabit cards, and has to be set manually on both ends to get the higher number. But since he was talking about fargs vs. no frags, it's a good bet that what he wanted was: MTU X MTU X/3 host1 <----------------> router <----------------> host 2 So that frags are created when transmitting data from host1 to host2. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 14:52:49 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 030ED37B401; Thu, 30 Jan 2003 14:52:48 -0800 (PST) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0D89643F43; Thu, 30 Jan 2003 14:52:47 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0073.cvx22-bradley.dialup.earthlink.net ([209.179.198.73] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18eNY1-00022R-00; Thu, 30 Jan 2003 14:52:42 -0800 Message-ID: <3E39AC64.5F0B763E@mindspring.com> Date: Thu, 30 Jan 2003 14:51:16 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: phk@freebsd.org Cc: wgrim@siue.edu, freebsd-hackers@freebsd.org Subject: Re: Network block device. References: <18887.1043939253@critter.freebsd.dk> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4d33235a1e39dd611df6ce6d8022a6978350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG In message <1043939002.3e393ebaaf94a@webmail1.isg.siue.edu>, wgrim@siue.edu writes: > I haven't been following this thread too closely, but I was hoping you could > clarify something for me. For what does GEOM mean/stand? GEOM is not an acronym, even though the last three letters are uppercase, as they would be with an acronym, like "LASER" or "RADAR", or a trademark, like "UNIX"), and technically, it should be written "Geom", instead. GEOM is the geometry manager that sits on top of the disk I/O subsystem. It's what handles partitions, slices, and other abstractions of the physical devices. In theory, it should be where CCD and other things, like RaidFrame and Vinum, perform volume management. It's very similar to what I suggested be done back in 1995/1996, except that I also suggested that the writing/removal of things like partition tables, disklabels, etc., take place via an ioctl to the device manager, and that a "partition type iterator" be created, so that you could ask the kernel what was supported, and get some metadata back from it. If that were done, then you would be able to replace all the disklabel, parted, and other programs with a single program that operated on metadata, and would not need modification as new GEOM modules were created. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 15:12:58 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 52D6037B408 for ; Thu, 30 Jan 2003 15:12:55 -0800 (PST) Received: from ms-smtp-03.southeast.rr.com (ms-smtp-03.southeast.rr.com [24.93.67.84]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6F68743F75 for ; Thu, 30 Jan 2003 15:12:49 -0800 (PST) (envelope-from bts@fake.com) Received: from mail5.nc.rr.com (fe5 [24.93.67.52]) by ms-smtp-03.southeast.rr.com (8.12.5/8.12.2) with ESMTP id h0UNBni6006928; Thu, 30 Jan 2003 18:11:49 -0500 (EST) Received: from this.is.fake.com ([24.74.172.220]) by mail5.nc.rr.com with Microsoft SMTPSVC(5.5.1877.757.75); Thu, 30 Jan 2003 18:10:56 -0500 Received: by this.is.fake.com (Postfix, from userid 111) id A191ABBC8; Thu, 30 Jan 2003 18:12:20 -0500 (EST) Content-Type: text/plain; charset="iso-8859-1" From: "Brian T. Schellenberger" To: Matthew Dillon Subject: Re: Random disk cache expiry Date: Thu, 30 Jan 2003 18:12:20 -0500 User-Agent: KMail/1.4.2 Cc: kientzle@acm.org, Sean Hamilton , hackers@freebsd.org References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301270904.43899.bschellenberger@nc.rr.com> <200301302222.h0UMMfFI090349@apollo.backplane.com> In-Reply-To: <200301302222.h0UMMfFI090349@apollo.backplane.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200301301812.20358.bschellenberger@nc.rr.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thursday 30 January 2003 05:22 pm, Matthew Dillon wrote: | Well, here's a counterpoint. Lets say you have an FTP | server with 1G of ram full of, say, pirated CDs at 600MB a | pop. | | Now lets say someone puts up a new madonna CD and suddenly | you have thousands of people from all over the world trying | to download a single 600MB file. | | Lets try another one. Lets say you have an FTP server with | 1G of ram full of hundreds of MPEG encoded pirated CDs at | 50MB a pop and you have thousands of people from all over the | world trying to download a core set of 25 CDs, which exceeds | the available ram you have to cache all of them. | | What I'm trying to illustrate here is the impossibility of | what you are asking. Your idea of 'sequential' access cache | restriction only works if there is just one process doing the | accessing. But if you have 25 processes accessing 25 different | files sequentially it doesn't work, and how is the system supposed to | detect the difference between 25 processes accessing 25 50MB files on | a 1G machine (which doesn't fit in the cache) verses 300 processes | accessing 15 50MB files on a 1G machine (which does fit). | Furthermore, how do you differentiate between 30 processes all | downloading the same 600MB CD verses 30 processes downloading two | different 600MB CD's, on a machine with 1G of cache? | | You can't. That's the problem. There is no magic number between | 0 and the amount of memory you have where you can say "I am going | to stop caching this sequential file" that covers even the more | common situations that come up. There is no algorithm that can | detect the above situations before the fact or on the fly. You | can analyize the situation after the fact, but by then it is too | late, and the situation may change from minute to minute. One minute | you have 300 people trying to download one CD, the next minute you | have 20 people trying to download 10 different CD's. | | -Matt You are absolutely right, and thank you for fullfilling my request to pillory me for that bit: | :Of course the trick here is waving my hands and saying "assume that | : you know how the file will be accessed in the future." You ought | : to pillory me for *that* bit. Even with hinting there are problems | : with this whole idea. Still with some hinting the algorithm could | : probably be a little more clever. | : | :(Actually, Terry Lambert *did* pillory me for that bit, just a bit, | : when he pointed out the impossibility of knowing whether the file | : is being used in the same way by other processes.) Though actually that was awful gentle for a pillory. All that said, there are *always* pathological cases for any algorithm; it is my intuition that somehow giving cache retention "priority" to processes that randomly access files would be likely to be an overall win. Not that I'm planning to put my money where my mouth is and write some code to test this contention. If you wanted to get *really* fancy you could add some code that tracked hits for given blocks over time and increased priority for blocks that were brought in often "recently" -- sort of an extended LRU list that wouldn't map a whole block in but would tell you what you'd tossed out lately and you could give priorty to blocks that were out of the "list cache" (or whatever we would call this secondary list of recently-used blocks). Um, in re-reading that paragraph it seems clear as mud . . . Whether any of these schemes would be of any practical benefit in any common situations is highly dubious; they wuld all increase overhead for one thing. I think that the definitive answer, really, is the one you gave earlier: If you have a specialized application or server which you know will make intense use of the file system in a predictable way, then you should customize the way it accesses the files because no general-purpose algorithm can ever be optimal for all cases. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 16: 7:17 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 17D6437B401 for ; Thu, 30 Jan 2003 16:07:17 -0800 (PST) Received: from mx5.mail.ru (mx5.mail.ru [194.67.57.15]) by mx1.FreeBSD.org (Postfix) with ESMTP id E7A0243F43 for ; Thu, 30 Jan 2003 16:07:15 -0800 (PST) (envelope-from shmukler@mail.ru) Received: from f2.int ([10.0.0.49] helo=f2.mail.ru) by mx5.mail.ru with esmtp (Exim MX.5) id 18eOi9-0000uf-00 for freebsd-hackers@freebsd.org; Fri, 31 Jan 2003 03:07:13 +0300 Received: from mail by f2.mail.ru with local (Exim FE.1) id 18eOi8-0005Yo-00 for freebsd-hackers@freebsd.org; Fri, 31 Jan 2003 03:07:12 +0300 Received: from [162.84.177.68] by win.mail.ru with HTTP; Fri, 31 Jan 2003 03:07:12 +0300 From: "Igor Shmukler" To: freebsd-hackers@freebsd.org Subject: random cache expiry Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: [162.84.177.68] Date: Fri, 31 Jan 2003 03:07:12 +0300 Reply-To: "Igor Shmukler" Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit Message-Id: Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I partially missed the discussion, but why would anyone want to implement random expiration, when there are better method to deal with issue. Including UBM which was already imemented in FreeBSD 2.2.8. (http://www.usenix.org/events/osdi2000/full_papers/kim/kim_html/) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 16: 7:44 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7E4E437B401 for ; Thu, 30 Jan 2003 16:07:42 -0800 (PST) Received: from kientzle.com (h-66-166-149-50.SNVACAID.covad.net [66.166.149.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 94D0843F79 for ; Thu, 30 Jan 2003 16:07:40 -0800 (PST) (envelope-from kientzle@acm.org) Received: from acm.org (big.x.kientzle.com [66.166.149.54]) by kientzle.com (8.11.3/8.11.3) with ESMTP id h0V074R31167; Thu, 30 Jan 2003 16:07:04 -0800 (PST) (envelope-from kientzle@acm.org) Message-ID: <3E39BE22.8050207@acm.org> Date: Thu, 30 Jan 2003 16:06:58 -0800 From: Tim Kientzle Reply-To: kientzle@acm.org User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:0.9.6) Gecko/20011206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Matthew Dillon Cc: "Brian T. Schellenberger" , Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301270019.44066.bschellenberger@nc.rr.com> <3E34C734.8010801@acm.org> <200301270904.43899.bschellenberger@nc.rr.com> <200301302222.h0UMMfFI090349@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon wrote: > Your idea of 'sequential' access cache restriction only > works if there is just one process doing the accessing. Not necessarily. I suspect that there is a strong tendency to access particular files in particular ways. E.g., in your example of a download server, those files are always read sequentially. You can make similar assertions about a lot of files: manpages, gzip files, C source code files, etc, are "always" read sequentially. If a file's access history were stored as a "hint" associated with the file, then it would be possible to make better up-front decisions about how to allocate cache space. The ideal would be to store such hints on disk (maybe as an extended attribute?), but it might also be useful to cache them in memory somewhere. That would allow the cache-management code to make much earlier decisions about how to handle a file. For example, if a process started to read a 10GB file that has historically been accessed sequentially, you could immediately decide to enable read-ahead for performance, but also mark those pages to be released as soon as they were read by the process. FWIW, a web search for "randomized caching" yields some interesting reading. Apparently, there are a few randomized cache-management algorithms for which the mathematics work out reasonably well, despite Terry's protestations to the contrary. ;-) I haven't yet found any papers describing experiences with real implementations, though. If only I had the time to spend poring over FreeBSD's cache-management code to see how these ideas might actually be implemented... Tim Kientzle To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 16:14:47 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0808C37B405 for ; Thu, 30 Jan 2003 16:14:46 -0800 (PST) Received: from HAL9000.homeunix.com (12-233-57-224.client.attbi.com [12.233.57.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id D552743F75 for ; Thu, 30 Jan 2003 16:14:39 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.6/8.12.5) with ESMTP id h0V0EbNt010911; Thu, 30 Jan 2003 16:14:37 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.6/8.12.5/Submit) id h0V0EaHm010910; Thu, 30 Jan 2003 16:14:36 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Date: Thu, 30 Jan 2003 16:14:36 -0800 From: David Schultz To: Matthew Dillon Cc: James Gritton , freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? Message-ID: <20030131001436.GA10856@HAL9000.homeunix.com> Mail-Followup-To: Matthew Dillon , James Gritton , freebsd-hackers@FreeBSD.ORG References: <20030130064448.GA7258@HAL9000.homeunix.com> <200301300719.h0U7JOfI086054@apollo.backplane.com> <20030130091419.GA7776@HAL9000.homeunix.com> <200301301923.h0UJNT0l089037@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200301301923.h0UJNT0l089037@apollo.backplane.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thus spake Matthew Dillon : > It's not possible to get a wholely accurate count no matter what > you do. For example, to truely know whether a process is using > a page you have to run through the process's page table (PMAP), > access the vm_page, then locate where in the shadow chain the VM object > the vm_page belongs to resides. But since hardware page tables are > throw-away, the system could very well have thrown away whole page > tables so this method is no more accurate then any other. Thanks for the explanations! I still don't understand why this doesn't work, assuming you don't care about nonresident pages: for each process p in the set for each map entry e in p->vmspace->vm_map for each page m in e->object.vm_object->memq if I haven't seen this m.phys_addr yet in the scan resident_pages++ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 16:24: 1 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9708037B401 for ; Thu, 30 Jan 2003 16:23:59 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0630F43F85 for ; Thu, 30 Jan 2003 16:23:59 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0V0Nr0i090964; Thu, 30 Jan 2003 16:23:53 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0V0NqAE090963; Thu, 30 Jan 2003 16:23:52 -0800 (PST) Date: Thu, 30 Jan 2003 16:23:52 -0800 (PST) From: Matthew Dillon Message-Id: <200301310023.h0V0NqAE090963@apollo.backplane.com> To: David Schultz Cc: James Gritton , freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? References: <20030130064448.GA7258@HAL9000.homeunix.com> <200301300719.h0U7JOfI086054@apollo.backplane.com> <20030130091419.GA7776@HAL9000.homeunix.com> <200301301923.h0UJNT0l089037@apollo.backplane.com> <20030131001436.GA10856@HAL9000.homeunix.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :Thus spake Matthew Dillon : :> It's not possible to get a wholely accurate count no matter what :> you do. For example, to truely know whether a process is using :> a page you have to run through the process's page table (PMAP), :> access the vm_page, then locate where in the shadow chain the VM object :> the vm_page belongs to resides. But since hardware page tables are :> throw-away, the system could very well have thrown away whole page :> tables so this method is no more accurate then any other. : :Thanks for the explanations! I still don't understand why this :doesn't work, assuming you don't care about nonresident pages: : :for each process p in the set : for each map entry e in p->vmspace->vm_map : for each page m in e->object.vm_object->memq : if I haven't seen this m.phys_addr yet in the scan : resident_pages++ That would get close, as long as the machine is not paging heavily. Think of it this way: If you have a lot of ram the above calculation will give you an upper bound on memory use, but some of the pages in the VM object's may be very old and not actually under active access by the process (for example, the pages might represent part of the program that was used during initialization and then never used again). If you do not have so much memory older pages will get flushed out or flushed to swap and the above calculation will represent more of a lower bound on the memory used by the group of processes. Consider a database. A database might be mapping a large table file and, if you have lots of memory, most of that table file might be cached (in the VM object's memq), but that doesn't mean the database has necessarily needed all that data recently. The database might run just as well with less memory available. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 16:33:51 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 12DD137B401 for ; Thu, 30 Jan 2003 16:33:49 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 88B4743F79 for ; Thu, 30 Jan 2003 16:33:48 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0V0Xk0i091014; Thu, 30 Jan 2003 16:33:46 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0V0XkRH091013; Thu, 30 Jan 2003 16:33:46 -0800 (PST) Date: Thu, 30 Jan 2003 16:33:46 -0800 (PST) From: Matthew Dillon Message-Id: <200301310033.h0V0XkRH091013@apollo.backplane.com> To: Tim Kientzle Cc: "Brian T. Schellenberger" , Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301270019.44066.bschellenberger@nc.rr.com> <3E34C734.8010801@acm.org> <200301270904.43899.bschellenberger@nc.rr.com> <200301302222.h0UMMfFI090349@apollo.backplane.com> <3E39BE22.8050207@acm.org> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :Not necessarily. I suspect that there is :a strong tendency to access particular files :in particular ways. E.g., in your example of :a download server, those files are always :read sequentially. You can make similar assertions :about a lot of files: manpages, gzip files, :C source code files, etc, are "always" read :sequentially. : :If a file's access history were stored as a "hint" :associated with the file, then it would :be possible to make better up-front decisions about :how to allocate cache space. The ideal would be to This has been tried. It works up to a point, but not to the extent that you want it to. The basic problem is that past history does not necessarily predict future behavior. With the web server example, different client loads will result in different access behaviors. They might still all be sequential, but the combinations of multiple users will change the behavior enough that you would not be able to use the history as a reliable metric to control the cache. There is also an issue of how to store the 'history'. It isn't a simple matter of storing when a block was last accessed. Analysis of the access history is just as important and a lot of the type of analysis we humans do is intuitive and just cannot be replicated by a computer. Basically it all devolves down into the case where if you know exactly how something is going to be accessed, or you need caching to work a certain way in order to guarentee a certain behavior, the foreknowledge you have of the access methodologies will allow you to cache the information manually far better then the system could cache it heuristically. :store such hints on disk (maybe as an extended :attribute?), but it might also be useful to cache :them in memory somewhere. That would allow the :cache-management code to make much earlier decisions :about how to handle a file. For example, if a process :started to read a 10GB file that has historically been :accessed sequentially, you could immediately decide :to enable read-ahead for performance, but also mark :those pages to be released as soon as they were read by the :process. : :FWIW, a web search for "randomized caching" yields :some interesting reading. Apparently, there are :a few randomized cache-management algorithms for :which the mathematics work out reasonably well, :despite Terry's protestations to the contrary. ;-) :I haven't yet found any papers describing experiences :with real implementations, though. : :If only I had the time to spend poring over FreeBSD's :cache-management code to see how these ideas might :actually be implemented... : :Tim Kientzle It should be noted that was already implement most of the heuristics you talk about. We have a heuristic that detects sequential access patterns, for example, and enables clustered read-ahead. The problem isn't detection, the problem is scale. These heuristics work wonderfully at a small scale (i.e. lets read 64K ahead verses trying to cache 64MB worth of the file). Just knowing something is sequential does not allow you to choose how much memory you should set aside to cache that object, for example. Automatically depressing the priority of pages read sequentially after they've been used can have as terrible a performance impact as it can a positive one, depending on the size of the object, the number of distinct objects being accessed in that manner, perceived latency by end users, number of end users, the speed of their connections (some objects may be accessed more slowly then others depending on the client's network bandwidth), and so forth. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 16:37:39 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1B11837B401 for ; Thu, 30 Jan 2003 16:37:38 -0800 (PST) Received: from ms-smtp-02.southeast.rr.com (ms-smtp-02.southeast.rr.com [24.93.67.83]) by mx1.FreeBSD.org (Postfix) with ESMTP id 60F5343E4A for ; Thu, 30 Jan 2003 16:37:32 -0800 (PST) (envelope-from bts@fake.com) Received: from mail4.nc.rr.com (fe4 [24.93.67.51]) by ms-smtp-02.southeast.rr.com (8.12.5/8.12.2) with ESMTP id h0V0aKo5006549; Thu, 30 Jan 2003 19:36:20 -0500 (EST) Received: from this.is.fake.com ([24.74.172.220]) by mail4.nc.rr.com with Microsoft SMTPSVC(5.5.1877.757.75); Thu, 30 Jan 2003 19:38:32 -0500 Received: by this.is.fake.com (Postfix, from userid 111) id A8B5EBB33; Thu, 30 Jan 2003 19:37:12 -0500 (EST) Content-Type: text/plain; charset="iso-8859-1" From: "Brian T. Schellenberger" To: kientzle@acm.org, Matthew Dillon Subject: Re: Random disk cache expiry Date: Thu, 30 Jan 2003 19:37:11 -0500 User-Agent: KMail/1.4.2 Cc: Sean Hamilton , hackers@freebsd.org References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301302222.h0UMMfFI090349@apollo.backplane.com> <3E39BE22.8050207@acm.org> In-Reply-To: <3E39BE22.8050207@acm.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200301301937.12407.bschellenberger@nc.rr.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Thursday 30 January 2003 07:06 pm, Tim Kientzle wrote: | Matthew Dillon wrote: | > Your idea of 'sequential' access cache restriction only | > | > works if there is just one process doing the accessing. | | Not necessarily. I suspect that there is | a strong tendency to access particular files | in particular ways. E.g., in your example of | a download server, those files are always | read sequentially. You can make similar assertions | about a lot of files: : : For example, if a process | started to read a 10GB file that has historically been | accessed sequentially, you could immediately decide | to enable read-ahead for performance, but also mark | those pages to be released as soon as they were read by the | process. I think you missed Matt's point, which is well-taken: Even if everybody accesses it sequentially, if you have 100 processes accessing it sequentially at the *same* time, then it would be to your benefit to leave the "old" pages around because even though *this* process won't access it again, the *next* process very well might, if it just happens to be reading it sequentially as well but is a little further behind on its sequential read. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 16:39:25 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 87DDF37B401 for ; Thu, 30 Jan 2003 16:39:24 -0800 (PST) Received: from mail.cruzio.com (mail.cruzio.com [63.249.95.37]) by mx1.FreeBSD.org (Postfix) with ESMTP id D3C2943F3F for ; Thu, 30 Jan 2003 16:39:23 -0800 (PST) (envelope-from brucem@dsl3-63-249-66-35.cruzio.com) Received: from dsl3-63-249-66-35.cruzio.com (dsl3-63-249-66-35.cruzio.com [63.249.66.35]) by mail.cruzio.com with ESMTP id QAA06178 for ; Thu, 30 Jan 2003 16:39:18 -0800 (PST) Received: (from brucem@localhost) by dsl3-63-249-66-35.cruzio.com (8.11.3/8.11.3) id h0V19Jo00850 for freebsd-hackers@freebsd.org; Thu, 30 Jan 2003 17:09:19 -0800 (PST) (envelope-from brucem) Date: Thu, 30 Jan 2003 17:09:19 -0800 (PST) From: "Bruce R. Montague" Message-Id: <200301310109.h0V19Jo00850@dsl3-63-249-66-35.cruzio.com> To: freebsd-hackers@freebsd.org Subject: Re: Random disk cache expiry Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi, re: > If a file's access history were stored as a "hint" > associated with the file, then it would > be possible to make better up-front decisions about > how to allocate cache space. I believe at one time this was a hot area, and now maybe it is back. I vaguely recall a recent PhD in this area, you might want to contact him and get some up to date pointers (I think Tom Kroger spent some time at CMU looking at a lot of file access traces in this regard). I believe Tom modified a Linux filesystem to do just this and did a fair amount of benchmarking (which, of course, proved it (or at least his PhD ;) was useful)! In any case you might get some interesting starting refs if you are interested in this area... http://www.cse.ucsc.edu/~tmk/ http://csl.cse.ucsc.edu/prediction.shtml http://csl.cse.ucsc.edu/acme.shtml - bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 17:26: 9 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 510A437B405 for ; Thu, 30 Jan 2003 17:26:05 -0800 (PST) Received: from wonka.esatclear.ie (wonka.esatclear.ie [194.145.128.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id AFFB243E4A for ; Thu, 30 Jan 2003 17:26:03 -0800 (PST) (envelope-from fergal@esatclear.ie) Received: from whizzo.loc (aa-airlock228.esatclear.ie [213.202.167.228]) by wonka.esatclear.ie (8.9.3/8.9.3) with ESMTP id BAA15987 for ; Fri, 31 Jan 2003 01:25:55 GMT Content-Type: text/plain; charset="us-ascii" From: Fergal Daly To: freebsd-hackers@freebsd.org Subject: Random disk cache expiry Date: Fri, 31 Jan 2003 01:27:38 +0000 User-Agent: KMail/1.4.3 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Message-Id: <200301310127.38826.fergal@esatclear.ie> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG kientzle@acm.org (Tim Kientzle) wrote in message=20 news:... > Personally, I think there's a lot of merit to _trying_ There's even more merit in only pretending to try...=20 Here's the results of a quick simulation of a cache using random replacem= ent.=20 I've also included a scheme for a caching algorithm that might solve the=20 problem. I have simulated multiple sequential reads of a file. There are two=20 parameters, the ratio of the file size to the cache size and the number o= f=20 times the file is read. For each combination of paramters, the simulation is averaged over 50 run= s. The size of the cache was 100 blocks, if you have a faster machine than m= ine,=20 you can turn that up but I'm pretty sure it'll make very little differenc= e. Since we are talking about files being read more than once, the first rea= ding=20 of the file (which will give only cache misses) is not included in the=20 statistics. What does each column mean? ratio: how big the file is compared to the cache size lock: the cache hit % for a cache that just locks the first x blocks of t= he=20 file into the cache and doesn't try to cache anything else once the cache= is=20 full. 1, 2, 3, 4, 5: the cache hit % for the random replacement cache with 1, 2= , 3,=20 4, 5 full reads of the file (not counting the initial read to warm up the= =20 cache) l:r: how much better the locking cache is compared to the random cache ratio=09lock=091=092=093=094=095=09l:r 1.0=09100=09100=09100=09100=09100=09100=09100 1.1=09 90=09 84=09 83=09 83=09 82=09 82=09107 1.2=09 83=09 72=09 69=09 69=09 69=09 69=09115 1.3=09 76=09 61=09 59=09 58=09 58=09 58=09124 1.4=09 71=09 52=09 51=09 50=09 49=09 49=09136 1.5=09 66=09 45=09 43=09 42=09 42=09 42=09146 1.6=09 62=09 39=09 38=09 36=09 36=09 36=09158 1.7=09 58=09 33=09 32=09 31=09 31=09 31=09173 1.8=09 55=09 29=09 28=09 27=09 27=09 27=09187 1.9=09 52=09 26=09 24=09 24=09 24=09 23=09198 2.0=09 50=09 22=09 21=09 20=09 20=09 20=09219 2.1=09 47=09 19=09 19=09 18=09 18=09 18=09238 2.2=09 45=09 17=09 16=09 16=09 16=09 15=09253 2.3=09 43=09 15=09 15=09 14=09 14=09 14=09280 2.4=09 41=09 13=09 12=09 12=09 12=09 12=09298 2.5=09 40=09 11=09 11=09 11=09 10=09 10=09336 2.6=09 38=09 10=09 9=09 10=09 9=09 9=09352 2.7=09 37=09 9=09 8=09 8=09 8=09 8=09394 2.8=09 35=09 8=09 8=09 8=09 7=09 7=09434 2.9=09 34=09 7=09 6=09 6=09 6=09 6=09472 3.0=09 33=09 6=09 6=09 6=09 6=09 6=09477 3.1=09 32=09 5=09 5=09 5=09 5=09 5=09556 3.2=09 31=09 5=09 5=09 5=09 4=09 4=09577 3.3=09 30=09 4=09 4=09 4=09 4=09 4=09645 3.4=09 29=09 4=09 3=09 3=09 3=09 3=09700 3.5=09 28=09 3=09 3=09 3=09 3=09 3=09714 3.6=09 27=09 3=09 3=09 3=09 3=09 3=09869 3.7=09 27=09 3=09 2=09 2=09 2=09 2=09886 3.8=09 26=09 2=09 2=09 2=09 2=09 2=09948 3.9=09 25=09 2=09 2=09 2=09 2=09 2=091035 4.0=09 25=09 2=09 2=09 2=09 2=09 2=091094 As you can see, the locking cache is always better than the random one an= d the=20 file doesn't have to be very big for it to be hugely better. Maybe this is well known already or maybe it is not compatible with the=20 current system but here's an idea for how you might implement the locking= =20 cache. When a program hints that a file will be read in this way, the cached dat= a for=20 that file is treated differently. Blocks to be uncached are selected in t= he=20 usual LRU way. Say block A is selected to be evicted but block A is part of one of these= =20 special files, so instead of evicting block A, we look for another block = in=20 the file, call it block B. Block B should be the block furthest from the = head=20 of the file. We evict block B and we give block B's timestamp to block A. To see this in action, imagine that blocks 1 to 20 of a file are in the c= ache.=20 The LRU cache decides to evict block 1, we see that it's part of a specia= l=20 file so instead we evict block 20 and give block 20's timestamp to block = 1.=20 Soon after, the LRU cache decides to evict block 2 but we evict block 19=20 instead and give its timestamp to block 2. Next our program reads block 2= 1=20 which is then cached. Soon after that the cache tries to evict block 3 bu= t=20 instead we evict block 21 and give it's timestamp to block 3. Taking the file as a whole, the cache neither prefers nor penalises it=20 compared to the current algorithm. However within the file, the blocks ne= ar=20 the head are given preference over the ones near the tail. If after 1 run through the file the cache has only kept 40% of the file. = Under=20 the current scheme this would be the final 40% of the file which is no us= e=20 for the next run. Under the new scheme it's got the initial 40% which is = as=20 good as we could hope for. There is the downside. The cache now has to keep an ordered list of the b= locks=20 contained in these special files owever there is no extra overhead for ot= her=20 files. One other possible problem is that the blocks at the head of the file wil= l=20 tend to have the newer timestamps and so when the file is read a second t= ime=20 the newer timstamps will be overwritten before the older ones. It might b= e=20 better to manage timestamps as a FIFO queue but on second thought, I thin= k=20 that will give this file an unfair advantage. It should be straight forward to extend this to groups of files by keepin= g a=20 single ordered list of all the blocks in several files. The Perl for the benchmark is available at http://www.fergaldaly.com/computer/randcache/bench F --=20 Do you need someone with lots of Unix sysadmin and/or lots of OO software= =20 development experience? Go on, giz a job. My CV - http://www.fergaldaly.com/cv.html To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 17:29: 3 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2FB7F37B401 for ; Thu, 30 Jan 2003 17:29:02 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id AA98B43F43 for ; Thu, 30 Jan 2003 17:29:01 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0V1Sx0i091269; Thu, 30 Jan 2003 17:28:59 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0V1Sxo5091268; Thu, 30 Jan 2003 17:28:59 -0800 (PST) Date: Thu, 30 Jan 2003 17:28:59 -0800 (PST) From: Matthew Dillon Message-Id: <200301310128.h0V1Sxo5091268@apollo.backplane.com> To: "Brian T. Schellenberger" Cc: kientzle@acm.org, Sean Hamilton , hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry References: <000501c2c4dd$f43ed450$16e306cf@slugabed.org> <200301302222.h0UMMfFI090349@apollo.backplane.com> <3E39BE22.8050207@acm.org> <200301301937.12407.bschellenberger@nc.rr.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :I think you missed Matt's point, which is well-taken: : :Even if everybody accesses it sequentially, if you have 100 processes :accessing it sequentially at the *same* time, then it would be to your :benefit to leave the "old" pages around because even though *this* :process won't access it again, the *next* process very well might, if :it just happens to be reading it sequentially as well but is a little :further behind on its sequential read. Right, and if the same 100 processes are accessing N files sequentially instead of just one, you get a different effect... you might blow out your cache if the aggregate size of the N files is too large. But then if some of those processes are accessing the same file, and other processes were accessing different files, you might want to cache that file but possibly not cache others, even though all the files are (for this example) the same size. But then what if some of the processes accessing some of those other files were from slow clients? You could get away with not caching those files and then you might possibly be able cache all the remaining files (being accessed by faster clients). And so on, and so forth. It gets even more complicated when you throw in read verses write verses read-modify-write accesses, and even more complicated when you add other load factors (e.g. the sheer number of connections might reduce the memory available for caching on the fly, or trying to balance executable pages verses data pages to optimize paging and program performance). So there is no 'perfect' caching algorithm. There are simply too many variables even in a well defined environment for even the best system heuristics to cover optimally. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 18:50: 4 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4BF7F37B401 for ; Thu, 30 Jan 2003 18:50:03 -0800 (PST) Received: from HAL9000.homeunix.com (12-233-57-224.client.attbi.com [12.233.57.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5C8BC43F43 for ; Thu, 30 Jan 2003 18:50:02 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.6/8.12.5) with ESMTP id h0V2o0Nt011417; Thu, 30 Jan 2003 18:50:00 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.6/8.12.5/Submit) id h0V2nqm1011416; Thu, 30 Jan 2003 18:49:52 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Date: Thu, 30 Jan 2003 18:49:52 -0800 From: David Schultz To: Matthew Dillon Cc: James Gritton , freebsd-hackers@FreeBSD.ORG Subject: Re: What's the memory footprint of a set of processes? Message-ID: <20030131024952.GA11372@HAL9000.homeunix.com> Mail-Followup-To: Matthew Dillon , James Gritton , freebsd-hackers@FreeBSD.ORG References: <20030130064448.GA7258@HAL9000.homeunix.com> <200301300719.h0U7JOfI086054@apollo.backplane.com> <20030130091419.GA7776@HAL9000.homeunix.com> <200301301923.h0UJNT0l089037@apollo.backplane.com> <20030131001436.GA10856@HAL9000.homeunix.com> <200301310023.h0V0NqAE090963@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200301310023.h0V0NqAE090963@apollo.backplane.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thus spake Matthew Dillon : > :Thanks for the explanations! I still don't understand why this > :doesn't work, assuming you don't care about nonresident pages: > : > :for each process p in the set > : for each map entry e in p->vmspace->vm_map > : for each page m in e->object.vm_object->memq > : if I haven't seen this m.phys_addr yet in the scan > : resident_pages++ > > That would get close, as long as the machine is not paging heavily. > Think of it this way: If you have a lot of ram the above calculation > will give you an upper bound on memory use, but some of the pages > in the VM object's may be very old and not actually under active > access by the process (for example, the pages might represent part > of the program that was used during initialization and then never used > again). If you do not have so much memory older pages will get flushed > out or flushed to swap and the above calculation will represent more > of a lower bound on the memory used by the group of processes. Yes, I understand this; that's why I said ``assuming you don't care about nonresident pages'' (a pretty big assumption, mind you.) I was just thinking about essentially calculating the physical memory usage for a set of processes, taking sharing into account, and I take it you were talking about calculating the total amount mapped. I imagine both metrics would be useful. For instance, a database might map a huge file but have a very small resident set. I don't know what the OP intended... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 19:47:36 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7C2F037B401 for ; Thu, 30 Jan 2003 19:47:35 -0800 (PST) Received: from HAL9000.homeunix.com (12-233-57-224.client.attbi.com [12.233.57.224]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0612F43F3F for ; Thu, 30 Jan 2003 19:47:35 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.6/8.12.5) with ESMTP id h0V3lVNt011615; Thu, 30 Jan 2003 19:47:31 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.6/8.12.5/Submit) id h0V3lVi4011614; Thu, 30 Jan 2003 19:47:31 -0800 (PST) (envelope-from dschultz@uclink.Berkeley.EDU) Date: Thu, 30 Jan 2003 19:47:31 -0800 From: David Schultz To: Fergal Daly Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Random disk cache expiry Message-ID: <20030131034731.GB11445@HAL9000.homeunix.com> Mail-Followup-To: Fergal Daly , freebsd-hackers@FreeBSD.ORG References: <200301310127.38826.fergal@esatclear.ie> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200301310127.38826.fergal@esatclear.ie> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Thus spake Fergal Daly : > kientzle@acm.org (Tim Kientzle) wrote in message > news:... > > Personally, I think there's a lot of merit to _trying_ > > There's even more merit in only pretending to try... Welcome to my quotes file. > As you can see, the locking cache is always better than the random one and the > file doesn't have to be very big for it to be hugely better. You have found an optimal replacement algorithm for the case of repeated sequential reads. In fact, if you know in advance what the access pattern is going to be, it is *always* possible to find an optimal replacement algorithm. Specifically, you always replace the block in the cache that will not be used for the longest time in the future. If you want the system to always cache the right thing in the general case, the only hint you would need to provide the system is a reference string specifying the predicted access pattern. (If I were to do this, my first reaction would be to encode it as an FSA.) If that proves to be infeasible, I'm sure there are ways to approximate the same thing. The hard parts, I think, would be teaching the VM system to use the new information, and gathering statistics from which you form your hints. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Thu Jan 30 22:49:12 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5F27B37B401 for ; Thu, 30 Jan 2003 22:49:11 -0800 (PST) Received: from mx9.mail.ru (mx9.mail.ru [194.67.57.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id 21F5E43F43 for ; Thu, 30 Jan 2003 22:49:10 -0800 (PST) (envelope-from shmukler@mail.ru) Received: from f13.int ([10.0.0.105] helo=f13.mail.ru) by mx9.mail.ru with esmtp (Exim MX.9) id 18eUz6-0004ZC-00 for freebsd-hackers@freebsd.org; Fri, 31 Jan 2003 09:49:08 +0300 Received: from mail by f13.mail.ru with local (Exim FE.1) id 18eUz6-0001D0-00 for freebsd-hackers@freebsd.org; Fri, 31 Jan 2003 09:49:08 +0300 Received: from [216.194.2.239] by win.mail.ru with HTTP; Fri, 31 Jan 2003 09:49:08 +0300 From: "Igor Shmukler" To: freebsd-hackers@freebsd.org Subject: Re: Random disk cache expiry Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: [216.194.2.239] Date: Fri, 31 Jan 2003 09:49:08 +0300 Reply-To: "Igor Shmukler" Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit Message-Id: Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > You have found an optimal replacement algorithm for the case of > repeated sequential reads. In fact, if you know in advance what > the access pattern is going to be, it is *always* possible to find > an optimal replacement algorithm. Specifically, you always > replace the block in the cache that will not be used for the > longest time in the future. Did everyone read UBM paper from OSDI? It presents one possible solution for dealing with sequentaly accessed files. Why is it not enough (at least to begin with)? To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Jan 31 9:46: 2 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3FBFC37B401 for ; Fri, 31 Jan 2003 09:46:00 -0800 (PST) Received: from mail.cruzio.com (mail.cruzio.com [63.249.95.37]) by mx1.FreeBSD.org (Postfix) with ESMTP id D982743F43 for ; Fri, 31 Jan 2003 09:45:59 -0800 (PST) (envelope-from brucem@dsl3-63-249-66-35.cruzio.com) Received: from dsl3-63-249-66-35.cruzio.com (dsl3-63-249-66-35.cruzio.com [63.249.66.35]) by mail.cruzio.com with ESMTP id JAA09983 for ; Fri, 31 Jan 2003 09:45:54 -0800 (PST) Received: (from brucem@localhost) by dsl3-63-249-66-35.cruzio.com (8.11.3/8.11.3) id h0VIFrW00567 for freebsd-hackers@freebsd.org; Fri, 31 Jan 2003 10:15:53 -0800 (PST) (envelope-from brucem) Date: Fri, 31 Jan 2003 10:15:53 -0800 (PST) From: "Bruce R. Montague" Message-Id: <200301311815.h0VIFrW00567@dsl3-63-249-66-35.cruzio.com> To: freebsd-hackers@freebsd.org Subject: Re: Random disk cache expiry Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG For those thinking of playing with predictive caching (likely an area of considerable student endeveour/interest these days at both filesystem and "web" level): --- Matthew Dillon: > So there is no 'perfect' caching algorithm. There > are simply too many variables even in a well defined > environment for even the best system heuristics to > cover optimally. --- David Schultz: > If that proves to be infeasible, I'm sure there are > ways to approximate the same thing. The hard parts, > I think, would be teaching the VM system to use the > new information, and gathering statistics from which > you form your hints. --- Right. It's easy if you know the complete future of the total system state, which of course you never will. Someone interested in this might try to apply the latest in machine learing techniques, classifiers, etc., to the online problem. Variants of this are receiving lots of attention in areas such as gene sequence prediction. I dunno, but it seems like a lot of the math ends up pretty similar to economics, and we all know how well those models work. Kind of funny, running an economic simulation in your kernel... but actually getting possible at some level, at least for research systems with modern machines. There was a time when you would be fired for putting floating-point in an OS. ---- http://csl.cse.ucsc.edu/acme.shtml : "Many cache replacement policies have been invented and some perform better than others under certain workload and network-topological conditions. It is impossible and sub-optimal to manually choose cache replacement policies for workloads and topologies that are under continuous change. We use machine learning algorithms to automatically select the best current policy or mixtures of policies from a policy (a.k.a expert) pool to provide an "adaptive caching" service." - bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Jan 31 13:31:44 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D126C37B401; Fri, 31 Jan 2003 13:31:41 -0800 (PST) Received: from yahoo.com (host81-134-41-205.in-addr.btopenworld.com [81.134.41.205]) by mx1.FreeBSD.org (Postfix) with SMTP id 7E9DA43FC7; Fri, 31 Jan 2003 13:31:36 -0800 (PST) (envelope-from newhsave@yahoo.com) Message-ID: <000410c5eb35$ccc25383$68615510@ljrpyit.rwa> From: To: Homeworker@FreeBSD.ORG Subject: Turn $25 into $45,000 MONTHLY, all automatic! 2588CMRd2-598mGjG7972Raq-23 Date: Fri, 31 Jan 2003 12:22:19 +0900 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit X-Priority: 3 X-Mailer: Microsoft Outlook Express 5.00.2615.200 Importance: Normal Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This is IT! Don't miss out in this amazing opportunity. Join now for $25 and earn up to $45,000 MONTHLY!!!! Yes, I said monthly AND IT IS ALL AUTOMATED. Get FREE information. Just click the link below. www.mlmontarget.com This is FREE information that will amaze you on how much MONEY YOU CAN EARN FOR ONLY $25 per month. Many join multiple times and it is ALL AUTOMATED. We do all the hard work. Click below now for FREE information. www.mlmontarget.com Start getting your MONEY today!!! 8454CbFV6-461bgof1523wBxl0-180ztPb3459XesT5-575PZdt8733Anqp8-647GKkY1078Nswl71 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Jan 31 15:12: 7 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B5EAB37B401 for ; Fri, 31 Jan 2003 15:12:06 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6438343E4A for ; Fri, 31 Jan 2003 15:12:06 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h0VNC5SJ007171 for ; Fri, 31 Jan 2003 15:12:06 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h0VNC5bQ007170; Fri, 31 Jan 2003 15:12:05 -0800 (PST) Date: Fri, 31 Jan 2003 15:12:05 -0800 (PST) From: Matthew Dillon Message-Id: <200301312312.h0VNC5bQ007170@apollo.backplane.com> To: hackers@freebsd.org Subject: Lower power SMP boxes? Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I've slowly been trying to trim down the power use in my machine room, oweing to astronomical PG&E bills :-(. I'm using those wonderful little EPIA series mini-itx motherboards (purchased from idot.com) as low/medium performance servers. They aren't all that fast but one will run a web site, pop/sendmail, and an ordb nameserver just dandy and can copy files over NFS at 7MBytes/s. That covers UP systems quite well. But MP is another story. At some point I would like to put together some SMP test boxes that don't cost the equivalent of rent on a small apartment in electricity use. They don't have to be super-fast, they just need to be SMP. I'm not talking about blade servers here, I'm talking about SMP boxes for testing purposes. Anyone have any ideas? -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Jan 31 21: 7:40 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B494A37B401 for ; Fri, 31 Jan 2003 21:07:39 -0800 (PST) Received: from mail.rpi.edu (mail.rpi.edu [128.113.2.7]) by mx1.FreeBSD.org (Postfix) with ESMTP id E808043F3F for ; Fri, 31 Jan 2003 21:07:38 -0800 (PST) (envelope-from drosih@rpi.edu) Received: from [128.113.24.47] (gilead.netel.rpi.edu [128.113.24.47]) by mail.rpi.edu (8.12.1/8.12.1) with ESMTP id h1157UOA069174; Sat, 1 Feb 2003 00:07:30 -0500 Mime-Version: 1.0 X-Sender: drosih@mail.rpi.edu Message-Id: In-Reply-To: <200301312312.h0VNC5bQ007170@apollo.backplane.com> References: <200301312312.h0VNC5bQ007170@apollo.backplane.com> Date: Sat, 1 Feb 2003 00:07:28 -0500 To: Matthew Dillon , hackers@FreeBSD.ORG From: Garance A Drosihn Subject: Re: Lower power SMP boxes? Content-Type: text/plain; charset="us-ascii" ; format="flowed" X-Scanned-By: MIMEDefang 2.3 (www dot roaringpenguin dot com slash mimedefang) Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG At 3:12 PM -0800 1/31/03, Matthew Dillon wrote: > But MP is another story. At some point I would like to put > together some SMP test boxes that don't cost the equivalent > of rent on a small apartment in electricity use. They don't > have to be super-fast, they just need to be SMP. I'm not > talking about blade servers here, I'm talking about SMP > boxes for testing purposes. > > Anyone have any ideas? The arrival of FreeBSD/PPC might give you some cheaper options. -- Garance Alistair Drosehn = gad@gilead.netel.rpi.edu Senior Systems Programmer or gad@freebsd.org Rensselaer Polytechnic Institute or drosih@rpi.edu To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Fri Jan 31 22: 4:22 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 24D2B37B401 for ; Fri, 31 Jan 2003 22:04:21 -0800 (PST) Received: from logopolis.mordacious.net (logopolis.mordacious.net [194.153.168.52]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7A8D543E4A for ; Fri, 31 Jan 2003 22:04:20 -0800 (PST) (envelope-from cg@ijcg.net) Received: from [10.1.0.2] (unknown [81.2.114.200]) (using TLSv1 with cipher EDH-RSA-DES-CBC3-SHA (168/168 bits)) (No client certificate requested) by logopolis.mordacious.net (Postfix) with ESMTP id 6133672F47A; Sat, 1 Feb 2003 06:05:26 +0000 (GMT) Date: Sat, 01 Feb 2003 06:04:14 +0000 From: cameron grant To: Matthew Dillon , hackers@freebsd.org Subject: Re: Lower power SMP boxes? Message-ID: <49151856.1044079454@[10.1.0.2]> In-Reply-To: <200301312312.h0VNC5bQ007170@apollo.backplane.com> References: <200301312312.h0VNC5bQ007170@apollo.backplane.com> X-Mailer: Mulberry/3.0.0 (Win32) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Content-Disposition: inline Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > But MP is another story. At some point I would like to put together > some SMP test boxes that don't cost the equivalent of rent on a small > apartment in electricity use. They don't have to be super-fast, they > just need to be SMP. I'm not talking about blade servers here, I'm > talking about SMP boxes for testing purposes. > > Anyone have any ideas? http://www.theinquirer.net/?article=7434 might be of interest. -cg To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 1:16:50 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7504737B401 for ; Sat, 1 Feb 2003 01:16:49 -0800 (PST) Received: from marvin.sko.mh.se (marvin.sko.mh.se [193.10.250.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id 58DFD43F3F for ; Sat, 1 Feb 2003 01:16:48 -0800 (PST) (envelope-from myrslok@marvin.sko.mh.se) Received: from localhost (myrslok@localhost) by marvin.sko.mh.se (8.11.6/8.11.6) with ESMTP id h119HNE16159; Sat, 1 Feb 2003 10:17:23 +0100 (CET) (envelope-from myrslok@marvin.sko.mh.se) Date: Sat, 1 Feb 2003 10:17:23 +0100 (CET) From: Mats Larsson To: Matthew Dillon Cc: hackers@FreeBSD.ORG Subject: Re: Lower power SMP boxes? In-Reply-To: <200301312312.h0VNC5bQ007170@apollo.backplane.com> Message-ID: <20030201101041.H16130@marvin.sko.mh.se> References: <200301312312.h0VNC5bQ007170@apollo.backplane.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Via just recently announced their new Nehemiah processor capable of smp, presumably slow as its precursor but also the lowest power consuming processor at the market (at least with standard socket fcpga motherboard) Check it out at: http://www.theinquirer.net/?article=7434 http://www.via.com.tw/en/viac3/c3.jsp // Mats On Fri, 31 Jan 2003, Matthew Dillon wrote: > I've slowly been trying to trim down the power use in my machine room, > oweing to astronomical PG&E bills :-(. I'm using those wonderful little > EPIA series mini-itx motherboards (purchased from idot.com) as low/medium > performance servers. They aren't all that fast but one will run a web > site, pop/sendmail, and an ordb nameserver just dandy and can copy files > over NFS at 7MBytes/s. That covers UP systems quite well. > > But MP is another story. At some point I would like to put together > some SMP test boxes that don't cost the equivalent of rent on a small > apartment in electricity use. They don't have to be super-fast, they > just need to be SMP. I'm not talking about blade servers here, I'm > talking about SMP boxes for testing purposes. > > Anyone have any ideas? > > -Matt > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 3: 2:15 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3F9C737B401; Sat, 1 Feb 2003 03:02:14 -0800 (PST) Received: from spider.deepcore.dk (cpe.atm2-0-56339.0x50c6aa0a.abnxx2.customer.tele.dk [80.198.170.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 121D343F3F; Sat, 1 Feb 2003 03:02:07 -0800 (PST) (envelope-from sos@spider.deepcore.dk) Received: (from sos@localhost) by spider.deepcore.dk (8.12.5/8.12.6) id h11B25xk077482; Sat, 1 Feb 2003 12:02:05 +0100 (CET) (envelope-from sos) From: Soeren Schmidt Message-Id: <200302011102.h11B25xk077482@spider.deepcore.dk> Subject: Request for info from SiS chipset owners To: current@freebsd.org, hackers@freebsd.org Date: Sat, 1 Feb 2003 12:02:05 +0100 (CET) X-Mailer: ELM [version 2.4ME+ PL98b (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=ISO-8859-1 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I'm currently in the midst of an ATA chipset support mega rewrite/update, and the last item on the list is SiS support. That where _you_ come into the picture, I need a pciconf -l from your SiS based system! Just reply to this message with the output from pciconf -l and you have helped me sort out the myriads of SiS chipsets out there. -Søren To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 3:29:42 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 142F037B401 for ; Sat, 1 Feb 2003 03:29:41 -0800 (PST) Received: from mail.gmx.net (mail.gmx.net [213.165.65.60]) by mx1.FreeBSD.org (Postfix) with SMTP id 891C843F43 for ; Sat, 1 Feb 2003 03:29:39 -0800 (PST) (envelope-from csharp@gmx.net) Received: (qmail 31313 invoked by uid 0); 1 Feb 2003 11:29:32 -0000 Received: from pD9E46590.dip.t-dialin.net (HELO dagon.cthulhu.net) (217.228.101.144) by mail.gmx.net (mp002-rz3) with SMTP; 1 Feb 2003 11:29:32 -0000 Received: from mephisto by dagon.cthulhu.net with local (Exim 4.12) id 18evqh-000599-00; Sat, 01 Feb 2003 12:30:15 +0100 Date: Sat, 1 Feb 2003 12:30:15 +0100 From: Christopher Sharp To: current@freebsd.org, hackers@freebsd.org Subject: Re: Request for info from SiS chipset owners Message-ID: <20030201113015.GA59313@gmx.net> Mail-Followup-To: current@freebsd.org, hackers@freebsd.org References: <200302011102.h11B25xk077482@spider.deepcore.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200302011102.h11B25xk077482@spider.deepcore.dk> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hope this helps: agp0@pci0:0:0: class=0x060000 card=0x00000000 chip=0x07351039 rev=0x01 hdr=0x00 pcib1@pci0:1:0: class=0x060400 card=0x00000000 chip=0x00011039 rev=0x00 hdr=0x01 isab0@pci0:2:0: class=0x060100 card=0x00000000 chip=0x00081039 rev=0x00 hdr=0x00 atapci0@pci0:2:5: class=0x010180 card=0x55131039 chip=0x55131039 rev=0xd0 hdr=0x00 bktr0@pci0:11:0: class=0x040000 card=0x13eb0070 chip=0x036e109e rev=0x11 hdr=0x00 none0@pci0:11:1: class=0x048000 card=0x13eb0070 chip=0x0878109e rev=0x11 hdr=0x00 ed0@pci0:13:0: class=0x020000 card=0x802910ec chip=0x802910ec rev=0x00 hdr=0x00 pcm0@pci0:17:0: class=0x040100 card=0x80401102 chip=0x00021102 rev=0x05 hdr=0x00 emujoy0@pci0:17:1: class=0x098000 card=0x00201102 chip=0x70021102 rev=0x05 hdr=0x00 none1@pci1:0:0: class=0x030000 card=0x88081462 chip=0x002d10de rev=0x15 hdr=0x00 - Chhristopher To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 5:24: 6 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E428537B401 for ; Sat, 1 Feb 2003 05:24:05 -0800 (PST) Received: from mail-out.ukr.net (mail-out.ukr.net [212.42.65.71]) by mx1.FreeBSD.org (Postfix) with ESMTP id BC9E243F75 for ; Sat, 1 Feb 2003 05:24:04 -0800 (PST) (envelope-from rtin@ukr.net) Received: from storage.ukr.net ([212.42.65.69] helo=storage.ukr.net) by mail-out.ukr.net with ESMTP id C00F8AD525 for ; Sat, 1 Feb 2003 15:24:00 +0200 (EET) Received: from mail by storage.ukr.net with local ID 18excm-000PdT-00 for hackers@FreeBSD.org; Sat, 01 Feb 2003 15:24:00 +0200 Received: from [195.123.254.35] by web.mail.ukr.net with HTTP; Sat, 01 Feb 2003 13:24:00 +0000 (GMT) From: "Kreiser Kirov" To: hackers@FreeBSD.org Subject: Hi!Dear FreeBSD! Mime-Version: 1.0 X-Mailer: mPOP Web-Mail 2.19 X-Originating-IP: cdu3-35.gu.kiev.ua [195.123.254.35] Reply-To: "Kreiser Kirov" Content-Type: text/plain; charset=koi8-r Content-Transfer-Encoding: 8bit Message-Id: Date: Sat, 01 Feb 2003 15:24:00 +0200 X-Scanner: exiscan for exim4 (http://duncanthrax.net/exiscan/) *18excm-000PdT-00*RXiVNhofe.Q* Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hello,Dear FreeBSD! My name is Gleb. I wrote to you from Ukraine(this country is situated in Europe,near Russia). I live in Kiev -the capital of Ukraine. I'm a student of National Politecnical University of Ukraine . I'm very sorry for my english,I know it only for reading some handbooks and man pages. In my computer class there are several mashins with FreeBSD installed for teaching students. But I don't want only work with FreeBSD.I want help to develop it's kernel,write programs,drivers. Please help find more doc's about architecture of it's kernel,how it works, kernel structs and functions. I had already read developers-handbook,but it is to hard for me to write something with this book.I know C and Assembler and good in writing applications. Help me join some simple project,I would work hardly!Thank you very much! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 5:50:48 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1E4C037B401; Sat, 1 Feb 2003 05:50:47 -0800 (PST) Received: from smtp-send.myrealbox.com (smtp-send.myrealbox.com [192.108.102.143]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9750A43F43; Sat, 1 Feb 2003 05:50:46 -0800 (PST) (envelope-from qhwt@myrealbox.com) Received: from localhost qhwt@smtp-send.myrealbox.com [220.97.114.198] by smtp-send.myrealbox.com with NetMail SMTP Agent $Revision: 3.24 $ on Novell NetWare; Sat, 01 Feb 2003 06:50:46 -0700 Date: Sat, 1 Feb 2003 22:51:36 +0900 From: qhwt@myrealbox.com To: Soeren Schmidt Cc: current@freebsd.org, hackers@freebsd.org Subject: Re: Request for info from SiS chipset owners Message-ID: <20030201135136.GA2222.qhwt@myrealbox.com> References: <200302011102.h11B25xk077482@spider.deepcore.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200302011102.h11B25xk077482@spider.deepcore.dk> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Feb 01, 2003 at 12:02:05PM +0100, Soeren Schmidt wrote: > Just reply to this message with the output from pciconf -l and you > have helped me sort out the myriads of SiS chipsets out there. agp0@pci0:0:0: class=0x060000 card=0x06481039 chip=0x06481039 rev=0x02 hdr=0x00 pcib1@pci0:1:0: class=0x060400 card=0x00000000 chip=0x00011039 rev=0x00 hdr=0x01 isab0@pci0:2:0: class=0x060100 card=0x00000000 chip=0x00081039 rev=0x04 hdr=0x00 atapci0@pci0:2:5: class=0x010180 card=0x55131039 chip=0x55131039 rev=0x00 hdr=0x00 none0@pci0:3:0: class=0x0c0310 card=0x70011039 chip=0x70011039 rev=0x0f hdr=0x00 none1@pci0:3:1: class=0x0c0310 card=0x70011039 chip=0x70011039 rev=0x0f hdr=0x00 none2@pci0:3:2: class=0x0c0310 card=0x70011039 chip=0x70011039 rev=0x0f hdr=0x00 none3@pci0:3:3: class=0x0c0320 card=0x50041458 chip=0x70021039 rev=0x00 hdr=0x00 dc0@pci0:11:0: class=0x020000 card=0x03151025 chip=0x00191011 rev=0x30 hdr=0x00 xl0@pci0:12:0: class=0x020000 card=0x00000000 chip=0x905010b7 rev=0x00 hdr=0x00 pcm0@pci0:13:0: class=0x040100 card=0x13711274 chip=0x13711274 rev=0x06 hdr=0x00 nvidia0@pci1:0:0: class=0x030000 card=0x00000000 chip=0x017110de rev=0xa3 hdr=0x00 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 7:45: 6 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CB69C37B401 for ; Sat, 1 Feb 2003 07:45:04 -0800 (PST) Received: from mail.gmx.net (mail.gmx.net [213.165.65.60]) by mx1.FreeBSD.org (Postfix) with SMTP id AD48043F43 for ; Sat, 1 Feb 2003 07:45:03 -0800 (PST) (envelope-from mdcki@gmx.net) Received: (qmail 4006 invoked by uid 0); 1 Feb 2003 15:45:02 -0000 Received: from cvpn020.gwdg.de (HELO gmx.net) (134.76.22.20) by mail.gmx.net (mp007-rz3) with SMTP; 1 Feb 2003 15:45:02 -0000 Message-ID: <3E3BEC87.80600@gmx.net> Date: Sat, 01 Feb 2003 16:49:27 +0100 From: Marcin Dalecki User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.2.1) Gecko/20021230 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Kreiser Kirov Cc: hackers@FreeBSD.org Subject: Re: Hi!Dear FreeBSD! References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Kreiser Kirov wrote: > Hello,Dear FreeBSD! > My name is Gleb. > I wrote to you from Ukraine(this country is situated in Europe,near Russia). Now you are explaning where the Ukraina is. Next time you will tell us where childs are comming from?! > I live in Kiev -the capital of Ukraine. > I'm a student of National Politecnical University of Ukraine . > I'm very sorry for my english,I know it only for reading some handbooks and man pages. > In my computer class there are several mashins with FreeBSD installed for teaching students. > But I don't want only work with FreeBSD.I want help to develop it's kernel,write programs,drivers. > Please help find more doc's about architecture of it's kernel,how it works, > kernel structs and functions. > I had already read developers-handbook,but it is to hard for me to write > something with this book.I know C and Assembler and good in writing applications. > Help me join some simple project,I would work hardly!Thank you very much! You could go hunt for the russian translation of the design of the UNIX operating system. It is freely available on the net as a bunch of plain text files. -- Marcin Dalecki To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 9:33:12 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D4F8437B401 for ; Sat, 1 Feb 2003 09:33:11 -0800 (PST) Received: from meketrex.pix.net (meketrex.pix.net [192.111.45.13]) by mx1.FreeBSD.org (Postfix) with ESMTP id 865AC43F3F for ; Sat, 1 Feb 2003 09:33:10 -0800 (PST) (envelope-from lidl@meketrex.pix.net) Received: (from lidl@localhost) by meketrex.pix.net (8.11.6/8.11.6) id h11HX3f06565; Sat, 1 Feb 2003 12:33:03 -0500 (EST) Date: Sat, 1 Feb 2003 12:33:03 -0500 From: "Kurt J. Lidl" To: Mats Larsson Cc: Matthew Dillon , hackers@FreeBSD.ORG Subject: Re: Lower power SMP boxes? Message-ID: <20030201123303.A6376@pix.net> References: <200301312312.h0VNC5bQ007170@apollo.backplane.com> <20030201101041.H16130@marvin.sko.mh.se> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20030201101041.H16130@marvin.sko.mh.se>; from myrslok@marvin.sko.mh.se on Sat, Feb 01, 2003 at 10:17:23AM +0100 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Feb 01, 2003 at 10:17:23AM +0100, Mats Larsson wrote: > Via just recently announced their new Nehemiah processor capable of smp, > presumably slow as its precursor but also the lowest power consuming > processor at the market (at least with standard socket fcpga motherboard) [...] > http://www.via.com.tw/en/viac3/c3.jsp It says "IO/APIC support in future versions". So, it's not an SMP option today, as I understand it. -Kurt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 10:45:55 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9061C37B401 for ; Sat, 1 Feb 2003 10:45:53 -0800 (PST) Received: from sccrmhc02.attbi.com (sccrmhc02.attbi.com [204.127.202.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id C3EAC43F79 for ; Sat, 1 Feb 2003 10:45:52 -0800 (PST) (envelope-from julian@elischer.org) Received: from InterJet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc02.attbi.com (sccrmhc02) with ESMTP id <2003020118455100200i2s7ne>; Sat, 1 Feb 2003 18:45:51 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id KAA55407; Sat, 1 Feb 2003 10:45:49 -0800 (PST) Date: Sat, 1 Feb 2003 10:45:48 -0800 (PST) From: Julian Elischer To: Marcin Dalecki Cc: Kreiser Kirov , hackers@FreeBSD.org Subject: Re: Hi!Dear FreeBSD! In-Reply-To: <3E3BEC87.80600@gmx.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 1 Feb 2003, Marcin Dalecki wrote: > Kreiser Kirov wrote: [stuff] Wouldn't it be best to put him in contact with some of the other developers in the area? Which brings the question.. who IS in the area.. maybe we should make some sort of geographical registration web page so that people can find each other? how many time shave you heard, "You're in XXXX? Hey, I'm in XXXX!" julian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 11:21:51 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 28F3037B401 for ; Sat, 1 Feb 2003 11:21:50 -0800 (PST) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id BEEF643E4A for ; Sat, 1 Feb 2003 11:21:49 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.12.6/8.12.6) with ESMTP id h11JLnSJ016624; Sat, 1 Feb 2003 11:21:49 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.12.6/8.12.6/Submit) id h11JLnrk016623; Sat, 1 Feb 2003 11:21:49 -0800 (PST) Date: Sat, 1 Feb 2003 11:21:49 -0800 (PST) From: Matthew Dillon Message-Id: <200302011921.h11JLnrk016623@apollo.backplane.com> To: "Kurt J. Lidl" Cc: Mats Larsson , hackers@FreeBSD.ORG Subject: Re: Lower power SMP boxes? References: <200301312312.h0VNC5bQ007170@apollo.backplane.com> <20030201101041.H16130@marvin.sko.mh.se> <20030201123303.A6376@pix.net> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG : :On Sat, Feb 01, 2003 at 10:17:23AM +0100, Mats Larsson wrote: :> Via just recently announced their new Nehemiah processor capable of smp, :> presumably slow as its precursor but also the lowest power consuming :> processor at the market (at least with standard socket fcpga motherboard) :[...] :> http://www.via.com.tw/en/viac3/c3.jsp : :It says "IO/APIC support in future versions". So, it's not an SMP option :today, as I understand it. : :-Kurt Although, this is more a deficiency in the way FreeBSD is designed. Using an APIC is nice, but not absolutely necessary. All we need are good specs on how VIA's SMP cpus interact with each other and we could support it. I like the 11 watts specified in the paper. That *is* low power for the class of system they are selling. I don't see a clock specification but I assume it is going to be at least as fast as the ~900MHz M-9000. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 11:25:21 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A3FD637B405 for ; Sat, 1 Feb 2003 11:25:19 -0800 (PST) Received: from falcon.midgard.homeip.net (h76n3fls20o913.telia.com [213.67.148.76]) by mx1.FreeBSD.org (Postfix) with SMTP id 3179743F93 for ; Sat, 1 Feb 2003 11:25:17 -0800 (PST) (envelope-from ertr1013@student.uu.se) Received: (qmail 13627 invoked by uid 1001); 1 Feb 2003 19:25:15 -0000 Date: Sat, 1 Feb 2003 20:25:15 +0100 From: Erik Trulsson To: Soeren Schmidt Cc: current@freebsd.org, hackers@freebsd.org Subject: Re: Request for info from SiS chipset owners Message-ID: <20030201192515.GA13542@falcon.midgard.homeip.net> Mail-Followup-To: Soeren Schmidt , current@freebsd.org, hackers@freebsd.org References: <200302011102.h11B25xk077482@spider.deepcore.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: <200302011102.h11B25xk077482@spider.deepcore.dk> User-Agent: Mutt/1.5.3i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Feb 01, 2003 at 12:02:05PM +0100, Soeren Schmidt wrote: >=20 > I'm currently in the midst of an ATA chipset support mega rewrite/update, > and the last item on the list is SiS support. >=20 > That where _you_ come into the picture, I need a pciconf -l from your > SiS based system! >=20 > Just reply to this message with the output from pciconf -l and you > have helped me sort out the myriads of SiS chipsets out there. >=20 > -S=F8ren chip0@pci0:0:0: class=3D0x060000 card=3D0x00000000 chip=3D0x55961039 rev=3D= 0x00 hdr=3D0x00 isab0@pci0:1:0: class=3D0x060100 card=3D0x00000000 chip=3D0x00081039 rev=3D= 0x01 hdr=3D0x00 atapci0@pci0:1:1: class=3D0x01018a card=3D0x00000000 chip=3D0x55131039 rev= =3D0x09 hdr=3D0x00 atapci1@pci0:11:0: class=3D0x018085 card=3D0x4d68105a chip=3D0x4d68105a rev= =3D0x02 hdr=3D0x00 none0@pci0:13:0: class=3D0x030000 card=3D0x63261569 chip=3D0x63261039 rev= =3D0x0b hdr=3D0x00 ed0@pci0:15:0: class=3D0x020000 card=3D0x802910ec chip=3D0x802910ec rev=3D0= x00 hdr=3D0x00 none1@pci0:20:0: class=3D0x030000 card=3D0x00000000 chip=3D0x02051039 rev= =3D0x11 hdr=3D0x00 The chipset used on this motherboard is a SiS 5596/5513. (Even though dmesg likes to claim that the ATA controller is a SiS 5591,=20 which it isn't. It is a SiS 5513.) --=20 Erik Trulsson ertr1013@student.uu.se To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 11:51:26 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 32A1F37B401 for ; Sat, 1 Feb 2003 11:51:25 -0800 (PST) Received: from meketrex.pix.net (meketrex.pix.net [192.111.45.13]) by mx1.FreeBSD.org (Postfix) with ESMTP id D24A243F43 for ; Sat, 1 Feb 2003 11:51:23 -0800 (PST) (envelope-from lidl@meketrex.pix.net) Received: (from lidl@localhost) by meketrex.pix.net (8.11.6/8.11.6) id h11JpLT09440; Sat, 1 Feb 2003 14:51:21 -0500 (EST) Date: Sat, 1 Feb 2003 14:51:21 -0500 From: "Kurt J. Lidl" To: Matthew Dillon Cc: hackers@FreeBSD.ORG Subject: Re: Lower power SMP boxes? Message-ID: <20030201145121.B9269@pix.net> References: <200301312312.h0VNC5bQ007170@apollo.backplane.com> <20030201101041.H16130@marvin.sko.mh.se> <20030201123303.A6376@pix.net> <200302011921.h11JLnrk016623@apollo.backplane.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <200302011921.h11JLnrk016623@apollo.backplane.com>; from dillon@apollo.backplane.com on Sat, Feb 01, 2003 at 11:21:49AM -0800 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Feb 01, 2003 at 11:21:49AM -0800, Matthew Dillon wrote: > :On Sat, Feb 01, 2003 at 10:17:23AM +0100, Mats Larsson wrote: > :> Via just recently announced their new Nehemiah processor capable of smp, > :> presumably slow as its precursor but also the lowest power consuming > :> processor at the market (at least with standard socket fcpga motherboard) > :[...] > :> http://www.via.com.tw/en/viac3/c3.jsp > : > :It says "IO/APIC support in future versions". So, it's not an SMP option > :today, as I understand it. > : > :-Kurt > > Although, this is more a deficiency in the way FreeBSD is designed. Using > an APIC is nice, but not absolutely necessary. All we need are good > specs on how VIA's SMP cpus interact with each other and we could > support it. How else are you going to do the physical interrupt steering? Unless they have gone through the effort of implementing a whole new and different steering mechanism -- which would fly in the face of having off-the-shelf OS support from the people in Redmond, at the very least. > I like the 11 watts specified in the paper. That *is* low power for > the class of system they are selling. I don't see a clock specification > but I assume it is going to be at least as fast as the ~900MHz M-9000. It says "the new generation VIA C3 processor is available at speeds starting at 1GHz" in the paragraph under "Enhanced Digital Media Performance". -Kurt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 11:56:32 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C64C937B401 for ; Sat, 1 Feb 2003 11:56:31 -0800 (PST) Received: from jive.SoftHome.net (jive.SoftHome.net [66.54.152.27]) by mx1.FreeBSD.org (Postfix) with SMTP id 277E943F3F for ; Sat, 1 Feb 2003 11:56:31 -0800 (PST) (envelope-from alhakeem@softhome.net) Received: (qmail 20044 invoked by uid 417); 1 Feb 2003 19:56:30 -0000 Received: from shunt-smtp-out-0 (HELO softhome.net) (172.16.3.12) by shunt-smtp-out-0 with SMTP; 1 Feb 2003 19:56:30 -0000 Received: from laptop ([62.30.180.59]) by softhome.net with esmtp; Sat, 01 Feb 2003 12:56:30 -0700 From: "Abdul Hakeem" To: freebsd-hackers@FreeBSD.org Subject: CD-ROM Date: Sat, 1 Feb 2003 19:56:54 -0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2627 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Hi, Could you pls tell me how to obtain the cd-rom or download the 64bit port of FreeBSD ? Regards, Abdul Hakeem To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 11:56:36 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D7F8237B407 for ; Sat, 1 Feb 2003 11:56:34 -0800 (PST) Received: from jive.SoftHome.net (jive.SoftHome.net [66.54.152.27]) by mx1.FreeBSD.org (Postfix) with SMTP id 677AC43F3F for ; Sat, 1 Feb 2003 11:56:34 -0800 (PST) (envelope-from alhakeem@softhome.net) Received: (qmail 19963 invoked by uid 417); 1 Feb 2003 19:56:29 -0000 Received: from shunt-smtp-out-0 (HELO softhome.net) (172.16.3.12) by shunt-smtp-out-0 with SMTP; 1 Feb 2003 19:56:29 -0000 Received: from laptop ([62.30.180.59]) by softhome.net with esmtp; Sat, 01 Feb 2003 12:56:28 -0700 From: "Abdul Hakeem" To: hackers@FreeBSD.org Subject: subscribe Date: Sat, 1 Feb 2003 19:56:54 -0000 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.2627 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2600.0000 Importance: Normal Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG subscribe To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 11:59:46 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E3AE537B401 for ; Sat, 1 Feb 2003 11:59:45 -0800 (PST) Received: from rhadamanth.submonkey.net (pc2-cdif2-5-cust247.cdif.cable.ntl.com [81.101.151.247]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7F9CE43F75 for ; Sat, 1 Feb 2003 11:59:44 -0800 (PST) (envelope-from setantae@submonkey.net) Received: from setantae by rhadamanth.submonkey.net with local (Exim 4.12) id 18f3ni-000GiQ-00; Sat, 01 Feb 2003 19:59:42 +0000 Date: Sat, 1 Feb 2003 19:59:42 +0000 From: Ceri Davies To: Julian Elischer Cc: hackers@FreeBSD.org Subject: Re: Hi!Dear FreeBSD! Message-ID: <20030201195942.GA52947@submonkey.net> Mail-Followup-To: Ceri Davies , Julian Elischer , hackers@FreeBSD.org References: <3E3BEC87.80600@gmx.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-message-flag: All your linuxconf-configured redhat are belong to us. X-message-flag-attribution: suresh, sdm. User-Agent: Mutt/1.5.3i Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, Feb 01, 2003 at 10:45:48AM -0800, Julian Elischer wrote: > maybe we should make some sort of geographical registration > web page so that people can find each other? ports/astro/xearth/files/freebsd.committers.markers ? Ceri -- The power of the heroic dwarves has come! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 12:17:11 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1BD7B37B401 for ; Sat, 1 Feb 2003 12:17:10 -0800 (PST) Received: from rwcrmhc53.attbi.com (rwcrmhc53.attbi.com [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id BCCB343F3F for ; Sat, 1 Feb 2003 12:17:09 -0800 (PST) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc53.attbi.com (rwcrmhc53) with ESMTP id <20030201201708053003lghje>; Sat, 1 Feb 2003 20:17:09 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA56011; Sat, 1 Feb 2003 12:17:08 -0800 (PST) Date: Sat, 1 Feb 2003 12:17:07 -0800 (PST) From: Julian Elischer To: Ceri Davies Cc: hackers@FreeBSD.org Subject: Re: Hi!Dear FreeBSD! In-Reply-To: <20030201195942.GA52947@submonkey.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sat, 1 Feb 2003, Ceri Davies wrote: > On Sat, Feb 01, 2003 at 10:45:48AM -0800, Julian Elischer wrote: > > > maybe we should make some sort of geographical registration > > web page so that people can find each other? > > ports/astro/xearth/files/freebsd.committers.markers ? I was going to add.. "and not xearth" but I decided it wouldnt be needed :-) I was thinking of something more like a simple webpage with country, city, name, and a way to have yourself added/deleted. > > Ceri > > -- > The power of the heroic dwarves has come! > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 13: 8:33 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B681E37B401 for ; Sat, 1 Feb 2003 13:08:31 -0800 (PST) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2D94643F43 for ; Sat, 1 Feb 2003 13:08:31 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0019.cvx22-bradley.dialup.earthlink.net ([209.179.198.19] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18f4rz-0004iP-00; Sat, 01 Feb 2003 13:08:12 -0800 Message-ID: <3E3C36E7.1A77AABA@mindspring.com> Date: Sat, 01 Feb 2003 13:06:47 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Matthew Dillon Cc: "Kurt J. Lidl" , Mats Larsson , hackers@FreeBSD.ORG Subject: Re: Lower power SMP boxes? References: <200301312312.h0VNC5bQ007170@apollo.backplane.com> <20030201101041.H16130@marvin.sko.mh.se> <20030201123303.A6376@pix.net> <200302011921.h11JLnrk016623@apollo.backplane.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4136153dbc9d384b72bb8768727881df23ca473d225a0f487350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Matthew Dillon wrote: > :It says "IO/APIC support in future versions". So, it's not an SMP option > :today, as I understand it. > > Although, this is more a deficiency in the way FreeBSD is designed. Using > an APIC is nice, but not absolutely necessary. All we need are good > specs on how VIA's SMP cpus interact with each other and we could > support it. You mean FreeBSD's requirement that Intel-based motherboards comply with the Intel Multiprocessing specification is a deficiency in FreeBSD? 8-). The best you can get is to modify the MMU and/or replace the L2 cache with hardware that performs negotiations. Even then, the coherency model would have to move from MESI, which is how it operates in the presence of an APIC, to MEI, instead, a *very significant* change, particularly for locking, which would require explicit signalling for synchronization. MEI is the model used by the BeBox and other PPC multiprocessors that were never intended to be SMP-ized: the PPC 601 and PPC 603 processors were never intended to be used in an SMP configuration, FWIW. > I like the 11 watts specified in the paper. That *is* low power for > the class of system they are selling. I don't see a clock specification > but I assume it is going to be at least as fast as the ~900MHz M-9000. The easiest thing to do is under-clock a very fast processor, if what you want to do is drop power requirements, as you've said. Running at a lower clock multiplier has other significant advantages, as well, since the interprocessor communication imposes a stall barrier in the arbitration logic (whether it's MESI, or MEI), so the closer the CPU is to the arbitration logic speed, the less the multiplier amplifies stalls into empty instruction pipelines. I'm pretty sure that Intel no longer sells the external APIC chips that used to be used to make 486 SMP systems that complied with the Intel MP Specification (the 486 never had an internal APIC, either), but that might also be an option. Were that the path taken, then there would be no serious changes required in FreeBSD (except perhaps the need to support MP Spec. 1.1 as well as 1.4, and FreeBSD did that successfully in the past). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 13:12: 1 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6A6A937B401 for ; Sat, 1 Feb 2003 13:12:00 -0800 (PST) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E7DF43F85 for ; Sat, 1 Feb 2003 13:12:00 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from pool0019.cvx22-bradley.dialup.earthlink.net ([209.179.198.19] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18f4vb-0005KJ-00; Sat, 01 Feb 2003 13:11:55 -0800 Message-ID: <3E3C37C7.3D6AFA2A@mindspring.com> Date: Sat, 01 Feb 2003 13:10:31 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer Cc: Ceri Davies , hackers@FreeBSD.org Subject: Re: Hi!Dear FreeBSD! References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4136153dbc9d384b719649e32fceecebd93caf27dac41a8fd350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Julian Elischer wrote: > On Sat, 1 Feb 2003, Ceri Davies wrote: > > On Sat, Feb 01, 2003 at 10:45:48AM -0800, Julian Elischer wrote: > > > > > maybe we should make some sort of geographical registration > > > web page so that people can find each other? > > > > ports/astro/xearth/files/freebsd.committers.markers ? > > I was going to add.. > "and not xearth" but I decided it wouldnt be needed :-) > > I was thinking of something more like a simple webpage with country, > city, name, and a way to have yourself added/deleted. So what you need is a program that takes a GIF of a Mercater Cylindrical Projection, and takes the xearth input files, and integrates red dots for committers, outputting a modified GIF, right? 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 13:37:13 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4D10737B401 for ; Sat, 1 Feb 2003 13:37:12 -0800 (PST) Received: from mailsrv.otenet.gr (mailsrv.otenet.gr [195.170.0.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 57FC443F3F for ; Sat, 1 Feb 2003 13:37:09 -0800 (PST) (envelope-from keramida@ceid.upatras.gr) Received: from gothmog.gr (patr530-b018.otenet.gr [195.167.121.146]) by mailsrv.otenet.gr (8.12.6/8.12.6) with ESMTP id h11Lb0Vq017398; Sat, 1 Feb 2003 23:37:01 +0200 (EET) Received: from gothmog.gr (gothmog [127.0.0.1]) by gothmog.gr (8.12.6/8.12.6) with ESMTP id h11LaxMP016151; Sat, 1 Feb 2003 23:36:59 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Received: (from giorgos@localhost) by gothmog.gr (8.12.6/8.12.6/Submit) id h11LaxHL016150; Sat, 1 Feb 2003 23:36:59 +0200 (EET) (envelope-from keramida@ceid.upatras.gr) Date: Sat, 1 Feb 2003 23:36:58 +0200 From: Giorgos Keramidas To: Terry Lambert Cc: Julian Elischer , Ceri Davies , hackers@FreeBSD.ORG Subject: Re: Hi!Dear FreeBSD! Message-ID: <20030201213658.GB15915@gothmog.gr> References: <3E3C37C7.3D6AFA2A@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3E3C37C7.3D6AFA2A@mindspring.com> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On 2003-02-01 13:10, Terry Lambert wrote: > Julian Elischer wrote: > > On Sat, 1 Feb 2003, Ceri Davies wrote: > > > On Sat, Feb 01, 2003 at 10:45:48AM -0800, Julian Elischer wrote: > > > > > > > maybe we should make some sort of geographical registration > > > > web page so that people can find each other? > > > > > > ports/astro/xearth/files/freebsd.committers.markers ? > > > > I was going to add.. > > "and not xearth" but I decided it wouldnt be needed :-) > > > > I was thinking of something more like a simple webpage with country, > > city, name, and a way to have yourself added/deleted. > > So what you need is a program that takes a GIF of a Mercater > Cylindrical Projection, and takes the xearth input files, and > integrates red dots for committers, outputting a modified GIF, > right? 8-). That would work, yes. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 15:14:47 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1894137B401 for ; Sat, 1 Feb 2003 15:14:46 -0800 (PST) Received: from sift.mirapoint.com (sift.mirapoint.com [63.107.133.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id AA3ED43F75 for ; Sat, 1 Feb 2003 15:14:45 -0800 (PST) (envelope-from zach@mirapoint.com) Received: from alpo.mirapoint.com (alpo.mirapoint.com [63.107.133.20]) by sift.mirapoint.com (Mirapoint Messaging Server MOS 3.3.2-CR) with ESMTP id ABL86961; Sat, 1 Feb 2003 15:14:40 -0800 (PST) Received: from 12.234.116.251 by alpo.mirapoint.com (Mirapoint Messaging Server MOS 3.3.2-CR) with HTTPS/1.1; Sat, 1 Feb 2003 15:14:39 -0800 Date: Sat, 1 Feb 2003 15:14:39 -0800 From: Zachary Amsden Subject: [4.1] Bug in calcru? To: freebsd-hackers@freebsd.org X-Mailer: Webmail Mirapoint Direct 3.3.2-CR MIME-Version: 1.0 Message-Id: <17f41996.4f9133b6.819db00@alpo.mirapoint.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG I believe that spl() protection is required here, in fact, that it should be extended to cover the read of p->p_runtime as well. These are all 64 bit integers, so on IA32, the worst case is two consecutive off-by one reads during overflow - i.e reading 0xffffffff 0x00000002 instead of 0x00000000 0x00000002. That is a rather significant difference, and since the numbers used in computation here become part of a kassert() later, a rather dangerous one. Alternatively, a 64 bit atomic read could be used. On FreeBSD-current, this entire operation is protected by a mutex, which looks safe to me, but in 4.1, this looks like a bug: void calcru(p, up, sp, ip) struct proc *p; struct timeval *up; struct timeval *sp; struct timeval *ip; { /* {user, system, interrupt, total} {ticks, usec}; previous tu: */ u_int64_t ut, uu, st, su, it, iu, tt, tu, ptu; int s; struct timeval tv; /* XXX: why spl-protect ? worst case is an off-by-one report */ s = splstatclock(); ut = p->p_uticks; st = p->p_sticks; it = p->p_iticks; splx(s); tt = ut + st + it; if (tt == 0) { st = 1; tt = 1; } tu = p->p_runtime; "A plague upon all your houses" - last words of Waldo Semon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message From owner-freebsd-hackers Sat Feb 1 21:58:36 2003 Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0831237B401 for ; Sat, 1 Feb 2003 21:58:36 -0800 (PST) Received: from 002.216-123-229-0.interbaun.com (002.216-123-229-0.interbaun.com [216.123.229.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7870443F43 for ; Sat, 1 Feb 2003 21:58:32 -0800 (PST) (envelope-from soralx@cydem.zp.ua) Received: from 128.216-123-229-0.interbaun.com ([192.168.0.224]) by 002.216-123-229-0.interbaun.com (8.11.6/8.11.6) with ESMTP id h1235cr60292 for ; Sat, 1 Feb 2003 20:05:39 -0700 (MST) (envelope-from soralx@cydem.zp.ua) Content-Type: text/plain; charset="iso-8859-1" From: To: hackers@FreeBSD.ORG Subject: Re: Lower power SMP boxes? Date: Sat, 1 Feb 2003 19:56:00 -0700 X-Mailer: KMail [version 1.4] References: <200301312312.h0VNC5bQ007170@apollo.backplane.com> <200302011921.h11JLnrk016623@apollo.backplane.com> <3E3C36E7.1A77AABA@mindspring.com> In-Reply-To: <3E3C36E7.1A77AABA@mindspring.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Message-Id: <200302011956.00791.soralx@cydem.zp.ua> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG > The easiest thing to do is under-clock a very fast processor, if > what you want to do is drop power requirements, as you've said. I would _really_ like to watch this process: you underclocking new Xeon 2GHz to 1GHz (minimum speed of VIA Nehemiah), trying to lower Vcc by 0.1V at most and get it working, and finally putting passvie radiator on it :) 50 Amps actually isn't too much, and 55W of heat - it's only 2 medium-power soldering irons... 8) 01.02.2003; 18:49:57 [SorAlx] http://cydem.zp.ua/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message