From owner-freebsd-arch Sun Jun 4 14: 9:14 2000 Delivered-To: freebsd-arch@freebsd.org Received: from peach.ocn.ne.jp (peach.ocn.ne.jp [210.145.254.87]) by hub.freebsd.org (Postfix) with ESMTP id 1B55537B50B for ; Sun, 4 Jun 2000 14:09:12 -0700 (PDT) (envelope-from dcs@newsguy.com) Received: from daniel.sobral (p09-dn03kiryunisiki.gunma.ocn.ne.jp [210.232.224.138]) by peach.ocn.ne.jp (8.9.1a/OCN/) with ESMTP id GAA17340 for ; Mon, 5 Jun 2000 06:09:09 +0900 (JST) Received: (from dcs@localhost) by daniel.sobral (8.9.3/8.9.3) id GAA00754 for arch@freebsd.org; Mon, 5 Jun 2000 06:10:19 +0900 (JST) (envelope-from dcs) From: "Daniel C. Sobral" Message-Id: <200006042110.GAA00754@daniel.sobral> Subject: regex(3) To: arch@freebsd.org Date: Mon, 5 Jun 2000 06:10:18 +0900 (JST) Disclaimer: Klaatu Barada Nikto! X-Mailer: ELM [version 2.4ME+ PL68 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I have been examining the regex(3) code for one of my projects, and saw some room for improvement on it. I'd like to ask: 1) Does anyone have any strong feelings about me hacking it? 2) Can anyone point me to recent regex standards? Our grep (GNU grep) supports a different syntax than our library. While that's normal for GNU add-ons, there are some stuff they *don't* support, or support with a different syntax. It got me wondering... At the very least, references to grep(1) ought to be removed from regex(3). -- Daniel C. Sobral (8-DCS) dcs@newsguy.com dcs@freebsd.org capo@there.is.no.bsdconspiracy.net Swipple's Rule of Order: He who shouts the loudest has the floor. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jun 4 15: 8:47 2000 Delivered-To: freebsd-arch@freebsd.org Received: from po4.glue.umd.edu (po4.glue.umd.edu [128.8.10.124]) by hub.freebsd.org (Postfix) with ESMTP id 7689437B64E for ; Sun, 4 Jun 2000 15:08:42 -0700 (PDT) (envelope-from howardjp@glue.umd.edu) Received: from y.glue.umd.edu (root@y.glue.umd.edu [128.8.10.68]) by po4.glue.umd.edu (8.9.3/8.9.3) with ESMTP id SAA22795; Sun, 4 Jun 2000 18:08:34 -0400 (EDT) Received: from y.glue.umd.edu (sendmail@localhost [127.0.0.1]) by y.glue.umd.edu (8.9.3/8.9.3) with SMTP id SAA20532; Sun, 4 Jun 2000 18:08:33 -0400 (EDT) Received: from localhost (howardjp@localhost) by y.glue.umd.edu (8.9.3/8.9.3) with ESMTP id SAA20528; Sun, 4 Jun 2000 18:08:32 -0400 (EDT) X-Authentication-Warning: y.glue.umd.edu: howardjp owned process doing -bs Date: Sun, 4 Jun 2000 18:08:32 -0400 (EDT) From: James Howard To: "Daniel C. Sobral" Cc: arch@FreeBSD.ORG Subject: Re: regex(3) In-Reply-To: <200006042110.GAA00754@daniel.sobral> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, 5 Jun 2000, Daniel C. Sobral wrote: > I have been examining the regex(3) code for one of my projects, and saw > some room for improvement on it. I'd like to ask: > > 1) Does anyone have any strong feelings about me hacking it? > > 2) Can anyone point me to recent regex standards? Please look at bin/14342. It something like quadruples the speed of regexec (or more, it was a long time ago). Jamie To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jun 4 17:49:55 2000 Delivered-To: freebsd-arch@freebsd.org Received: from sharmas.dhs.org (c62443-a.frmt1.sfba.home.com [24.0.69.165]) by hub.freebsd.org (Postfix) with ESMTP id 93A6537BBA6 for ; Sun, 4 Jun 2000 17:49:47 -0700 (PDT) (envelope-from adsharma@sharmas.dhs.org) Received: (from adsharma@localhost) by sharmas.dhs.org (8.9.3/8.9.3) id RAA28581; Sun, 4 Jun 2000 17:49:36 -0700 Date: Sun, 4 Jun 2000 17:49:36 -0700 From: Arun Sharma To: "Daniel C. Sobral" Cc: arch@FreeBSD.ORG Subject: Re: regex(3) Message-ID: <20000604174936.A28574@sharmas.dhs.org> References: <200006042110.GAA00754@daniel.sobral> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i In-Reply-To: <200006042110.GAA00754@daniel.sobral>; from Daniel C. Sobral on Mon, Jun 05, 2000 at 06:10:18AM +0900 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, Jun 05, 2000 at 06:10:18AM +0900, Daniel C. Sobral wrote: > 2) Can anyone point me to recent regex standards? I don't know if it's a standard or not, but it could be useful to you: http://jakarta.apache.org/regexp/index.html -Arun To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sun Jun 4 23:47:37 2000 Delivered-To: freebsd-arch@freebsd.org Received: from relay.nuxi.com (nuxi.cs.ucdavis.edu [169.237.7.38]) by hub.freebsd.org (Postfix) with ESMTP id 1BBC437B597 for ; Sun, 4 Jun 2000 23:47:35 -0700 (PDT) (envelope-from obrien@NUXI.com) Received: from dragon.nuxi.com (root@[209.152.133.57]) by relay.nuxi.com (8.9.3/8.9.3) with ESMTP id XAA62772; Sun, 4 Jun 2000 23:47:33 -0700 (PDT) (envelope-from obrien@dragon.nuxi.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.9.3/8.9.1) id XAA34755; Sun, 4 Jun 2000 23:47:51 -0700 (PDT) (envelope-from obrien) Date: Sun, 4 Jun 2000 23:47:51 -0700 From: "David O'Brien" To: "Daniel C. Sobral" Cc: arch@freebsd.org Subject: Re: regex(3) Message-ID: <20000604234751.A34712@dragon.nuxi.com> Reply-To: obrien@freebsd.org References: <200006042110.GAA00754@daniel.sobral> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <200006042110.GAA00754@daniel.sobral>; from dcs@newsguy.com on Mon, Jun 05, 2000 at 06:10:18AM +0900 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-PGP-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Keyid: 34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, Jun 05, 2000 at 06:10:18AM +0900, Daniel C. Sobral wrote: > I have been examining the regex(3) code for one of my projects, and saw > some room for improvement on it. I'd like to ask: > > 1) Does anyone have any strong feelings about me hacking it? Yes. I've been communicating with Henry Spencer about his new regex library in order to updated ours. -- -- David (obrien@FreeBSD.org) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jun 5 2:31:32 2000 Delivered-To: freebsd-arch@freebsd.org Received: from peach.ocn.ne.jp (peach.ocn.ne.jp [210.145.254.87]) by hub.freebsd.org (Postfix) with ESMTP id 62BE837B50E; Mon, 5 Jun 2000 02:31:30 -0700 (PDT) (envelope-from dcs@newsguy.com) Received: from newsguy.com (p17-dn01kiryunisiki.gunma.ocn.ne.jp [211.0.245.18]) by peach.ocn.ne.jp (8.9.1a/OCN/) with ESMTP id SAA05458; Mon, 5 Jun 2000 18:31:29 +0900 (JST) Message-ID: <393B73B6.AD5BCA2B@newsguy.com> Date: Mon, 05 Jun 2000 18:32:38 +0900 From: "Daniel C. Sobral" X-Mailer: Mozilla 4.7 [en] (Win98; I) X-Accept-Language: en,pt-BR,ja MIME-Version: 1.0 To: obrien@freebsd.org Cc: arch@freebsd.org Subject: Re: regex(3) References: <200006042110.GAA00754@daniel.sobral> <20000604234751.A34712@dragon.nuxi.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG David O'Brien wrote: > > On Mon, Jun 05, 2000 at 06:10:18AM +0900, Daniel C. Sobral wrote: > > I have been examining the regex(3) code for one of my projects, and saw > > some room for improvement on it. I'd like to ask: > > > > 1) Does anyone have any strong feelings about me hacking it? > > Yes. I've been communicating with Henry Spencer about his new regex > library in order to updated ours. Ah, good, then. Do you have an url for his new library? -- Daniel C. Sobral (8-DCS) dcs@newsguy.com dcs@freebsd.org capo@yet.another.bsdconspiracy.org Hmmm - I have to go check this. My reality assumptions are shattered. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jun 5 6:35:31 2000 Delivered-To: freebsd-arch@freebsd.org Received: from smtprich.nortel.com (smtprich.nortel.com [192.135.215.8]) by hub.freebsd.org (Postfix) with ESMTP id EA87137BCCA for ; Mon, 5 Jun 2000 06:35:24 -0700 (PDT) (envelope-from malachai@iname.com) Received: from ertpg15e1.nortelnetworks.com (actually zrtph06n) by smtprich.nortel.com; Mon, 5 Jun 2000 08:25:48 -0500 Received: from zrtpd004.us.nortel.com (actually zrtpd004) by ertpg15e1.nortelnetworks.com; Mon, 5 Jun 2000 09:33:48 -0400 Received: from nhmbd14.us.nortel.com ([47.120.192.112]) by zrtpd004.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id L3HQCLP8; Mon, 5 Jun 2000 09:33:45 -0400 Received: from cs163.humb.nt.com (nhmbn472.us.nortel.com [47.100.194.163]) by nhmbd14.us.nortel.com with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id L1MV6M1Y; Mon, 5 Jun 2000 09:33:38 -0400 Date: Mon, 5 Jun 2000 09:33:10 -0400 X-Sybari-Space: 00000000 00000000 00000000 From: Shawn Halpenny To: "Daniel C. Sobral" Cc: arch@FreeBSD.ORG Subject: Re: regex(3) Message-ID: <20000605093310.A22370@cs163.humb.nt.com> References: <200006042110.GAA00754@daniel.sobral> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0.1us In-Reply-To: <200006042110.GAA00754@daniel.sobral> X-Orig: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, Jun 04, 2000 at 05:10:18PM EDT > I have been examining the regex(3) code for one of my projects, and saw > some room for improvement on it. I'd like to ask: > > 1) Does anyone have any strong feelings about me hacking it? > > 2) Can anyone point me to recent regex standards? > > Our grep (GNU grep) supports a different syntax than our library. While > that's normal for GNU add-ons, there are some stuff they *don't* > support, or support with a different syntax. It got me wondering... > > At the very least, references to grep(1) ought to be removed from > regex(3). While not entirely related, I'm considering http://ourworld.compuserve.com/homepages/John_Maddock/regexpp.htm for a regex-heavy project I'm involved in. -- Shawn Halpenny | Maniacal@I Ache, Ohm | "Universal Danger!" +- - - - - - - - - - - - - - - - + - - - - - - - - - - - - - - \ | vi:G3kfM~lxfAPXh~l~2x2FirllpfcxlrifaprmfOX~Xp2hr.lrcelyl2p - - - - - - - -| fU~X~refsPprnlxppri2lxlpr,pFrpprrfaPlpfiprgllxp~3Xlpfndw To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jun 5 10:20:10 2000 Delivered-To: freebsd-arch@freebsd.org Received: from gratis.grondar.za (grouter.grondar.za [196.7.18.65]) by hub.freebsd.org (Postfix) with ESMTP id 7742F37C0A2 for ; Mon, 5 Jun 2000 10:20:04 -0700 (PDT) (envelope-from mark@grimreaper.grondar.za) Received: from grimreaper.grondar.za (gratis.grondar.za [196.7.18.133]) by gratis.grondar.za (8.9.3/8.9.3) with ESMTP id TAA18713 for ; Mon, 5 Jun 2000 19:20:00 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200006051720.TAA18713@gratis.grondar.za> To: arch@FreeBSD.org Subject: (2nd iteration) New /dev/(random|null|zero) - review, please From: Mark Murray Date: Mon, 05 Jun 2000 19:20:11 +0200 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG [ Sent to arch at the suggestion of David O'Brien ] (Some improvements have been done - 2nd iteration) Hi I have finished doing a MI /dev/null and /dev/zero, and I have got a new /dev/random. I'm looking for reviewers. The code is in freefall:~markm/NEWDEVS/*. There is a tar file and diffs (all for the sys/ area). Some other supplementary patches are needed in userland, these are not included. 2nd Iteration Improvements: o /dev/null and /dev/zero or no longer optional; they are "standard". o /dev/zero uses malloc(9) to get its space, not a hard-coded block of zeros. Malloc is done once; at device startup. NOTES: o The devices are (can be) modules, or by setting options, they can be hard coded into the kernel. I would like to make them autoload somehow, but I'm not sure how. o I'd like to make the devices "pseudo-devices", rather than options. Comments? o The random number generator will give random-looking output, but does absolutely no harvesting of entropy at the moment. Because I want it to be a loadable module, I need some way of registering the entropy harvesting routines. Something like weak-symboled routines that are overridden when the module is loaded would be ideal. Suggestions? o I am using Brice Schneier's "Yarrow" algorithm for the RNG; I have only supplied enough of it now to give "sort of" random numbers. As I solve the harvesting problem, I'll improve on that. o The RNG is slow; the others are much faster than their originals. o I intend to use sysctl(9) to set most of yarrow's "tweakables". Thanks to Jeroen van Gelderen for some excellent ideas on optimization! M To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Mon Jun 5 11:17:11 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cypherpunks.ai (cypherpunks.ai [209.88.68.47]) by hub.freebsd.org (Postfix) with ESMTP id 75BAA37B763; Mon, 5 Jun 2000 11:17:07 -0700 (PDT) (envelope-from jeroen@vangelderen.org) Received: from vangelderen.org (grolsch.ai [209.88.68.214]) by cypherpunks.ai (Postfix) with ESMTP id 262B949; Mon, 5 Jun 2000 14:17:07 -0400 (AST) Message-ID: <393BEE84.BBAD3E82@vangelderen.org> Date: Mon, 05 Jun 2000 14:16:36 -0400 From: "Jeroen C. van Gelderen" X-Mailer: Mozilla 4.72 [en] (X11; I; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Mark Murray Cc: arch@FreeBSD.org, phk@freebsd.org Subject: Re: (2nd iteration) New /dev/(random|null|zero) - review, please References: <200006051720.TAA18713@gratis.grondar.za> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Mark Murray wrote: > > [ Sent to arch at the suggestion of David O'Brien ] > (Some improvements have been done - 2nd iteration) > > Hi > > I have finished doing a MI /dev/null and /dev/zero, and I have got a > new /dev/random. I'm looking for reviewers. I've already submitted a MI /dev/[null|zero] for commit to PHK. He said he would commit them after testing. It's the same driver you have already seen at http://jeroen.vangelderen.org/FreeBSD . > o The random number generator will give random-looking output, but does > absolutely no harvesting of entropy at the moment. Because I want > it to be a loadable module, I need some way of registering the entropy > harvesting routines. Something like weak-symboled routines that are > overridden when the module is loaded would be ideal. Suggestions? Split-level. Entropy sources should export an entropy device. Yarrow should bind to all available entropy devices and use those. This would allow for - entropy devices in KLDs. - dynamic addition/removal of entropy sources (USB). - separation of RNG policy (Yarrow) from entropy gathering. - dynamic IRQs not affecting RNG security. > o The RNG is slow; the others are much faster than their originals. Can be tweaked. Use a 256-bit cipher like Rijndael and build a hash out of it. Would improve security too as the entropy pool would hold 256 bits. You can also pre-generate a few KB of random bits. Cheers, Jeroen -- Jeroen C. van Gelderen o _ _ _ jeroen@vangelderen.org _o /\_ _ \\o (_)\__/o (_) _< \_ _>(_) (_)/<_ \_| \ _|/' \/ (_)>(_) (_) (_) (_) (_)' _\o_ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 1:14:39 2000 Delivered-To: freebsd-arch@freebsd.org Received: from gratis.grondar.za (grouter.grondar.za [196.7.18.65]) by hub.freebsd.org (Postfix) with ESMTP id 57BE937BF71; Tue, 6 Jun 2000 01:14:31 -0700 (PDT) (envelope-from mark@grimreaper.grondar.za) Received: from grimreaper.grondar.za (gratis.grondar.za [196.7.18.133]) by gratis.grondar.za (8.9.3/8.9.3) with ESMTP id KAA21235; Tue, 6 Jun 2000 10:14:18 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200006060814.KAA21235@gratis.grondar.za> To: "Jeroen C. van Gelderen" Cc: Mark Murray , arch@FreeBSD.org, phk@FreeBSD.org Subject: Re: (2nd iteration) New /dev/(random|null|zero) - review, please References: <393BEE84.BBAD3E82@vangelderen.org> In-Reply-To: <393BEE84.BBAD3E82@vangelderen.org> ; from "Jeroen C. van Gelderen" "Mon, 05 Jun 2000 14:16:36 -0400." Date: Tue, 06 Jun 2000 10:14:23 +0200 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > I have finished doing a MI /dev/null and /dev/zero, and I have got a > > new /dev/random. I'm looking for reviewers. > > I've already submitted a MI /dev/[null|zero] for commit to PHK. He > said he would commit them after testing. It's the same driver you > have already seen at http://jeroen.vangelderen.org/FreeBSD . I stole lots of ideas from you :-); I think mine is faster... > > o The random number generator will give random-looking output, but does > > absolutely no harvesting of entropy at the moment. Because I want > > it to be a loadable module, I need some way of registering the entropy > > harvesting routines. Something like weak-symboled routines that are > > overridden when the module is loaded would be ideal. Suggestions? > > Split-level. Entropy sources should export an entropy device. Yarrow > should bind to all available entropy devices and use those. This would > allow for > - entropy devices in KLDs. > - dynamic addition/removal of entropy sources (USB). > - separation of RNG policy (Yarrow) from entropy gathering. > - dynamic IRQs not affecting RNG security. Makes sense, but I need an actual mechanism for achiving this. BDE suggested the module event handler; I need to investigate. > > o The RNG is slow; the others are much faster than their originals. > > Can be tweaked. Use a 256-bit cipher like Rijndael and build a hash > out of it. Would improve security too as the entropy pool would hold > 256 bits. You can also pre-generate a few KB of random bits. Hmmm.... Sounds good! Got a Rijndael reference? M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 3:41:38 2000 Delivered-To: freebsd-arch@freebsd.org Received: from grisu.bik-gmbh.de (grisu.bik-gmbh.de [194.233.237.82]) by hub.freebsd.org (Postfix) with ESMTP id 8CCE237B75F for ; Tue, 6 Jun 2000 03:41:34 -0700 (PDT) (envelope-from cracauer@counter.bik-gmbh.de) Received: from counter.bik-gmbh.de (counter.bik-gmbh.de [194.233.237.131]) by grisu.bik-gmbh.de (8.9.3/8.9.3) with ESMTP id MAA25101; Tue, 6 Jun 2000 12:41:29 +0200 (MEST) Received: (from cracauer@localhost) by counter.bik-gmbh.de (8.9.3/8.8.8) id MAA30518; Tue, 6 Jun 2000 12:41:17 +0200 (CEST) (envelope-from cracauer) Date: Tue, 6 Jun 2000 12:41:17 +0200 From: Martin Cracauer To: "David O'Brien" Cc: arch@FreeBSD.ORG Subject: Re: Plans to change our debugging format to DWARF2 Message-ID: <20000606124116.A16993@cons.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.4i Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG "David O'Brien" wrote: > The GDB developers have encourage me to change our native debugging > format from STABS to DWARF2 (for ELF binaries). This is your chance to > comment before I make my decision. Thoughts: - 5.0 has a long time until it must be release-ready. - So far we were happy with radical changes like this (to a major part due to your work, thanks!). - Although I don't like it, C++ is still growing importance from my point of view. Any idea what the Linux folks are going to do? When they switch, we have to do it anyway (to get continued support by GNU maintainers). If we switch before them, we have the opportunity to influence the development of the GNU tools before zillions of Linuxers make noise (not saying that all Linuxers are like this). Martin P.S. I'm sticking my nose into binutils for other reasons anyway and would be available for responsive reviews. -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer http://www.bik-gmbh.de/~cracauer/ FreeBSD - where you want to go. Today. http://www.freebsd.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 3:53:14 2000 Delivered-To: freebsd-arch@freebsd.org Received: from grisu.bik-gmbh.de (grisu.bik-gmbh.de [194.233.237.82]) by hub.freebsd.org (Postfix) with ESMTP id EB6A637C007 for ; Tue, 6 Jun 2000 03:53:08 -0700 (PDT) (envelope-from cracauer@counter.bik-gmbh.de) Received: from counter.bik-gmbh.de (counter.bik-gmbh.de [194.233.237.131]) by grisu.bik-gmbh.de (8.9.3/8.9.3) with ESMTP id MAA25603; Tue, 6 Jun 2000 12:53:03 +0200 (MEST) Received: (from cracauer@localhost) by counter.bik-gmbh.de (8.9.3/8.8.8) id MAA40047; Tue, 6 Jun 2000 12:52:33 +0200 (CEST) (envelope-from cracauer) Date: Tue, 6 Jun 2000 12:52:32 +0200 From: Martin Cracauer To: "\"Daniel C. Sobral\"" Cc: arch@FreeBSD.ORG Subject: Re: regex(3) Message-ID: <20000606125232.A33659@cons.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.4i In-Reply-To: ; from "Daniel C. Sobral" on Mon, Jun 05, 2000 at 12:13:24PM +0200 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In , "Daniel C. Sobral" wrote: > I have been examining the regex(3) code for one of my projects, and saw > some room for improvement on it. I'd like to ask: > > 1) Does anyone have any strong feelings about me hacking it? > > 2) Can anyone point me to recent regex standards? > > Our grep (GNU grep) supports a different syntax than our library. While > that's normal for GNU add-ons, there are some stuff they *don't* > support, or support with a different syntax. It got me wondering... We have to obey to standards and if they say that some of the tools differ... Here's the official stuff (free login required): http://www.opengroup.org/onlinepubs/007908799/xsh/regcomp.html http://www.opengroup.org/onlinepubs/007908799/xbd/re.html http://www.opengroup.org/onlinepubs/007908799/xcu/grep.html There is no best and fastest and design issues like prefer-longest-match vs. speed must be understood. The O'Reilly on regex is a must-read for this kind of work. Martin -- %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% Martin Cracauer http://www.bik-gmbh.de/~cracauer/ FreeBSD - where you want to go. Today. http://www.freebsd.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 8: 0:17 2000 Delivered-To: freebsd-arch@freebsd.org Received: from relay.nuxi.com (nuxi.cs.ucdavis.edu [169.237.7.38]) by hub.freebsd.org (Postfix) with ESMTP id 54EF437B76E for ; Tue, 6 Jun 2000 08:00:14 -0700 (PDT) (envelope-from obrien@NUXI.com) Received: from dragon.nuxi.com (root@[209.152.133.57]) by relay.nuxi.com (8.9.3/8.9.3) with ESMTP id IAA70362; Tue, 6 Jun 2000 08:00:11 -0700 (PDT) (envelope-from obrien@dragon.nuxi.com) Received: (from obrien@localhost) by dragon.nuxi.com (8.9.3/8.9.1) id IAA12681; Tue, 6 Jun 2000 08:00:31 -0700 (PDT) (envelope-from obrien) Date: Tue, 6 Jun 2000 08:00:31 -0700 From: "David O'Brien" To: Martin Cracauer Cc: arch@FreeBSD.ORG Subject: Re: Plans to change our debugging format to DWARF2 Message-ID: <20000606080031.F78380@dragon.nuxi.com> Reply-To: obrien@NUXI.com References: <20000606124116.A16993@cons.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <20000606124116.A16993@cons.org>; from cracauer@cons.org on Tue, Jun 06, 2000 at 12:41:17PM +0200 X-Operating-System: FreeBSD 5.0-CURRENT Organization: The NUXI BSD group X-PGP-Fingerprint: B7 4D 3E E9 11 39 5F A3 90 76 5D 69 58 D9 98 7A X-Pgp-Keyid: 34F9F9D5 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Tue, Jun 06, 2000 at 12:41:17PM +0200, Martin Cracauer wrote: > Any idea what the Linux folks are going to do? When they switch, we > have to do it anyway (to get continued support by GNU maintainers). Sparc-linux has made the switch. Windriver(sp?) has made the change. And DWARF2 is the only debugging format supported by the IA-64 ABI. -- -- David (obrien@NUXI.com) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 10: 6:29 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cypherpunks.ai (cypherpunks.ai [209.88.68.47]) by hub.freebsd.org (Postfix) with ESMTP id B030937BA32; Tue, 6 Jun 2000 10:06:24 -0700 (PDT) (envelope-from jeroen@vangelderen.org) Received: from vangelderen.org (grolsch.ai [209.88.68.214]) by cypherpunks.ai (Postfix) with ESMTP id 563C84F; Tue, 6 Jun 2000 13:06:16 -0400 (AST) Message-ID: <393D2F67.EFC00B39@vangelderen.org> Date: Tue, 06 Jun 2000 13:05:43 -0400 From: "Jeroen C. van Gelderen" X-Mailer: Mozilla 4.72 [en] (X11; I; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Mark Murray Cc: Mark Murray , arch@FreeBSD.org, phk@FreeBSD.org Subject: Re: (2nd iteration) New /dev/(random|null|zero) - review, please References: <393BEE84.BBAD3E82@vangelderen.org> <200006060814.KAA21235@gratis.grondar.za> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Mark Murray wrote: > > > I have finished doing a MI /dev/null and /dev/zero, and I have got a > > > new /dev/random. I'm looking for reviewers. > > > > I've already submitted a MI /dev/[null|zero] for commit to PHK. He > > said he would commit them after testing. It's the same driver you > > have already seen at http://jeroen.vangelderen.org/FreeBSD . > > I stole lots of ideas from you :-); I think mine is faster... There is no way to tell as I can't look at your sources but I'd be interested to find out. Why didn't you submit patches if you knew I had already done the work? > > Split-level. Entropy sources should export an entropy device. Yarrow > > should bind to all available entropy devices and use those. This would > > allow for > > - entropy devices in KLDs. > > - dynamic addition/removal of entropy sources (USB). > > - separation of RNG policy (Yarrow) from entropy gathering. > > - dynamic IRQs not affecting RNG security. > > Makes sense, but I need an actual mechanism for achiving this. BDE > suggested the module event handler; I need to investigate. kobj and kqueue. Entropy devices should be kobjects. Kernel keeps a pool of those and the Yarrow module can pull the ones it uses. The Yarrow module would register with the pool to receive arrival events and DTRT. kobjects should have a type so that you can use the pool for different device types. Something tells me that such a thing is already done for USB and pccard but I don't know for sure. > > > o The RNG is slow; the others are much faster than their originals. > > > > Can be tweaked. Use a 256-bit cipher like Rijndael and build a hash > > out of it. Would improve security too as the entropy pool would hold > > 256 bits. You can also pre-generate a few KB of random bits. > > Hmmm.... Sounds good! Got a Rijndael reference? http://www.esat.kuleuven.ac.be/~rijmen/rijndael/ . I have a 256-bit version (block and key). It's pretty easy to construct a 256-bit hash out of that, using a MD{45}-like padding scheme. This would give us Yarrow-256 as opposed to Yarrow-160 described in Bruce's paper. Cheers, Jeroen -- Jeroen C. van Gelderen o _ _ _ jeroen@vangelderen.org _o /\_ _ \\o (_)\__/o (_) _< \_ _>(_) (_)/<_ \_| \ _|/' \/ (_)>(_) (_) (_) (_) (_)' _\o_ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 12: 6:29 2000 Delivered-To: freebsd-arch@freebsd.org Received: from gratis.grondar.za (grouter.grondar.za [196.7.18.65]) by hub.freebsd.org (Postfix) with ESMTP id 29C8B37B580; Tue, 6 Jun 2000 12:06:18 -0700 (PDT) (envelope-from mark@grimreaper.grondar.za) Received: from grimreaper.grondar.za (gratis.grondar.za [196.7.18.133]) by gratis.grondar.za (8.9.3/8.9.3) with ESMTP id VAA22939; Tue, 6 Jun 2000 21:06:05 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200006061906.VAA22939@gratis.grondar.za> To: "Jeroen C. van Gelderen" Cc: Mark Murray , arch@FreeBSD.org, phk@FreeBSD.org Subject: Re: (2nd iteration) New /dev/(random|null|zero) - review, please References: <393D2F67.EFC00B39@vangelderen.org> In-Reply-To: <393D2F67.EFC00B39@vangelderen.org> ; from "Jeroen C. van Gelderen" "Tue, 06 Jun 2000 13:05:43 -0400." Date: Tue, 06 Jun 2000 21:06:07 +0200 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > I stole lots of ideas from you :-); I think mine is faster... > > There is no way to tell as I can't look at your sources but I'd be > interested to find out. Why didn't you submit patches if you knew > I had already done the work? Erm - I had already written 90% of it; I had announced my intentions, and _then_ you gave me a URL to your work (with no other explanation). I posted the usual (and sorry - I didn't know you were not a committer :-) ). Nag me and I'll send you a copy next time I roll it (tomorrow). > > Makes sense, but I need an actual mechanism for achiving this. BDE > > suggested the module event handler; I need to investigate. > > kobj and kqueue. Entropy devices should be kobjects. Kernel keeps a > pool of those and the Yarrow module can pull the ones it uses. The > Yarrow module would register with the pool to receive arrival events > and DTRT. kobjects should have a type so that you can use the pool > for different device types. Cool! Looks good! > Something tells me that such a thing is already done for USB and > pccard but I don't know for sure. I'll look... > > Hmmm.... Sounds good! Got a Rijndael reference? > > http://www.esat.kuleuven.ac.be/~rijmen/rijndael/ . I have a 256-bit > version (block and key). It's pretty easy to construct a 256-bit hash > out of that, using a MD{45}-like padding scheme. This would give us > Yarrow-256 as opposed to Yarrow-160 described in Bruce's paper. Thanks! (We could of course use Blowfish256 - already in the kernel) M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 13: 1:49 2000 Delivered-To: freebsd-arch@freebsd.org Received: from spirit.jaded.net (shortbus.jaded.net [216.94.132.8]) by hub.freebsd.org (Postfix) with ESMTP id 61D5237BA6D; Tue, 6 Jun 2000 13:01:39 -0700 (PDT) (envelope-from dan@spirit.jaded.net) Received: (from dan@localhost) by spirit.jaded.net (8.9.3/8.9.3) id QAA03583; Tue, 6 Jun 2000 16:01:18 -0400 (EDT) (envelope-from dan) Date: Tue, 6 Jun 2000 16:01:18 -0400 From: Dan Moschuk To: "Jeroen C. van Gelderen" Cc: Mark Murray , arch@FreeBSD.org, phk@FreeBSD.org Subject: Re: (2nd iteration) New /dev/(random|null|zero) - review, please Message-ID: <20000606160118.C3351@spirit.jaded.net> References: <200006051720.TAA18713@gratis.grondar.za> <393BEE84.BBAD3E82@vangelderen.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <393BEE84.BBAD3E82@vangelderen.org>; from jeroen@vangelderen.org on Mon, Jun 05, 2000 at 02:16:36PM -0400 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG | > o The random number generator will give random-looking output, but does | > absolutely no harvesting of entropy at the moment. Because I want | > it to be a loadable module, I need some way of registering the entropy | > harvesting routines. Something like weak-symboled routines that are | > overridden when the module is loaded would be ideal. Suggestions? | | Split-level. Entropy sources should export an entropy device. Yarrow | should bind to all available entropy devices and use those. This would | allow for | - entropy devices in KLDs. | - dynamic addition/removal of entropy sources (USB). | - separation of RNG policy (Yarrow) from entropy gathering. | - dynamic IRQs not affecting RNG security. I have a driver for the i82802 chipset (Intel Thermal Noise RNG) that needs to be newbus-ified before committing. Anyone that can help me with this, it would be appreciated. Having hooks in various drivers to export entropy to yarrow is a great idea. It would certainly give us a nice framework to be able to secure other areas in the kernel, such as random pid generation, src ports and sequence numbers. | > o The RNG is slow; the others are much faster than their originals. | | Can be tweaked. Use a 256-bit cipher like Rijndael and build a hash | out of it. Would improve security too as the entropy pool would hold | 256 bits. You can also pre-generate a few KB of random bits. Because of the significant speed decrease in using Yarrow, I'd like to see us keep the current implementation around, and having Yarrow as an option or psuedo-device to be used instead. -- Dan Moschuk (TFreak!dan@freebsd.org) "Don't get even -- get odd!" To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 13:22:12 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cypherpunks.ai (cypherpunks.ai [209.88.68.47]) by hub.freebsd.org (Postfix) with ESMTP id B7B9537B722; Tue, 6 Jun 2000 13:22:02 -0700 (PDT) (envelope-from jeroen@vangelderen.org) Received: from vangelderen.org (grolsch.ai [209.88.68.214]) by cypherpunks.ai (Postfix) with ESMTP id 12F525D; Tue, 6 Jun 2000 16:21:59 -0400 (AST) Message-ID: <393D5D46.6BCACDE4@vangelderen.org> Date: Tue, 06 Jun 2000 16:21:26 -0400 From: "Jeroen C. van Gelderen" X-Mailer: Mozilla 4.72 [en] (X11; I; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Dan Moschuk Cc: Mark Murray , arch@FreeBSD.org, phk@FreeBSD.org Subject: Re: (2nd iteration) New /dev/(random|null|zero) - review, please References: <200006051720.TAA18713@gratis.grondar.za> <393BEE84.BBAD3E82@vangelderen.org> <20000606160118.C3351@spirit.jaded.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Dan Moschuk wrote: > | > o The RNG is slow; the others are much faster than their originals. > | > | Can be tweaked. Use a 256-bit cipher like Rijndael and build a hash > | out of it. Would improve security too as the entropy pool would hold > | 256 bits. You can also pre-generate a few KB of random bits. > > Because of the significant speed decrease in using Yarrow, I'd like to see > us keep the current implementation around, and having Yarrow as an > option or psuedo-device to be used instead. Yarrow -when finished- is not noticably slower than our current implementation of /dev/[u]random. Yarrow does one block encryption for every output block and a generator gate every 10 blocks. This would allow for at least 40 mbit/s output on a 200 Mhz PPro when using Rijndael/256/256. Cheers, Jeroen -- Jeroen C. van Gelderen o _ _ _ jeroen@vangelderen.org _o /\_ _ \\o (_)\__/o (_) _< \_ _>(_) (_)/<_ \_| \ _|/' \/ (_)>(_) (_) (_) (_) (_)' _\o_ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 14: 3:51 2000 Delivered-To: freebsd-arch@freebsd.org Received: from spirit.jaded.net (shortbus.jaded.net [216.94.132.8]) by hub.freebsd.org (Postfix) with ESMTP id C62CA37BA93; Tue, 6 Jun 2000 14:03:43 -0700 (PDT) (envelope-from dan@spirit.jaded.net) Received: (from dan@localhost) by spirit.jaded.net (8.9.3/8.9.3) id RAA03766; Tue, 6 Jun 2000 17:03:36 -0400 (EDT) (envelope-from dan) Date: Tue, 6 Jun 2000 17:03:36 -0400 From: Dan Moschuk To: "Jeroen C. van Gelderen" Cc: Dan Moschuk , Mark Murray , arch@FreeBSD.org, phk@FreeBSD.org Subject: Re: (2nd iteration) New /dev/(random|null|zero) - review, please Message-ID: <20000606170336.D3351@spirit.jaded.net> References: <200006051720.TAA18713@gratis.grondar.za> <393BEE84.BBAD3E82@vangelderen.org> <20000606160118.C3351@spirit.jaded.net> <393D5D46.6BCACDE4@vangelderen.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: <393D5D46.6BCACDE4@vangelderen.org>; from jeroen@vangelderen.org on Tue, Jun 06, 2000 at 04:21:26PM -0400 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG | > Because of the significant speed decrease in using Yarrow, I'd like to see | > us keep the current implementation around, and having Yarrow as an | > option or psuedo-device to be used instead. | | Yarrow -when finished- is not noticably slower than our current | implementation of /dev/[u]random. Yarrow does one block encryption | for every output block and a generator gate every 10 blocks. This | would allow for at least 40 mbit/s output on a 200 Mhz PPro when | using Rijndael/256/256. It was my understanding that we weren't using Rijndael/256 yet. Is this going to change? -- Dan Moschuk (TFreak!dan@freebsd.org) "Don't get even -- get odd!" To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 23: 8: 8 2000 Delivered-To: freebsd-arch@freebsd.org Received: from gratis.grondar.za (grouter.grondar.za [196.7.18.65]) by hub.freebsd.org (Postfix) with ESMTP id 9E16437B82C; Tue, 6 Jun 2000 23:08:01 -0700 (PDT) (envelope-from mark@grimreaper.grondar.za) Received: from grimreaper.grondar.za (gratis.grondar.za [196.7.18.133]) by gratis.grondar.za (8.9.3/8.9.3) with ESMTP id IAA24428; Wed, 7 Jun 2000 08:07:47 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200006070607.IAA24428@gratis.grondar.za> To: "Jeroen C. van Gelderen" Cc: Dan Moschuk , Mark Murray , arch@FreeBSD.org, phk@FreeBSD.org Subject: Re: (2nd iteration) New /dev/(random|null|zero) - review, please References: <393D5D46.6BCACDE4@vangelderen.org> In-Reply-To: <393D5D46.6BCACDE4@vangelderen.org> ; from "Jeroen C. van Gelderen" "Tue, 06 Jun 2000 16:21:26 -0400." Date: Wed, 07 Jun 2000 08:07:46 +0200 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > Because of the significant speed decrease in using Yarrow, I'd like to see > > us keep the current implementation around, and having Yarrow as an > > option or psuedo-device to be used instead. > > Yarrow -when finished- is not noticably slower than our current > implementation of /dev/[u]random. Yarrow does one block encryption > for every output block and a generator gate every 10 blocks. This > would allow for at least 40 mbit/s output on a 200 Mhz PPro when > using Rijndael/256/256. I tend to agree; I am currently using SHA1 and DES3, and it is quite slow, mostly in the proportion of DES3::MD5 speeds, which makes sense as the existing implementation uses MD5. M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Tue Jun 6 23:11:59 2000 Delivered-To: freebsd-arch@freebsd.org Received: from gratis.grondar.za (grouter.grondar.za [196.7.18.65]) by hub.freebsd.org (Postfix) with ESMTP id 2F74C37B6EE; Tue, 6 Jun 2000 23:11:52 -0700 (PDT) (envelope-from mark@grimreaper.grondar.za) Received: from grimreaper.grondar.za (gratis.grondar.za [196.7.18.133]) by gratis.grondar.za (8.9.3/8.9.3) with ESMTP id IAA24440; Wed, 7 Jun 2000 08:11:48 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200006070611.IAA24440@gratis.grondar.za> To: Dan Moschuk Cc: "Jeroen C. van Gelderen" , arch@FreeBSD.org, phk@FreeBSD.org From: Mark Murray Subject: Re: (2nd iteration) New /dev/(random|null|zero) - review, please References: <20000606170336.D3351@spirit.jaded.net> In-Reply-To: <20000606170336.D3351@spirit.jaded.net> ; from Dan Moschuk "Tue, 06 Jun 2000 17:03:36 -0400." Date: Wed, 07 Jun 2000 08:11:46 +0200 Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > | Yarrow -when finished- is not noticably slower than our current > | implementation of /dev/[u]random. Yarrow does one block encryption > | for every output block and a generator gate every 10 blocks. This > | would allow for at least 40 mbit/s output on a 200 Mhz PPro when > | using Rijndael/256/256. > > It was my understanding that we weren't using Rijndael/256 yet. Is this > going to change? The original plan was for SHA1/DES3; I am listening to counter-arguments. I may use Rijndael; I may use Blowfish. M -- Mark Murray Join the anti-SPAM movement: http://www.cauce.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Wed Jun 7 12: 7:56 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cypherpunks.ai (cypherpunks.ai [209.88.68.47]) by hub.freebsd.org (Postfix) with ESMTP id BC3B737BA09; Wed, 7 Jun 2000 12:07:52 -0700 (PDT) (envelope-from jeroen@vangelderen.org) Received: from vangelderen.org (grolsch.ai [209.88.68.214]) by cypherpunks.ai (Postfix) with ESMTP id C0AD451; Wed, 7 Jun 2000 15:07:48 -0400 (AST) Message-ID: <393E9D84.4026FC0E@vangelderen.org> Date: Wed, 07 Jun 2000 15:07:48 -0400 From: "Jeroen C. van Gelderen" X-Mailer: Mozilla 4.72 [en] (X11; I; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Mark Murray Cc: Dan Moschuk , Mark Murray , arch@FreeBSD.org Subject: Re: (2nd iteration) New /dev/(random|null|zero) - review, please References: <393D5D46.6BCACDE4@vangelderen.org> <200006070607.IAA24428@gratis.grondar.za> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Mark Murray wrote: > > > > Because of the significant speed decrease in using Yarrow, I'd like to see > > > us keep the current implementation around, and having Yarrow as an > > > option or psuedo-device to be used instead. > > > > Yarrow -when finished- is not noticably slower than our current > > implementation of /dev/[u]random. Yarrow does one block encryption > > for every output block and a generator gate every 10 blocks. This > > would allow for at least 40 mbit/s output on a 200 Mhz PPro when > > using Rijndael/256/256. > > I tend to agree; I am currently using SHA1 and DES3, and it is quite > slow, mostly in the proportion of DES3::MD5 speeds, which makes sense > as the existing implementation uses MD5. Rijndael is quite a bit faster than single DES which will do 28 Mbit/s on a PPro 200. Assuming a generator gate every 10 blocks Rijndael would do 11 encryptions per 10 blocks and hence run at least at 25.5 Mbit/s. In reality we could see around 50 Mbit/s performance, depending on the Rijndael implementation. Of course I'm not counting reseeds but those can be done in a lazy fashion. You can generate hundreds of MBytes before you *need* to reseed, depending on your security policy. So the bottleneck is not the PRNG mechanism, it's the security policy and your entropy sources. If you require a reseed every hundred blocks or so it becomes much more expensive. But that holds for our current /dev/[u]random implementation too. Cheers, Jeroen -- Jeroen C. van Gelderen o _ _ _ jeroen@vangelderen.org _o /\_ _ \\o (_)\__/o (_) _< \_ _>(_) (_)/<_ \_| \ _|/' \/ (_)>(_) (_) (_) (_) (_)' _\o_ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jun 8 0:19: 0 2000 Delivered-To: freebsd-arch@freebsd.org Received: from njord.bart.nl (njord.bart.nl [194.158.170.15]) by hub.freebsd.org (Postfix) with ESMTP id B042D37BAED for ; Thu, 8 Jun 2000 00:18:54 -0700 (PDT) (envelope-from asmodai@wxs.nl) Received: from daemon.ninth-circle.org (lucifer.is.an.elder.of.the.ninth-circle.org [195.38.216.226]) by njord.bart.nl (8.10.1/8.10.1) with ESMTP id e587Ipg93953; Thu, 8 Jun 2000 09:18:51 +0200 (CEST) Received: (from asmodai@localhost) by daemon.ninth-circle.org (8.9.3/8.9.3) id JAA13148; Thu, 8 Jun 2000 09:15:07 +0200 (CEST) (envelope-from asmodai) Date: Thu, 8 Jun 2000 09:15:07 +0200 From: Jeroen Ruigrok/Asmodai To: "David O'Brien" Cc: Martin Cracauer , arch@FreeBSD.ORG Subject: Re: Plans to change our debugging format to DWARF2 Message-ID: <20000608091507.E1587@daemon.ninth-circle.org> References: <20000606124116.A16993@cons.org> <20000606080031.F78380@dragon.nuxi.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2i In-Reply-To: <20000606080031.F78380@dragon.nuxi.com>; from obrien@NUXI.com on Tue, Jun 06, 2000 at 08:00:31AM -0700 Organisation: Ninth-Circle Enterprises Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG -On [20000606 20:01], David O'Brien (obrien@NUXI.com) wrote: >On Tue, Jun 06, 2000 at 12:41:17PM +0200, Martin Cracauer wrote: >> Any idea what the Linux folks are going to do? When they switch, we >> have to do it anyway (to get continued support by GNU maintainers). > >Sparc-linux has made the switch. Windriver(sp?) has made the change. >And DWARF2 is the only debugging format supported by the IA-64 ABI. I'd say go for it. But of course we want to MFC this to 4.x as well some point in the future. Along with all the other compiler changes. -- Jeroen Ruigrok vd Werven/Asmodai asmodai@[wxs.nl|bart.nl|freebsd.org] Documentation nutter/C-rated Coder BSD: Technical excellence at its best The BSD Programmer's Documentation Project Haste makes waste... To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jun 8 13:40:22 2000 Delivered-To: freebsd-arch@freebsd.org Received: from brs.com.br (brs.com.br [192.41.24.32]) by hub.freebsd.org (Postfix) with ESMTP id 6574D37C152 for ; Thu, 8 Jun 2000 13:40:20 -0700 (PDT) (envelope-from email@carlos.eti.br) Received: from fraga.carlos.eti.br ([200.241.0.6]) by brs.com.br (8.8.5) id SAA11478; Thu, 8 Jun 2000 18:40:19 -0200 (GMT+2) X-Authentication-Warning: brs.com.br: Host [200.241.0.6] claimed to be fraga.carlos.eti.br Message-Id: <4.3.2.7.2.20000608174119.00b33c00@carlos.eti.br> X-Sender: email@carlos.eti.br X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Thu, 08 Jun 2000 17:41:24 -0300 To: freebsd-arch@FreeBSD.ORG From: Carlos Fraga Subject: Re: It's worth ! Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG A site that pays you to receive some e-mails. No more than that. Nothing to buy, just to receive the e-mail and click on the link to visit the site. Don't you believe it exists ? Yes, it exists. And I have already received a US$ 50,00 check. Will you say that you don't want some money ? It's up to you to subscribe and start receiving e-mails and money ! Follow the link: http://www.sendmoreinfo.com/id/871883 See you. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jun 8 14: 0:10 2000 Delivered-To: freebsd-arch@freebsd.org Received: from grimreaper.grondar.za (grimreaper.grondar.za [196.7.18.138]) by hub.freebsd.org (Postfix) with ESMTP id 746B837C09B; Thu, 8 Jun 2000 14:00:01 -0700 (PDT) (envelope-from mark@grondar.za) Received: from grimreaper.grondar.za (localhost [127.0.0.1]) by grimreaper.grondar.za (8.9.3/8.9.3) with ESMTP id WAA01487; Thu, 8 Jun 2000 22:58:43 +0200 (SAST) (envelope-from mark@grimreaper.grondar.za) Message-Id: <200006082058.WAA01487@grimreaper.grondar.za> To: arch@FreeBSD.ORG Cc: bde@FreeBSD.ORG, dfr@FreeBSD.ORG, pkh@FreeBSD.ORG, jeroen@vangelderen.org Subject: (3rd iteration) New /dev/(random|null|zero) - review, please Date: Thu, 08 Jun 2000 22:58:43 +0200 From: Mark Murray Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG (Some more improvements have been done - 3rd iteration) Hi I have finished doing a MI /dev/null and /dev/zero, and I have got a new /dev/random. I'm looking for reviewers. The code is in http://freefall.freebsd.org/~markm/. There is a tar file and diffs (all for the sys/ area). Some other supplementary patches are needed in userland, these are not included. I like to think that this is a commit candidate. Please review as such. NOTES: 3rd Iteration Improvements: o Jeroen van Gelderen properly credited, as I stole^wused a lot of his very good ideas. o Much better module system (no SYSINIT, rather DEV_MODULE). o In anticipation of different cryptosystems, use Blowfish instead of SHA1/DES3. I am open to the use of other algorithms; I used Blowfish because 1) its already in the kernel and 2) _I_ have not yet seen a decent cryptanalysis of it. (This may change) o Add the beginnings of sysctl(3) framework to tweak the running Yarrow algorithm. 2nd Iteration Improvements: o /dev/null and /dev/zero or no longer optional; they are "standard". o /dev/zero uses malloc(9) to get its space, not a hard-coded block of zeros. Malloc is done once; at device startup. Original: o The devices are (can be) modules, or by setting options, they can be hard coded into the kernel. I would like to make them autoload somehow, but I'm not sure how. o I'd like to make the devices "pseudo-devices", rather than options. Comments? o The random number generator will give random-looking output, but does absolutely no harvesting of entropy at the moment. Because I want it to be a loadable module, I need some way of registering the entropy harvesting routines. Something like weak-symboled routines that are overridden when the module is loaded would be ideal. Suggestions? o I am using Brice Schneier's "Yarrow" algorithm for the RNG; I have only supplied enough of it now to give "sort of" random numbers. As I solve the harvesting problem, I'll improve on that. o The RNG is slow; the others are much faster than their originals. o I intend to use sysctl(9) to set most of yarrow's "tweakables". Thanks to Jeroen van Gelderen for some excellent ideas on optimization! M To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message ------- End of Forwarded Message To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jun 8 18: 5:58 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cypherpunks.ai (cypherpunks.ai [209.88.68.47]) by hub.freebsd.org (Postfix) with ESMTP id 9255437B71F; Thu, 8 Jun 2000 18:05:54 -0700 (PDT) (envelope-from jeroen@vangelderen.org) Received: from vangelderen.org (grolsch.ai [209.88.68.214]) by cypherpunks.ai (Postfix) with ESMTP id 8664188; Thu, 8 Jun 2000 21:05:53 -0400 (AST) Message-ID: <394042F1.7CDDC16D@vangelderen.org> Date: Thu, 08 Jun 2000 21:05:53 -0400 From: "Jeroen C. van Gelderen" X-Mailer: Mozilla 4.72 [en] (X11; I; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Mark Murray Cc: arch@FreeBSD.ORG, bde@FreeBSD.ORG, dfr@FreeBSD.ORG, pkh@FreeBSD.ORG Subject: Re: (3rd iteration) New /dev/(random|null|zero) - review, please References: <200006082058.WAA01487@grimreaper.grondar.za> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Mark Murray wrote: > > (Some more improvements have been done - 3rd iteration) > > Hi > > I have finished doing a MI /dev/null and /dev/zero, and I have got a > new /dev/random. I'm looking for reviewers. > > The code is in http://freefall.freebsd.org/~markm/. There is a tar > file and diffs (all for the sys/ area). Some other supplementary patches > are needed in userland, these are not included. > > I like to think that this is a commit candidate. Please review as such. I think you should wait until Yarrow is ready and actually gathers entropy. The /dev/[null|zero] bits should go in though. [...] > o Much better module system (no SYSINIT, rather DEV_MODULE). Thanks. > o In anticipation of different cryptosystems, use Blowfish instead > of SHA1/DES3. I am open to the use of other algorithms; I used > Blowfish because 1) its already in the kernel and 2) _I_ have > not yet seen a decent cryptanalysis of it. (This may change) The rule generally is: if there is no decent cryptanalysis, don't use the algorithm; Not the other way around. I pointed out in an earlier email that Blowfish has very low key agility and as such is not a good candidate for Yarrow because there is a factor 53(!) overhead for each block you output. If you want to use an algorithm that's already in the kernel, use CAST5. An alternative is to import one of the 5 AES finalists and use it for the time being (on the premise that AES will go into the kernel when it's chosen). AES candidates have a 128-bit blocksize which is better than 64 in this case. This would be my recommendation. Cheers, Jeroen -- Jeroen C. van Gelderen o _ _ _ jeroen@vangelderen.org _o /\_ _ \\o (_)\__/o (_) _< \_ _>(_) (_)/<_ \_| \ _|/' \/ (_)>(_) (_) (_) (_) (_)' _\o_ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Thu Jun 8 18:11:40 2000 Delivered-To: freebsd-arch@freebsd.org Received: from cypherpunks.ai (cypherpunks.ai [209.88.68.47]) by hub.freebsd.org (Postfix) with ESMTP id 7A9C137BD37; Thu, 8 Jun 2000 18:11:37 -0700 (PDT) (envelope-from jeroen@vangelderen.org) Received: from vangelderen.org (grolsch.ai [209.88.68.214]) by cypherpunks.ai (Postfix) with ESMTP id 4A8D288; Thu, 8 Jun 2000 21:11:36 -0400 (AST) Message-ID: <39404448.4435649A@vangelderen.org> Date: Thu, 08 Jun 2000 21:11:36 -0400 From: "Jeroen C. van Gelderen" X-Mailer: Mozilla 4.72 [en] (X11; I; Linux 2.2.12 i386) X-Accept-Language: en MIME-Version: 1.0 To: Mark Murray Cc: arch@FreeBSD.ORG, bde@FreeBSD.ORG, dfr@FreeBSD.ORG, pkh@FreeBSD.ORG Subject: Re: (3rd iteration) New /dev/(random|null|zero) - review, please References: <200006082058.WAA01487@grimreaper.grondar.za> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Mark Murray wrote: > > (Some more improvements have been done - 3rd iteration) > > Hi > > I have finished doing a MI /dev/null and /dev/zero, and I have got a > new /dev/random. I'm looking for reviewers. > > The code is in http://freefall.freebsd.org/~markm/. There is a tar > file and diffs (all for the sys/ area). Some other supplementary patches > are needed in userland, these are not included. > > I like to think that this is a commit candidate. Please review as such. The module allows unloads even when users have the devices open. I think you need to do reference counting. The misc device already does that and has been tested already. You know the URL. The device is named nulldev. PHK suggested that misc is the preferred name. I'd personally prefer 'rathole' though ;-) Cheers, Jeroen -- Jeroen C. van Gelderen o _ _ _ jeroen@vangelderen.org _o /\_ _ \\o (_)\__/o (_) _< \_ _>(_) (_)/<_ \_| \ _|/' \/ (_)>(_) (_) (_) (_) (_)' _\o_ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jun 9 1:35:17 2000 Delivered-To: freebsd-arch@freebsd.org Received: from tele-post-20.mail.demon.net (tele-post-20.mail.demon.net [194.217.242.20]) by hub.freebsd.org (Postfix) with ESMTP id BF6F637C260 for ; Fri, 9 Jun 2000 01:35:04 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from nlsys.demon.co.uk ([158.152.125.33] helo=herring.nlsystems.com) by tele-post-20.mail.demon.net with esmtp (Exim 2.12 #2) id 130KFr-0009a7-0K for arch@freebsd.org; Fri, 9 Jun 2000 08:35:03 +0000 Received: from salmon.nlsystems.com (salmon.nlsystems.com [10.0.0.3]) by herring.nlsystems.com (8.9.3/8.8.8) with ESMTP id JAA84560 for ; Fri, 9 Jun 2000 09:04:22 +0100 (BST) (envelope-from dfr@nlsystems.com) Date: Fri, 9 Jun 2000 09:07:58 +0100 (BST) From: Doug Rabson To: arch@freebsd.org Subject: Syscalls and execve Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I was experimenting the other day optimising the return path for syscall on the alpha. The current code treats syscall return the same as any other exception (restoring *all* registers, making extra checks for returning to user mode etc). This is extra bogus since the syscall entry point rightly doesn't bother to save the values of any temporary registers so on return, those temporaries will be loaded with whatever was lying around on the kernel stack. I decided to streamline the syscall exit by knowing that it will be returning to user mode and the ipl will be zero. Also there is not much point in restoring temporaries since the caller should only expect it to preserve $s0-$6. This worked fairly well but there is a question in my mind about execve(). The MD function setregs() (very badly misnamed, should probably be called cpu_exec() or something similar) tries to arrange for the new image to start with all registers set to zero. It does this by zeroing the trapframe and assuming that the syscall return will reload all the registers. Optimising the syscall path means that the new image will start with random values in most of the registers. Is this acceptable behaviour for exec? Are there any security implications for 'leaking' the values which the kernel left in the temporary registers on syscall exit (both regular syscalls and execve)? One possibility is to explicitly zero the temporaries on syscall exit, which would be faster than reloading them from the trapframe. -- Doug Rabson Mail: dfr@nlsystems.com Nonlinear Systems Ltd. Phone: +44 20 8442 9037 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jun 9 1:55:38 2000 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id 30B5337C260 for ; Fri, 9 Jun 2000 01:55:32 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id BAA10749; Fri, 9 Jun 2000 01:50:31 -0700 (PDT) (envelope-from dillon) Date: Fri, 9 Jun 2000 01:50:31 -0700 (PDT) From: Matthew Dillon Message-Id: <200006090850.BAA10749@apollo.backplane.com> To: Doug Rabson Cc: arch@FreeBSD.ORG Subject: Re: Syscalls and execve References: Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :returning to user mode and the ipl will be zero. Also there is not much :point in restoring temporaries since the caller should only expect it to :preserve $s0-$6. : :This worked fairly well but there is a question in my mind about :execve(). The MD function setregs() (very badly misnamed, should probably :be called cpu_exec() or something similar) tries to arrange for the new :image to start with all registers set to zero. It does this by zeroing the :trapframe and assuming that the syscall return will reload all the :registers. Optimising the syscall path means that the new image will start :with random values in most of the registers. : :Is this acceptable behaviour for exec? Are there any security implications :for 'leaking' the values which the kernel left in the temporary registers :on syscall exit (both regular syscalls and execve)? One possibility is to :explicitly zero the temporaries on syscall exit, which would be faster :than reloading them from the trapframe. : :-- :Doug Rabson Mail: dfr@nlsystems.com :Nonlinear Systems Ltd. Phone: +44 20 8442 9037 I would say "no" for exec... there are certainly security implications, but I would worry more about starting a new program with inconsistent state. It's not a good idea even if the state is supposed to be unused. Why not have the new exec()'d process, when it gets the cpu in supervisor mode, clear the registers in supervisor mode before returning to user mode? e.g. near the end of kern/kern_exec.c's execve(). (or somewhere similar). Then at least the 'garbage' will be more like what you see on return from a syscall rather then something inherited from another process. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jun 9 5:16:11 2000 Delivered-To: freebsd-arch@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id F1E3137B544 for ; Fri, 9 Jun 2000 05:16:08 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.10.1/8.10.1) id e59CG7I07757; Fri, 9 Jun 2000 08:16:07 -0400 (EDT) Date: Fri, 9 Jun 2000 08:16:07 -0400 (EDT) From: Luoqi Chen Message-Id: <200006091216.e59CG7I07757@lor.watermarkgroup.com> To: dfr@nlsystems.com, dillon@apollo.backplane.com Subject: Re: Syscalls and execve Cc: arch@FreeBSD.ORG Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > Why not have the new exec()'d process, when it gets the cpu in supervisor > mode, clear the registers in supervisor mode before returning > to user mode? e.g. near the end of kern/kern_exec.c's execve(). > (or somewhere similar). Then at least the 'garbage' will be more > like what you see on return from a syscall rather then something > inherited from another process. > Here the current process *is* the process calling exec() (unlike fork()), so why not just zero those registers in setregs()? -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jun 9 5:18:58 2000 Delivered-To: freebsd-arch@freebsd.org Received: from gidora.zeta.org.au (gidora.zeta.org.au [203.26.10.25]) by hub.freebsd.org (Postfix) with SMTP id C8FDC37B544 for ; Fri, 9 Jun 2000 05:18:53 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: (qmail 4893 invoked from network); 9 Jun 2000 12:18:49 -0000 Received: from unknown (HELO bde.zeta.org.au) (203.2.228.102) by gidora.zeta.org.au with SMTP; 9 Jun 2000 12:18:49 -0000 Date: Fri, 9 Jun 2000 22:18:45 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Doug Rabson Cc: arch@FreeBSD.ORG Subject: Re: Syscalls and execve In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 9 Jun 2000, Doug Rabson wrote: > ... > This worked fairly well but there is a question in my mind about > execve(). The MD function setregs() (very badly misnamed, should probably > be called cpu_exec() or something similar) tries to arrange for the new > image to start with all registers set to zero. It does this by zeroing the > trapframe and assuming that the syscall return will reload all the > registers. Optimising the syscall path means that the new image will start > with random values in most of the registers. > > Is this acceptable behaviour for exec? Are there any security implications > for 'leaking' the values which the kernel left in the temporary registers > on syscall exit (both regular syscalls and execve)? One possibility is to > explicitly zero the temporaries on syscall exit, which would be faster > than reloading them from the trapframe. There are minor security implementations. The registers might contain part of a password string or some other secret data, although this is unlikely and we probably have several lonstanding leaks of this kind from copying out partially filled structs. Not saving the registers in a consistent place for syscalls causes problems with procfs, ptrace and debuggers. /proc/regs, ptrace() and accessing registers in the "u area" using ptrace() now work for processes in the kernel, independently of how they entered the kernel. In FreeBSD-1.0, the registers were saved in a slightly different place for syscalls vs for traps, and everything that accessed them had to know about this complication in order to find them. If you don't save the registers on syscall entry, then they will be too hard to find. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Fri Jun 9 5:39:12 2000 Delivered-To: freebsd-arch@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id 1520E37BD0F for ; Fri, 9 Jun 2000 05:39:11 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.10.1/8.10.1) id e59Cd9T08096; Fri, 9 Jun 2000 08:39:10 -0400 (EDT) Date: Fri, 9 Jun 2000 08:39:10 -0400 (EDT) From: Luoqi Chen Message-Id: <200006091239.e59Cd9T08096@lor.watermarkgroup.com> To: dfr@nlsystems.com, dillon@apollo.backplane.com Subject: Re: Syscalls and execve Cc: arch@FreeBSD.ORG Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > > Why not have the new exec()'d process, when it gets the cpu in supervisor > > mode, clear the registers in supervisor mode before returning > > to user mode? e.g. near the end of kern/kern_exec.c's execve(). > > (or somewhere similar). Then at least the 'garbage' will be more > > like what you see on return from a syscall rather then something > > inherited from another process. > > > Here the current process *is* the process calling exec() (unlike fork()), so > why not just zero those registers in setregs()? > > -lq > I take back what I've just said, this won't guarantee zeros in those volatile registers. Moreover doesn't the new process image expect to see argc, argv, envp in the argument registers? Exec() needs special treatment, maybe an exec_trampoline()? -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jun 10 1:35:11 2000 Delivered-To: freebsd-arch@freebsd.org Received: from finch-post-12.mail.demon.net (finch-post-12.mail.demon.net [194.217.242.41]) by hub.freebsd.org (Postfix) with ESMTP id D16C737B74A for ; Sat, 10 Jun 2000 01:34:54 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from nlsys.demon.co.uk ([158.152.125.33] helo=herring.nlsystems.com) by finch-post-12.mail.demon.net with esmtp (Exim 2.12 #1) id 130gj7-000Dz0-0C; Sat, 10 Jun 2000 08:34:47 +0000 Received: from salmon.nlsystems.com (salmon.nlsystems.com [10.0.0.3]) by herring.nlsystems.com (8.9.3/8.8.8) with ESMTP id JAA96653; Sat, 10 Jun 2000 09:35:11 +0100 (BST) (envelope-from dfr@nlsystems.com) Date: Sat, 10 Jun 2000 09:39:06 +0100 (BST) From: Doug Rabson To: Bruce Evans Cc: arch@FreeBSD.ORG Subject: Re: Syscalls and execve In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 9 Jun 2000, Bruce Evans wrote: > On Fri, 9 Jun 2000, Doug Rabson wrote: > > > ... > > This worked fairly well but there is a question in my mind about > > execve(). The MD function setregs() (very badly misnamed, should probably > > be called cpu_exec() or something similar) tries to arrange for the new > > image to start with all registers set to zero. It does this by zeroing the > > trapframe and assuming that the syscall return will reload all the > > registers. Optimising the syscall path means that the new image will start > > with random values in most of the registers. > > > > Is this acceptable behaviour for exec? Are there any security implications > > for 'leaking' the values which the kernel left in the temporary registers > > on syscall exit (both regular syscalls and execve)? One possibility is to > > explicitly zero the temporaries on syscall exit, which would be faster > > than reloading them from the trapframe. > > There are minor security implementations. The registers might contain > part of a password string or some other secret data, although this is > unlikely and we probably have several lonstanding leaks of this kind > from copying out partially filled structs. > > Not saving the registers in a consistent place for syscalls causes > problems with procfs, ptrace and debuggers. /proc/regs, ptrace() and > accessing registers in the "u area" using ptrace() now work for > processes in the kernel, independently of how they entered the kernel. > In FreeBSD-1.0, the registers were saved in a slightly different place > for syscalls vs for traps, and everything that accessed them had to > know about this complication in order to find them. If you don't save > the registers on syscall entry, then they will be too hard to find. The trapframe which is created for syscall is identical to the trapframe for exceptions and interrupts, its just not fully populated with register values. -- Doug Rabson Mail: dfr@nlsystems.com Nonlinear Systems Ltd. Phone: +44 20 8442 9037 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jun 10 1:37:29 2000 Delivered-To: freebsd-arch@freebsd.org Received: from finch-post-11.mail.demon.net (finch-post-11.mail.demon.net [194.217.242.39]) by hub.freebsd.org (Postfix) with ESMTP id 5BB2837B74A for ; Sat, 10 Jun 2000 01:37:18 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from nlsys.demon.co.uk ([158.152.125.33] helo=herring.nlsystems.com) by finch-post-11.mail.demon.net with esmtp (Exim 2.12 #1) id 130glS-000K5K-0B; Sat, 10 Jun 2000 08:37:10 +0000 Received: from salmon.nlsystems.com (salmon.nlsystems.com [10.0.0.3]) by herring.nlsystems.com (8.9.3/8.8.8) with ESMTP id JAA00714; Sat, 10 Jun 2000 09:38:15 +0100 (BST) (envelope-from dfr@nlsystems.com) Date: Sat, 10 Jun 2000 09:42:11 +0100 (BST) From: Doug Rabson To: Luoqi Chen Cc: dillon@apollo.backplane.com, arch@FreeBSD.ORG Subject: Re: Syscalls and execve In-Reply-To: <200006091239.e59Cd9T08096@lor.watermarkgroup.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 9 Jun 2000, Luoqi Chen wrote: > > > Why not have the new exec()'d process, when it gets the cpu in supervisor > > > mode, clear the registers in supervisor mode before returning > > > to user mode? e.g. near the end of kern/kern_exec.c's execve(). > > > (or somewhere similar). Then at least the 'garbage' will be more > > > like what you see on return from a syscall rather then something > > > inherited from another process. > > > > > Here the current process *is* the process calling exec() (unlike fork()), so > > why not just zero those registers in setregs()? > > > > -lq > > > I take back what I've just said, this won't guarantee zeros in those volatile > registers. Moreover doesn't the new process image expect to see argc, argv, > envp in the argument registers? Exec() needs special treatment, maybe an > exec_trampoline()? The entry point for execve on alpha expects $pc and $pv to point to the start instruction, $a0 to be the new stack pointer and $a3 to point at ps_strings. Values for argc, argv etc are calculated by examining the stack. -- Doug Rabson Mail: dfr@nlsystems.com Nonlinear Systems Ltd. Phone: +44 20 8442 9037 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jun 10 1:40:18 2000 Delivered-To: freebsd-arch@freebsd.org Received: from anchor-post-30.mail.demon.net (anchor-post-30.mail.demon.net [194.217.242.88]) by hub.freebsd.org (Postfix) with ESMTP id 13C6537B74A for ; Sat, 10 Jun 2000 01:40:11 -0700 (PDT) (envelope-from dfr@nlsystems.com) Received: from nlsys.demon.co.uk ([158.152.125.33] helo=herring.nlsystems.com) by anchor-post-30.mail.demon.net with esmtp (Exim 2.12 #1) id 130goA-0003qc-0U; Sat, 10 Jun 2000 09:39:58 +0100 Received: from salmon.nlsystems.com (salmon.nlsystems.com [10.0.0.3]) by herring.nlsystems.com (8.9.3/8.8.8) with ESMTP id JAA02023; Sat, 10 Jun 2000 09:41:03 +0100 (BST) (envelope-from dfr@nlsystems.com) Date: Sat, 10 Jun 2000 09:44:59 +0100 (BST) From: Doug Rabson To: Luoqi Chen Cc: dillon@apollo.backplane.com, arch@FreeBSD.ORG Subject: Re: Syscalls and execve In-Reply-To: <200006091239.e59Cd9T08096@lor.watermarkgroup.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 9 Jun 2000, Luoqi Chen wrote: > > > Why not have the new exec()'d process, when it gets the cpu in supervisor > > > mode, clear the registers in supervisor mode before returning > > > to user mode? e.g. near the end of kern/kern_exec.c's execve(). > > > (or somewhere similar). Then at least the 'garbage' will be more > > > like what you see on return from a syscall rather then something > > > inherited from another process. > > > > > Here the current process *is* the process calling exec() (unlike fork()), so > > why not just zero those registers in setregs()? > > > > -lq > > > I take back what I've just said, this won't guarantee zeros in those volatile > registers. Moreover doesn't the new process image expect to see argc, argv, > envp in the argument registers? Exec() needs special treatment, maybe an > exec_trampoline()? I think an exec_trampoline might well be the best solution. I can't quite see how to work it though. -- Doug Rabson Mail: dfr@nlsystems.com Nonlinear Systems Ltd. Phone: +44 20 8442 9037 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jun 10 6:10:13 2000 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 1C54A37B708 for ; Sat, 10 Jun 2000 06:10:11 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id IAA07662; Sat, 10 Jun 2000 08:56:43 -0400 (EDT) Date: Sat, 10 Jun 2000 08:56:42 -0400 (EDT) From: Daniel Eischen To: Doug Rabson Cc: Bruce Evans , arch@FreeBSD.ORG Subject: Re: Syscalls and execve In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, 10 Jun 2000, Doug Rabson wrote: > The trapframe which is created for syscall is identical to the trapframe > for exceptions and interrupts, its just not fully populated with register > values. On a related subject, the alpha sigcontext is different than the trapframe. This makes implementing {get,set}context slightly more difficult because they have to know which frame is in the machine context. For the new threads architecture (similar to scheduler activations), the context of threads that become unblocked in the kernel is passed out to the user threads library. I want to add {get,set}context as library routines and use them to handle both signal contexts and trapframes as passed out of the kernel. This isn't a big deal, we can just have a different set of routines to handle trapframes for the alpha, but if there is an opportunity to make trapframes and signal frames consistent (as they are on i386)... -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jun 10 6:55:51 2000 Delivered-To: freebsd-arch@freebsd.org Received: from gidora.zeta.org.au (gidora.zeta.org.au [203.26.10.25]) by hub.freebsd.org (Postfix) with SMTP id 8135B37B81D for ; Sat, 10 Jun 2000 06:55:48 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: (qmail 20104 invoked from network); 10 Jun 2000 13:55:45 -0000 Received: from unknown (HELO bde.zeta.org.au) (203.2.228.102) by gidora.zeta.org.au with SMTP; 10 Jun 2000 13:55:45 -0000 Date: Sat, 10 Jun 2000 23:55:41 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Daniel Eischen Cc: Doug Rabson , arch@FreeBSD.ORG Subject: Re: Syscalls and execve In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, 10 Jun 2000, Daniel Eischen wrote: > On Sat, 10 Jun 2000, Doug Rabson wrote: > > The trapframe which is created for syscall is identical to the trapframe > > for exceptions and interrupts, its just not fully populated with register > > values. > > On a related subject, the alpha sigcontext is different than the > trapframe. This makes implementing {get,set}context slightly more > difficult because they have to know which frame is in the machine > context. For the new threads architecture (similar to scheduler It's different on i386's too. It is certain to be different if the floating point state is separate and too expensive to save in the trap frame. The i386 also has complications for vm86 mode. > activations), the context of threads that become unblocked in > the kernel is passed out to the user threads library. I want to > add {get,set}context as library routines and use them to handle > both signal contexts and trapframes as passed out of the kernel. I don't see how they can be implemented as pure library routines, since they will have to get and set kernel state. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jun 10 11:16:50 2000 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 3FFBB37BEFE for ; Sat, 10 Jun 2000 11:16:47 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id OAA08510; Sat, 10 Jun 2000 14:16:28 -0400 (EDT) Date: Sat, 10 Jun 2000 14:16:28 -0400 (EDT) From: Daniel Eischen To: Bruce Evans Cc: Doug Rabson , arch@FreeBSD.ORG Subject: Re: Syscalls and execve In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, 10 Jun 2000, Bruce Evans wrote: > On Sat, 10 Jun 2000, Daniel Eischen wrote: > > > On Sat, 10 Jun 2000, Doug Rabson wrote: > > > The trapframe which is created for syscall is identical to the trapframe > > > for exceptions and interrupts, its just not fully populated with register > > > values. > > > > On a related subject, the alpha sigcontext is different than the > > trapframe. This makes implementing {get,set}context slightly more > > difficult because they have to know which frame is in the machine > > context. For the new threads architecture (similar to scheduler > > It's different on i386's too. It is certain to be different if the > floating point state is separate and too expensive to save in the > trap frame. The i386 also has complications for vm86 mode. I was thinking that the user threads library would handle floating point state separately. Right now, the threads library only saves floating point state when a signal occurs. Normal context switches do not save floating point state; it is only when a thread is preempted as a result of a signal that the floating point state is saved. A question I've been wanting to ask is: When a system call is made, can we assume that the floating point registers are still needed after return from the system call? > > activations), the context of threads that become unblocked in > > the kernel is passed out to the user threads library. I want to > > add {get,set}context as library routines and use them to handle > > both signal contexts and trapframes as passed out of the kernel. > > I don't see how they can be implemented as pure library routines, > since they will have to get and set kernel state. Why? They don't now (except for getting/setting the signal mask which can be totally avoided). -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jun 10 12:14:41 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mailman.zeta.org.au (mailman.zeta.org.au [203.26.10.16]) by hub.freebsd.org (Postfix) with ESMTP id 8EBC737BAF4 for ; Sat, 10 Jun 2000 12:14:37 -0700 (PDT) (envelope-from bde@zeta.org.au) Received: from bde.zeta.org.au (bde.zeta.org.au [203.2.228.102]) by mailman.zeta.org.au (8.8.7/8.8.7) with ESMTP id EAA07736; Sun, 11 Jun 2000 04:59:47 +1000 Date: Sun, 11 Jun 2000 04:59:43 +1000 (EST) From: Bruce Evans X-Sender: bde@besplex.bde.org To: Daniel Eischen Cc: Doug Rabson , arch@FreeBSD.ORG Subject: Re: Syscalls and execve In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sat, 10 Jun 2000, Daniel Eischen wrote: > On Sat, 10 Jun 2000, Bruce Evans wrote: > > A question I've been wanting to ask is: When a system call is > made, can we assume that the floating point registers are still > needed after return from the system call? I'm not sure what you mean by "needed". On i386's, the FP registers aren't guaranteed to have any particular values on return from a function that doesn't return an FP value, in particular for syscalls (except the control word must be preserved except by functions that are supposed to change it, and the status word and tag word must not be changed significantly). Syscalls on i386's don't change any of the FP state in practice. (After a context switch, the physical state may have changed on return from a syscall, but applications can't tell because they see a virtual state.) > > > activations), the context of threads that become unblocked in > > > the kernel is passed out to the user threads library. I want to > > > add {get,set}context as library routines and use them to handle > > > both signal contexts and trapframes as passed out of the kernel. > > > > I don't see how they can be implemented as pure library routines, > > since they will have to get and set kernel state. > > Why? They don't now (except for getting/setting the signal mask > which can be totally avoided). There is that, and maybe the FP state (or a kernel flag that tells you where the FP state is saved), and the alternative signal stack. On second thoughts, I guess there is no problem with the FP state for the current process -- a process should be able to see its own state using machine instructions. It is quite different from kernel state like the signal mask and the altstack -- these must be tracked by adding a layer to the syscalls that set them. Bruce To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jun 10 17:59:54 2000 Delivered-To: freebsd-arch@freebsd.org Received: from lor.watermarkgroup.com (lor.watermarkgroup.com [207.202.73.33]) by hub.freebsd.org (Postfix) with ESMTP id 9E53537BE13 for ; Sat, 10 Jun 2000 17:59:52 -0700 (PDT) (envelope-from luoqi@watermarkgroup.com) Received: (from luoqi@localhost) by lor.watermarkgroup.com (8.10.1/8.10.1) id e5B0xpk14371; Sat, 10 Jun 2000 20:59:51 -0400 (EDT) Date: Sat, 10 Jun 2000 20:59:51 -0400 (EDT) From: Luoqi Chen Message-Id: <200006110059.e5B0xpk14371@lor.watermarkgroup.com> To: dfr@nlsystems.com Subject: Re: Syscalls and execve Cc: arch@FreeBSD.ORG Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG > I think an exec_trampoline might well be the best solution. I can't quite > see how to work it though. > > -- > Doug Rabson Mail: dfr@nlsystems.com > Nonlinear Systems Ltd. Phone: +44 20 8442 9037 > I would put the trampoline code right below the arguments, so it won't waste any stack space. I would first place $pc, $a0, $a3 values into $ra, $s0, $s1 registers in the shorter trapframe, and in the trampoline code, copy from $s0/1 to $a0/3, clear $s0/1 and other volatile registers, return. -lq To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message From owner-freebsd-arch Sat Jun 10 19: 7:45 2000 Delivered-To: freebsd-arch@freebsd.org Received: from pcnet1.pcnet.com (pcnet1.pcnet.com [204.213.232.3]) by hub.freebsd.org (Postfix) with ESMTP id 2D65137BD65 for ; Sat, 10 Jun 2000 19:07:38 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.8.7/PCNet) id WAA22407; Sat, 10 Jun 2000 22:07:16 -0400 (EDT) Date: Sat, 10 Jun 2000 22:07:15 -0400 (EDT) From: Daniel Eischen To: Bruce Evans Cc: Doug Rabson , arch@FreeBSD.ORG Subject: Re: Syscalls and execve In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Sun, 11 Jun 2000, Bruce Evans wrote: > On Sat, 10 Jun 2000, Daniel Eischen wrote: > > > On Sat, 10 Jun 2000, Bruce Evans wrote: > > > > A question I've been wanting to ask is: When a system call is > > made, can we assume that the floating point registers are still > > needed after return from the system call? > > I'm not sure what you mean by "needed". On i386's, the FP registers > aren't guaranteed to have any particular values on return from a > function that doesn't return an FP value, in particular for syscalls > (except the control word must be preserved except by functions that > are supposed to change it, and the status word and tag word must not > be changed significantly). Syscalls on i386's don't change any of > the FP state in practice. (After a context switch, the physical > state may have changed on return from a syscall, but applications > can't tell because they see a virtual state.) I'd like to avoid saving the FP state (except perhaps for the control word) when a syscall blocks in the kernel. After a syscall blocks, the threads library gets an upcall and switches to another thread without saving the previous (blocked) threads FP state. There is also the possiblility that the upcall happens in a different process/processor (when there are multiple cooperating processes). In this case, the process handling the upcall doesn't have the FP state available unless the kernel passes it back out. > > > > activations), the context of threads that become unblocked in > > > > the kernel is passed out to the user threads library. I want to > > > > add {get,set}context as library routines and use them to handle > > > > both signal contexts and trapframes as passed out of the kernel. > > > > > > I don't see how they can be implemented as pure library routines, > > > since they will have to get and set kernel state. > > > > Why? They don't now (except for getting/setting the signal mask > > which can be totally avoided). > > There is that, and maybe the FP state (or a kernel flag that tells you > where the FP state is saved), and the alternative signal stack. On > second thoughts, I guess there is no problem with the FP state for > the current process -- a process should be able to see its own state > using machine instructions. It is quite different from kernel state > like the signal mask and the altstack -- these must be tracked by > adding a layer to the syscalls that set them. I have library level routines for {get,set,make,swap}context and modified ucontext_t to add a flag indicating whether the FP regs were valid or not (perhaps this should be a flag in mcontext_t on second thought). As for alternative signal stacks, POSIX doesn't recognize them (at least in the context of pthreads). I was planning on using the threads own stack to deliver signals. Signals could still be delivered to the process on a sigalt stack, but the threads library would add a signal handling frame on the top of the target threads stack instead of trying to handle the signal on the signal stack. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message