From owner-freebsd-chat Sun Jan 12 8:56:49 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5531A37B401; Sun, 12 Jan 2003 08:56:48 -0800 (PST) Received: from picard.skynet.be (picard.skynet.be [195.238.3.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id D49A243F6D; Sun, 12 Jan 2003 08:56:46 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [12.27.220.113] (ip-26.shub-internet.org [194.78.144.26] (may be forged)) by picard.skynet.be (8.11.6/8.11.6/Skynet-OUT-2.20) with ESMTP id h0CGuf915834; Sun, 12 Jan 2003 17:56:41 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> X-Grok: +++ath X-WebTV-Stationery: Standard; BGColor=black; TextColor=black Reply-By: Wed, 1 Jan 1984 12:34:56 +0100 Date: Sun, 12 Jan 2003 17:45:17 +0100 To: Doug Barton From: Brad Knowles Subject: Re: Need advice on PHP and MySQL books Cc: Terry Lambert , freebsd-chat@FreeBSD.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 2:55 PM -0800 2003/01/11, Doug Barton wrote: > Eventually, I also want to be able to > run bind 9 plugged directly into the database (or a similar database) > without zone files at all. That's not too hard. Check out the bind-dlz page at . This is not quite the full implementation that you'd need, as there is some protocol-level support that needs to be added for creating new zones and distributing that information to the secondaries. However, for the work that is done, there's no sense in reinventing the wheel. -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Sun Jan 12 8:56:56 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2025D37B401; Sun, 12 Jan 2003 08:56:54 -0800 (PST) Received: from picard.skynet.be (picard.skynet.be [195.238.3.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id BF5A143F3F; Sun, 12 Jan 2003 08:56:52 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [12.27.220.113] (ip-26.shub-internet.org [194.78.144.26] (may be forged)) by picard.skynet.be (8.11.6/8.11.6/Skynet-OUT-2.20) with ESMTP id h0CGui915901; Sun, 12 Jan 2003 17:56:44 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <3E20D1D6.E9FC60B1@mindspring.com> References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> <3E20D1D6.E9FC60B1@mindspring.com> X-Grok: +++ath X-WebTV-Stationery: Standard; BGColor=black; TextColor=black Reply-By: Wed, 1 Jan 1984 12:34:56 +0100 Date: Sun, 12 Jan 2003 17:53:00 +0100 To: Terry Lambert From: Brad Knowles Subject: Re: Need advice on PHP and MySQL books Cc: Doug Barton , freebsd-chat@FreeBSD.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 6:24 PM -0800 2003/01/11, Terry Lambert wrote: > This is actually exactly what we did for the IBM Web Connections > NOC (we used Perl for the CGI to populate the MySQL database, and > we used Java to generate the DNS and SMTP configuration files). I have heard of a number of places that did their own DNS management tools along this sort of lines. Not necessarily using these particular languages, but basically this same idea. Some of those have been re-packaged and turned into commercial systems like QIP from Lucent or QuickDNS Manager from Men & Mice -- the cool thing about QuickDNS Manager is that it will work with the nameserver they wrote themselves as well as stock ISC BIND. However, in terms of the work of this sort that is publicly available, the most advanced project I know of is bind-dlz (see ). > The next generation would have been able to update in realtime, > without needing to kick servers in the head, or to explicitly > derive the data. Really? I'd love to hear more about that. > It would be very east to do this with an LDAP directory, I think; > much harder to do with a relation database, unless you establish > record references internal to the database which were, themselves, > hierarchical. BIND 9 already provides hooks for back-end databases, and there are already defined profiles for working with PostgreSQL, and a Berkeley DB implementation should be nearly trivial. The problem is that any alternative back-end database you could choose would almost certainly be much, much slower than the built-in in-memory red/black tree that is already used within BIND. The only question should be whether or not the replacement back-end database gives you enough other advantages to make up for the loss in performance. > IMO, you are much better off just sending notifications of changes > to the DNS server (via DNSUPDAT), and modifying the DNS server to > permit zone creation in the primary (minimally). Thinking about this a bit more, if you had multiple primaries pulling data off the same PostgreSQL dictionary, and/or you had your also-notify setting configured to hit the other "primary" servers, then you wouldn't have to worry about the synchronization between the "primary" and the "secondaries", because all machines would be primary. -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Sun Jan 12 16:18:49 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0F88537B401; Sun, 12 Jan 2003 16:18:48 -0800 (PST) Received: from badboy.mail.pas.earthlink.net (badboy.mail.pas.earthlink.net [207.217.120.20]) by mx1.FreeBSD.org (Postfix) with ESMTP id 753F543F18; Sun, 12 Jan 2003 16:18:47 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from heron (heron.mail.pas.earthlink.net [207.217.120.189]) by badboy.mail.pas.earthlink.net (8.11.6+Sun/8.11.6) with ESMTP id h0CNh2k15594; Sun, 12 Jan 2003 15:43:02 -0800 (PST) Received: from pool0338.cvx22-bradley.dialup.earthlink.net ([209.179.199.83] helo=mindspring.com) by heron with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18Xrkc-0005fw-00; Sun, 12 Jan 2003 15:42:46 -0800 Message-ID: <3E21FD22.38CD81BB@mindspring.com> Date: Sun, 12 Jan 2003 15:41:22 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Brad Knowles Cc: Doug Barton , freebsd-chat@FreeBSD.org Subject: Re: Need advice on PHP and MySQL books References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a422a5d9d5aac9e78ac9f134522e7a3792666fa475841a1c7a350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Brad Knowles wrote: > That's not too hard. Check out the bind-dlz page at > . Thanks for this pointer, BTW; I was unaware that this project existed. 8-). -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Sun Jan 12 16:23:20 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 273E137B401; Sun, 12 Jan 2003 16:23:17 -0800 (PST) Received: from vador.skynet.be (vador.skynet.be [195.238.3.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id AFB6F43F13; Sun, 12 Jan 2003 16:23:15 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.2] (ip-26.shub-internet.org [194.78.144.26] (may be forged)) by vador.skynet.be (8.11.6/8.11.6/Skynet-OUT-2.20) with ESMTP id h0D0N1m00565; Mon, 13 Jan 2003 01:23:02 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <3E21FCA1.C4C63219@mindspring.com> References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> <3E20D1D6.E9FC60B1@mindspring.com> <3E21FCA1.C4C63219@mindspring.com> X-Grok: +++ath X-WebTV-Stationery: Standard; BGColor=black; TextColor=black Reply-By: Wed, 1 Jan 1984 12:34:56 +0100 Date: Mon, 13 Jan 2003 01:19:07 +0100 To: Terry Lambert From: Brad Knowles Subject: Re: Need advice on PHP and MySQL books Cc: Brad Knowles , Doug Barton , freebsd-chat@FreeBSD.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 3:39 PM -0800 2003/01/12, Terry Lambert wrote: > Without this, you would be required to extendively modify your > server software; bere little DNS software, no matter who writes > it, has the ability to shed load very well. I disagree. The Caching Name Server (CNS) and Authoritative Name Server (ANS) software from Nominum looks like it will handle load very, very well. Rick Jones has done some preliminary benchmarks that show it capable of handling 50,000+ DNS queries per second, and I believe that it may well be even faster than this. I'll be shortly doing some testing of my own, which I will be presenting at RIPE 44 in Amsterdam at the end of January. Neverthless, the lesson we learned really well at AOL is that if you get enough ankle-biters together, they can still take down the biggest elephant in the world. > The (relatively) > recent attack on the root servers had no effect only because it > was not sustained until more than 50% of the TTL had passed, at > which point the overall degradation would have been exponential; > as it was, less than 1% of the Internet even knew there was an > attack in progress. If they had tried to sustain the attack that long, I think the perpetrators would have been found and caught, and dealt with very, very harshly. I think they knew this, which is why they stopped so soon. > If you added a really long latency, effectively increasing the > pool retention time for requests, then you decrease the total > number of requests you can handle in a given period of time > before the servers become vulnerable by becomining more limiting > than the pipe size. At that point, you've introduced a serious > vulnerability to the system, that hasn't been addressed, and can't > be addressed merely by throwing more resources at it. I'm sorry, I don't quite understand how you arrive at this conclusion. Could you elaborate? > This is really wrong. > > The problem with this model is the communications channel between > the primaries is vulnerable. There is no security weakness that did not already exist, because you forward to them the same update packet that you receive yourself. At least, this is how I think it would be best to handle the "multiple primaries" problem. > As I stated on namedroppers years ago, when DJB first argued this > approach, and that inter-server synchronization was "left as an > exercise for the student" (his suggestion was rsync or some other > protocol for synchronization, over SSH, etc.), the problem is that > DNS data should always flow over the DNS protocol. I couldn't agree more. However, there are some cases where you may have to deal with out-of-band data, such as when you're replacing the back-end database. > The main secondary reason is, I think, that you don't want to > open up a communications channel from an insecure server (the DNS > server is public-facing) to a secure zone server (the database, > whatever it is, is vulnerable. > > Basically, this argues that data should be pushed from the secure > area to the insecure area, rather than pulled from the insecure > area from the secure... unless that pull occurs over an already > established secure tunnel. Also agreed. See above regarding the forwarding of the same update packet as was received. > Thus my recommendation would be to replicate a DNS server contents > in the customer-facing server in the insecure area from another, > non-customer-facing, server in the secure area. Fair enough. I can't really argue with the rest. ;-) -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Sun Jan 12 16:23:21 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A9FC037B405; Sun, 12 Jan 2003 16:23:17 -0800 (PST) Received: from vador.skynet.be (vador.skynet.be [195.238.3.236]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7A6F543EB2; Sun, 12 Jan 2003 16:23:16 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.2] (ip-26.shub-internet.org [194.78.144.26] (may be forged)) by vador.skynet.be (8.11.6/8.11.6/Skynet-OUT-2.20) with ESMTP id h0D0N6m00643; Mon, 13 Jan 2003 01:23:07 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <3E21FD22.38CD81BB@mindspring.com> References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> <3E21FD22.38CD81BB@mindspring.com> X-Grok: +++ath X-WebTV-Stationery: Standard; BGColor=black; TextColor=black Reply-By: Wed, 1 Jan 1984 12:34:56 +0100 Date: Mon, 13 Jan 2003 01:22:43 +0100 To: Terry Lambert From: Brad Knowles Subject: Re: Need advice on PHP and MySQL books Cc: Brad Knowles , Doug Barton , freebsd-chat@FreeBSD.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 3:41 PM -0800 2003/01/12, Terry Lambert wrote: >> That's not too hard. Check out the bind-dlz page at >> . > > Thanks for this pointer, BTW; I was unaware that this project > existed. 8-). I like to think of myself as reasonably well-informed when it comes to things related to the DNS, and I didn't even know about this project until it was proposed as a poster talk for SANE 2002, and I wouldn't have even known about it then except that I was on the program committee. Now, while we're on the subject of DNS-related things that Stichting NLnet is supporting, you may also want to check out NSD -- Name Server Daemon at . Yet another thing I learned about as a poster at SANE 2002.... -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Sun Jan 12 16:41:40 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5AC9F37B401; Sun, 12 Jan 2003 16:41:35 -0800 (PST) Received: from soulshock.mail.pas.earthlink.net (soulshock.mail.pas.earthlink.net [207.217.120.130]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9743B43F43; Sun, 12 Jan 2003 16:41:34 -0800 (PST) (envelope-from tlambert2@mindspring.com) Received: from heron (heron.mail.pas.earthlink.net [207.217.120.189]) by soulshock.mail.pas.earthlink.net (8.11.6+Sun/8.11.6) with ESMTP id h0CNg4H17925; Sun, 12 Jan 2003 15:42:04 -0800 (PST) Received: from pool0338.cvx22-bradley.dialup.earthlink.net ([209.179.199.83] helo=mindspring.com) by heron with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 18Xrjb-0005Y6-00; Sun, 12 Jan 2003 15:41:44 -0800 Message-ID: <3E21FCA1.C4C63219@mindspring.com> Date: Sun, 12 Jan 2003 15:39:13 -0800 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Brad Knowles Cc: Doug Barton , freebsd-chat@FreeBSD.org Subject: Re: Need advice on PHP and MySQL books References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> <3E20D1D6.E9FC60B1@mindspring.com> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a422a5d9d5aac9e78a2fbc6a49d97850d6548b785378294e88350badd9bab72f9c350badd9bab72f9c Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Brad Knowles wrote: > However, in terms of the work of this sort that is publicly > available, the most advanced project I know of is bind-dlz (see > ). I was unaware of this project; it looks like there is a single developer (common, for Source Forge projects, unfortunately), but the last patch was in December, so I guess it's still somewhat active. > > The next generation would have been able to update in realtime, > > without needing to kick servers in the head, or to explicitly > > derive the data. > > Really? I'd love to hear more about that. Basically, it would have used the DNSUPDAT mechanism to push data into the standard bind. Following a discussion on namedroppers (the DNSEXT mailing list), I modified the bind code to permit creation of zones in a primary, via DNSUPDAT. The specific reason for disallowing creation of zones via DNSUPDAT was lack of a security association. But there is really not a model issue -- it's prefectly reasonable to disable the check that precents this in the source code, if the zone is being created as a primary. Creating zones in a secondary is much harder... doing that would require a security association. Also, the most common method of operation is "stealth primary", for this purpose, so that the secondaries (which appear to the world as primaries) are the only machines banging your main DNS server. This is desirable for (hopefully) obvious reasons. > > It would be very east to do this with an LDAP directory, I think; > > much harder to do with a relation database, unless you establish > > record references internal to the database which were, themselves, > > hierarchical. > > BIND 9 already provides hooks for back-end databases, and there > are already defined profiles for working with PostgreSQL, and a > Berkeley DB implementation should be nearly trivial. The problem is > that any alternative back-end database you could choose would almost > certainly be much, much slower than the built-in in-memory red/black > tree that is already used within BIND. Precisely. This is a result of the "impedance mismatch" that I noted, which comes from trying to map a hierarchical relationship into a relational database, which assumes flat relations. This is why a hierarchical database (e.g. LDAP) or even a file system, works better as a data source. It's also why DNSUPDAT is preferred. The primary harm these days is actually "cached data"; I've been considerening for several years now, whether or not to write a canonical "Cached Data Considered Harmful". I think I can make a good case. In any case, with explicit update, the caching problem is solved, as is the non-native performance problem. The same would be true for DNS served out of an already hierarchical data store, like LDAP, because it has the same issues to resolve that the red/black tree solves in DNS. Rather than solving this twice (and ending up with a very fast LDAP server -- but a difficult task 8-)), my take was to update the DNS data in situ. The biggest issue was the creation of caching files, so that the DNS data remained cached to disk in a way that allowed a recovery after a server reboot (again, zone creation is the problem, since a zone cache file, once it exists, will be maintained properly with no additional work). So really, this was about two modifications to bind. > The only question should be whether or not the replacement > back-end database gives you enough other advantages to make up for > the loss in performance. IMO, "no". The point of having high performance is to have a stomach that's bigger than your eyes, so to speak: you want to be able to handle as much load as it's possible to shove down all the pipes to your server, and then add overage, so if you are DDOS'ed, the limiting factor is your pipe, not your server. Without this, you would be required to extendively modify your server software; bere little DNS software, no matter who writes it, has the ability to shed load very well. The (relatively) recent attack on the root servers had no effect only because it was not sustained until more than 50% of the TTL had passed, at which point the overall degradation would have been exponential; as it was, less than 1% of the Internet even knew there was an attack in progress. If you added a really long latency, effectively increasing the pool retention time for requests, then you decrease the total number of requests you can handle in a given period of time before the servers become vulnerable by becomining more limiting than the pipe size. At that point, you've introduced a serious vulnerability to the system, that hasn't been addressed, and can't be addressed merely by throwing more resources at it. > > IMO, you are much better off just sending notifications of changes > > to the DNS server (via DNSUPDAT), and modifying the DNS server to > > permit zone creation in the primary (minimally). > > Thinking about this a bit more, if you had multiple primaries > pulling data off the same PostgreSQL dictionary, and/or you had your > also-notify setting configured to hit the other "primary" servers, > then you wouldn't have to worry about the synchronization between the > "primary" and the "secondaries", because all machines would be > primary. This is really wrong. The problem with this model is the communications channel between the primaries is vulnerable. As I stated on namedroppers years ago, when DJB first argued this approach, and that inter-server synchronization was "left as an exercise for the student" (his suggestion was rsync or some other protocol for synchronization, over SSH, etc.), the problem is that DNS data should always flow over the DNS protocol. There are a lot of reasons for this, but the top ones, to my mind, are standardization of installations, and the inability to override corporate policy and poke holes in firewalls. The main secondary reason is, I think, that you don't want to open up a communications channel from an insecure server (the DNS server is public-facing) to a secure zone server (the database, whatever it is, is vulnerable. Basically, this argues that data should be pushed from the secure area to the insecure area, rather than pulled from the insecure area from the secure... unless that pull occurs over an already established secure tunnel. Thus my recommendation would be to replicate a DNS server contents in the customer-facing server in the insecure area from another, non-customer-facing, server in the secure area. Basically, this boils down to an implementation of: database server primary DNS secondary DNS ------------> <----------- contact ------------> -----------> data flow | | barriers green yellow red "IBM-ese" Most often, your database is actually your customer database. Effectively, this means that the database contains data which has been normalized for business processes, rather than for efficiency of serving DNS requests, even if you've magically addressed all the security concerns. One of the biggest problesm IBM has, actually, from an insider's perspective, is that it only ever expends effort on systems that are customer-facing and/or customer-visible. Everything else is held together with spit an bailing wire. It doesn't matter how hard it is for an IBM employeee to do their job, so long as it's easy for the customer. This actually results in the evolution of systems with an extremely high marginal cost on a per incident basis, for customer interactions, with the products IBM offers being limited to either front-loaded costs, or ongoing consulting costs, at margins on the order of 40%; they could actually cut out 20% of overhead (increasing profits by 20%) by working *on* their business. In any case, what this came down to in our case was a lot of manual effort on the part of an employee, when it came to account setup and teardown. Effectively, the systems ran independently, like you were suggesting they might, such that the data can be normalized to how it's going to be used, instead of for a business process. This will not be the common case, out in the world, where you're not able to charge another 20% for the name "IBM", or you can charge for it, but you want that 20% to go into profits, instead of into operations costs. -- Terry To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Mon Jan 13 3:22:32 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DCF4037B401 for ; Mon, 13 Jan 2003 03:22:31 -0800 (PST) Received: from shale.csir.co.za (shale.csir.co.za [146.64.46.5]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7032443ED8 for ; Mon, 13 Jan 2003 03:22:25 -0800 (PST) (envelope-from reg@shale.csir.co.za) Received: (from reg@localhost) by shale.csir.co.za (8.11.6/8.11.6) id h0DBMHL28655 for freebsd-chat@freebsd.org; Mon, 13 Jan 2003 13:22:17 +0200 (SAT) (envelope-from reg) Date: Mon, 13 Jan 2003 13:22:17 +0200 From: Jeremy Lea To: freebsd-chat@freebsd.org Subject: Looking for an ISP in Cape Town Message-ID: <20030113112216.GA28632@shale.csir.co.za> Mail-Followup-To: Jeremy Lea , freebsd-chat@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi, If you're not a South African, you can probably ignore this... I'm looking for a FreeBSD friendly ISP in Cape Town, since I'm going to be moving down there soon... Basically an ISP that doesn't mind fairly high volumes of mail, uses normal PPP, etc. If I can get shell access and procmail on their mail server, all the better... Regards, -Jeremy -- FreeBSD - Because the best things in life are free... http://www.freebsd.org/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Mon Jan 13 5:21:53 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 22F2237B405; Mon, 13 Jan 2003 05:21:52 -0800 (PST) Received: from picard.skynet.be (picard.skynet.be [195.238.3.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id A123F43F18; Mon, 13 Jan 2003 05:21:50 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.2] (ip-26.shub-internet.org [194.78.144.26] (may be forged)) by picard.skynet.be (8.11.6/8.11.6/Skynet-OUT-2.20) with ESMTP id h0DDLJ901310; Mon, 13 Jan 2003 14:21:21 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <20030113112216.GA28632@shale.csir.co.za> References: <20030113112216.GA28632@shale.csir.co.za> X-Grok: +++ath X-WebTV-Stationery: Standard; BGColor=black; TextColor=black Reply-By: Wed, 1 Jan 1984 12:34:56 +0100 Date: Mon, 13 Jan 2003 14:20:46 +0100 To: Jeremy Lea From: Brad Knowles Subject: Re: Looking for an ISP in Cape Town Cc: freebsd-chat@freebsd.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 1:22 PM +0200 2003/01/13, Jeremy Lea wrote: > I'm looking for a FreeBSD friendly ISP in Cape Town, since I'm going to > be moving down there soon... Basically an ISP that doesn't mind fairly > high volumes of mail, uses normal PPP, etc. If I can get shell access > and procmail on their mail server, all the better... My experience is that most providers in Africa are much more Linux-oriented, if they aren't using either Microsoft or something ancient and proprietary like SCO. However, even though they may choose Linux instead of *BSD, they may still be more friendly to your type of usage. Have you looked at the providers at or ? -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Mon Jan 13 6:15:26 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 63C0B37B401 for ; Mon, 13 Jan 2003 06:15:25 -0800 (PST) Received: from bastet.rfc822.net (bastet.rfc822.net [64.81.113.233]) by mx1.FreeBSD.org (Postfix) with ESMTP id E612243E4A for ; Mon, 13 Jan 2003 06:15:24 -0800 (PST) (envelope-from pde@bastet.rfc822.net) Received: by bastet.rfc822.net (Postfix, from userid 1001) id AA9019F048; Mon, 13 Jan 2003 08:15:42 -0600 (CST) Date: Mon, 13 Jan 2003 08:15:42 -0600 From: Pete Ehlke To: freebsd-chat@FreeBSD.org Subject: Re: Need advice on PHP and MySQL books Message-ID: <20030113141542.GC2260@rfc822.net> References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> <3E21FD22.38CD81BB@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.1i Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Mon, Jan 13, 2003 at 01:22:43AM +0100, Brad Knowles wrote: > > Now, while we're on the subject of DNS-related things that > Stichting NLnet is supporting, you may also want to check out NSD -- > Name Server Daemon at . > Yet another thing I learned about as a poster at SANE 2002.... > Does anyone know the story about NSD? I've looked at it several times, run it and played with it locally quite a bit, and found it extremely interesting. But I've had a third-hand report that RIPE folks have said (third hand, but this is the direct quote I got...) "the damn thing just didn't work". Haven't been able to get more than that. -Pete To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Mon Jan 13 7:35:39 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4B92537B401 for ; Mon, 13 Jan 2003 07:35:37 -0800 (PST) Received: from excalibur.skynet.be (excalibur.skynet.be [195.238.3.90]) by mx1.FreeBSD.org (Postfix) with ESMTP id D450B43F3F for ; Mon, 13 Jan 2003 07:35:34 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.2] (ip-26.shub-internet.org [194.78.144.26] (may be forged)) by excalibur.skynet.be (8.11.6/8.11.6/Skynet-OUT-2.20) with ESMTP id h0DFYud00465; Mon, 13 Jan 2003 16:34:57 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <20030113141542.GC2260@rfc822.net> References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> <3E21FD22.38CD81BB@mindspring.com> <20030113141542.GC2260@rfc822.net> X-Grok: +++ath X-WebTV-Stationery: Standard; BGColor=black; TextColor=black Reply-By: Wed, 1 Jan 1984 12:34:56 +0100 Date: Mon, 13 Jan 2003 15:51:43 +0100 To: Pete Ehlke From: Brad Knowles Subject: Re: Need advice on PHP and MySQL books Cc: freebsd-chat@FreeBSD.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 8:15 AM -0600 2003/01/13, Pete Ehlke wrote: > Does anyone know the story about NSD? I've looked at it several times, > run it and played with it locally quite a bit, and found it extremely > interesting. But I've had a third-hand report that RIPE folks have said > (third hand, but this is the direct quote I got...) "the damn thing just > didn't work". Haven't been able to get more than that. I've benchmarked it, and it is *damn* bloody fast (see ). However, the way it works is to pre-calculate every possible query for the zone in question, and then to pre-generate every possible answer (coalescing as many questions and answers together as it can), and then to index all the answers from all the questions. They throw out all the "normal" authoritative name server tasks that would normally be done that are not strictly required for the kind of operations you would expect to see at a root name server. So, for a moderately large zone, it is incredibly fast, but can return somewhat strange answers (due to the way it optimizes queries and may ignore domain name compression or implement it differently than you would expect), and it certainly doesn't do anything like round-robin, etc.... But for much larger zones (like .nl, and especially the signed "test" version at .nl.nl), you just can't feed it enough memory. Or so I have been told, and by people at least one step closer to that process. Unfortunately, I learned some things about the way it works *after* the presentation I did at LISA 2002, so now my view of the program is not nearly so positive as it was then. Personally, I'd be much more interested to learn how the folks at Nominum managed to get CNS and ANS to be as incredibly fast as they are, and still manage to provide "full service" DNS. Sure beats the crap out of anything else I've ever seen. -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Mon Jan 13 7:49:56 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5498137B401 for ; Mon, 13 Jan 2003 07:49:55 -0800 (PST) Received: from bastet.rfc822.net (bastet.rfc822.net [64.81.113.233]) by mx1.FreeBSD.org (Postfix) with ESMTP id AF99643F5B for ; Mon, 13 Jan 2003 07:49:54 -0800 (PST) (envelope-from pde@bastet.rfc822.net) Received: by bastet.rfc822.net (Postfix, from userid 1001) id ECE149F035; Mon, 13 Jan 2003 09:50:11 -0600 (CST) Date: Mon, 13 Jan 2003 09:50:11 -0600 From: Pete Ehlke To: freebsd-chat@FreeBSD.org Subject: Re: Need advice on PHP and MySQL books Message-ID: <20030113155011.GA2719@rfc822.net> References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> <3E21FD22.38CD81BB@mindspring.com> <20030113141542.GC2260@rfc822.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.1i Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Mon, Jan 13, 2003 at 03:51:43PM +0100, Brad Knowles wrote: > At 8:15 AM -0600 2003/01/13, Pete Ehlke wrote: > > > Does anyone know the story about NSD? I've looked at it several times, > > run it and played with it locally quite a bit, and found it extremely > > interesting. But I've had a third-hand report that RIPE folks have said > > (third hand, but this is the direct quote I got...) "the damn thing just > > didn't work". Haven't been able to get more than that. > > I've benchmarked it, and it is *damn* bloody fast (see > ). > Agreed. It simply blew me away when I tried it out here at home. It was doing utterly unheard of numbers on old, low-end PC hardware. For those who might want to play with it, /usr/ports/net/nsd > However, the way it works is to pre-calculate every possible > query for the zone in question, and then to pre-generate every > possible answer (coalescing as many questions and answers together as > it can), and then to index all the answers from all the questions. > They throw out all the "normal" authoritative name server tasks that > would normally be done that are not strictly required for the kind of > operations you would expect to see at a root name server. > Right. IIRC (and I may not; the coffee hasn't really kicked in yet...) djb tried the same thing in early versions of his authoritative server, and gave up on the idea. > But for much larger zones (like .nl, and especially the signed > "test" version at .nl.nl), you just can't feed it enough memory. Or > so I have been told, and by people at least one step closer to that > process. Ah, right, that makes sense. -P. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Mon Jan 13 8:42: 2 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8ED1B37B401 for ; Mon, 13 Jan 2003 08:41:58 -0800 (PST) Received: from riker.skynet.be (riker.skynet.be [195.238.3.89]) by mx1.FreeBSD.org (Postfix) with ESMTP id CE62443F1E for ; Mon, 13 Jan 2003 08:41:56 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.2] (ip-26.shub-internet.org [194.78.144.26] (may be forged)) by riker.skynet.be (8.11.6/8.11.6/Skynet-OUT-2.20) with ESMTP id h0DGf8I28494; Mon, 13 Jan 2003 17:41:08 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <20030113155011.GA2719@rfc822.net> References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> <3E21FD22.38CD81BB@mindspring.com> <20030113141542.GC2260@rfc822.net> <20030113155011.GA2719@rfc822.net> X-Grok: +++ath X-WebTV-Stationery: Standard; BGColor=black; TextColor=black Reply-By: Wed, 1 Jan 1984 12:34:56 +0100 Date: Mon, 13 Jan 2003 17:16:30 +0100 To: Pete Ehlke From: Brad Knowles Subject: Re: Need advice on PHP and MySQL books Cc: freebsd-chat@FreeBSD.org Content-Type: text/plain; charset="iso-8859-1" ; format="flowed" Content-Transfer-Encoding: 8bit Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 9:50 AM -0600 2003/01/13, Pete Ehlke wrote: >> I've benchmarked it, and it is *damn* bloody fast (see >> ). >> > Agreed. It simply blew me away when I tried it out here at home. It was > doing utterly unheard of numbers on old, low-end PC hardware. Yup. Using a private copy of the root zone and the .tv zone, I saw it doing 1400+ DNS queries per second, using queryperf (from BIND 9) on the same ultra-low end Compaq Armada 4131T laptop that I "inherited" from my wife when she upgraded (Pentium 133, 48MB RAM, 384MB virtual, 10GB IBM Travelstar 20GN, Asanté FriendlyNET AL1011 "Prism2" 802.11b WiFi NIC). I believe that my limitation here was how fast queryperf could generate the packets, not how fast NSD could respond. I tried doing some testing on my PowerBook G3 "Pismo" laptop running MacOS X 10.2.1 (400Mhz G3 processor, 1GB RAM, 2GB virtual, 48GB IBM Travelstar 40GH, Apple Airport 802.11b WiFi NIC), and I recall seeing NSD doing over 5000 DNS queries per second. Again, I believe that my limit was how fast queryperf could generate the questions, not how fast NSD could answer them. Simply unbelievable. > For those who might want to play with it, /usr/ports/net/nsd Excellent! How long has this been there? > Right. IIRC (and I may not; the coffee hasn't really kicked in yet...) > djb tried the same thing in early versions of his authoritative server, > and gave up on the idea. He's definitely done some whacked-out things. He's also claiming some equally unbelievable numbers. However, I have yet to see any proof of those claims. I'm a strong believer in the "show me" principle when it comes to claims of high levels of performance. In the case of NSD, I believe the numbers that I have personally seen, although I believe that the methods they used are not suitable for a general-purpose authoritative-only server, nor do they seem to be suitable for larger special-case situations such TLD nameservers and possibly root nameservers. -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Mon Jan 13 8:47:15 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A600F37B406 for ; Mon, 13 Jan 2003 08:47:14 -0800 (PST) Received: from bastet.rfc822.net (bastet.rfc822.net [64.81.113.233]) by mx1.FreeBSD.org (Postfix) with ESMTP id 27FF643F43 for ; Mon, 13 Jan 2003 08:47:14 -0800 (PST) (envelope-from pde@bastet.rfc822.net) Received: by bastet.rfc822.net (Postfix, from userid 1001) id 31A1C9F004; Mon, 13 Jan 2003 10:47:32 -0600 (CST) Date: Mon, 13 Jan 2003 10:47:32 -0600 From: Pete Ehlke To: Brad Knowles Cc: freebsd-chat@FreeBSD.org Subject: Re: Need advice on PHP and MySQL books Message-ID: <20030113164732.GA2998@rfc822.net> References: <20030110234309.R12065@2-234-22-23.pyvrag.nggov.pbz> <3E1FF12B.5390D978@mindspring.com> <20030111144619.X22424@2-234-22-23.pyvrag.nggov.pbz> <3E21FD22.38CD81BB@mindspring.com> <20030113141542.GC2260@rfc822.net> <20030113155011.GA2719@rfc822.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.1i Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Mon, Jan 13, 2003 at 05:16:30PM +0100, Brad Knowles wrote: > At 9:50 AM -0600 2003/01/13, Pete Ehlke wrote: > > > For those who might want to play with it, /usr/ports/net/nsd > > Excellent! How long has this been there? > bastet[/usr/ports/net/nsd]$ head -3 Makefile # New ports collection makefile for: nsd # Date created: 08 August 2002 # Whom: alexis bastet[/usr/ports/net/nsd]$ ;-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Thu Jan 16 21:45: 4 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C2F4E37B401 for ; Thu, 16 Jan 2003 21:45:02 -0800 (PST) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id 424C043F6D for ; Thu, 16 Jan 2003 21:45:02 -0800 (PST) (envelope-from crist.clark@attbi.com) Received: from blossom.cjclark.org (12-234-89-252.client.attbi.com[12.234.89.252]) by rwcrmhc52.attbi.com (rwcrmhc52) with ESMTP id <20030117054501052008uld5e>; Fri, 17 Jan 2003 05:45:01 +0000 Received: from blossom.cjclark.org (localhost. [127.0.0.1]) by blossom.cjclark.org (8.12.6/8.12.3) with ESMTP id h0H5j0eq050497 for ; Thu, 16 Jan 2003 21:45:00 -0800 (PST) (envelope-from crist.clark@attbi.com) Received: (from cjc@localhost) by blossom.cjclark.org (8.12.6/8.12.6/Submit) id h0H5ixj1050493 for freebsd-chat@freebsd.org; Thu, 16 Jan 2003 21:44:59 -0800 (PST) X-Authentication-Warning: blossom.cjclark.org: cjc set sender to crist.clark@attbi.com using -f Date: Thu, 16 Jan 2003 21:44:59 -0800 From: "Crist J. Clark" To: freebsd-chat@freebsd.org Subject: Script Challenge Message-ID: <20030117054458.GA50247@blossom.cjclark.org> Reply-To: "Crist J. Clark" Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i X-URL: http://people.freebsd.org/~cjc/ Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org I ran into a little problem today and needed to write a quick script. The problem isn't to tough, and I got it done with a 15 line-or-so Perl script, but there must be a more elegant way to do this. Here's the basic problem: I have a flat file that contains a database. Items in the database can contain sub-items which in turn contain sub-items with no hard limit to the recursion. The format of the file is, :b_item (value) :a_item ( :z_subitem(value) :x_subitem(value) :y_subitem( :r_subsubitem(value) :t_subsubitem(vaule) :s_subsubitem(value) ) :w_subitem(value) ) :c_item(value) That is, "values" whether individual items or lists, are surrounded by parentheses. (You need not worry about parentheses in "values" or the closing parentheses of a list not showing up on its own line, it always does.) I want to sort each level. That is, the above would become, :a_item ( :w_subitem(value) :x_subitem(value) :y_subitem( :r_subsubitem(value) :s_subsubitem(value) :t_subsubitem(vaule) ) :z_subitem(value) ) :b_item (value) :c_item(value) Like I said, I used a Perl script with a recursive function to do it, but it ain't all that pretty. I suspect I'm missing a really cool way to do it. Any suggestions? (Any language will do.) -- Crist J. Clark | cjclark@alum.mit.edu | cjclark@jhu.edu http://people.freebsd.org/~cjc/ | cjc@freebsd.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message From owner-freebsd-chat Fri Jan 17 1:40:15 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4AE9F37B401; Fri, 17 Jan 2003 01:40:13 -0800 (PST) Received: from dreadnought.cnchost.com (dreadnought.cnchost.com [207.155.248.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id E80DC43EB2; Fri, 17 Jan 2003 01:40:12 -0800 (PST) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (adsl-209-204-185-216.sonic.net [209.204.185.216]) by dreadnought.cnchost.com id EAA14712; Fri, 17 Jan 2003 04:40:12 -0500 (EST) [ConcentricHost SMTP Relay 1.15] Message-ID: <200301170940.EAA14712@dreadnought.cnchost.com> To: "Crist J. Clark" Cc: freebsd-chat@FreeBSD.ORG Subject: Re: Script Challenge In-reply-to: Your message of "Thu, 16 Jan 2003 21:44:59 PST." <20030117054458.GA50247@blossom.cjclark.org> Date: Fri, 17 Jan 2003 01:40:11 -0800 From: Bakul Shah Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org > Items in the database can contain sub-items which in turn contain > sub-items with no hard limit to the recursion. ... > I want to sort each level. ... > Like I said, I used a Perl script with a recursive function to do > it, but it ain't all that pretty. I suspect I'm missing a really cool > way to do it. Any suggestions? (Any language will do.) Attached below is a small scheme script (using Aubrey Jaffer's scm and slib -- see /usr/ports/lang/scm). When it is fed your sample input :b_item (value) :a_item ( :z_subitem(value) :x_subitem(value) :y_subitem( :r_subsubitem(value) :t_subsubitem(vaule) :s_subsubitem(value) ) :w_subitem(value) ) :c_item(value) It produces ((:a_item (:w_subitem value) (:x_subitem value) (:y_subitem (:r_subsubitem value) (:s_subsubitem value) (:t_subsubitem vaule)) (:z_subitem value)) (:b_item value) (:c_item value)) As you can see the output is formatted as one big list. Such "lispy" formats are trivial to read in and easy to process. But if you don't like that, you can write a simple converter back to your format or convert this script to linenoise perl. Note: the input can have any amount of whitespace and there is virtually no limit on nesting. But the script does assume Scheme input syntax. If not, you have to replace (read) with your own. It also assumes all lists have either 1 item or an even number of items (i.e. no errors). -- bakul cat > sortdb.scm <string (car a)) (symbol->string (car b)))) (define (sort-value l) (if (or (null? l) (null? (cdr l))) l (sort-list l '()))) (define (sort-list l result) (if (null? l) (sort result less?) (sort-list (cddr l) (cons (cons (car l) (sort-value (cadr l))) result)))) (define (read-db p) (let loop ((result '()) (item (read p))) (if (eof-object? item) (sort-value (reverse result)) (loop (cons item result) (read p))))) (pretty-print (read-db (current-input-port))) EOF chmod +x sortdb.scm sortdb.scm < tst-db > sorted-tst-db To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message