Date: Tue, 22 Jul 2003 12:33:18 +0300 (EEST) From: Evren Yurtesen <yurtesen@finland.ispro.net.tr> To: Peter McGarvey <fbsd-x@packet.org.uk> Cc: xaphod@techie.com Subject: managing dns and emails Message-ID: <20030722120555.H84851@finland.ispro.net.tr> In-Reply-To: <20030722085237.GB28071@packet.org.uk> References: <20030721165525.L21521@finland.ispro.net.tr> <20030721193402.GA9925@greatmachine.diogenes.intranet> <20030722085237.GB28071@packet.org.uk>
next in thread | previous in thread | raw e-mail | index | archive | help
A few days ago I was writing a script for passing all user data from tacacs to freeradius. It worked fine! :) Now I am also under quite stress trying to figure out how to migrate from sendmail to qmail+vpopmail. :) I have prepared the mail server qmail-tls, smtp authentication,pop3,imap,pop3-ssl,imap-ssl (with courier-imap), vqadmin,qmailadmin,amavis :) and even make documentation of how to install everything :) Now the problem is that I have many domains in virtusertable file and there was no script that I could find which could convert that and insert into vpopmail which use mysql. It seemed easy to write at first but then I realized some users in virtusertable are aliased in alias file and some are even aliased to themselves etc. :) So it became quite a challenge to figure out for which user I should create an email and for which I should make alias and for which I should make forward file. Then detect the rest of the users and insert them to a default domain. I made a script which seems to work anyhow, it is 17kbyte though :) and it seems to work but it will be clear on saturday if it is really working. About the dns, if you pass to a normal BIND then you should create 35000 zone files? How will you manage that? About that script you mentioned is it something like this? http://www.inter7.com/dnsadmin.html It seems that it only misses the customer interface. I am of course interested in keeping things tidy and clean. In time the things go out of synchronization. For example I know that I have many domain files left over because somebody deleted the domain zone information from named.conf but forgot to delete the file. Or somebody forgot to remove the domain from the web server etc. At least, now I can delete emails of users who doesnt touch their inboxes for 6 month, automatically :) The rest is to come. I would be interested to evaluate your scripts too and make changes or additions as needed. It is a lot easier than starting from scratch. If you can make them available then I would download of course. By the way, have you ever thought of starting a project in sourceforge or something? That way we can have many people contributing to this tidyness project :) Lets keep in touch then, if you have an ICQ number, mine is 913003 Evren On Tue, 22 Jul 2003, Peter McGarvey wrote: > * Evren Yurtesen <yurtesen@ispro.net.tr> [2003-07-22 07:07:27 BST]: > > Excellent script! I have few questions and perhaps good news :) > > > > I have already modified your script to read named.conf file and parse it > > quite properly! :) Meaning nicely skipping commends and finding zones and > > making an array of domain names. > > > > Then of course checking against the array was easy. I now continue to add > > more functionality :) > > Yeah, well the script was a modification of my original hack which > generated lists to tell me which of the 20 nameservers I'm responsible > for are authoritative for the domain. Allowing me to redelegate to the > standard platform in a somewhat ordered manner. > > The original script also checks the MNAME of the parent zone to allow me > to detect (and therefore ignore) sub-domains of zones we also hold. > > Writing something to keep a single DNS platform neat and tidy is on my > list of things to do... but it's way down my list of priorities. So > I'd definately be interested in any modifications you make to acheive > such a thing. > > > > > Now my question is something funny perhaps. Normally I store the domain > > files at /etc/namedb if you have 350,000 domains, then wouldnt it mean > > you should have files of those domains somewhere? :) What kind of > > structure do you use? > > Ooops, my bad, 35,000 domains. I'd spent the day trying to work out how > to migrate 350,000 mail accounts onto a volume half the size of the > current one. I don't think my brain had quite recovered. :-( > > But anyway, we store the zonedata in an Oracle DB, which exports to a > MySQL DB, with a cutomised BIND pulling all this info out of the DB > directly. A (non-technical) domains team of 4 do most of the day-to-day > changes, transfers, etc, etc. wia a webby front-end. All the backend > admin is done by me. > > Sounds rather kewl, but it's broke like you wouldn't believe. I mean, > just to get the damn thing to detect deleted RRs requires a stop-start > of the daemon - nasty. > > As soon as my boss gives me the go ahead, I'll be replacing the > customised BIND with a standard one. I've got a perl script (a nice > one, not a hack) that pulls the zonedata out of the MySQL DB, generates > zonefiles (if required) and a named.conf (if required), and does an > "rndc reload", a "rndc reload {zone}", or nothing depending on the > requirements. Run from cron - and I'll never need to restart the daemon > ever again.... :-) > > I've got a mate who works for another ISP who's used this script (and > the associated schema) as the basis for his new DNS platform (which > allows customers to maintian their own domains). If you feel this may > be useful then let me know and I'll write some doc etc. and make it > available for download. > > > > > Evren > > > > On Mon, 21 Jul 2003, Peter McGarvey wrote: > > > > > * Evren Yurtesen <yurtesen@ispro.net.tr> [2003-07-21 14:57:35 BST]: > > > > I have about 1000 domain names in my named.conf file. > > > > I wonder if it would be possible to check the validity of these domains > > > > or even if they are pointing to my name server or not easily... > > > > Is there a ready script or something somebody knows here? > > > > > > > > > > Well, I'm trying to combine the DNS platforms of 4 different companies > > > onto a single standard platform. I've got over 350,000 domains.... all > > > of which need redelegating. And just to make life fun these 4 different > > > platforms have all been left to rot for years. > > > > > > None of the tools I've found have provided me with any of the > > > information I need to speed up the process. So I've resorted to using a > > > variety of tools, and writing my own. > > > > > > For checking the actual zonefiles, I tend to use named-checkzone which > > > is part of the bind distribution - but not part of the BSD install > > > unfortunately. > > > > > > As for my scripts... well they are rather nasty :-( > > > > > > But I've attached a re-hacked hack of a hack script of mine that may be > > > useful. > > > > > > cat domainlist | perl auths.pl > > > > > > > > > > > > -- > > > TTFN, FNORD > > > > > > Peter McGarvey > > > Freelance FreeBSD Hacker > > > (will work for bandwidth) > > > > > #!/usr/bin/perl > > > > # > > # > > # Creation Date: 21st July 2003 > > # > > # Author: > > # Peter P. McGarvey <xaphod@techie.com> > > # > > # Additional Code: > > # Evren Yurtesen <yurtesen@ispro.net.tr> > > # > > # VERSION HISTORY > > # > > # format x.y.z > > # - x = Major version number > > # - y = Minior version number > > # - z = Burst number > > # I work in bursts. Weekend here, couple of hours a night > > # there, then a night (a week etc.) off to think. The > > # `burst number' helps me keep track of different versions > > # methinks CVS would be a good idea one day. > > # > > # 0.0.1 2003-07-21 - Peter P. McGarvey <xaphod@techie.com> > > # * A NASTY hack of a perl script. > > # 0.1.1 2003-07-22 - Evren Yurtesen <yurtesen@ispro.net.tr> > > # * Added named.conf file parsing > > # * Added automatical nameserver checking > > # > > use vars qw($VERSION); > > $VERSION = '0.1.1'; > > # > > # ============================================================================= > > > > #location of named.conf file > > $named_conf="named.conf"; > > > > #the name servers you expect to see > > @nameservers=('ns1.ispro.net','ns2.ispro.net','dns1.ispro.net.tr','dns2.ispro.net.tr'); > > > > #debugging level (the higher, the more text!) > > $debug=20; > > > > # ============================================================================= > > # Starting the Code > > > > > > use Net::DNS; > > > > > > my $timeout=0; > > > > my @domainlist = &get_domain_list; > > > > foreach my $zone (&get_domain_list) { > > #while(my $zone = <domainlist>) { > > print "# zone: $zone\n"; > > my ($parent_zone) = $zone =~ /^[^\.]*\.(.*)/; > > print "# parent of $zone is $parent_zone\n"; > > my $parent_auth_primary = &get_auth_primary($parent_zone, $timeout); > > if ($parent_auth_primary == -1) { > > print "# DEAD ZONE!\n# ABORT\n\n"; > > print DEAD "# $zone\n"; > > next; > > } > > print "# parent auth primary is: $parent_auth_primary\n"; > > my @auth_servers = &get_ns_list($zone, $parent_auth_primary, $timeout); > > if (scalar(@auth_servers) == 0) { > > print "# No auth servers, DEAD ZONE!\n"; > > print DEAD "$zone\n"; > > next; > > } > > foreach my $ns (@auth_servers) { > > $ns = lc($ns); > > print $ns, "\n"; > > } > > } > > > > # ============================================================================= > > # End of the Code - Start of the Sub-Functions > > # &get_domain_list - Returns the list of domains from named.conf.file > > # &get_auth_primary - Takes the name of a zone > > # Return the MNAME field from the SOA > > # &get_ns_list - Returns the list of NS records reported by that server > > > > # ============================================================================= > > # Returns the list of domains from named.conf file > > sub get_domain_list { > > open(NAMED_CONF,"< $named_conf") or die "Can not open $named_conf"; > > #set some variables to 0 > > $comment_block_starting=0; > > $comment_block_started=0; > > #make a list of domains > > while($line=<NAMED_CONF>) { > > #find if there is double backslash and use only left of it > > #we must put newline > > if($line =~ /\/\//) { > > $line=(split /\/\//,$line)[0]."\n"; > > } > > #find if there is both /* my text */ in $line and use only outside > > if ($line =~ /\/\*/ && $line =~ /\*\//) { > > @values1=split(/\/\*/,$line); > > @values2=split(/\*\//,$line); > > $line=$values1[0].$values2[1]; > > } else { #find if there is only one of them? > > #if there is only /* take left side and wait until finding */ > > if($line =~ /\/\*/) { > > $line=(split /\/\*/,$line)[0]; > > #count the occurances in case if they are nested > > $comment_block_starting+=1; > > } > > #if there is only */ then stop waiting and take right side > > if($line =~ /\*\//) { > > $line=(split /\*\//,$line)[1]; > > if(($comment_block_starting-=1) eq 0) { > > $comment_block_started=0; > > } > > } > > } > > if (($comment_block_started=$comment_block_starting) eq 0) { > > if($line =~ /zone.+\".+\"/ && ! ($line =~ /zone.+\"."/ || $line =~ /zone.+\".*ARPA\"/i)) { > > $line=(split /\"/,$line)[1]; > > #cut out left and right spaces just to be sure > > $line =~ s/^\s+|\s+$//g; > > push(@domainlist,$line); > > } > > } > > } > > close(NAMED_CONF); > > return sort @domainlist; > > } > > > > # ============================================================================= > > # Takes the name of a zone > > # Return the MNAME field from the SOA > > sub get_auth_primary { > > my $zone = shift; > > my $tcp_timeout = shift; > > > > my $res = new Net::DNS::Resolver; > > my $packet = new Net::DNS::Packet($zone, "SOA", "IN"); > > > > $res->tcp_timeout($tcp_timeout) if $tcp_timeout; > > > > my $req = $res->send($packet); > > > > return -1 > > unless defined($req); > > return -1 > > unless ( ($req->header->ancount >= 1) > > && (($req->answer)[0]->type eq "SOA") ); > > return ($req->answer)[0]->mname . "."; > > # Return the FQDN > > } > > > > # ============================================================================= > > # Returns the list of NS records reported by that server > > sub get_ns_list { > > my $zone = shift; > > my $server = shift; > > my $tcp_timeout = shift; > > > > my @ns_list; > > > > my $res = new Net::DNS::Resolver; > > > > $res->tcp_timeout($tcp_timeout) if $tcp_timeout; > > > > #die "FAIL: NS has no address (" . $res->errorstring . ")" > > return @ns_list > > unless $res->nameservers($server); > > > > my $req = $res->query($zone, "NS"); > > > > #die "WARN: NS query results were bad (" . $res->errorstring . ")" > > return @ns_list > > unless ( defined($req) > > && ($req->header->ancount > 0) ); > > > > foreach my $rr_ns ($req->answer) { > > push @ns_list, $rr_ns->nsdname . "."; #FQDN! > > } > > return sort @ns_list; > > } > > > _______________________________________________ > > freebsd-isp@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-isp > > To unsubscribe, send any mail to "freebsd-isp-unsubscribe@freebsd.org" > > > -- > TTFN, FNORD > > Peter McGarvey > Freelance FreeBSD Hacker > (will work for bandwidth) >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030722120555.H84851>