From owner-freebsd-isp@FreeBSD.ORG Thu May 22 19:24:39 2003 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2F77137B401 for ; Thu, 22 May 2003 19:24:39 -0700 (PDT) Received: from mercury.jorsm.com (mercury.jorsm.com [207.112.128.9]) by mx1.FreeBSD.org (Postfix) with ESMTP id 711E643F93 for ; Thu, 22 May 2003 19:24:38 -0700 (PDT) (envelope-from jeff@mercury.jorsm.com) Received: from 127.0.0.1 (localhost [127.0.0.1]) by localhost (Postfix) with SMTP id DED4F1E1796; Thu, 22 May 2003 21:24:37 -0500 (CDT) Received: by mercury.jorsm.com (Postfix, from userid 101) id 9BA831E1A61; Thu, 22 May 2003 21:24:31 -0500 (CDT) Received: from localhost (localhost [127.0.0.1]) by mercury.jorsm.com (Postfix) with ESMTP id 98D111E1A5B; Thu, 22 May 2003 21:24:31 -0500 (CDT) Date: Thu, 22 May 2003 21:24:31 -0500 (CDT) From: Jeff Lynch To: Etienne Ledoux In-Reply-To: <200305222205.42277.etienne@unix.za.org> Message-ID: <20030522204601.T87632-100000@mercury.jorsm.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-isp@freebsd.org Subject: Re: migrating users from an exchange server to a unix mailserver. X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 May 2003 02:24:39 -0000 On Thu, 22 May 2003, Etienne Ledoux wrote: > Greetings, > > I'm busy with project which involves merging different mail servers with > differnet setups *transparently* into one "beeg" mail enviroment. One of > the problems I am currently facing is merging a MS exchange server with > thousands of users authenticating using domain\username as their username. We did something similar, maybe I can help with part of it... > > So far I'm considering using postfix + mysql + courier-imap + vmailadmin. > I've based my setup on the following guides. More complex than ours, not sure about mailbox storage format. Is it /var/mail/username (mbox format?) > > - http://www.probsd.net/vmail/ > - http://www.marlow.dk/postfix/ > > But it gets a little tricky with the merge of the exchange server. This is where we ran into trouble (exchange that is) > > - Is it possible to authenticate exchange type usernames ( domain\username ) > in a unix mail enviroment. If so what should I use etc. Not clear on domail\username, we ignored any domain issues, maybe our situation wasn't as complex, but we just needed username and password. Most of the pws were un-available to us so we used Loftcrack to get the usernames and pws in plaintext. > - I would like to do this and still keep the user info in the mysql db, this > will eventually be linked to the rest of the customer db with billing info > etc. which is also kept in a mysql db. We didn't do this, but you can export the exchange user information (less passwd) in csv format. Maybe you could use that and cut the fields you don't need and import using phpMyAdmin? > > Any hints ? > Has anybody else done something similar, maybe my setup is wrong for this > kind > of thing and I should be looking at something else ? > > Any help will be appreciated. > > I've tried adding an account with domain\username into the mysql db but it > doesn't seem to be able to authenticate with courier-pop3/imap We were able to auth the exchange user accounts with username and password so I hacked a perl script to step through a text file with username:plaintext-password and used POP3 to grab the mailbox from the exchange server. There were some issues with exchange format so I cleaned it by running through formail (from the procmail port) and appended mbox format data for each user to be moved. Here is the script, names changed to protect the innocent. It was not designed for production environment or general distribution. If you need a different inbox format than mbox, this could get the mailboxes first and then you can convert (such as mbox2maildir conversions out there) from there. It's been a long time since we did this so I am a little fuzzy on the other details. I'm pretty sure we did things in this order: 1) created new accounts on the bigass mailserver for users to be moved 2) moved the IP address from exchange-server and aliased on bigass server (avoid DNS ttl issues for MX change) 3) allowed bigass mailserver to accept new mail for accounts in progress of being moved (UIDL support handles messages already read or not) 4) ran the script on bigass server which appends the mbox data to the newly created mail account "inbox" (/var/mail/username) 6) shut down the exchange server (woohoo!), be careful, no file locking is done by this script, but was negligible risk for us. If you use this script, you might want to change the paths to write temporary files and check things out before going live with this. ----cut here------ #!/usr/bin/perl # # use at your own risk, I disclaim all warranties of merchantability # of fitness for any particular purpose # --jeff lynch (jorsm.net) # use Mail::POP3Client; # Default pop3 host to retrive accounts from $host = "exchange-server.domain.com"; # Input file containing list of usernames and passwords one each line # of the form: # username:password $pwfile = "pop3logins.txt"; # mbox mail spool path $mail = "/var/mail"; # We use formail to make sure we write in mbox format (Exchange 5.5 does # not include a leading "From ..." line. $formail = "/usr/local/bin/formail -ds"; # Debugging $debug = 1; #$debug = 0; open(LOG, ">pop3movembox.log") or die "Can't create logfile\n"; print LOG "pop3movembox.pl started `date`\n"; open(PWF, "<$pwfile") or die "Can't open password list file $pwfile\n"; while () { chomp; ($user,$pass) = split(":",$_); # Check if we've already got a mailbox for this user @uidls = (); if ( open(MBOX,"<$mail/$user") ) { # yes, create a list of the message unique id lables print LOG "$user - mbox exists, checking UIDLS\n" if $debug; while () { chomp; $label = $1 and push(@uidls, $label) if /^X-UIDL:\s+(\w+)/; } print LOG "$user - has these UIDLS: " if $debug; foreach $j (@uidls) { print LOG $j, ", " if $debug; } print LOG "\n" if $debug; } else { print LOG "$user - mbox not found, skip local UIDL check\n" if $debug; } close(MBOX); # initiate pop3 login $pop = new Mail::POP3Client( USER => $user, PASSWORD => $pass, HOST => $host ); # if login yes if( $pop->State =~ "TRANSACTION" ) { # open mbox append mode open(MBOX, "| $formail >> $mail/$user") or die "formail died on $user\n"; # Step through messages for ($i = 1; $i <= $pop->Count; $i++) { # Ask the POP server for the message X-UIDL: header field foreach ($pop->Uidl($i)) { $ruidl = $2 if /^(\w+)\s+(\w+)/; } print LOG "$user - get msg $i UIDL: $ruidl?\n" if $debug; # Determine if current message UIDL is in the list of known UIDLs from # mbox on this server $missing = "yes"; foreach $id (@uidls) { if( $id =~ $ruidl ) { $missing = "no"; } } # If message on remote server is not in local mbox, download it if( $missing =~ /yes/ ) { # print LOG "$user - downloading msg $i, UIDL: $ruidl\n" if $debug; # Print message header foreach ($pop->Head($i)) { print MBOX $_, "\n"; } # Add the UIDL field to message header print MBOX "X-UIDL: $ruidl\n\n"; # Print the message body foreach ($pop->Body($i)) { print MBOX $_, "\n"; } # Blank line between messages print MBOX "\n"; } else { print LOG "$user - has msg $i UIDL: $ruidl, skipping\n" if $debug; } } $pop->Close(); close(MBOX); } else { # Problem logging in to remote POP server print LOG "Login failed for: $user\n"; } } close(LOG); close(PWF); ============================================================================ Jeffrey A. Lynch | JORSM Internet, Regional Internet Services email: jeff@jorsm.com | 7 Area Codes in Chicagoland and NW Indiana Voice: (219)322-2180 | 100Mbps+ Connectivity, 56K-DS3, V.90, ISDN Autoresponse: info@jorsm.com | Quality Service, Affordable Prices http://www.jorsm.com | Serving Gov, Biz, Residential Since 1995