FreeBSD Mail Archives

Date:      Thu, 22 May 2003 21:24:31 -0500 (CDT)
From:      Jeff Lynch <jeff@mercury.jorsm.com>
To:        Etienne Ledoux <etienne@unix.za.org>
Cc:        freebsd-isp@freebsd.org
Subject:   Re: migrating users from an exchange server to a unix mailserver.
Message-ID:  <20030522204601.T87632-100000@mercury.jorsm.com>
In-Reply-To: <200305222205.42277.etienne@unix.za.org>

On Thu, 22 May 2003, Etienne Ledoux wrote:

> Greetings,
>
> I'm busy with project which involves merging different mail servers with
> differnet setups *transparently* into one "beeg" mail enviroment. One of
> the problems I am currently facing is merging a MS exchange server with
> thousands of users authenticating using domain\username as their username.

We did something similar, maybe I can help with part of it...

>
> So far I'm considering using postfix + mysql + courier-imap + vmailadmin.
> I've based my setup on the following guides.

More complex than ours, not sure about mailbox storage format. Is it
/var/mail/username (mbox format?)

>
> - http://www.probsd.net/vmail/
> - http://www.marlow.dk/postfix/
>
> But it gets a little tricky with the merge of the exchange server.

This is where we ran into trouble (exchange that is)

>
> - Is it possible to authenticate exchange type usernames ( domain\username )
> in a unix mail enviroment. If so what should I use etc.

Not clear on domail\username, we ignored any domain issues, maybe our
situation wasn't as complex, but we just needed username and password.
Most of the pws were un-available to us so we used Loftcrack to
get the usernames and pws in plaintext.

> - I would like to do this and still keep the user info in the mysql db, this
> will eventually be linked to the rest of the customer db with billing info
> etc. which is also kept in a mysql db.

We didn't do this, but you can export the exchange user information (less
passwd) in csv format. Maybe you could use that and cut the fields you
don't need and import using phpMyAdmin?

>
> Any hints ?
> Has anybody else done something similar, maybe my setup is wrong for this
>  kind
> of thing and I should be looking at something else ?
>
> Any help will be appreciated.
>
> I've tried adding an account with domain\username into the mysql db but it
> doesn't seem to be able to authenticate with courier-pop3/imap

We were able to auth the exchange user accounts with username and password
so I hacked a perl script to step through a text file with
username:plaintext-password and used POP3 to grab the mailbox from the
exchange server. There were some issues
with exchange format so I cleaned it by running through formail (from
the procmail port) and appended mbox format data for each user to be moved.

Here is the script, names changed to protect the innocent. It was
not designed for production environment or general distribution. If
you need a different inbox format than mbox, this could get the
mailboxes first and then you can convert (such as mbox2maildir
conversions out there) from there. It's  been a long time since
we did this so I am a little fuzzy on the other details.

I'm pretty sure we did things in this order:
1) created new accounts on the bigass mailserver for users to be moved
2) moved the IP address from exchange-server and aliased on bigass server
(avoid DNS ttl issues for MX
change)
3) allowed bigass mailserver to accept new mail for accounts in
progress of being moved (UIDL support handles messages already read or
not)
4) ran the script on bigass server which appends the mbox data to the
newly created mail account "inbox" (/var/mail/username) 6) shut down the
exchange server (woohoo!), be careful, no file locking is done by this
script, but was negligible risk for us.

If you use this script, you might want to change the paths to
write temporary files and check things out before going live with
this.

----cut here------
#!/usr/bin/perl
#
# use at your own risk, I disclaim all warranties of merchantability
# of fitness for any particular purpose
# --jeff lynch (jorsm.net)
#

use Mail::POP3Client;

# Default pop3 host to retrive accounts from
$host = "exchange-server.domain.com";

# Input file containing list of usernames and passwords one each line
# of the form:
# username:password
$pwfile = "pop3logins.txt";

# mbox mail spool path
$mail = "/var/mail";

# We use formail to make sure we write in mbox format (Exchange 5.5 does
# not include a leading "From ..." line.
$formail = "/usr/local/bin/formail -ds";

# Debugging
$debug = 1;
#$debug = 0;

open(LOG, ">pop3movembox.log") or die "Can't create logfile\n";
print LOG "pop3movembox.pl started `date`\n";
open(PWF, "<$pwfile") or die "Can't open password list file $pwfile\n";
while (<PWF>) {
  chomp;
  ($user,$pass) = split(":",$_);

  # Check if we've already got a mailbox for this user
  @uidls = ();
  if ( open(MBOX,"<$mail/$user") ) {
	# yes, create a list of the message unique id lables
	print LOG "$user - mbox exists, checking UIDLS\n" if $debug;

	while (<MBOX>) {
	  chomp;
	  $label = $1 and push(@uidls, $label) if /^X-UIDL:\s+(\w+)/;
	}
	print LOG "$user - has these UIDLS: " if $debug;
	foreach $j (@uidls) {
	  print LOG $j, ", " if $debug;
	}
	print LOG "\n" if $debug;
  } else {
	print LOG "$user - mbox not found, skip local UIDL check\n" if $debug;
  }
  close(MBOX);

  # initiate pop3 login
  $pop = new Mail::POP3Client( USER     => $user,
                               PASSWORD => $pass,
                               HOST     => $host );

  # if login yes
  if( $pop->State =~ "TRANSACTION" ) {
    # open mbox append mode
    open(MBOX, "| $formail >> $mail/$user") or die "formail died on $user\n";

    # Step through messages
    for ($i = 1; $i <= $pop->Count; $i++) {
      # Ask the POP server for the message X-UIDL: header field
      foreach ($pop->Uidl($i)) {
	$ruidl = $2 if /^(\w+)\s+(\w+)/;
      }
      print LOG "$user - get msg $i UIDL: $ruidl?\n" if $debug;

      # Determine if current message UIDL is in the list of known UIDLs from
      # mbox on this server
      $missing = "yes";
      foreach $id (@uidls) {
	if( $id =~ $ruidl ) {
	  $missing = "no";
	}
      }
      # If message on remote server is not in local mbox, download it
      if( $missing =~ /yes/ ) {
	#
	print LOG "$user - downloading msg $i, UIDL: $ruidl\n" if $debug;
        # Print message header
        foreach ($pop->Head($i)) {
          print MBOX $_, "\n";
        }

        # Add the UIDL field to message header
	print MBOX "X-UIDL: $ruidl\n\n";

        # Print the message body
	foreach ($pop->Body($i)) {
	  print MBOX $_, "\n";
	}

	# Blank line between messages
	print MBOX "\n";

      } else {
	print LOG "$user - has msg $i UIDL: $ruidl, skipping\n" if $debug;
      }
    }
    $pop->Close();
    close(MBOX);

  } else {
    # Problem logging in to remote POP server
    print LOG "Login failed for: $user\n";
  }

}

close(LOG);
close(PWF);

============================================================================
Jeffrey A. Lynch		| JORSM Internet, Regional Internet Services
email: jeff@jorsm.com		| 7 Area Codes in Chicagoland and NW Indiana
Voice: (219)322-2180		| 100Mbps+ Connectivity, 56K-DS3, V.90, ISDN
Autoresponse: info@jorsm.com	| Quality Service, Affordable Prices
http://www.jorsm.com		| Serving Gov, Biz, Residential Since 1995

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030522204601.T87632-100000>

Header And Logo

Peripheral Links

Site Navigation

Header And Logo

Peripheral Links

Search

Site Navigation