Date: Fri, 5 Apr 2002 13:18:42 -0700 From: "Samuel Chow" <cyschow@shaw.ca> To: <freebsd-doc@freebsd.org> Subject: Some email concepts Message-ID: <001201c1dcdf$14d97d60$2784412f@ca.nortel.com>
next in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format.
------=_NextPart_000_000F_01C1DCA4.685913A0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: 7bit
Hi there,
Attached is something I put together quickly explaining
how email can be setup in some very general terms.
Believe me, it was a struggle just to understand enough
to make sense out of the documentation of various email
components. Once the general concept is understood,
it generally is not too difficult actually perform the
configuration.
Feedback is most welcomed. Thanks.
---
Samuel Chow
cyschow@shaw.ca
This message is displayed using recycled electrons.
Segmentation Fault (core dumped)
------=_NextPart_000_000F_01C1DCA4.685913A0
Content-Type: text/plain;
name="Email.txt"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
filename="Email.txt"
=0A=
=0A=
How does email work?=0A=
--------------------=0A=
=0A=
Let say Alice is about to send an email to her friend Bob. She starts =0A=
up her favorite email application and starts to compose the message. =0A=
When she is done composing, she hits the send button, and the email is =0A=
delivered to Bob's mailbox. Bob reads the email and replies back, and =0A=
the email shows up in Alice's favorite email application. It is simple, =0A=
isn't it?=0A=
=0A=
Or is it? Let's examine how email works in details.=0A=
=0A=
The email application Alice is using is known as the Mail User =0A=
Agent (MUA). The main purpose of the MUA is to allow a user to compose =0A=
the email, and to pass the completed email on to the Mail Transport =0A=
Agent (MTA). The MTA is also known as the SMTP server or outgoing =0A=
mail server, and it can be running on the local machine, or on =0A=
some machines over the network.=0A=
=0A=
The following is a partial list of the more popular MUAs:=0A=
- Microsoft Outlook and Outlook Express=0A=
- Pegasus Mail=0A=
- Eudora=0A=
- pine=0A=
- mutt=0A=
=0A=
The MTA is responsible to figure out how to deliver the email coming =0A=
from the MUA. First, the MTA determines whether the email can be=0A=
delivered locally or not. If so, the MTA goes through the local=0A=
delivery procedure, and considers its job done.=0A=
=0A=
Let say the email is addressed to bob@someisp.com, and that someisp.com =0A=
is not the local domain name. The local MTA first asks the Domain Name =0A=
Service (DNS) which IP addresses are responsible for handling incoming =0A=
email for the someisp.com domain (the MX records). Next, the local MTA =0A=
initiates a TCP connection to the remote MTA running on one of the IP =0A=
addresses returned by DNS. Assuming the connection is made =
successfully, =0A=
the local MTA dumps the email message onto the remote MTA, and figures =0A=
its job is completed. Exactly how the email is delivered on the remote =0A=
end is not of any concern to the local MTA.=0A=
=0A=
The following is a partial list of the more popular MTAs:=0A=
- sendmail=0A=
- qmail=0A=
=0A=
Let's turn our attention to Bob's reply email to Alice. The local MTA=0A=
accepts a TCP connection from the Internet, and an email is now =
requiring =0A=
delivery. The local MTA inspects the To: address and decides that Alice =0A=
is local to its domain. It starts the local delivery procedure. =0A=
=0A=
The local MTA first checks to see if Alice has any local delivery =0A=
instructions. These delivery instructions are stored in files =0A=
specific to the local MTA. Examples of the local delivery instruction=0A=
files are .forward for sendmail, and .qmail for qmail. If no =0A=
delivery instruction are found, the local MTA writes to Alice's=0A=
mailbox and finish its task; otherwise, it follows the instruction,=0A=
and possibly invokes some email filtering applications, such as=0A=
procmail and maildrop.=0A=
=0A=
Alice's favorite email application checks her mailbox periodically,=0A=
and notices a new email has arrived. It makes that familiar "You've=0A=
Got Mail" sound and let Alice reads Bob's reply.=0A=
=0A=
If you now think email is really complicated, well, it is. Fortunately,=0A=
your ISP has it all sorted out. Depending on what you choose to =
implement,=0A=
on your FreeBSD system, you may be able to utilize some of the =0A=
functionalities of your ISP's mail system.=0A=
=0A=
=0A=
Protocols Related to Email=0A=
--------------------------=0A=
=0A=
There are a number of important protocols used in the delivery of email. =
=0A=
They are SMTP, POP3, and IMAP.=0A=
=0A=
SMTP (Simple Mail Transfer Protocol) is used mainly between the MTAs. It=0A=
is also used by the MUAs to pass composed email to MTAs for delivery. In=0A=
other words, this protocol is used for outgoing mail.=0A=
=0A=
POP3 (Post Office Protocol Version 3) is used by MUAs to retrieve =
incoming=0A=
emails. Typically, emails are downloaded and then deleted from the =
server.=0A=
=0A=
IMAP (Internet Message Access Protocol) is used by MUAs to access email =0A=
on a server. Emails are stored on a server, and they are not never =0A=
downloaded (in theory).=0A=
=0A=
POP3 and IMAP are both for incoming emails, but they are designed for=0A=
different uses. POP3 are excellent for downloading all messages for=0A=
offline consumption on a single computer. IMAP, on the other hand, =0A=
is for accessing the same email storage using different computers=0A=
and/or operation systems.=0A=
=0A=
=0A=
Mail User Agents=0A=
----------------=0A=
=0A=
MUAs are the email clients. Most modern MUAs supports the POP3 protocol,=0A=
while only a few supports the IMAP protocol. Furthermore, IMAP clients=0A=
do things a little differently, and are not totally compatible with=0A=
each others.=0A=
=0A=
Personally, I tried Mahogany on FreeBSD and didn't like it. Then,=0A=
I installed slypheed, and felt in love with it the minute I start using =0A=
it. When I am at work being forced to use Microsoft Windows, I use=0A=
Outlook Express to access IMAP server at home. Other people swear by=0A=
mutt and pine. The bottom line is don't take my words for it, try=0A=
it all and pick the one you like most.=0A=
=0A=
=0A=
Simplest Case=0A=
-------------=0A=
=0A=
The simplest case of configuring email is to use the SMTP server of your =0A=
ISP for outgoing mail, and the POP3 server of your ISP for incoming mail.=0A=
Simply configure your MUAs to use the mail server given to you by=0A=
your ISP. This setting avoid any email configuration on your FreeBSD=0A=
machine.=0A=
=0A=
=0A=
Local MTA=0A=
---------=0A=
=0A=
Let assume we have a number of users on the same network,=0A=
and they wish to send each other emails. While technically it is=0A=
possible to use the SMTP server and POP3 server of the ISP, it sounds=0A=
really funny. It is also insecure -- sensitive emails are sent out =0A=
onto the Internet for the world to see before it is delivered to =0A=
the intended recipient.=0A=
=0A=
The answer is to setup a MTA running within the local protected network.=0A=
Referring to the first section, we are providing additional SMTP server =0A=
and the POP3 server with our own software.=0A=
=0A=
In order to configure a MTA, a domain name is required. This domain=0A=
name can be the fully qualified domain name (FQDN) of the domain. This =0A=
allow remote MTA to deliver email directly once the DNS MX record=0A=
is configured properly. More on DNS MX record later.=0A=
=0A=
If your network does not have a FQDN, you can use any name, with=0A=
the understanding that the domain name is invalid and invisible outside =0A=
of your network. This means no remote MTA can ever be delivering email=0A=
directly to your domain. =0A=
=0A=
The next topic of discussion is relaying. Relaying means the local MTA=0A=
accepts email deliveries not destined for the local domain, and =0A=
attempts to deliver the emails to their destinations. Years ago,=0A=
MTAs were shipped as open relays, meaning the MTAs were accepting =0A=
emails from anybody. Then came the spammers. They deliver one email=0A=
message with a long list of recipients to the open relay, and the =0A=
victim MTA are using its resources to attempt to deliver the spam to=0A=
thousands and thousands of recipients.=0A=
=0A=
So, MTAs should not be relays, and should only accept delivery for=0A=
its local domain, right? Now, when Alice's MUA passes an email=0A=
with Bob's address bob@someisp.com, the MTA is going to reject on=0A=
the basis of not relaying any email. Umm...=0A=
=0A=
The correct answer is to configure the MTA to relay based on either=0A=
1) the From: address -- if it is from some trusted domain, accept=0A=
and relay the email; =0A=
2) the IP address -- if the SMTP connection is from trusted =0A=
hosts, accept and relay the email;=0A=
3) requiring login to the SMTP server.=0A=
=0A=
Keep in mind that option 1 is rather insecure. The From: address=0A=
of an email can easily be forged. =0A=
=0A=
=0A=
Local Delivery Instruction=0A=
--------------------------=0A=
=0A=
A number of things can be specified in the local delivery instruction,=0A=
including the forward email address, and program delivery. Forwarding=0A=
email is straight forward -- it, well, forwards the email to a =0A=
different address. Program delivery, on the other hand, is a powerful =0A=
feature and can be complicated.=0A=
=0A=
Program delivery means the email is passed onto a program to determine=0A=
what to do with that particular piece of email. The following are a =0A=
partial list of the more popular application for program deliveries:=0A=
- procmail=0A=
- mailfilter=0A=
=0A=
Why is program delivery such as powerful features? Here is a list=0A=
of thing that can do by program delivery:=0A=
- inspects the From: address. If the mail is from a luser,=0A=
redirect to /dev/null; if it is from my boss, reply=0A=
with a form letter saying I am in a meeting.=0A=
- inspects the Subject: of the email. If it resembles =0A=
"earn money quick", forwards to the authorities (not=0A=
recommended).=0A=
- inspects the Sender: header. If it matches FreeBSD.ORG,=0A=
file it into the FreeBSD folders.=0A=
- inspects the To: address. If it does not contain my address,=0A=
file it into the junk mail folder (be careful with this one).=0A=
=0A=
The list of things that can be done with program delivery =0A=
keeps growing.=0A=
=0A=
=0A=
Mailbox Format=0A=
--------------=0A=
=0A=
As mentioned in the first section, the local MTA delivers emails to =0A=
mailboxes for the MUAs. There are two major types of mailboxes, namely =0A=
the Unix local mailbox and Maildir. =0A=
=0A=
Unix local mailbox is the default. It consists of two parts. All =0A=
unexamined email are stored in the file /var/mail/userid, while all=0A=
examined email are stored in ~/mbox in the home directory. Since all =0A=
emails are in one file, proper locking is required to prevent mailbox =0A=
corruption. NFS mounting Unix local mailbox is not recommended, because =0A=
file locking over NFS is problematic.=0A=
=0A=
Maildir is used by qmail. In this scheme, each email is stored as a =0A=
separate file. There are also separate directories for new email and =0A=
email delivery-in-progress. No locking of any files are required. =
Hence, =0A=
it is safe to NFS mounting the Maildir. The downside is very few =
software =0A=
support this Maildir format. However, if qmail is chosen as the MTA, =0A=
Maildir must be used.=0A=
=0A=
=0A=
Accessing Email=0A=
---------------=0A=
=0A=
If local Unix mailbox is used, and all of the following conditions=0A=
are true, no additional protocol are required to access email:=0A=
a) client machines (if exists) NFS mounting the /var/mail directory, =0A=
and locking over NFS works; and=0A=
b) MUAs understand local Unix mailboxes.=0A=
=0A=
If Maildir is used, and all of the following conditions are true,=0A=
no additional protocols are required to access email:=0A=
a) client machines (if exists) NFS mounting the ~/Maildir directory.=0A=
b) MUAs understand Maildir format mailbox.=0A=
=0A=
In realities, the following practical reasons dictate addition protocols=0A=
be used:=0A=
a) locking over NFS is problematic;=0A=
b) Microsoft Windows typically does not do NFS mounting;=0A=
c) Win32 MUAs do not understand Unix mailboxes;=0A=
d) Most MUAs do not understand Maildir mailboxes;=0A=
e) Security -- no one will export filesystems to the Internet, if=0A=
traveling clients are to be served.=0A=
=0A=
Referring to the Protocols section, one can choose between the POP3=0A=
and IMAP protocols. Here are some scenarios and reasons why one=0A=
protocol is chosen over the other:=0A=
=0A=
Scenario Reasons=0A=
=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A=
Reading email on one POP3. Easiest to setup and supported=0A=
computer or operating system by most MUAs.=0A=
only. Example server: qpopper=0A=
=0A=
Reading email on multiple IMAP. The protocol is designed for=0A=
computers and/or operating this purpose. Only a few MUAs =0A=
systems. support IMAP protocol. MUAs are not=0A=
totally compatible among themselves.=0A=
A separate server machine serving =
email =0A=
is required.=0A=
Example server: courier-imapd=0A=
=0A=
Have exactly one computer. Out of luck. Can't bake your cake=0A=
Reading email on multiple and have it too. Sorry.=0A=
operating system.=0A=
=0A=
=0A=
Additional Utilities=0A=
--------------------=0A=
=0A=
So, after installing the SMTP server and the POP3/IMAP server, all=0A=
the MUAs can be reconfigured to use the in house servers. Wait a =0A=
minute, what happens to the email delivering to the ISP account?=0A=
Well, MUAs can be configure to check both accounts, that is one=0A=
of the solutions.=0A=
=0A=
A better solution requires the use of an additional mail utility=0A=
known as fetchmail. Fetchmail retrieves emails from remote mail servers,=0A=
massages retrieved email according some instruction, and forwards=0A=
the emails to some SMTP servers.=0A=
=0A=
In this case, fetchmail can be instructed to fetch email from the=0A=
ISP mail server, to convert the external domain name into the=0A=
internal domain name (if you do not have FQDN), and to forward=0A=
to the local MTA. The local MTA accepts the email and deliver=0A=
them locally as usual.=0A=
=0A=
=0A=
Access Email Remotely=0A=
---------------------=0A=
=0A=
If the IMAP protocol is chosen, remote access of the email storage=0A=
is allowed. This means one can login from work/school, and check=0A=
his email anywhere. To perform this task securely, the concept=0A=
of tunneling must be used.=0A=
=0A=
The term tunneling is not specific to email, and generally means=0A=
embedding one protocol within another procotol. In this particular=0A=
case, the IMAP and SMTP protocol is being tunneled within the =0A=
SSH protocol.=0A=
=0A=
The SSH protocol encrypts the traffic and allow a secure connection=0A=
between two location over the insecure Internet. The idea is to=0A=
use the port forwarding feature of the SSH protocol to allow=0A=
access to the IMAP server and the SMTP server remotely. The=0A=
advantage of doing so is that all the traffic is being encrypted,=0A=
and nobody (in theory) on the Internet can be eavesdropping on your=0A=
email.=0A=
=0A=
------=_NextPart_000_000F_01C1DCA4.685913A0--
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-doc" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?001201c1dcdf$14d97d60$2784412f>