Constructing E-mail – Your Own Mail Server

Jump to the Examples

Disclaimer: Should you decide to act upon any information on this website, you do so at your own risk! A mail server that is configured badly can be abused to send spam in your name, can get your address blocklisted, and can lose your mail. Test with throw-away addresses first, and never move your only address to a server you have not tested.

E-mail is older than the web, older than most of the protocols we use every day, and it is still the "master key" of the Internet: almost every account you own can be reset by sending a message to your mailbox. It is also one of the few services that is truly federated: anyone with a mail server can send to anyone else, without asking a company for permission. Below you will find a bit of history, what was used and what is used today, and then a practical part with a mail server of your own. Where the topic touches other notes, there are links: the machine itself in Constructing a Cloud, DNS, ports and firewalls in Computer Networks, the dangers in Security Considerations, and the terminal commands in Communication with the Machines.

How a Message Travels

Before the history, it helps to know the parts, because the old and the new software are named after the jobs they do. Sending e-mail is like the postal system described in Computer Networks: you write a letter, a local post office accepts it, and sorting centres pass it on until the recipient's post office puts it in a mailbox that the recipient opens whenever they want.

Sender's program (MUA)          e.g. Thunderbird, a phone app, a web page
        |  port 587 or 465 (submission, login required)
        v
[ Submission server (MSA) ]     checks who you are, accepts the message
        |
        v
[ Sending server (MTA) ] ---- DNS lookup of the recipient domain's MX record
        |  port 25 (SMTP between servers)
        v
[ Receiving server (MTA) ]      checks SPF, DKIM, DMARC, spam filters
        |
        v
[ Delivery agent (MDA) ] --> Mailbox on disk (Maildir)
                                   ^
                                   |  port 993 (IMAP over TLS)
                          Recipient's program (MUA)

The roles and the usual software (my own selection, not a complete list):

A Bit of E-mail History

Messages between users of the same computer existed in the 1960s, on time-sharing systems (the same era as in Constructing a Cloud). The step that made it e-mail was crossing the network: in 1971 Ray Tomlinson sent a message between two machines on the ARPANET and chose the @ sign to separate the user from the host. Within a few years e-mail was the most used service on the network, long before the web (the early Internet is described in Computer Networks). The first widely discussed unwanted mass message, to hundreds of ARPANET users, came in 1978 – an early sign of a problem that still shapes every mail server today.

The format and the transport were standardised in the early 1980s. RFC 821 (SMTP, the Simple Mail Transfer Protocol, by Jon Postel) and RFC 822 (the message format with its From, To and Subject headers) were published in 1982, and Eric Allman's Sendmail brought SMTP to Unix systems shortly after. In 1986 RFC 974 tied mail to the Domain Name System: the MX record tells the sender which server accepts the mail for a domain, and that is still how mail is routed. For people who were not on the Internet there were other networks, such as UUCP and Usenet-style dial-up systems, and bridges between them, with addresses that listed the whole route ("bang paths"). Competing standards also existed, for instance the formal international X.400, which lost to the simpler SMTP.

Reading mail from a server needed its own protocols. POP (1984) simply downloaded the messages to one computer, and POP3 (1988, revised in 1996) is still around. IMAP, designed by Mark Crispin from 1986, keeps the mail on the server and lets many devices see the same folders, which matters once you have a phone, a laptop and a tablet. In 1992 MIME made it possible to send more than plain ASCII text: attachments, pictures and other alphabets (the final form is in RFC 2045 to 2049, 1996). The 1990s were also the decade of a new generation of server software. Exim and qmail appeared in 1995, Postfix in 1998 as a safer, simpler replacement for Sendmail, and Dovecot in 2002. Meanwhile, in 1996, Hotmail showed that mail could live entirely in a web page, and Microsoft Exchange became the standard mail and calendar server of many companies.

Spam and abuse changed the rules. In the 1990s most servers happily forwarded mail for anybody, and these "open relays" were used for spam, so mail servers started to check where the connection comes from and who is allowed to send. A whole family of standards was added on top of the old protocol, to prove that a message is really from its domain: SPF (2006, now RFC 7208), DKIM (2007, now RFC 6376) and DMARC (RFC 7489, 2015). In the same years encryption between servers became possible with STARTTLS (RFC 3207, 2002), and in 2018 RFC 8314 declared that clients should use TLS from the first byte.

In 2004 Gmail offered a gigabyte of storage, which was huge at the time, and the era of a few very large providers began. Today most people, and many companies, use one of those providers. The strength of a large provider is also its weakness: it knows who writes to whom, and it decides what is spam. In 2024 Google and Yahoo started to require that senders of large volumes authenticate their mail with SPF, DKIM and DMARC and offer an easy way to unsubscribe, and Microsoft followed with similar rules for Outlook.com in 2025. For a small self-hosted server these rules are good news in one respect (a message without proper authentication is rejected anyway) and a warning in another: running your own server means meeting the standards of the biggest players.

A short timeline:

Some resources for history:

• Email (Wikipedia)
• RFC 5321 – the current SMTP specification
• RFC 5322 – the current message format
• Email spam (Wikipedia)
• Computer History Museum

Protocols and Software – What Was and What Is

Some that have mostly disappeared or should be avoided:

Ports and protocols in use today:

Some newer ideas are worth knowing. JMAP (RFC 8620 and related) is a modern, JSON-over-HTTP replacement for IMAP, supported by a few servers and clients. MTA-STS (RFC 8461) and DANE (RFC 7672) tell other servers that they must use TLS when sending to you, and not to fall back to plain text. End-to-end encryption with OpenPGP or S/MIME protects the content of a message even from the servers in between, but not the headers, the sender or the time, and it is still hard to use (Autocrypt tries to simplify it). For secrets, a messenger with built-in end-to-end encryption is usually a better tool than e-mail, as discussed in Security Considerations.

Software you can use today (my own selection, not a ranking):

Some resources for protocols and software:

• Postfix documentation
• Dovecot documentation
• Rspamd documentation
• RFC 8314 – cleartext considered obsolete for mail
• IANA – port number registry

Why It Is Hard: Receiving Is Easy, Sending Is Not

Setting up the software is the easy half. Everyone can publish an MX record and accept mail for a domain, and a server that only receives is not difficult to run. Sending is where small servers fail, because every large provider decides case by case whether to believe you:

• IP address reputation. Providers keep lists of addresses that sent spam or that belong to ranges where ordinary servers should not exist. The address ranges of home connections and of some hosting companies are on such lists in advance, such as the Spamhaus policy lists. Check your address at check.spamhaus.org before you build anything.
• Port 25. Many home providers and many cloud and hosting providers block outgoing connections on port 25 by default, to stop infected computers from sending spam. Some unblock it on request, some never. Ask or read the terms before you pay.
• Reverse DNS (PTR). The address of your server must resolve back to a name, and that name must resolve to the same address. Only the owner of the address can set this, usually through a form at the hosting provider. Without it, much of your mail will be rejected.
• Authentication. SPF, DKIM and DMARC records and a matching HELO name are not optional any more.
• CGNAT and dynamic addresses. If your provider shares one public address between customers or changes yours from time to time (see Computer Networks), incoming mail will not reach a server at home reliably, and the address will never have a reputation.
• Reliability. A server that is off for a week loses nothing from senders that retry (most servers keep trying for several days), but you must be able to fix it, and the box that receives your password resets must never be the one you cannot reach.

The realistic options, from the least to the most work (my own judgement, not test results):

Planning: Domain, DNS and Names

The domain is what you really own. If you use your own domain with any provider, you can later move to another provider, or to your own server, without telling anyone that your address has changed. Make sure the domain is registered in your name, that you have two-factor authentication on the registrar account, and that the DNS provider has an interface or an API for editing records. Then decide on one name for the mail server (for example mail.example.org); this name goes into the certificate, the HELO greeting, the reverse DNS and the MX record.

The records of a typical setup (documentation addresses and example.org used as placeholders):

Examples: Building a Mail Server

The examples build the same service in different ways. They are a starting point, not a tested recipe for your exact system: the configuration of Postfix and Dovecot changes between versions (Dovecot 2.4 renamed many settings), so always check the documentation of the version that you install. Use test addresses, and do not use the real domain of a business until everything works.

Overview

Example 1 – Debian with Postfix, Dovecot and Rspamd

A single domain, a few virtual mailboxes, TLS everywhere. Prepare a Debian server as described in Constructing a Cloud (SSH keys, firewall, automatic updates), set the DNS records from the table above and open ports 25, 465 or 587, and 993 in the firewall (the nftables example in Computer Networks shows how to allow them).

1. Install the packages and create a user that owns the mail:

   sudo apt update
   sudo apt install postfix dovecot-imapd dovecot-lmtpd rspamd certbot
   sudo useradd --system --uid 5000 --home-dir /var/vmail --create-home --shell /usr/sbin/nologin vmail

   Choose "Internet Site" if the Postfix installer asks and enter the mail name mail.example.org.
2. Get a certificate for the mail host name (port 80 must be reachable for the simple method) and reload the services after each renewal:

   sudo certbot certonly --standalone -d mail.example.org --deploy-hook "systemctl reload postfix dovecot"

3. Postfix – the main settings in /etc/postfix/main.cf (the rest stays as installed):

   # /etc/postfix/main.cf (excerpt)
   myhostname = mail.example.org
   mydestination = localhost
   mynetworks = 127.0.0.0/8 [::1]/128
   inet_interfaces = all
   
   # virtual mailboxes: Postfix only checks that the recipient exists, Dovecot stores the mail
   virtual_mailbox_domains = example.org
   virtual_mailbox_maps = hash:/etc/postfix/vmailbox
   virtual_alias_maps = hash:/etc/postfix/valias
   virtual_transport = lmtp:unix:private/dovecot-lmtp
   
   # TLS: opportunistic towards other servers, certificate from Let's Encrypt
   smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.org/fullchain.pem
   smtpd_tls_key_file = /etc/letsencrypt/live/mail.example.org/privkey.pem
   smtpd_tls_security_level = may
   smtp_tls_security_level = may
   smtpd_tls_protocols = >=TLSv1.2
   smtp_tls_protocols = >=TLSv1.2
   
   # authentication for submission goes through Dovecot
   smtpd_sasl_type = dovecot
   smtpd_sasl_path = private/auth
   
   # never be an open relay: only logged-in users or local networks may send out
   smtpd_relay_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination
   smtpd_helo_required = yes
   disable_vrfy_command = yes
   
   # Rspamd: spam filtering and DKIM signing
   smtpd_milters = inet:localhost:11332
   non_smtpd_milters = inet:localhost:11332
   milter_default_action = accept
   milter_mail_macros = i {mail_addr} {client_addr} {client_name} {auth_authen}

4. The submission service for your own programs, in /etc/postfix/master.cf (remove the comment signs in front of the existing "submission" line and set these options; port 465 is the same with smtpd_tls_wrappermode=yes):

   submission inet n - y - - smtpd
     -o syslog_name=postfix/submission
     -o smtpd_tls_security_level=encrypt
     -o smtpd_tls_auth_only=yes
     -o smtpd_sasl_auth_enable=yes
     -o smtpd_relay_restrictions=permit_sasl_authenticated,reject

5. The mailboxes and aliases – one line per address (the value after the address is ignored), and the addresses that every domain must have:

   # /etc/postfix/vmailbox
   ana@example.org    OK
   
   # /etc/postfix/valias
   postmaster@example.org    ana@example.org
   abuse@example.org         ana@example.org
   
   sudo postmap /etc/postfix/vmailbox /etc/postfix/valias

6. Dovecot – mailboxes in the Maildir format, passwords in a hashed file and sockets for Postfix. The syntax below is for Dovecot 2.3; version 2.4 renamed several settings, so compare with its documentation:

   # /etc/dovecot/conf.d/10-mail.conf
   mail_location = maildir:/var/vmail/%d/%n
   
   # /etc/dovecot/conf.d/10-auth.conf
   disable_plaintext_auth = yes
   auth_mechanisms = plain login
   passdb {
     driver = passwd-file
     args = scheme=ARGON2ID username_format=%u /etc/dovecot/users
   }
   userdb {
     driver = static
     args = uid=vmail gid=vmail home=/var/vmail/%d/%n
   }
   
   # /etc/dovecot/conf.d/10-ssl.conf
   ssl = required
   ssl_cert = </etc/letsencrypt/live/mail.example.org/fullchain.pem
   ssl_key = </etc/letsencrypt/live/mail.example.org/privkey.pem
   
   # /etc/dovecot/conf.d/10-master.conf (inside the existing service blocks)
   service lmtp {
     unix_listener /var/spool/postfix/private/dovecot-lmtp {
       mode = 0600
       user = postfix
       group = postfix
     }
   }
   service auth {
     unix_listener /var/spool/postfix/private/auth {
       mode = 0660
       user = postfix
       group = postfix
     }
   }

   Create the password hash with doveadm pw -s ARGON2ID (use BLF-CRYPT if your build does not offer it) and put one line per user into /etc/dovecot/users, in the form ana@example.org:{ARGON2ID}hash. Keep that file readable only by root and Dovecot.
7. DKIM signing with Rspamd. Generate a key, tell Rspamd to use it, and publish the public part as the TXT record from the table:

   sudo mkdir -p /var/lib/rspamd/dkim
   sudo rspamadm dkim_keygen -s default -b 2048 -d example.org -k /var/lib/rspamd/dkim/example.org.default.key
   sudo chown -R _rspamd:_rspamd /var/lib/rspamd/dkim
   
   # /etc/rspamd/local.d/dkim_signing.conf
   path = "/var/lib/rspamd/dkim/$domain.$selector.key";
   selector = "default";
   allow_username_mismatch = true;

   The keygen command prints the TXT record. On Debian the Rspamd user is called _rspamd, but check on your system.
8. Check and start:

   sudo postfix check
   sudo systemctl restart postfix dovecot rspamd
   sudo systemctl enable postfix dovecot rspamd
   sudo tail -f /var/log/mail.log        # or: sudo journalctl -u postfix -f

   Then continue with Example 4 and do not use the server for anything important before the tests pass.

Example 2 – The same in containers with docker-mailserver

docker-mailserver packs Postfix, Dovecot, Rspamd and the glue into one maintained container image, with a command for adding users and for creating the DKIM key. The container runtime is installed as in the first example of Constructing a Cloud, and the container keeps its data in directories that you can back up. Start from the official compose.yaml and mailserver.env of the project, because the settings change between versions. The idea:

services:
  mailserver:
    image: ghcr.io/docker-mailserver/docker-mailserver:latest   # pin a version number in real use
    container_name: mailserver
    hostname: mail.example.org
    ports:
      - "25:25"
      - "465:465"
      - "587:587"
      - "993:993"
    volumes:
      - ./docker-data/dms/mail-data/:/var/mail/
      - ./docker-data/dms/mail-state/:/var/mail-state/
      - ./docker-data/dms/mail-logs/:/var/log/mail/
      - ./docker-data/dms/config/:/tmp/docker-mailserver/
      - /etc/letsencrypt:/etc/letsencrypt:ro
      - /etc/localtime:/etc/localtime:ro
    environment:
      - SSL_TYPE=letsencrypt
      - ENABLE_RSPAMD=1
      - ENABLE_CLAMAV=0          # needs a lot of memory
      - ENABLE_FAIL2BAN=1
    cap_add:
      - NET_ADMIN
    restart: always

docker compose up -d
docker exec -ti mailserver setup email add ana@example.org
docker exec -ti mailserver setup alias add postmaster@example.org ana@example.org
docker exec -ti mailserver setup config dkim

The last command creates the DKIM key; the project's documentation tells you where the public record is stored for your version, and you publish it in DNS as in the table above. Set the DNS records, the PTR record and the firewall as in the first example. The all-in-one packages mailcow and Mailu add a web interface for users, domains and quotas; they need more memory (check their requirements) and a server dedicated to them. Where a container runtime is a hard requirement of the project, Podman can usually replace Docker, as shown for the Red Hat family in Constructing a Cloud.

Example 3 – Receive yourself, send through a relay

If your address has a bad reputation, port 25 is blocked, or you simply do not want to manage deliverability, you can keep the mailboxes on your server and hand the outgoing mail to an authenticated relay (the "smarthost") that has a good reputation. Your own server still receives the mail through the MX record, stores it and serves it by IMAP; only outgoing mail goes through the relay. Keep SPF and DKIM consistent with the relay's instructions, because the relay will say what to put into the SPF record. In Postfix this means a few lines:

# /etc/postfix/main.cf (added to the configuration of Example 1)
relayhost = [smtp.relay.example.net]:587
smtp_sasl_auth_enable = yes
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_sasl_security_options = noanonymous
smtp_tls_security_level = encrypt

# /etc/postfix/sasl_passwd
[smtp.relay.example.net]:587    relay-user:relay-password

sudo chmod 600 /etc/postfix/sasl_passwd
sudo postmap /etc/postfix/sasl_passwd
sudo apt install libsasl2-modules
sudo systemctl reload postfix

A variation for a server at home behind CGNAT: rent a very small server with a public address to act as the visible front (the MX target), and carry the mail to your server at home through a WireGuard tunnel, as described for VPNs in Computer Networks. The small server can also send the outgoing mail. This keeps the mailboxes at home with only the thin front exposed, but it is also another machine to keep updated.

Example 4 – Test it from the terminal and from the outside

Most of the work is testing. The commands below only read, or send a single message. They work in any terminal, the same way as in Communication with the Machines (on Windows, use PowerShell and install the tools, or run them from the server).

# DNS: are the records where they should be?
dig +short MX example.org
dig +short TXT example.org
dig +short TXT default._domainkey.example.org
dig +short TXT _dmarc.example.org
dig +short -x 203.0.113.10               # reverse DNS: must return mail.example.org.

# Is the TLS certificate valid? (IMAP over TLS)
openssl s_client -connect mail.example.org:993 -servername mail.example.org

# Send a test message through submission with a login
swaks --to your-other-address@example.net --from ana@example.org \
      --server mail.example.org --port 587 --tls \
      --auth LOGIN --auth-user ana@example.org

# On the server: the queue and the log
mailq
sudo tail -n 50 /var/log/mail.log

swaks asks for the password if you do not give one. From a different network, also test that you cannot relay: connect to port 25, say that you are an unknown sender, and try to send a message to an address in another domain. The server must refuse it. Then check the whole result with an outside service: mail-tester.com scores a test message and explains what is missing, MXToolbox checks the records and the blocklists, and Google Postmaster Tools shows what Gmail thinks of your domain once you send a regular volume. Send a first message to an address at a large provider, find out whether it arrives in the inbox or in the spam folder, and only then start to move any real address.

Finally set up the programs. In Thunderbird or on a phone, enter the address, the server name mail.example.org, IMAP on port 993 with TLS, and SMTP on port 587 with STARTTLS (or 465 with TLS), and the full address as the user name. A webmail such as Roundcube can be added later behind a reverse proxy with HTTPS, as in the services table of Constructing a Cloud.

Keeping the Mail Healthy

• Expose only what is needed: ports 25, 465 or 587, and 993. Do not publish the administration interfaces of mailcow, Roundcube or Rspamd to the Internet without TLS and a second factor, and keep SSH on keys only. A fail2ban rule or a rate limit in the firewall stops the constant guessing of passwords.
• Never run an open relay. Test it after every change of the configuration. A mail server that relays for everybody is blocklisted within hours.
• Update regularly. Mail software faces the whole Internet. Use automatic security updates as described in Constructing a Cloud, and subscribe to the security announcements of the project that you use.
• Back up the mailboxes, the configuration, the DKIM keys and the list of users. A mailbox is not a backup of itself, and a lost DKIM key means a new key and a new DNS record. Use the 3-2-1 rule and test the restore.
• Renew certificates and watch them. Certbot renews by itself, but the services must reload the new certificate (the deploy hook in Example 1). An expired certificate is the most common reason for a mailbox that "suddenly stopped working".
• Watch the queue and the reports. Look at mailq now and then, read the DMARC reports that arrive at the address in your record (a free service can summarise them), and set up an independent check that tells you when the server is down.
• Be careful what you send. A single infected computer or a forgotten script that sends many messages damages the reputation that you worked to build. Keep the volume low, send only what recipients expect, and never send bulk mail from a personal server.
• Know the law. E-mail contains personal data of other people. If you host mail for a family, a club or a company, the GDPR (see Security Considerations) may apply to you, including how long you keep mail and what you do about a breach.
• Have a way out. Because your address is on your own domain, you can point the MX record to a hosted provider if the server fails. Keep the credentials of the domain registrar and of the DNS provider separate from the mail server, so that you can do that from anywhere.

Some resources:

• Postfix documentation
• Dovecot documentation
• Rspamd documentation
• docker-mailserver documentation
• Google – email sender guidelines
• RFC 7208 – SPF, RFC 6376 – DKIM and RFC 7489 – DMARC
• mail-tester.com and MXToolbox
• Spamhaus – IP and domain reputation checker

A Few More Thoughts

E-mail survived more than fifty years because it is open, simple at the core and extended in small steps. It also shows a pattern that repeats in the other notes of this garden: a service that anyone can run slowly becomes a service that a few can run well, because the defence against abuse is expensive. The result is a trade-off. You can use a large provider and get reliability and convenience in exchange for your data and your dependence, or run your own server and get control and knowledge in exchange for work and responsibility.

For most people the sensible first step is the domain: use your own name for your address, and everything else can change later. Running the server yourself is not for everyone, but even one evening spent on a test domain teaches more about DNS, TLS, authentication and spam than any text, including this one. If you also want to see the bigger picture of running your own services, continue with Constructing a Cloud.