← Back to DevBytes
AI July 11, 2026 12 min read

How to Set Up a Mail Server with Postfix and Dovecot

What Are Postfix and Dovecot?

Postfix is a high-performance, open-source mail transfer agent (MTA) responsible for routing and delivering email across networks. Originally designed as a secure replacement for Sendmail, Postfix handles outgoing mail delivery, accepts incoming messages via SMTP, and manages queue processing with a modular architecture that isolates individual components for enhanced security.

Dovecot is an IMAP and POP3 mail delivery agent (MDA) that allows end users to retrieve email from the server using standard mail clients like Thunderbird, Outlook, or mobile mail apps. Dovecot also includes a robust authentication layer and can optionally handle local mail delivery through its LMTP (Local Mail Transfer Protocol) service.

Together, Postfix and Dovecot form a complete mail server stack capable of sending, receiving, storing, and serving email to authenticated users across multiple devices.

Why Running Your Own Mail Server Matters

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

In an era dominated by centralized email providers like Gmail and Outlook, self-hosting a mail server offers compelling advantages:

Prerequisites and Environment

Before beginning, ensure you have the following in place:

Update your system packages before starting:

sudo apt update && sudo apt upgrade -y
sudo apt install -y postfix dovecot-core dovecot-imapd dovecot-pop3d dovecot-lmtpd

DNS Configuration Fundamentals

Proper DNS records are critical for mail delivery. Incomplete or incorrect DNS is the leading cause of mail server failures. Configure the following records at your DNS provider:

MX Record

The MX record tells the world where to deliver mail for your domain:

example.com.    IN    MX    10    mail.example.com.

The priority value (10) allows multiple MX records for redundancy — lower numbers receive mail first.

A and PTR Records

Create an A record for your mail hostname and ensure reverse DNS matches:

mail.example.com.    IN    A    198.51.100.10
10.100.51.198.in-addr.arpa.    IN    PTR    mail.example.com.

Verify your PTR record with dig -x 198.51.100.10 — many receiving servers will reject mail if the PTR does not resolve to a valid hostname.

SPF Record

SPF (Sender Policy Framework) declares which servers are authorized to send mail on behalf of your domain:

example.com.    IN    TXT    "v=spf1 mx a:mail.example.com -all"

The -all mechanism is strict — only the MX and specified A records may send. Use ~all (soft fail) during initial testing if desired.

DKIM and DMARC Preparation

While DKIM signing requires additional setup with OpenDKIM, note that a basic DMARC policy record should be added once your server is operational:

_dmarc.example.com.    IN    TXT    "v=DMARC1; p=quarantine; rua=mailto:postmaster@example.com"

This instructs receiving servers to quarantine messages that fail authentication and sends aggregate reports to your postmaster address.

Postfix Installation and Core Configuration

Install Postfix along with supplemental packages for TLS and submission support:

sudo apt install -y postfix postfix-mysql postfix-pcre libsasl2-modules

During the interactive installation dialog, select "Internet Site" and enter your domain name. After installation, edit the main configuration file:

sudo nano /etc/postfix/main.cf

Replace the content with a clean, secure baseline configuration. Here is a complete main.cf tuned for a single-domain setup:

# ========== Basic Identity ==========
myhostname = mail.example.com
mydomain = example.com
myorigin = $mydomain

# ========== Network Interfaces ==========
inet_interfaces = all
inet_protocols = ipv4

# ========== Trusted Networks ==========
mynetworks = 127.0.0.0/8, [::1]/128

# ========== Mail Delivery Domain ==========
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain

# ========== Relay Control ==========
mynetworks_style = host
relay_domains = $mydestination
relayhost =

# ========== Recipient Restrictions ==========
relay_recipient_maps = hash:/etc/postfix/relay_recipients

# ========== TLS Configuration ==========
smtpd_tls_cert_file = /etc/letsencrypt/live/mail.example.com/fullchain.pem
smtpd_tls_key_file = /etc/letsencrypt/live/mail.example.com/privkey.pem
smtpd_tls_security_level = may
smtpd_tls_auth_only = no
smtpd_tls_received_header = yes
smtpd_tls_loglevel = 1

smtp_tls_cert_file = $smtpd_tls_cert_file
smtp_tls_key_file = $smtpd_tls_key_file
smtp_tls_security_level = may
smtp_tls_loglevel = 1

# ========== SASL Authentication ==========
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth
smtpd_sasl_security_options = noanonymous
smtpd_sasl_tls_security_options = noanonymous
smtpd_sender_restrictions = permit_mynetworks, permit_sasl_authenticated

# ========== Submission Port (587) ==========
smtpd_recipient_restrictions =
  permit_mynetworks,
  permit_sasl_authenticated,
  reject_unauth_destination

# ========== Anti-Spam Measures ==========
smtpd_helo_required = yes
disable_vrfy_command = yes
smtpd_delay_reject = no
strict_rfc821_envelopes = yes

# ========== Limits ==========
smtpd_client_connection_count_limit = 20
smtpd_client_message_rate_limit = 50
smtpd_message_size_limit = 26214400
mailbox_size_limit = 0
message_size_limit = 26214400

# ========== Aliases and Maps ==========
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases

Next, configure the master.cf file to enable the submission and services properly:

sudo nano /etc/postfix/master.cf

Ensure the submission service (port 587) is uncommented and configured:

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o smtpd_sender_restrictions=permit_sasl_authenticated,reject
  -o milter_macro_daemon_name=ORIGINATING

Also enable the smtps service on port 465 if you have legacy clients that require implicit TLS:

smtps     inet  n       -       y       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  -o smtpd_sender_restrictions=permit_sasl_authenticated,reject

Apply changes and verify the configuration:

sudo postfix check
sudo systemctl restart postfix
sudo systemctl status postfix

Dovecot Installation and Configuration

Dovecot handles IMAP/POP3 access and authenticates users for Postfix via SASL. Install the full Dovecot suite:

sudo apt install -y dovecot-core dovecot-imapd dovecot-pop3d dovecot-lmtpd dovecot-sieve dovecot-managesieve

Dovecot uses a modular configuration directory structure. Edit the main configuration file:

sudo nano /etc/dovecot/dovecot.conf

Start with the core directives:

# ========== Core Protocol Support ==========
protocols = imap pop3 lmtp sieve

# ========== Server Listening ==========
listen = *, ::

# ========== Mail Location ==========
mail_location = maildir:~/Maildir

# ========== Authentication ==========
auth_mechanisms = plain login

# ========== TLS Configuration ==========
ssl = required
ssl_cert = </etc/letsencrypt/live/mail.example.com/fullchain.pem
ssl_key = </etc/letsencrypt/live/mail.example.com/privkey.pem
ssl_min_protocol = TLSv1.2
ssl_cipher_list = ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384
ssl_prefer_server_ciphers = yes

# ========== Logging ==========
log_path = syslog
mail_debug = no

# ========== Namespace ==========
namespace inbox {
  inbox = yes
  mailbox Trash {
    auto = subscribe
    special_use = \Trash
  }
  mailbox Drafts {
    auto = subscribe
    special_use = \Drafts
  }
  mailbox Sent {
    auto = subscribe
    special_use = \Sent
  }
  mailbox Junk {
    auto = subscribe
    special_use = \Junk
  }
  mailbox Archive {
    auto = subscribe
    special_use = \Archive
  }
}

# ========== LMTP Delivery ==========
lmtp_rcpt_check_quota = yes
lmtp_hdr_add_quota = yes

Now configure the authentication source. Edit the authentication configuration:

sudo nano /etc/dovecot/conf.d/10-auth.conf

Key settings to verify or modify:

auth_mechanisms = plain login
!include auth-system.conf.ext

For a simple setup using system users, ensure the system auth file is enabled. If you plan to use virtual users with a database, you would create a custom passdb and userdb file, but system users work perfectly for small-scale deployments. Edit the system auth configuration:

sudo nano /etc/dovecot/conf.d/auth-system.conf.ext

Ensure it contains:

passdb {
  driver = pam
  args = dovecot
}

userdb {
  driver = passwd
  args = uid=5000 gid=5000 home=/var/mail/vhosts/%d/%n allow_all_users=yes
}

For virtual mail users (recommended for multi-domain setups), create a dedicated structure:

sudo mkdir -p /var/mail/vhosts/example.com
sudo groupadd -g 5000 vmail
sudo useradd -g vmail -u 5000 -d /var/mail/vhosts -m vmail
sudo chown -R vmail:vmail /var/mail/vhosts

Configuring the SASL Socket for Postfix

Postfix communicates with Dovecot for authentication through a UNIX socket. Configure this in Dovecot:

sudo nano /etc/dovecot/conf.d/10-master.conf

Add or modify the service auth block:

service auth {
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }

  unix_listener auth-userdb {
    mode = 0600
    user = vmail
    group = vmail
  }
}

Restart Dovecot and verify the socket exists:

sudo systemctl restart dovecot
ls -la /var/spool/postfix/private/auth

You should see the socket file with owner postfix:postfix and permissions srw-rw----.

Creating Mail Users

With system-based authentication, each mail user corresponds to a Linux system account. Create a user specifically for email:

sudo useradd -m -s /sbin/nologin jane.doe
sudo passwd jane.doe

The /sbin/nologin shell prevents SSH login while allowing Dovecot authentication. For virtual users with a flat-file database, create a password file:

sudo nano /etc/dovecot/users.passwd

Generate a salted password hash:

echo -n "SecurePassword123" | doveadm pw -s SHA512-CRYPT -r 500000

Add the user entry:

jane.doe@example.com:{SHA512-CRYPT}$6$...hash...:5000:5000::/var/mail/vhosts/example.com/jane.doe::

Then update Dovecot authentication to use this file instead of PAM/passwd.

Enabling Sieve for Server-Side Mail Filtering

Sieve allows users to create server-side filters for organizing incoming mail into folders. Enable it in the LMTP delivery process:

sudo nano /etc/dovecot/conf.d/20-lmtp.conf

Add the sieve plugin to the LMTP protocol:

protocol lmtp {
  mail_plugins = $mail_plugins sieve
}

Configure the sieve plugin settings:

sudo nano /etc/dovecot/conf.d/90-sieve.conf

Essential configuration:

plugin {
  sieve = file:~/sieve;active=~/.dovecot.sieve
  sieve_before = /var/mail/sieve/before.sieve
  sieve_dir = ~/sieve
  sieve_global_dir = /var/mail/sieve
}

Create a global sieve script to filter spam or add common rules:

sudo mkdir -p /var/mail/sieve
sudo nano /var/mail/sieve/before.sieve

Example global filter:

require ["fileinto", "regex"];

if header :contains "X-Spam-Flag" "YES" {
  fileinto "Junk";
  stop;
}

Compile the sieve script:

sudo sievec /var/mail/sieve/before.sieve

Obtaining and Configuring SSL/TLS Certificates

Let's Encrypt provides free, automated certificates. Install Certbot and obtain a certificate:

sudo apt install -y certbot
sudo certbot certonly --standalone -d mail.example.com --agree-tos -m admin@example.com

Certbot places certificates in /etc/letsencrypt/live/mail.example.com/. Configure automatic renewal:

sudo nano /etc/cron.d/certbot-renewal

Add a renewal cron job that reloads both services:

0 3 * * 1 root certbot renew --quiet --deploy-hook "systemctl reload postfix dovecot"

Verify TLS is working on all ports:

openssl s_client -connect mail.example.com:25 -starttls smtp
openssl s_client -connect mail.example.com:587 -starttls smtp
openssl s_client -connect mail.example.com:993

Each command should show a valid certificate chain and negotiated TLS protocol.

Testing the Complete Mail Stack

Test SMTP submission and delivery manually using OpenSSL and base64-encoded authentication:

# Connect to submission port
openssl s_client -connect mail.example.com:587 -starttls smtp -quiet
# You should see "220 mail.example.com ESMTP Postfix"

# EHLO greeting
EHLO test-client.local

# Authenticate (generate base64 credentials)
echo -ne '\0jane.doe\0SecurePassword123' | base64
# Output: AGphbmUuZG9lAFNlY3VyZVBhc3N3b3JkMTIz

# In the OpenSSL session:
AUTH PLAIN AGphbmUuZG9lAFNlY3VyZVBhc3N3b3JkMTIz

# Compose and send a message
MAIL FROM:<jane.doe@example.com>
RCPT TO:<test@example.org>
DATA
Subject: Test Message
From: jane.doe@example.com
To: test@example.org

This is a test of the mail server configuration.
.
QUIT

Verify Dovecot IMAP access:

openssl s_client -connect mail.example.com:993 -quiet
# After connection:
A1 LOGIN jane.doe SecurePassword123
A2 LIST "" "*"
A3 SELECT INBOX
A4 LOGOUT

Check mail logs for delivery confirmation:

sudo tail -f /var/log/mail.log

Look for lines containing postfix/smtp with status=sent and dovecot entries showing successful authentication.

Configuring a Mail Client

After confirming the server works, configure a desktop or mobile mail client with these settings:

Test sending an email to an external address and receiving a reply to verify end-to-end functionality.

Best Practices for Production Mail Servers

1. Implement DKIM Signing

Install OpenDKIM and sign all outgoing messages to improve deliverability:

sudo apt install -y opendkim opendkim-tools
sudo mkdir -p /etc/opendkim/keys/example.com
cd /etc/opendkim/keys/example.com
sudo opendkim-genkey -s default -d example.com
sudo chown opendkim:opendkim default.private

Add the DKIM TXT record to your DNS (the public key is in default.txt), then integrate OpenDKIM with Postfix via the milter interface in main.cf:

milter_default_action = accept
milter_protocol = 6
smtpd_milters = inet:127.0.0.1:8891
non_smtpd_milters = $smtpd_milters

2. Enable Strict TLS Encryption

After testing with opportunistic TLS (may), switch to mandatory encryption for submission:

smtpd_tls_security_level = encrypt
smtpd_tls_auth_only = yes

3. Configure Rate Limiting and Connection Controls

Protect your server from abuse with sensible limits in main.cf:

smtpd_client_connection_rate_limit = 10
anvil_rate_time_unit = 60s
smtpd_error_sleep_time = 5s
smtpd_soft_error_limit = 5
smtpd_hard_error_limit = 10

4. Set Up Monitoring and Alerts

Install logwatch or mailgraph to monitor mail flow. Configure Postfix to send bounce notifications to a responsible address:

sudo nano /etc/postfix/sender_bcc_maps

Add:

postmaster@example.com    archive@example.com

Then run postmap /etc/postfix/sender_bcc_maps and reference it in main.cf with sender_bcc_maps = hash:/etc/postfix/sender_bcc_maps.

5. Regular Backups of Mail Storage

Maildir storage is file-based and easy to back up with rsync:

sudo rsync -avz --delete /var/mail/vhosts/ backup-server:/backups/mail/

Schedule this daily and verify restoration procedures periodically.

6. Firewall and Port Hardening

Only expose necessary ports to the internet. Using ufw:

sudo ufw allow 25/tcp
sudo ufw allow 587/tcp
sudo ufw allow 993/tcp
sudo ufw allow 22/tcp
sudo ufw default deny incoming
sudo ufw enable

7. Test Your Configuration Thoroughly

Use external testing tools to validate your setup:

# Check SMTP configuration
sudo apt install -y mailutils
echo "Test body" | mail -s "Test Subject" external-address@example.org

# Use online validators like:
# - Mail Tester (https://www.mail-tester.com)
# - MX Toolbox (https://mxtoolbox.com)
# - DKIM Validator
# - SPF Validator

These services analyze your server's behavior and score deliverability based on SPF, DKIM, DMARC, TLS, and reverse DNS compliance.

Troubleshooting Common Issues

Mail Loops and Relay Denied Errors

If you see relay access denied in logs, verify that mydestination includes your domain and that SASL authentication is working:

sudo postfix reload
sudo tail -f /var/log/mail.log | grep "SASL"

TLS Certificate Errors

Ensure file paths in both Postfix and Dovecot configurations point to the correct certificate files. Check permissions:

sudo ls -la /etc/letsencrypt/live/mail.example.com/
sudo chmod 755 /etc/letsencrypt/live /etc/letsencrypt/archive

Dovecot Authentication Failures

Test authentication directly:

sudo doveadm auth test jane.doe SecurePassword123
# Expected output: passdb lookup succeeded, userdb lookup succeeded

If this fails, check your passdb configuration and ensure the user exists in the configured backend.

Conclusion

Setting up a complete mail server with Postfix and Dovecot gives you full control over your email infrastructure. You have configured a robust SMTP submission system with TLS encryption, SASL authentication, IMAP access with server-side filtering via Sieve, and proper DNS records including SPF and MX. The stack you built handles mail submission, local delivery, remote relay, and retrieval — all secured with modern TLS ciphers and authenticated access controls.

With the best practices outlined — DKIM signing, rate limiting, monitoring, regular backups, and firewall hardening — your mail server is ready for production use. Continue to monitor deliverability scores, keep certificates renewed, and test your configuration against evolving standards. The knowledge gained from this hands-on setup extends far beyond this single implementation, giving you deep insight into how internet email actually works at the protocol level.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles