Introduction: What is IMAP?
IMAP (Internet Message Access Protocol) is a standard protocol for accessing email messages stored on a remote mail server. Unlike POP3, which downloads and deletes messages from the server, IMAP allows clients to manipulate messages and mailboxes on the server while keeping them stored remotely. This enables multi-device synchronization, server-side search, and folder management.
IMAP operates over TCP (typically port 143 for plaintext, 993 for SSL/TLS). It uses a command/response pattern with tagged commands and server replies. Understanding IMAP is crucial for building email clients, automated mail processing systems, or any application that needs to interact with a mail store programmatically.
Why IMAP Matters for Developers
🚀 Deploy your AI agent in 10 minutes
Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.
Try it free →Developers encounter IMAP when building:
- Email clients (desktop, web, mobile) that need full mailbox access
- Migration and backup tools to copy messages between servers
- Automated email processors that categorize, archive, or respond to incoming mail
- Monitoring and analytics systems that extract data from mail traffic
Implementing IMAP from scratch or using libraries requires understanding the protocol's state machine, command syntax, and response parsing to handle edge cases, connection drops, and large mailboxes efficiently.
Core IMAP Concepts
Before writing code, grasp these fundamentals:
- Mailboxes (Folders): Hierarchical collections of messages (INBOX, Sent, Drafts).
- Message Sequence Numbers vs. UIDs: Sequence numbers are 1-based positions that change when messages are deleted. Unique IDs (UIDs) are permanent identifiers assigned to messages, never reused.
- Flags: System flags like \Seen, \Answered, \Flagged, \Deleted, \Draft, \Recent, plus custom keywords.
- IMAP States: The connection is always in one of several states: Non-Authenticated, Authenticated, Selected, Logout. Commands are only valid in certain states (e.g., SEARCH only in Selected).
- Responses: Each command yields a tagged response (OK, NO, BAD) plus optional untagged data like EXISTS, EXPUNGE, FETCH.
Getting Started: Connecting to an IMAP Server
Let's build a minimal IMAP client using Python’s imaplib from the standard library. We'll also show raw socket interaction to illustrate the protocol.
Using Python’s imaplib (High-Level)
import imaplib
import ssl
# Connect to Gmail IMAP over SSL
imap_host = 'imap.gmail.com'
imap_port = 993
username = 'user@gmail.com'
password = 'app-specific-password' # Use OAuth2 for production
# Create an SSL context (modern TLS)
context = ssl.create_default_context()
client = imaplib.IMAP4_SSL(host=imap_host, port=imap_port, ssl_context=context)
try:
client.login(username, password)
print('Login successful')
except imaplib.IMAP4.error as e:
print(f'Login failed: {e}')
client.logout()
exit()
# List mailboxes
status, mailboxes = client.list()
if status == 'OK':
for mb in mailboxes:
print(mb.decode())
# Select INBOX
status, _ = client.select('INBOX')
print(f'INBOX selected: {status}')
client.logout()
Always use SSL/TLS (IMAP4_SSL). For plaintext connections, upgrade with STARTTLS after connection. We'll cover STARTTLS later.
Raw Socket Example (Theory in Practice)
To understand the protocol, let's send raw IMAP commands over a socket:
import socket
import ssl
# Connect and wrap with SSL
sock = socket.create_connection(('imap.gmail.com', 993))
ssl_context = ssl.create_default_context()
ssl_sock = ssl_context.wrap_socket(sock, server_hostname='imap.gmail.com')
def send_command(tag, cmd):
ssl_sock.sendall(f'{tag} {cmd}\r\n'.encode())
response = b''
while True:
chunk = ssl_sock.recv(4096)
response += chunk
# Check for completion tagged line
if f'{tag} OK'.encode() in response or f'{tag} NO'.encode() in response or f'{tag} BAD'.encode() in response:
break
return response.decode()
# Read initial greeting
greeting = ssl_sock.recv(1024)
print('Server:', greeting.decode())
# Login
tag = 'A001'
response = send_command(tag, 'LOGIN user@gmail.com app-password')
print(response)
# List mailboxes
tag = 'A002'
response = send_command(tag, 'LIST "" "*"')
print(response)
# Select INBOX
tag = 'A003'
response = send_command(tag, 'SELECT INBOX')
print(response)
# Logout
tag = 'A004'
response = send_command(tag, 'LOGOUT')
print(response)
ssl_sock.close()
The raw approach clarifies that every command is prefixed with a unique tag (alphanumeric string like 'A001') and terminated by CRLF. The server responds with untagged lines (like '* 1 EXISTS') and finally a tagged response line with the same tag.
Implementing IMAP Commands: A Step-by-Step Guide
After login and selecting a mailbox, you can perform actions. We'll use imaplib for clarity but reference raw commands.
Fetching Messages (FETCH)
Retrieve message envelopes, headers, or entire bodies. IMAP FETCH allows selective fetching to avoid downloading huge attachments unnecessarily.
# Fetch all message UIDs in INBOX
status, data = client.uid('search', None, 'ALL')
if status != 'OK':
print('Search failed')
return
uids = data[0].split()
latest_uid = uids[-1] # newest message
# Fetch envelope (From, Subject, Date) and internal date
status, data = client.uid('fetch', latest_uid, '(FLAGS INTERNALDATE BODY.PEEK[HEADER.FIELDS (FROM SUBJECT DATE)])')
print('Fetch result:', data)
# Fetch full message body (headers + body, without setting \Seen flag)
status, data = client.uid('fetch', latest_uid, '(BODY.PEEK[])')
raw_email = data[0][1]
print('Raw email length:', len(raw_email))
BODY.PEEK is crucial: it retrieves body parts without marking the message as read (doesn't set \Seen flag). Use BODY.PEEK[HEADER] for headers, BODY.PEEK[TEXT] for body text, BODY.PEEK[] for full message.
Raw command equivalent: A004 UID FETCH 1234 (FLAGS BODY.PEEK[])
Searching Messages (SEARCH)
IMAP search is powerful. Combine criteria: since date, unseen, from, subject, body text, etc.
# Search for unseen messages from a specific sender since a date
criteria = '(UNSEEN FROM "newsletter@example.com" SINCE "01-Jan-2025")'
status, data = client.uid('search', None, criteria)
uids = data[0].split()
print(f'Found {len(uids)} unseen messages')
Raw: A005 UID SEARCH UNSEEN FROM "newsletter@example.com" SINCE 01-Jan-2025
Note: IMAP date format is DD-Mmm-YYYY (e.g., 01-Jan-2025). Always use UID SEARCH to get persistent identifiers.
Managing Flags and Deletions
Mark messages read, flag them, or delete them.
# Mark a message as read (add \Seen flag)
client.uid('store', latest_uid, '+FLAGS', '\\Seen')
# Delete a message: add \Deleted flag, then expunge
client.uid('store', uid_to_delete, '+FLAGS', '\\Deleted')
client.expunge() # Permanently remove messages with \Deleted flag
Raw: A006 UID STORE 1234 +FLAGS (\Seen), A007 EXPUNGE. The EXPUNGE command also causes untagged EXPUNGE responses that shift sequence numbers; using UIDs avoids this issue.
Mailbox Management
Create, rename, delete, and subscribe to mailboxes.
# Create a new folder
client.create('Projects/IMAP-Tutorial')
# Rename folder
client.rename('Projects/IMAP-Tutorial', 'Projects/IMAP-Guide')
# Delete folder (must be empty)
client.delete('Projects/IMAP-Guide')
# Subscribe/unsubscribe (some servers require subscription for visibility)
client.subscribe('INBOX/Subfolder')
client.unsubscribe('OldFolder')
Handling Responses and Parsing
IMAP responses can be complex: FETCH responses may contain nested literals, body parts, and MIME structure. imaplib parses into tuples, but understanding raw parsing is essential for robust clients.
Parsing BODYSTRUCTURE
Fetching the MIME structure of a message without downloading attachments:
# Fetch BODYSTRUCTURE
status, data = client.uid('fetch', uid, '(BODYSTRUCTURE)')
bodystructure = data[0] # nested list describing MIME parts
print(bodystructure)
# Example: ((("text" "plain" ("charset" "us-ascii") NIL NIL "7bit" 42 1) ("text" "html" ...)) ...)
This allows selecting specific parts for download (e.g., only plain text alternative).
Raw: A008 UID FETCH 1234 BODYSTRUCTURE
Handling Literal Strings and Synchronization
When a command expects large input (APPEND), IMAP uses a synchronization mechanism: the server sends a continuation request "+". The client sends the literal data. imaplib handles this internally via append method.
# Append a message to a mailbox (raw RFC822 text)
raw_message = b"From: me@example.com\r\nSubject: Test\r\n\r\nBody here\r\n"
# The library sends: APPEND mailbox (\Seen) {size}
# Server responds with + ready for data
status = client.append('INBOX', None, None, raw_message)
print('Append status:', status)
Understanding this helps when implementing custom APPEND or when dealing with non-standard extensions.
Advanced: IMAP IDLE for Real-Time Updates
IDLE command allows the server to push updates (new mail, flag changes) without polling. Essential for real-time email clients.
# Using IDLE with imaplib (requires manual handling)
# Select INBOX
client.select('INBOX')
# Send IDLE command
client.send(b'%s IDLE\r\n' % (client._new_tag().encode())) # custom tag
# Wait for server response
while True:
try:
# Set a timeout for socket response
response = client._get_response()
if response:
# If untagged response like '* 1 EXISTS', print it
print('Update:', response)
# Break on any update, then DONE to stop IDLE
break
except socket.timeout:
# Heartbeat: some servers require periodic noop or DONE
pass
# Send DONE to exit IDLE
client.send(b'DONE\r\n')
client._get_response() # consume continuation
Note: imaplib doesn't expose IDLE directly; you need to use low-level _new_tag() and send. A better approach is using an async library like aiosmtpd or imapclient. In practice, use imaplib with threads for IDLE or switch to a dedicated library.
Best Practices for IMAP Client Development
- Use UIDs, not sequence numbers: Sequence numbers change after expunge; UIDs are stable. Always use
UID FETCH,UID STORE,UID SEARCHto avoid race conditions. - Prefer BODY.PEEK: Avoid inadvertently marking messages as read unless explicitly desired. Use
BODY.PEEKfor reading. - Handle connection drops gracefully: Implement reconnection with state synchronization. After reconnect, re-select mailbox and re-fetch recent changes using
UID FETCH ... (FLAGS)since last known UID. - Respect server limits and bandwidth: Fetch in chunks (e.g., 50 messages at a time). Avoid fetching all UIDs in huge mailboxes; use
SEARCHwith date ranges or limit the number of results. - Use TLS/SSL always: Never transmit credentials in plaintext. Use STARTTLS if connecting to port 143. Validate certificates.
- Implement OAuth 2.0 for modern providers: Gmail, Outlook require OAuth. Use
XOAUTH2SASL mechanism. imaplib supports it viaauthenticatemethod. - Parse responses defensively: Servers may vary in format (e.g., quoting). Use library functions when possible; when parsing raw, expect literals, atoms, and quoted strings.
- Log commands and responses for debugging: Enable debugging in imaplib:
imaplib.Debug = 4or log your own raw traffic. - Respect IMAP states: Don't issue SEARCH before SELECT. After LOGOUT, don't send further commands.
- Handle IDLE correctly: Send DONE to exit, and expect continuation responses. Use threading to keep IDLE responsive.
Conclusion
Implementing IMAP from scratch or integrating it into your application requires understanding its stateful nature, UID persistence, and efficient fetching patterns. By mastering the command structure and using libraries like imaplib combined with low-level knowledge, you can build robust email processing pipelines, full-featured mail clients, or automated migration tools. Start with simple fetch and search operations, then layer on IDLE for real-time updates, and always adhere to best practices for security and performance. With this tutorial, you’re equipped to move from IMAP theory to practical, production-ready code.