Understanding the SMTP Protocol
What is SMTP?
The Simple Mail Transfer Protocol (SMTP) is the backbone of email delivery across the internet. Defined originally in RFC 821 and updated by RFC 5321, SMTP governs how email messages are transmitted from a sending mail client (or Mail User Agent) to a receiving mail server, and then relayed hop-by-hop until they reach the recipient's mailbox. It operates over TCP, traditionally on port 25 for relay, port 587 for message submission, and port 465 for SMTP over TLS (SMTPS).
At its core, SMTP is a text-based request-response protocol. The client initiates a TCP connection, the server greets it with a banner, and the two parties exchange commands and responses in a structured dialogue. Each command is a simple ASCII string terminated by CRLF (\r\n), and each response begins with a three-digit numeric code followed by a human-readable text explanation. This simplicity is by design — it allows SMTP to be implemented on virtually any platform with minimal overhead.
Why SMTP Matters for Developers
Understanding SMTP at the protocol level gives you complete control over email generation, delivery, and debugging. While production applications typically use high-level libraries like smtplib in Python or nodemailer in Node.js, knowing what happens under the hood enables you to:
- Diagnose delivery problems by reading raw server responses
- Implement custom mail clients or automated notification systems without external dependencies
- Build email testing and mocking frameworks
- Understand security mechanisms like SPF, DKIM, and DMARC that build on SMTP
- Handle edge cases like internationalized email addresses (EAI/SMTPUTF8)
- Create lightweight embedded system alerts that communicate via SMTP directly
In this tutorial, you will implement a complete SMTP client from scratch using Python's socket module. You will learn the protocol flow, MIME encoding, TLS security, and authentication — everything needed to send a rich HTML email with attachments through any modern SMTP server.
How SMTP Works: The Theory
🚀 Deploy your AI agent in 10 minutes
Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.
Try it free →The Client-Server Conversation Model
An SMTP session follows a strict sequence of phases. Understanding this sequence is critical before writing any code:
- Connection Establishment: Client opens a TCP socket to the server. The server responds with a greeting (code 220), indicating it is ready to receive commands.
- Capability Advertisement (EHLO): The client identifies itself using the EHLO (Extended Hello) command. The server responds with a 250 code and lists supported extensions like STARTTLS, AUTH, SIZE, and 8BITMIME.
- TLS Negotiation (optional): If the server advertised STARTTLS and the client wants encryption, the client issues STARTTLS, the server confirms with 220, and a TLS handshake upgrades the connection.
- Authentication (optional): The client authenticates using AUTH LOGIN or AUTH PLAIN, providing credentials encoded in Base64. The server responds with 235 on success.
- Envelope Specification: The client specifies the sender with MAIL FROM and one or more recipients with RCPT TO. Each command receives a 250 response if accepted.
- Message Transmission (DATA): The client issues the DATA command. The server responds with 354, signaling the client to transmit the message body line-by-line. The body is terminated by a single dot on a line by itself (
.CRLF). The server acknowledges with 250. - Session Termination: The client sends QUIT. The server responds with 221 and closes the connection.
Essential SMTP Commands and Response Codes
Here is a reference table of the most important commands you will implement:
EHLO <domain>— Extended Hello, identifies the client and requests server capabilitiesSTARTTLS— Requests TLS encryption upgrade on the current connectionAUTH LOGIN— Initiates Base64-encoded username/password authenticationAUTH PLAIN— Single-step Base64 authentication with username and password togetherMAIL FROM:<sender>— Defines the envelope sender addressRCPT TO:<recipient>— Defines one envelope recipient; may be repeated for multiple recipientsDATA— Signals the start of the message body transmissionQUIT— Gracefully terminates the session
Response codes follow a simple pattern: codes in the 200-399 range indicate success or intermediate acceptance, 400-499 indicate temporary failures (try again later), and 500-599 indicate permanent failures. The most common codes you will encounter are:
- 220 — Service ready (greeting banner)
- 250 — Requested action completed
- 334 — Server challenge for authentication (expects Base64 input)
- 235 — Authentication successful
- 354 — Start mail input; end with
.on a line by itself - 221 — Service closing transmission channel
- 535 — Authentication credentials invalid
- 550 — Mailbox unavailable or relaying denied
Hands-On Implementation: Building an SMTP Client
Setting Up a Basic Socket Connection
We begin by establishing a raw TCP connection and reading the server's greeting banner. This is the foundation upon which everything else is built. The following code connects to a specified SMTP server on port 587 (the standard submission port) and reads the initial 220 response:
import socket
import ssl
import base64
import re
from typing import List, Tuple, Optional
class SMTPClient:
"""
A complete SMTP client implementation from scratch.
Supports EHLO, STARTTLS, AUTH LOGIN/PLAIN, MAIL FROM,
RCPT TO, DATA, and QUIT commands.
"""
def __init__(self, server: str, port: int = 587, timeout: int = 30):
self.server = server
self.port = port
self.timeout = timeout
self.sock: Optional[socket.socket] = None
self.ssl_context: Optional[ssl.SSLContext] = None
self._read_buffer = b""
def connect(self) -> Tuple[int, str]:
"""
Establish a TCP connection and read the server greeting.
Returns a tuple of (response_code, response_text).
"""
# Create a TCP socket
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.settimeout(self.timeout)
# Resolve hostname and connect
addr_info = socket.getaddrinfo(self.server, self.port,
socket.AF_UNSPEC, socket.SOCK_STREAM)
for family, socktype, proto, canonname, sockaddr in addr_info:
try:
self.sock.connect(sockaddr)
break
except Exception:
continue
# Read the server's greeting banner
code, message = self._read_response()
if code != 220:
raise Exception(f"Expected 220 greeting, got {code}: {message}")
return code, message
def _send_line(self, line: str) -> None:
"""Send a single line (CRLF terminated) to the server."""
data = (line + "\r\n").encode("ascii", errors="replace")
self.sock.sendall(data)
def _read_response(self, expect_multiline: bool = False) -> Tuple[int, str]:
"""
Read a complete SMTP response, handling multi-line responses.
Returns (response_code, full_message).
"""
response_lines = []
while True:
# Read data in chunks until we have at least one complete line
chunk = self.sock.recv(4096)
if not chunk:
raise Exception("Connection closed by server")
self._read_buffer += chunk
# Process complete lines from the buffer
while b"\n" in self._read_buffer:
line_end = self._read_buffer.index(b"\n")
line = self._read_buffer[:line_end].decode("ascii", errors="replace")
# Remove trailing \r if present
if line.endswith("\r"):
line = line[:-1]
# Remove the processed line from the buffer
self._read_buffer = self._read_buffer[line_end + 1:]
# Parse the response line
if len(line) >= 3 and line[0:3].isdigit():
code = int(line[0:3])
text = line[4:] if len(line) > 3 and line[3] == " " else line[3:]
response_lines.append((code, text))
# Check if this is the last line of a multi-line response
# In multi-line responses, lines have '-' after the code except the last
is_last = True
if len(line) > 3 and line[3] == "-":
is_last = False
if is_last:
full_message = "\n".join(text for _, text in response_lines)
return code, full_message
else:
# Continuation lines without a code (rare but possible)
if response_lines:
response_lines.append((None, line))
# Should never reach here due to the return inside the loop
raise Exception("Incomplete response received")
def ehlo(self, client_domain: str = "localhost") -> List[str]:
"""
Send the EHLO command and parse server capabilities.
Returns a list of advertised extension keywords.
"""
self._send_line(f"EHLO {client_domain}")
code, message = self._read_response(expect_multiline=True)
if code != 250:
raise Exception(f"EHLO failed with code {code}: {message}")
# Parse capabilities from the multi-line response
capabilities = []
# The message contains all the response lines; we need to parse the raw
# response more carefully. We'll re-read and parse properly.
# For simplicity, we'll extract keywords from the full message.
for line in message.split("\n"):
line = line.strip()
# Skip the greeting line that repeats the domain
if line.startswith("250"):
parts = line.split()
for part in parts[1:]:
if part.isupper() or part in ("SIZE", "AUTH", "STARTTLS",
"8BITMIME", "SMTPUTF8"):
capabilities.append(part)
return capabilities
def starttls(self) -> None:
"""
Issue STARTTLS and upgrade the connection to TLS.
"""
self._send_line("STARTTLS")
code, message = self._read_response()
if code != 220:
raise Exception(f"STARTTLS failed with code {code}: {message}")
# Create a default SSL context for client connections
# For production, you might want to specify certificate verification
self.ssl_context = ssl.create_default_context()
# Wrap the existing socket with TLS
self.sock = self.ssl_context.wrap_socket(
self.sock,
server_hostname=self.server
)
print("TLS connection established successfully.")
def auth_login(self, username: str, password: str) -> bool:
"""
Authenticate using the AUTH LOGIN mechanism.
AUTH LOGIN sends username and password as separate Base64-encoded strings.
"""
self._send_line("AUTH LOGIN")
code, message = self._read_response()
if code != 334:
raise Exception(f"AUTH LOGIN not accepted, got code {code}: {message}")
# Decode the server's prompt (it contains Base64-encoded "Username:")
# Actually the server sends "334 VXNlcm5hbWU6" which decodes to "Username:"
# We just send our Base64-encoded username
encoded_username = base64.b64encode(username.encode("utf-8")).decode("ascii")
self._send_line(encoded_username)
code, message = self._read_response()
if code != 334:
raise Exception(f"Username rejected with code {code}: {message}")
# Send Base64-encoded password
encoded_password = base64.b64encode(password.encode("utf-8")).decode("ascii")
self._send_line(encoded_password)
code, message = self._read_response()
if code != 235:
raise Exception(f"Authentication failed with code {code}: {message}")
print("Authentication successful.")
return True
def auth_plain(self, username: str, password: str) -> bool:
"""
Authenticate using the AUTH PLAIN mechanism.
AUTH PLAIN sends a single Base64-encoded string: '\\0username\\0password'
"""
# Build the plain auth string: \0username\0password
auth_string = f"\0{username}\0{password}"
encoded_auth = base64.b64encode(auth_string.encode("utf-8")).decode("ascii")
self._send_line("AUTH PLAIN")
code, message = self._read_response()
if code != 334:
raise Exception(f"AUTH PLAIN not accepted, got code {code}: {message}")
self._send_line(encoded_auth)
code, message = self._read_response()
if code != 235:
raise Exception(f"Authentication failed with code {code}: {message}")
print("Authentication successful (PLAIN).")
return True
def mail_from(self, sender: str) -> None:
"""
Set the envelope sender address.
The sender should be a bare email address like 'user@example.com'
or a display-name formatted address like 'User <user@example.com>'.
"""
# Strip display name if present, keep only the address
match = re.search(r'<(.+?)>', sender)
if match:
sender_addr = match.group(1)
else:
sender_addr = sender.strip()
self._send_line(f"MAIL FROM:<{sender_addr}>")
code, message = self._read_response()
if code != 250:
raise Exception(f"MAIL FROM rejected with code {code}: {message}")
def rcpt_to(self, recipient: str) -> None:
"""
Add an envelope recipient.
Like sender, can be a bare address or display-name format.
"""
match = re.search(r'<(.+?)>', recipient)
if match:
rcpt_addr = match.group(1)
else:
rcpt_addr = recipient.strip()
self._send_line(f"RCPT TO:<{rcpt_addr}>")
code, message = self._read_response()
if code != 250:
raise Exception(f"RCPT TO rejected for {rcpt_addr} with code {code}: {message}")
def data(self, message_body: str) -> None:
"""
Send the DATA command and transmit the full message body.
The message_body should be a complete RFC 5322 formatted message,
including headers and body, with lines separated by CRLF.
"""
self._send_line("DATA")
code, message = self._read_response()
if code != 354:
raise Exception(f"DATA not accepted, got code {code}: {message}")
# Send the message body
# Ensure lines end with CRLF
lines = message_body.split("\n")
for line in lines:
# Handle dot-stuffing: if a line starts with '.', prepend another '.'
if line.startswith("."):
line = "." + line
self.sock.sendall((line + "\r\n").encode("ascii", errors="replace"))
# Send the termination sequence: a single dot on a line by itself
self._send_line(".")
code, message = self._read_response()
if code != 250:
raise Exception(f"Message transmission failed with code {code}: {message}")
print("Message accepted for delivery.")
def quit(self) -> None:
"""Gracefully terminate the SMTP session."""
self._send_line("QUIT")
code, message = self._read_response()
if code != 221:
# Non-fatal: the server is closing anyway
print(f"QUIT response: {code} {message}")
if self.sock:
self.sock.close()
def close(self) -> None:
"""Close the connection without QUIT (abort)."""
if self.sock:
self.sock.close()This foundational class handles the low-level socket communication, response parsing (including multi-line responses where lines are separated by hyphens), and the basic command-response cycle. The _read_response method is particularly important: it correctly handles the SMTP convention where multi-line responses use a hyphen after the status code on all lines except the last, which uses a space.
Parsing Server Responses
Understanding how to parse SMTP responses correctly is essential. A multi-line EHLO response looks like this:
250-smtp.example.com Hello client [192.168.1.1]
250-SIZE 52428800
250-8BITMIME
250-STARTTLS
250 AUTH LOGIN PLAINNotice the hyphens on lines 1-4 and the space on line 5. The _read_response method in our implementation detects this pattern and returns the complete set of response lines as a single message string, from which we can extract capability keywords.
Implementing the Full SMTP Dialogue
Now we can assemble a complete email sending function that orchestrates the entire SMTP conversation. This function handles connection, capability discovery, TLS upgrade, authentication, envelope setup, and message transmission:
def send_email_smtp(
smtp_server: str,
port: int,
username: str,
password: str,
sender: str,
recipients: List[str],
subject: str,
body_html: str,
body_plain: Optional[str] = None,
use_tls: bool = True,
auth_method: str = "LOGIN"
) -> bool:
"""
Send an email via SMTP with full protocol control.
Args:
smtp_server: SMTP server hostname (e.g., 'smtp.gmail.com')
port: Port number (587 for submission, 465 for SMTPS)
username: SMTP authentication username
password: SMTP authentication password
sender: Envelope sender address
recipients: List of envelope recipient addresses
subject: Email subject line
body_html: HTML body content
body_plain: Plain text fallback (generated automatically if None)
use_tls: Whether to upgrade to TLS via STARTTLS
auth_method: 'LOGIN' or 'PLAIN'
Returns:
True if the email was accepted for delivery
"""
client = SMTPClient(smtp_server, port, timeout=30)
try:
# Phase 1: Connect and read greeting
print("Connecting to server...")
code, greeting = client.connect()
print(f"Server greeting: [{code}] {greeting.strip()}")
# Phase 2: EHLO to get capabilities
print("Sending EHLO...")
capabilities = client.ehlo("mydomain.com")
print(f"Server capabilities: {capabilities}")
# Phase 3: Upgrade to TLS if available and requested
if use_tls and "STARTTLS" in [c.upper() for c in capabilities]:
print("Upgrading connection to TLS...")
client.starttls()
# After TLS, re-issue EHLO (required by RFC)
capabilities = client.ehlo("mydomain.com")
print(f"Post-TLS capabilities: {capabilities}")
# Phase 4: Authenticate
print(f"Authenticating using AUTH {auth_method}...")
if auth_method.upper() == "LOGIN":
client.auth_login(username, password)
elif auth_method.upper() == "PLAIN":
client.auth_plain(username, password)
else:
raise ValueError(f"Unsupported auth method: {auth_method}")
# Phase 5: Envelope
print(f"Setting envelope sender: {sender}")
client.mail_from(sender)
for recipient in recipients:
print(f"Adding recipient: {recipient}")
client.rcpt_to(recipient)
# Phase 6: Construct the message
message = build_mime_message(sender, recipients, subject,
body_html, body_plain)
# Phase 7: Transmit via DATA
print("Transmitting message body...")
client.data(message)
print("Email accepted by server.")
return True
except Exception as e:
print(f"Error during SMTP transaction: {e}")
raise
finally:
print("Closing connection...")
client.quit()Adding MIME Support for Rich Emails
Modern emails contain HTML, plain text alternatives, and potentially attachments. To transmit these, we must construct a properly formatted MIME message. The following function builds a complete RFC 5322 message with multipart/alternative structure and proper headers:
import uuid
from datetime import datetime
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from email.mime.base import MIMEBase
from email import encoders
from email.utils import formataddr
import os
def build_mime_message(
sender: str,
recipients: List[str],
subject: str,
body_html: str,
body_plain: Optional[str] = None,
attachments: Optional[List[str]] = None
) -> str:
"""
Build a complete MIME email message ready for SMTP DATA transmission.
Returns the message as a string with proper CRLF line endings.
"""
# Create the top-level MIME container
msg = MIMEMultipart("alternative")
# Generate a unique Message-ID
msg_id = f"<{uuid.uuid4().hex}@{sender.split('@')[-1]}>"
# Add essential headers
msg["Message-ID"] = msg_id
msg["Date"] = datetime.now().strftime("%a, %d %b %Y %H:%M:%S %z")
msg["From"] = formataddr(("Sender Name", sender)) if " " not in sender else sender
msg["To"] = ", ".join(recipients)
msg["Subject"] = subject
msg["MIME-Version"] = "1.0"
# Add plain text part (required for deliverability)
if body_plain is None:
# Simple HTML-to-text stripping (production would use html2text or similar)
body_plain = body_html.replace("
", "\n").replace("", "\n\n")
body_plain = re.sub(r'<[^>]+>', '', body_plain)
body_plain = body_plain.strip()
plain_part = MIMEText(body_plain, "plain", "utf-8")
msg.attach(plain_part)
# Add HTML part
html_part = MIMEText(body_html, "html", "utf-8")
msg.attach(html_part)
# Handle attachments if provided
if attachments:
for filepath in attachments:
if not os.path.exists(filepath):
print(f"Warning: Attachment {filepath} not found, skipping.")
continue
filename = os.path.basename(filepath)
with open(filepath, "rb") as f:
data = f.read()
# Determine content type based on extension
content_type = _guess_mime_type(filename)
main_type, sub_type = content_type.split("/", 1)
attachment_part = MIMEBase(main_type, sub_type)
attachment_part.set_payload(data)
encoders.encode_base64(attachment_part)
attachment_part.add_header(
"Content-Disposition",
"attachment",
filename=filename
)
msg.attach(attachment_part)
# Convert the message to a string with proper CRLF endings
# Python's email library uses native line endings; we convert to CRLF
raw_message = msg.as_string()
# Replace lone \n with \r\n, but avoid doubling existing \r\n
raw_message = raw_message.replace("\r\n", "\n").replace("\n", "\r\n")
return raw_message
def _guess_mime_type(filename: str) -> str:
"""Simple MIME type guesser based on file extension."""
ext = os.path.splitext(filename)[1].lower()
mime_map = {
".pdf": "application/pdf",
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".gif": "image/gif",
".txt": "text/plain",
".html": "text/html",
".csv": "text/csv",
".zip": "application/zip",
".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
".xlsx": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
}
return mime_map.get(ext, "application/octet-stream")This MIME builder creates a proper multipart/alternative structure where the plain text part provides a fallback for text-only clients, and the HTML part delivers the rich experience. When attachments are included, the structure becomes multipart/mixed containing the alternative parts plus the attachment parts, each Base64-encoded with appropriate content-type headers.
Adding TLS Security
Security is non-negotiable in modern SMTP. Our client already supports STARTTLS via the starttls() method. However, for servers that expect implicit TLS on port 465 (SMTPS), we need a different approach — wrapping the socket with TLS before any SMTP communication. Here is how to extend our client to support both modes:
def send_via_implicit_tls(
smtp_server: str,
username: str,
password: str,
sender: str,
recipients: List[str],
subject: str,
body_html: str,
body_plain: Optional[str] = None
) -> bool:
"""
Send email via implicit TLS (SMTPS on port 465).
In this mode, TLS is established before any SMTP commands.
"""
# Create a socket and wrap it with TLS immediately
raw_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
raw_sock.settimeout(30)
ssl_context = ssl.create_default_context()
ssl_sock = ssl_context.wrap_socket(raw_sock, server_hostname=smtp_server)
# Connect through the SSL-wrapped socket
ssl_sock.connect((smtp_server, 465))
# Now create our SMTPClient but replace its socket with the TLS socket
client = SMTPClient.__new__(SMTPClient)
client.server = smtp_server
client.port = 465
client.timeout = 30
client.sock = ssl_sock
client.ssl_context = ssl_context
client._read_buffer = b""
try:
# Read greeting (already over TLS)
code, greeting = client._read_response()
print(f"TLS greeting: [{code}] {greeting.strip()}")
# EHLO
capabilities = client.ehlo("mydomain.com")
print(f"Capabilities: {capabilities}")
# Authenticate (PLAIN often preferred on implicit TLS)
client.auth_plain(username, password)
# Envelope
client.mail_from(sender)
for rcpt in recipients:
client.rcpt_to(rcpt)
# Message
message = build_mime_message(sender, recipients, subject,
body_html, body_plain)
client.data(message)
print("Email sent via implicit TLS.")
return True
finally:
client.quit()Note the critical difference: in explicit TLS (STARTTLS on port 587), the connection begins in plaintext and is upgraded after capabilities are exchanged. In implicit TLS (port 465), the TLS handshake completes before any SMTP bytes are transmitted. Both methods are valid, but STARTTLS on port 587 is the IETF-recommended approach for message submission.
Best Practices for SMTP Implementation
Authentication and Security
When implementing SMTP in production, security must be your top priority:
- Always use TLS: Never transmit credentials or message content over plaintext channels. Modern SMTP servers may reject authentication attempts on unencrypted connections outright.
- Validate certificates: The default SSL context created by
ssl.create_default_context()performs proper certificate chain validation. Do not disable this withssl._create_unverified_context()in production code — doing so opens the door to man-in-the-middle attacks. - Prefer OAuth 2.0 when available: For providers like Gmail and Microsoft 365, use OAuth 2.0 tokens with the XOAUTH2 SASL mechanism instead of storing plaintext passwords. This limits credential exposure and allows scoped access.
- Sanitize all inputs: Email headers, especially
From,To, andSubject, must be properly encoded if they contain non-ASCII characters. Useemail.header.Headeror theformataddrfunction to handle internationalized display names correctly. - Implement SPF, DKIM, and DMARC awareness: While these are server-side concerns, your client should use a sender domain that has these records properly configured. Messages from domains without SPF records are increasingly likely to be rejected or marked as spam.
Error Handling and Retries
SMTP servers can and do fail. A robust implementation handles errors gracefully:
import time
from typing import Callable
def send_with_retry(
send_func: Callable,
max_retries: int = 3,
backoff_factor: float = 2.0,
retryable_codes: Tuple[int, ...] = (421, 450, 451, 452)
) -> bool:
"""
Execute an SMTP send function with exponential backoff retry logic.
Args:
send_func: A callable that performs the SMTP send (returns True on success)
max_retries: Maximum number of retry attempts
backoff_factor: Multiplier for delay between retries (delay = factor^attempt)
retryable_codes: SMTP codes that indicate temporary failures worth retrying
Returns:
True if successful, raises last exception if all retries exhausted
"""
last_exception = None
for attempt in range(max_retries + 1):
try:
result = send_func()
if result:
return True
except Exception as e:
last_exception = e
# Check if the error is retryable
error_str = str(e)
is_retryable = any(f"code {code}" in error_str for code in retryable_codes)
if not is_retryable and attempt > 0:
# Permanent failure — don't retry
raise
if attempt < max_retries:
delay = backoff_factor ** attempt
print(f"Attempt {attempt + 1} failed: {e}")
print(f"Retrying in {delay:.1f} seconds...")
time.sleep(delay)
else:
raise last_exception
raise last_exceptionKey retryable SMTP error codes include: 421 (service not available, closing transmission channel — often due to rate limiting), 450 (mailbox busy), 451 (local error in processing — temporary), and 452 (insufficient system storage). Permanent failures like 550 (mailbox unavailable) or 535 (authentication failed) should never be retried.
Respecting Rate Limits and Batching
When sending bulk emails, implement proper throttling and connection reuse:
class Bulk