LangChain Architecture: Design Patterns and Project Structure

What Is LangChain Architecture?

LangChain is a framework that simplifies building applications powered by large language models (LLMs). Its architecture is not a monolithic engine but a modular, composable set of abstractions that developers connect together to form pipelines, agents, and advanced reasoning systems. The core components include:

• Models – Wrappers around LLM providers (OpenAI, Anthropic, local models) and chat models, exposing uniform interfaces.
• Prompts – Templates and prompt management (PromptTemplate, ChatPromptTemplate) with input validation and serialisation.
• Chains – Ordered sequences of calls (to models, tools, or other chains) that form the backbone of any LangChain application.
• Indexes / Retrievers – Utilities for loading, splitting, embedding, and retrieving documents, enabling RAG (Retrieval-Augmented Generation).
• Memory – Stateful mechanisms (buffers, summaries) that allow chains and agents to remember previous interactions.
• Agents – Dynamic, decision-making components that use an LLM to choose a sequence of actions (tools) to take.
• Callbacks – A hook system for logging, streaming, and monitoring every step in a chain execution.

These components are glued together by well-known design patterns that solve recurring problems in LLM application development. Understanding these patterns and how to structure a project around them is critical for building maintainable, scalable, and testable AI applications.

Why Design Patterns Matter

Working with LLMs introduces unique challenges: non-deterministic outputs, context window limits, latency, and the need to combine multiple model calls with external tools and data. Simply throwing a prompt at an LLM is rarely enough for production systems. Design patterns provide:

• Reusability – Proven recipes that can be adapted across different domains (e.g., customer support, data analysis).
• Reliability – Structured error handling, fallbacks, and validation that reduce unexpected failures.
• Separation of concerns – Clear boundaries between reasoning, data access, and tool execution, making code easier to test.
• Observability – Patterns naturally expose intermediate steps, simplifying logging and debugging via callbacks.

LangChain’s design is centred around these patterns. The framework provides ready-made classes that implement them, but the real power lies in understanding how to compose and customise them for your own use cases.

Core Design Patterns in LangChain

The Chain Pattern

The simplest and most fundamental pattern is the Chain. It encapsulates a sequence of operations where the output of one step feeds into the input of the next. LangChain provides several chain types: LLMChain (a single prompt + model call), SequentialChain (multiple steps), and TransformChain (pure Python transformations).

A classic example is a two-step chain that first summarises a document, then translates the summary into another language.

from langchain.chains import LLMChain, SequentialChain
from langchain.prompts import PromptTemplate
from langchain_openai import ChatOpenAI

# Define two prompts
summarise_prompt = PromptTemplate(
    input_variables=["document"],
    template="Summarise the following document in 3 sentences:\n\n{document}"
)
translate_prompt = PromptTemplate(
    input_variables=["summary"],
    template="Translate the following summary to French:\n\n{summary}"
)

# Create two LLMChains
llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
summarise_chain = LLMChain(llm=llm, prompt=summarise_prompt, output_key="summary")
translate_chain = LLMChain(llm=llm, prompt=translate_prompt, output_key="french_summary")

# Compose into a SequentialChain
pipeline = SequentialChain(
    chains=[summarise_chain, translate_chain],
    input_variables=["document"],
    output_variables=["summary", "french_summary"],
    verbose=True
)

# Run
result = pipeline.invoke({"document": "LangChain is a framework for building LLM applications..."})
print(result["french_summary"])

The Router Pattern

When a single prompt cannot handle all intents, the Router pattern dynamically selects the most appropriate chain based on the input. LangChain offers LLMRouterChain, MultiPromptChain, and MultiRouteChain for this purpose. The router uses an LLM call to classify the input and pick the next step.

from langchain.chains.router import MultiPromptChain
from langchain.chains.router.llm_router import LLMRouterChain, RouterOutputParser
from langchain.prompts import PromptTemplate
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)

# Define specialised prompts
billing_prompt = PromptTemplate(
    input_variables=["input"],
    template="You are a billing specialist. Help the user with their question:\n{input}"
)
technical_prompt = PromptTemplate(
    input_variables=["input"],
    template="You are a technical support engineer. Answer the user's technical question:\n{input}"
)

# Router prompt and parsing
router_template = """
Given the user input, choose the most appropriate destination: 'billing' or 'technical'.
Input: {input}
Destination:
"""
router_prompt = PromptTemplate(
    template=router_template,
    input_variables=["input"],
    output_parser=RouterOutputParser()
)
router_chain = LLMRouterChain.from_llm(llm, router_prompt)

# Build the multi-prompt chain
multi_chain = MultiPromptChain(
    router_chain=router_chain,
    destination_chains={"billing": LLMChain(llm=llm, prompt=billing_prompt),
                        "technical": LLMChain(llm=llm, prompt=technical_prompt)},
    default_chain=LLMChain(llm=llm, prompt=PromptTemplate(template="{input}", input_variables=["input"])),
    verbose=True
)

query = "I was charged twice last month, can you help?"
print(multi_chain.invoke(query))

The ReAct (Reason + Act) Pattern

The ReAct pattern interleaves reasoning and action. The agent thinks step by step, decides which tool to use, observes the result, and repeats until it can answer the user. LangChain’s AgentExecutor implements this pattern, combining an LLM, a set of tools, and an agent scratchpad.

from langchain.agents import AgentExecutor, create_react_agent
from langchain.tools import Tool
from langchain_openai import ChatOpenAI
from langchain.prompts import PromptTemplate
from langchain import hub

llm = ChatOpenAI(model="gpt-4", temperature=0)

# Define custom tools (e.g., a calculator and a search)
def calculator(expression: str) -> str:
    """Evaluates a mathematical expression."""
    return str(eval(expression))

def search(query: str) -> str:
    """Simulates a search engine."""
    # In production, replace with a real search API
    return f"Results for '{query}': [some relevant information...]"

tools = [
    Tool(name="Calculator", func=calculator, description="Useful for math calculations"),
    Tool(name="Search", func=search, description="Useful for finding factual information")
]

# Get the ReAct prompt from the hub (or define your own)
prompt = hub.pull("hwchase17/react")

# Create the agent
agent = create_react_agent(llm, tools, prompt)

# Wrap in an executor
executor = AgentExecutor(agent=agent, tools=tools, verbose=True, handle_parsing_errors=True)

response = executor.invoke({"input": "What is the square root of 256, and who is the current US president?"})
print(response["output"])

The Retrieval-Augmented Generation (RAG) Pattern

RAG combines a retriever that fetches relevant documents with a generator that crafts an answer based on that context. LangChain provides RetrievalQA, ConversationalRetrievalChain, and many helper classes for document loading, splitting, and embedding.

from langchain.chains import RetrievalQA
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain.vectorstores import Chroma
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.document_loaders import TextLoader

# Load and split documents
loader = TextLoader("knowledge_base.txt")
documents = loader.load()
splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
docs = splitter.split_documents(documents)

# Create vector store and retriever
embeddings = OpenAIEmbeddings()
vectorstore = Chroma.from_documents(docs, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

# Build the QA chain
llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0)
qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",  # or "map_reduce", "refine"
    retriever=retriever,
    return_source_documents=True,
    verbose=True
)

query = "What is the refund policy?"
result = qa_chain.invoke({"query": query})
print(result["result"])
for doc in result["source_documents"]:
    print("Source:", doc.metadata.get("source"))

The Memory Pattern

Most conversational systems need to remember previous turns. LangChain’s Memory classes (ConversationBufferMemory, ConversationSummaryMemory, etc.) store history and inject it into prompts. They can be attached to chains or agents seamlessly.

from langchain.chains import ConversationChain
from langchain.memory import ConversationBufferMemory
from langchain_openai import ChatOpenAI
from langchain.prompts import ChatPromptTemplate, MessagesPlaceholder

llm = ChatOpenAI(model="gpt-3.5-turbo", temperature=0.7)

# Create memory
memory = ConversationBufferMemory(return_messages=True)

# Build a prompt that includes a placeholder for history
prompt = ChatPromptTemplate.from_messages([
    ("system", "You are a friendly assistant."),
    MessagesPlaceholder(variable_name="history"),
    ("human", "{input}")
])

# ConversationChain automatically manages memory
conversation = ConversationChain(
    llm=llm,
    prompt=prompt,
    memory=memory,
    verbose=True
)

conversation.predict(input="Hi, my name is Alice.")
conversation.predict(input="What's my name?")
# Output will include "Alice" because memory retains the context

Project Structure Best Practices

As your LangChain application grows, a well-organised project structure becomes essential. Below is a recommended layout that separates concerns, promotes reusability, and simplifies testing.

Directory Layout

my_langchain_app/
├── config/
│   ├── settings.yaml          # LLM providers, API keys, chunk sizes, etc.
│   └── prompts/
│       ├── __init__.py
│       └── system_prompts.py  # PromptTemplate definitions and versions
├── chains/
│   ├── __init__.py
│   ├── summarisation.py       # Factory functions returning chain instances
│   ├── qa.py
│   └── router.py
├── agents/
│   ├── __init__.py
│   ├── tools.py               # Custom tool definitions
│   └── agent.py               # Agent builder (e.g., create_my_agent)
├── indexes/
│   ├── __init__.py
│   ├── loader.py              # Document loading and preprocessing
│   └── retriever.py           # Vector store and retriever setup
├── memory/
│   ├── __init__.py
│   └── session_memory.py      # Memory factory (e.g., per-user sessions)
├── utils/
│   ├── callbacks.py           # Custom callback handlers for logging
│   └── logger.py              # Centralised logging configuration
├── main.py                    # Entry point (CLI, API, or demo script)
└── tests/
    ├── test_chains.py
    ├── test_agents.py
    └── conftest.py            # Shared fixtures (e.g., mock LLM responses)

Configuration Management

Never hardcode API keys or model names. Use environment variables and a configuration object (e.g., a pydantic.BaseSettings class) to load and validate settings. This makes it easy to switch between development, staging, and production environments.

# config/settings.py
from pydantic_settings import BaseSettings

class Settings(BaseSettings):
    openai_api_key: str
    model_name: str = "gpt-3.5-turbo"
    temperature: float = 0.0
    chunk_size: int = 500
    vector_store_path: str = "./chroma_db"

    class Config:
        env_file = ".env"

Chain Factory Functions

Instead of building chains directly in your main script, create factory functions in dedicated modules. They accept configuration and return fully configured chain objects. This improves testability and keeps your entry point clean.

# chains/qa.py
from langchain.chains import RetrievalQA
from langchain_openai import ChatOpenAI

def build_qa_chain(retriever, settings):
    llm = ChatOpenAI(
        model=settings.model_name,
        temperature=settings.temperature,
        openai_api_key=settings.openai_api_key
    )
    return RetrievalQA.from_chain_type(
        llm=llm,
        retriever=retriever,
        chain_type="stuff"
    )

Testing Strategies

Test individual components in isolation. Use mock LLM responses (LangChain provides FakeLLM and FakeChatModel) to avoid actual API calls during unit tests. Integration tests can hit a local vector store or a test-specific LLM endpoint. Keep prompts in version-controlled files to track changes and automatically validate their output schemas.

# tests/test_chains.py
from langchain.llms.fake import FakeListLLM
from my_langchain_app.chains.summarisation import build_summarisation_chain

def test_summarisation():
    fake_llm = FakeListLLM(responses=["This is a short summary."])
    chain = build_summarisation_chain(fake_llm)  # factory that accepts an LLM
    result = chain.invoke({"document": "Long text..."})
    assert "summary" in result
    assert result["summary"] == "This is a short summary."

How to Use These Patterns: A Step-by-Step Example

Let’s build a complete, memory-enabled document QA bot that combines RAG, memory, and a router. The system will:

• Load a knowledge base from PDFs
• Create a retriever for document search
• Route between a document-based QA chain and a general conversation chain (with memory)
• Maintain chat history across turns

# main.py
from config.settings import Settings
from indexes.loader import load_and_split_pdfs
from indexes.retriever import create_retriever
from chains.qa import build_qa_chain
from chains.router import build_router_chain
from memory.session_memory import get_or_create_memory
from langchain_openai import ChatOpenAI
from langchain.chains import ConversationChain

settings = Settings()

# 1. Load and index documents
docs = load_and_split_pdfs("data/", chunk_size=settings.chunk_size)
retriever = create_retriever(docs, settings.vector_store_path)

# 2. Build QA chain
qa_chain = build_qa_chain(retriever, settings)

# 3. Build a general conversation chain with memory
llm = ChatOpenAI(model=settings.model_name, temperature=0.7, openai_api_key=settings.openai_api_key)
memory = get_or_create_memory(session_id="user123")  # per-user memory
conversation_chain = ConversationChain(llm=llm, memory=memory, verbose=True)

# 4. Route between the two based on intent
router = build_router_chain(llm, qa_chain, conversation_chain)

# Interactive loop
while True:
    user_input = input("You: ")
    if user_input.lower() == "quit":
        break
    response = router.invoke({"input": user_input})
    print(f"Bot: {response['output']}")

The build_router_chain function might use LLMRouterChain to classify whether the question needs document retrieval ("qa") or is a casual chat ("conversation"). This composition demonstrates how patterns work together: RAG for knowledge, Memory for continuity, and Router for orchestration.

Best Practices Summary

• Keep chains focused – Each chain should do one thing well. Compose complex behaviour from simple chains.
• Use Pydantic for prompts – Validate input variables and output schemas to catch errors early.
• Leverage callbacks – Attach custom handlers to log token usage, latencies, or to stream intermediate results.
• Separate concerns – Place document loading, embedding, and chain building in distinct modules.
• Prefer async where possible – Use ainvoke, astream to improve throughput in web services.
• Handle errors gracefully – Add fallback chains, retry logic, and parsing error handlers to agents.
• Version your prompts – Store them in files or a registry and treat them like code (review, test, and roll back).
• Write tests – Unit-test chains with fake LLMs, integration-test retrieval with sample documents.

Conclusion

LangChain’s architecture is a rich ecosystem of modular components built around proven design patterns. By understanding the Chain, Router, ReAct, RAG, and Memory patterns, you can tackle almost any LLM-based task with confidence. Adopting a clean project structure with separated concerns, factory functions, and rigorous testing ensures your application remains maintainable as it grows. Start small, compose patterns incrementally, and always keep observability and error handling at the core of your design. With these foundations, you’re ready to build robust, production-grade AI applications.