Designing a Search Engine with Cassandra

Designing a Search Engine with Apache Cassandra

What It Is

Building a search engine on top of Apache Cassandra means leveraging Cassandra's distributed, highly available architecture to store and retrieve data through carefully crafted index patterns. Unlike traditional search solutions like Elasticsearch or Solr that use inverted indices based on term frequency and relevance scoring, a Cassandra-based search engine relies on denormalized data models, wide rows, and composite partition keys to enable fast lookups across massive datasets. The core idea is to pre-compute and store query results in the shape your application needs, effectively treating Cassandra as a distributed, persistent cache of searchable data organized by access patterns.

Cassandra is not a full-text search engine out of the box. It does not tokenize text, compute TF-IDF scores, or handle fuzzy matching natively. Instead, you design a search engine on Cassandra by building custom indexing structures — inverted indices, time-ordered event logs, prefix trees, and trigram indexes — all modeled as Cassandra tables. The result is a highly scalable, write-optimized search system that can handle millions of writes per second while maintaining predictable query latency through partition-based lookups.

Why It Matters

Modern applications generate staggering volumes of data — log events, user activities, IoT sensor readings, financial transactions — and they demand the ability to search through this data in real time. Deploying a separate search infrastructure introduces operational complexity: you must maintain two distributed systems, handle data synchronization, and manage failure modes across both. By building search capabilities directly into Cassandra, you get:

• Unified infrastructure — one database for both primary storage and search
• Linear scalability — add nodes to handle more queries and more data with no single point of failure
• Zero synchronization latency — writes go directly to the search indexes within the same batch or transaction (using logged batches)
• Operational simplicity — backups, repairs, and monitoring apply uniformly to all data
• Predictable performance — queries hit well-defined partition boundaries, avoiding full-table scans

This approach is particularly valuable for time-series search (finding events within a time window), tag-based search (filtering by multiple attributes), and prefix/autocomplete search where the query patterns are known in advance and can be modeled into Cassandra's partition structure.

Core Design Patterns

Pattern 1: Inverted Index with Wide Rows

The inverted index is the foundational pattern for building search on Cassandra. For each searchable term, you create a partition that contains all document IDs (or document payloads) containing that term. Since Cassandra partitions can grow to billions of cells, this scales naturally.

Schema design:

-- The primary document store
CREATE TABLE documents (
    doc_id uuid PRIMARY KEY,
    title text,
    body text,
    created_at timestamp
);

-- Inverted index: term -> list of matching doc_ids
CREATE TABLE term_index (
    term text,
    doc_id uuid,
    created_at timestamp,
    PRIMARY KEY (term, doc_id)
) WITH CLUSTERING ORDER BY (doc_id ASC);

-- Optional: store a snippet or relevance signal directly in the index
CREATE TABLE term_index_enriched (
    term text,
    doc_id uuid,
    title_snippet text,
    created_at timestamp,
    PRIMARY KEY (term, created_at, doc_id)
) WITH CLUSTERING ORDER BY (created_at DESC, doc_id ASC);

Inserting data:

-- Insert a document and populate the inverted index in a logged batch
BEGIN BATCH
    INSERT INTO documents (doc_id, title, body, created_at)
    VALUES (now(), 'Cassandra Performance Tuning', 'Optimizing read paths...', '2025-01-15 10:00:00');
    
    INSERT INTO term_index (term, doc_id, created_at)
    VALUES ('cassandra', now(), '2025-01-15 10:00:00');
    
    INSERT INTO term_index (term, doc_id, created_at)
    VALUES ('performance', now(), '2025-01-15 10:00:00');
    
    INSERT INTO term_index (term, doc_id, created_at)
    VALUES ('tuning', now(), '2025-01-15 10:00:00');
    
    INSERT INTO term_index (term, doc_id, created_at)
    VALUES ('optimizing', now(), '2025-01-15 10:00:00');
APPLY BATCH;

Querying the index:

-- Find all documents containing the term 'cassandra'
SELECT doc_id FROM term_index WHERE term = 'cassandra';

-- Multi-term intersection: retrieve matching doc_ids and intersect client-side
-- or use a materialized view approach (covered later)

For multi-term queries, you fetch the doc_id lists for each term from Cassandra and perform intersection (AND) or union (OR) in your application. Cassandra gives you sub-millisecond partition reads, so fetching 3–5 term partitions and intersecting them in memory is extremely fast even for millions of documents, provided each term partition is not excessively large. For very popular terms (the "stop word" problem), consider bucketing.

Pattern 2: Time-Partitioned Event Search

When searching through time-series data — logs, metrics, financial ticks — the primary access pattern is almost always a time range. Cassandra's composite partition keys allow you to partition by time bucket (e.g., per hour or per day) and cluster by timestamp for ordered retrieval.

-- Time-series search table with tag filtering
CREATE TABLE events_by_time (
    bucket text,           -- e.g., '2025-01-15-14' for hourly bucket
    event_time timestamp,
    event_id uuid,
    event_type text,
    severity text,
    payload text,
    PRIMARY KEY ((bucket), event_time, event_id)
) WITH CLUSTERING ORDER BY (event_time DESC, event_id ASC);

Writing events:

-- Derive bucket from the event timestamp
INSERT INTO events_by_time (
    bucket, event_time, event_id, event_type, severity, payload
) VALUES (
    '2025-01-15-14',
    '2025-01-15 14:23:45',
    now(),
    'ERROR',
    'CRITICAL',
    'OutOfMemoryError in node-7'
);

Querying a time window:

-- Fetch all events between 14:00 and 14:30 on Jan 15, 2025
SELECT event_time, event_id, event_type, severity, payload
FROM events_by_time
WHERE bucket = '2025-01-15-14'
  AND event_time >= '2025-01-15 14:00:00'
  AND event_time <= '2025-01-15 14:30:00'
ORDER BY event_time DESC
LIMIT 100;

To search across multiple buckets (e.g., a 6-hour window), issue parallel queries for each bucket and merge results. Cassandra's driver-side token-aware routing ensures each query hits the optimal replica.

Pattern 3: Tag-Based Search with Composite Partitions

Many search use cases involve filtering by multiple discrete attributes — tags, categories, statuses. You can model this by building a table with a composite partition key that includes all filterable dimensions, or by creating secondary index tables that group entities by their attribute combinations.

-- Product catalog searchable by category and brand
CREATE TABLE products_by_category_brand (
    category text,
    brand text,
    product_id uuid,
    name text,
    price decimal,
    stock int,
    PRIMARY KEY ((category, brand), product_id)
);

-- Query: all products in 'electronics' by 'apple'
SELECT * FROM products_by_category_brand
WHERE category = 'electronics' AND brand = 'apple';

For multi-attribute search where any combination of filters may be applied, you can create a dedicated search table that uses a composite partition key with a "shard" component to allow partial key queries (though Cassandra requires the full partition key for efficient queries). The practical solution is to maintain multiple index tables for common filter combinations and fall back to a primary index with client-side filtering for rare combinations.

-- A flexible search table with a generated filter signature
CREATE TABLE products_filtered (
    filter_signature text,   -- e.g., 'category:electronics|brand:apple'
    updated_at timestamp,
    product_id uuid,
    name text,
    price decimal,
    PRIMARY KEY (filter_signature, updated_at, product_id)
) WITH CLUSTERING ORDER BY (updated_at DESC);

-- When inserting a product, compute all common filter signatures
INSERT INTO products_filtered (filter_signature, updated_at, product_id, name, price)
VALUES ('category:electronics', '2025-01-15 10:00:00', now(), 'MacBook Pro', 2499.00);

INSERT INTO products_filtered (filter_signature, updated_at, product_id, name, price)
VALUES ('brand:apple', '2025-01-15 10:00:00', now(), 'MacBook Pro', 2499.00);

INSERT INTO products_filtered (filter_signature, updated_at, product_id, name, price)
VALUES ('category:electronics|brand:apple', '2025-01-15 10:00:00', now(), 'MacBook Pro', 2499.00);

The application selects the most specific filter_signature that matches the user's query, retrieves that partition, and applies any remaining filters in memory. This gives you sub-10ms P99 latency for most queries while maintaining write scalability.

Pattern 4: Prefix Search and Autocomplete

Implementing autocomplete (type-ahead suggestions) on Cassandra requires modeling all prefixes of searchable tokens. For each token you want to suggest, insert rows for every prefix, and use a clustering key to order suggestions by frequency or recency.

-- Autocomplete suggestions table
CREATE TABLE suggestions (
    prefix text,
    frequency int,
    token text,
    PRIMARY KEY (prefix, frequency, token)
) WITH CLUSTERING ORDER BY (frequency DESC, token ASC);

-- Populate prefixes for the token 'cassandra'
-- Insert rows for prefixes: 'c', 'ca', 'cas', 'cass', 'cassa', ...
INSERT INTO suggestions (prefix, frequency, token) VALUES ('c', 1000, 'cassandra');
INSERT INTO suggestions (prefix, frequency, token) VALUES ('ca', 1000, 'cassandra');
INSERT INTO suggestions (prefix, frequency, token) VALUES ('cas', 1000, 'cassandra');
INSERT INTO suggestions (prefix, frequency, token) VALUES ('cass', 1000, 'cassandra');
INSERT INTO suggestions (prefix, frequency, token) VALUES ('cassa', 1000, 'cassandra');
-- Continue for all prefixes...

-- Query: top 10 suggestions for prefix 'cas'
SELECT token, frequency FROM suggestions
WHERE prefix = 'cas'
ORDER BY frequency DESC
LIMIT 10;

This pattern creates significant write amplification (a token of length N generates N writes), but Cassandra handles this beautifully with its log-structured storage and memtable flushes. The reads are single-partition, extremely fast, and scale linearly with cluster size.

Pattern 5: Full-Text Search Approximation with Trigram Indexing

For substring search and fuzzy matching, you can implement trigram indexing. Break text into overlapping 3-character sequences and index each trigram. At query time, break the query string into trigrams, fetch matching document partitions, and intersect the results.

-- Trigram index table
CREATE TABLE trigram_index (
    trigram text,
    doc_id uuid,
    title text,
    PRIMARY KEY (trigram, doc_id)
);

-- Function to generate trigrams (implemented in application code)
-- For 'hello': generate ['hel', 'ell', 'llo'] and insert each

-- Example inserts for document containing 'hello'
INSERT INTO trigram_index (trigram, doc_id, title) VALUES ('hel', doc_id, 'Greeting Guide');
INSERT INTO trigram_index (trigram, doc_id, title) VALUES ('ell', doc_id, 'Greeting Guide');
INSERT INTO trigram_index (trigram, doc_id, title) VALUES ('llo', doc_id, 'Greeting Guide');

-- Search for documents containing substring 'ell'
-- Break 'ell' into trigrams: ['ell'] (since it's exactly 3 chars)
SELECT doc_id, title FROM trigram_index WHERE trigram = 'ell';

For longer query strings, generate trigrams, fetch doc_id sets for each, and compute the intersection client-side. This approximates full-text search with good recall for substring matching, though it lacks linguistic awareness (stemming, synonym expansion). You can layer a lightweight application-side tokenizer to improve matching quality.

Handling Hot Partitions and Term Skew

In any search system, some terms are vastly more popular than others. A term like "error" in a log search system might appear in millions of documents, creating a hot partition. Cassandra's solution is partition bucketing — appending a shard number to the partition key.

-- Bucketed inverted index to distribute hot terms
CREATE TABLE bucketed_term_index (
    term text,
    bucket int,           -- 0 to N-1, where N is a configurable shard count
    doc_id uuid,
    created_at timestamp,
    PRIMARY KEY ((term, bucket), doc_id)
);

-- Write: choose a bucket randomly (or by hash of doc_id % N)
INSERT INTO bucketed_term_index (term, bucket, doc_id, created_at)
VALUES ('error', 7, now(), '2025-01-15 10:00:00');

-- Read: query ALL buckets for the term in parallel and merge
-- For N=10 buckets, issue 10 queries, one for each (term, 0..9)
SELECT doc_id FROM bucketed_term_index WHERE term = 'error' AND bucket = 0;
SELECT doc_id FROM bucketed_term_index WHERE term = 'error' AND bucket = 1;
-- ... up to bucket 9

This distributes the load across multiple partitions at the cost of N queries per term. Choose N based on the expected popularity of the term — 10 for common terms, 100 for extremely hot terms. The driver's asynchronous query capabilities make this pattern efficient.

Data Consistency and Batch Operations

Maintaining consistency between your primary document table and search indexes is critical. Use logged batches for atomicity across multiple tables when the tables share the same partition (or at least the same coordinator). For cross-partition atomicity, accept eventual consistency and implement a repair mechanism.

-- Logged batch ensures atomic writes across tables
BEGIN BATCH
    INSERT INTO documents (doc_id, title, body, created_at)
    VALUES (d1a2b3c4-...uuid..., 'My Article', 'Content here...', toTimestamp(now()));
    
    INSERT INTO term_index (term, doc_id, created_at) VALUES ('article', d1a2b3c4-..., toTimestamp(now()));
    INSERT INTO term_index (term, doc_id, created_at) VALUES ('content', d1a2b3c4-..., toTimestamp(now()));
APPLY BATCH;

For bulk re-indexing or repair, use unlogged batches with caution — they are not atomic but offer higher throughput. Always implement an idempotent reconciliation process that can be safely re-run.

Compaction and Tombstone Management

Search indexes on Cassandra experience frequent updates and deletes as documents are modified or removed. This generates tombstones — deletion markers that must be retained until GCGraceSeconds expires (default 10 days). Excessive tombstones slow down reads and increase disk usage. Mitigation strategies:

• Use Time-To-Live (TTL) for index entries that have a natural expiration, avoiding explicit deletes
• Set appropriate GCGraceSeconds — lower it for tables where you do frequent deletes and can afford losing a delete marker during repair
• Use TWCS (Time Window Compaction Strategy) for time-series search tables to naturally expire old data
• Avoid deleting individual cells — instead, delete entire partitions when the document is removed

-- Create an index table with TTL so entries auto-expire
CREATE TABLE ephemeral_term_index (
    term text,
    doc_id uuid,
    PRIMARY KEY (term, doc_id)
) WITH default_time_to_live = 86400;  -- 24 hours in seconds

-- Insert with row-level TTL override
INSERT INTO ephemeral_term_index (term, doc_id)
VALUES ('temporary-tag', d1a2b3c4-...)
USING TTL 3600;  -- expires in 1 hour

Production Deployment Considerations

Sizing partitions: Keep partitions under 100MB for optimal performance. For inverted indexes, monitor term frequency — if a single term generates more than 100MB of data, implement bucketing. Use nodetool cfstats to inspect partition sizes.

Read repair and consistency levels: For search queries, use LOCAL_QUORUM for production to balance consistency and availability. For background indexing jobs, LOCAL_ONE may be acceptable if you have a reconciliation process.

-- Configuring consistency in your application (Java driver example)
// Set statement-level consistency
SimpleStatement query = SimpleStatement.builder(
    "SELECT doc_id FROM term_index WHERE term = ?"
).addPositionalValue("cassandra")
 .setConsistencyLevel(DefaultConsistencyLevel.LOCAL_QUORUM)
 .build();

Monitoring: Track read/write latencies per table, tombstone counts, and partition sizes. Set alerts for partitions exceeding size thresholds. Cassandra's virtual tables (system_views.tombstones) provide visibility into tombstone accumulation.

Best Practices Summary

• Model queries first, then schema — design tables to answer specific search queries with a single partition read
• Denormalize aggressively — storage is cheap; multiple queries or complex joins are expensive
• Pre-compute filter signatures — for faceted search, materialize common filter combinations as partition keys
• Use asynchronous driver APIs — issue parallel partition reads and merge results client-side
• Monitor and bucket hot terms — a single overloaded partition degrades the entire search experience
• Leverage TTL for ephemeral data — auto-expiring indexes reduce operational burden
• Test with production-grade data volumes — simulate millions of documents and thousands of queries per second to validate your partition design
• Keep a reconciliation pipeline — eventual consistency in cross-partition batches means you need a periodic repair job to ensure index integrity

Advanced: Hybrid Search with Lucene-Indexed Cassandra

For use cases requiring true full-text search with relevance ranking, stemming, and phrase queries, consider integrating Stratio's Cassandra Lucene Index plugin. This embeds a Lucene index directly within Cassandra, mapping each Cassandra partition to a Lucene document. It supports:

• Full-text search with Boolean queries, fuzzy matching, and range queries
• Relevance scoring (TF-IDF, BM25)
• Faceted search and spatial queries
• Automatic index updates on writes

-- Creating a Lucene index on a Cassandra table (using Stratio plugin)
CREATE CUSTOM INDEX documents_lucene_idx ON documents ()
USING 'com.stratio.cassandra.lucene.Index'
WITH OPTIONS = {
    'schema': '{
        fields: {
            title: {type: "text", analyzer: "english"},
            body: {type: "text", analyzer: "english"},
            created_at: {type: "date"}
        }
    }'
};

-- Querying with Lucene syntax
SELECT * FROM documents
WHERE expr(documents_lucene_idx, '{
    query: {
        type: "boolean",
        must: [{type: "match", field: "body", value: "Cassandra performance"}],
        filter: [{type: "range", field: "created_at", from: "2025-01-01", to: "2025-12-31"}]
    }
}')
LIMIT 50;

This hybrid approach gives you the best of both worlds: Cassandra's distributed, fault-tolerant storage with Lucene's mature text analysis and relevance ranking. The trade-off is additional memory overhead per node and slower write performance due to Lucene indexing.

Conclusion

Designing a search engine with Apache Cassandra is fundamentally about embracing denormalization, pre-computing access patterns, and structuring data to match the exact queries your application needs. Cassandra excels as a search platform when you can partition your search space by time, term, or filter signature — turning what would be a scatter-gather scan in a relational database into a collection of highly efficient, bounded partition reads. By combining inverted indices, time-bucketed event storage, prefix trees, and trigram indexes, you can build a search system that handles billions of documents, sustains millions of writes per second, and delivers sub-millisecond query responses — all within a single, operationally simple distributed database. The patterns described here form a proven foundation that scales linearly with your cluster, giving you the confidence to treat Cassandra as a first-class search engine in your architecture.