Git Partial Clone and Sparse Checkout

What Are Git Partial Clone and Sparse Checkout?

Partial Clone and Sparse Checkout are two distinct but complementary Git features designed to dramatically reduce the amount of data you fetch and store when working with large repositories. They solve different parts of the problem: Partial Clone limits the objects transferred over the network during clone and fetch operations, while Sparse Checkout limits the working tree to only the files you actually need.

Partial Clone was introduced in Git 2.19 and lets you clone with a filter that omits certain types of objects (usually large blobs) from the initial clone. The omitted objects are fetched lazily on demand when you check out a commit that needs them. This means your local repository starts much smaller and grows only as needed.

Sparse Checkout has been available in Git for years, but the modern git sparse-checkout command (standardised in Git 2.25) makes it easy to restrict the visible files in your working directory to a subset of the repository’s tree. Instead of checking out every file, you can instruct Git to only populate your working copy with specific directories or patterns.

Used together, you can clone a 50GB repository, download only commit and tree history (maybe a few hundred megabytes), and then checkout only the two folders your team works on. The result: a local clone that feels lightweight, fast to update, and easy to work with even on machines with limited disk space.

Key Concepts

• Partial Clone filters: blob:none, blob:limit=<size>, tree:0, and others.
• Lazy fetching / on-demand backfill: Missing objects are automatically fetched when a checkout or merge needs them.
• Promisor remotes: The remote from which the clone was made is marked as a "promisor" that can supply missing objects.
• Sparse-checkout patterns: Define which directories or paths are present in the working tree using git sparse-checkout set.
• Cone mode: A restricted pattern set that dramatically improves performance by limiting Git’s tree traversal to specified directories.

Why It Matters

Many modern codebases live in monorepos containing thousands of projects, enormous binary assets, or years of accumulated history. Cloning such repositories fully can take tens of minutes, consume gigabytes of disk, and waste bandwidth on files you’ll never touch.

CI/CD pipelines suffer when every job clones the entire repo just to build a single service. Developers with laptops face slow git fetch and painful disk usage. Partial Clone and Sparse Checkout together turn this from a heavyweight operation into a lean, targeted workflow.

• Reduced clone times: Skip downloading huge blobs you don’t need yet.
• Lower disk footprint: Only the blobs you actually use are stored locally; the working tree contains only the directories you specify.
• Faster CI builds: Clone a subset of history and checkout only the relevant path.
• Better developer experience: Quick git fetch and git status even in enormous repositories.

How to Use Git Partial Clone

Partial Clone is controlled by the --filter option during git clone (and also available in git fetch). The most common filter is blob:none, which omits all file blobs (the actual file contents) and only downloads commits and trees.

Cloning with a Filter

To create a partial clone that skips all blobs:

# Clone a repository without downloading any file contents initially
git clone --filter=blob:none https://github.com/example/large-repo.git
cd large-repo

This downloads every commit and tree object (the directory structure) but none of the file blobs. The clone will report a much smaller .git directory than a full clone.

Other Useful Filters

• blob:limit=1m — omit blobs larger than 1 megabyte. Smaller blobs are downloaded normally.
• blob:limit=10m — useful for skipping large binaries but retaining small source files immediately.
• tree:0 — omit all tree objects as well; only commit history is transferred. Extremely minimal, but any checkout will need to fetch trees on demand.

# Clone only commits, no trees and no blobs
git clone --filter=tree:0 https://github.com/example/large-repo.git

How Missing Objects Are Fetched

When you run a command that needs a missing blob (e.g., git checkout or git diff), Git automatically requests it from the remote. This is transparent; you might see a slight delay the first time, but the object is then stored locally permanently.

# Checkout a branch – missing blobs will be fetched on demand
git checkout main
# If a file's blob is missing, Git fetches it automatically.
# The command completes once the required objects arrive.

Checking Your Clone's Filter

To see which objects are missing and verify your filter settings:

# List the current partial clone filter for the remote
git remote get-url origin
git config remote.origin.partialclonefilter

# Show which objects are missing (promisor objects)
git rev-list --filter-print-objects --all

Updating a Partial Clone

Regular git fetch works normally and respects the filter. Only new commits and trees are fetched; blobs remain omitted until needed.

git fetch origin
# Fetches only the commit/tree objects that match the filter

How to Use Sparse Checkout

Sparse checkout controls which parts of the repository are actually present in your working directory. The modern way to set this up is with the git sparse-checkout command.

Initialising Sparse Checkout

You should always use cone mode unless you have a specific reason not to. Cone mode restricts patterns to directory-level matches and greatly speeds up Git operations.

# Enter sparse-checkout mode (cone mode recommended)
git sparse-checkout init --cone

# By default, the root directory is checked out.
# Now you can restrict the working tree to specific directories.

Setting Sparse Checkout Patterns

Use git sparse-checkout set to define the directories you want. Any directory not matching these patterns will be removed from the working tree (but remains in the index and can be restored).

# Checkout only the 'src' and 'docs' directories
git sparse-checkout set src docs

# Checkout only the 'frontend/webapp' subtree
git sparse-checkout set frontend/webapp

# Add more directories later
git sparse-checkout add tests

After running set or add, Git automatically updates your working tree. Files outside the specified patterns are deleted from disk but are still tracked by Git; you can bring them back by expanding the patterns.

Listing and Managing Patterns

# Show the current sparse-checkout patterns
git sparse-checkout list

# Reapply the sparse-checkout rules (useful after manual index changes)
git sparse-checkout reapply

# Disable sparse checkout entirely and restore a full working tree
git sparse-checkout disable

Older Method (Without cone mode)

Before Git 2.25, sparse checkout was configured via core.sparseCheckout and a file .git/info/sparse-checkout. The modern command is strongly preferred, but the legacy method is still supported. Avoid mixing them.

Combining Partial Clone with Sparse Checkout

The real power emerges when you use both techniques together. The typical workflow is:

1. Clone with a filter to avoid downloading blobs.
2. Use --no-checkout to prevent Git from immediately checking out any files (which would trigger blob fetches).
3. Initialise sparse checkout and set the directories you actually need.
4. Perform the initial checkout, which will now only populate the selected directories and fetch only the blobs for those directories.

Step-by-Step Example

Assume a huge monorepo at https://github.com/example/monorepo. You only need the services/payments and libs/common folders.

# 1. Partial clone with no blobs and no initial checkout
git clone --filter=blob:none --no-checkout https://github.com/example/monorepo.git
cd monorepo

# 2. Enable sparse checkout in cone mode
git sparse-checkout init --cone

# 3. Set the directories you want to work with
git sparse-checkout set services/payments libs/common

# 4. Now perform the initial checkout. Only the selected directories appear.
git checkout main
# Blobs needed for services/payments and libs/common are fetched on demand.

After this, ls shows only services/payments and libs/common. The .git directory contains commit and tree history but only the blobs actually used. Disk usage is minimal, and future git fetch operations are fast because no blobs are fetched until they are needed.

Working with Branches

# Switch to a different branch – only missing blobs for the sparse directories are fetched
git checkout feature/new-integration

# Create a new branch and make changes within the sparse directories
git checkout -b my-feature
# Work normally – only the sparse-checkout paths are present

Updating the Sparse Set After Cloning

You can expand or shrink the sparse checkout at any time. Adding a new directory may trigger on-demand blob fetches for files in that directory.

# Add the 'infra/scripts' directory to the checkout
git sparse-checkout add infra/scripts
# Git fetches any missing blobs for infra/scripts automatically

Best Practices

• Always use cone mode (git sparse-checkout init --cone) for performance. Non‑cone mode can lead to very slow git status and git diff in large repos.
• Combine with --filter=blob:none as your default partial clone filter. It’s the safest option and gives you maximum space savings while keeping all history and tree structure available.
• Use --no-checkout during clone to avoid an immediate full checkout (and unnecessary blob fetches). Then set up sparse checkout before the first git checkout.
• In CI pipelines, clone with minimal history if possible: add --depth=1 for a shallow clone along with the filter to get only the latest commit and no blobs. Example: git clone --filter=blob:none --depth=1 --no-checkout <url> then sparse-checkout the needed path.
• Monitor your patterns: Run git sparse-checkout list regularly to ensure your working tree matches your expectations. Use git rev-list --filter-print-objects HEAD to see which blobs are still missing.
• Avoid overly aggressive filters like tree:0 unless you are certain you won’t need tree objects frequently. It can cause many small fetch pauses during normal operations.
• Set the filter on existing remotes if you forgot during clone: git config remote.origin.partialclonefilter "blob:none" and then git fetch --refilter (Git 2.29+) to re-fetch with the filter applied.
• Be aware of git gc and git repack: They work normally but will only operate on objects you already have locally. Missing objects stay missing.

Conclusion

Git Partial Clone and Sparse Checkout are transformative for teams working in large repositories. By fetching only the history you need and checking out only the files you work on, you turn a monolithic repository into a fast, manageable workspace. The combination is especially powerful for monorepos, CI/CD systems, and developers on bandwidth- or disk-constrained machines. Start with git clone --filter=blob:none --no-checkout, enable cone-mode sparse checkout, and enjoy a dramatically lighter Git experience.