← Back to DevBytes

Git Security: Preventing Secret Leaks

Understanding Secret Leaks in Git

Secret leaks occur when sensitive information—API keys, database credentials, private certificates, authentication tokens, or proprietary code—is accidentally committed to a Git repository. Once pushed to a remote repository, these secrets become part of the permanent version history and can be exposed to unauthorized parties. Even if you delete the file in a subsequent commit, the sensitive data remains accessible in the commit history unless explicitly purged.

What Constitutes a Secret?

In software development, secrets include any piece of information that should never be publicly accessible or shared outside a controlled environment. Common examples include:

Why Preventing Secret Leaks Matters

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

The consequences of a secret leak can be devastating and far-reaching. Security researchers estimate that thousands of secrets are leaked on public GitHub repositories every day. Once a secret is exposed, attackers can automate scanning of repositories to find and exploit these credentials within minutes. The impact includes unauthorized access to production infrastructure, data breaches, financial losses from compromised cloud accounts, and severe reputational damage. In regulated industries, a leak can also result in compliance violations under frameworks like GDPR, HIPAA, or PCI-DSS, potentially leading to substantial fines.

Critically, rewriting Git history after a leak is not enough—the secret must be rotated immediately because any cached or logged copies remain valid. Prevention is therefore exponentially more valuable than remediation.

How Secrets End Up in Repositories

Developers rarely commit secrets maliciously. The most common vectors are accidental and often stem from workflow oversights:

Pre-Commit Prevention Strategies

Using .gitignore Effectively

The first line of defense is a well-configured .gitignore file. This should be set up before the first commit in any repository. Create a .gitignore file in the root of your project and add patterns for files that commonly contain secrets:

# Environment files with secrets
.env
.env.*
*.env

# Credential files
credentials.json
*.pem
*.key
*.p12
*.pfx

# Configuration with secrets
config/secrets.yml
secrets.yaml

# IDE-specific settings that may contain tokens
.idea/
.vscode/token

# Terraform state files
*.tfstate
*.tfstate.backup

# Kubernetes secrets
secrets/
*.secrets.yaml

However, .gitignore alone is insufficient. It only prevents untracked files from being staged. If a file is already tracked, modifications to it will still be committed regardless of .gitignore rules. For already-tracked sensitive files, you must use git rm --cached to remove them from tracking before the ignore rules take effect.

Git Hooks: Pre-Commit Scanning

Git hooks are scripts that run automatically at specific points in the Git lifecycle. The pre-commit hook executes before a commit is finalized and can inspect the staged changes for secrets. If a secret is detected, the hook can reject the commit entirely. Here is a practical pre-commit hook script that scans for high-entropy strings resembling secrets:

#!/bin/bash
# .git/hooks/pre-commit
# Scans staged changes for potential secrets

# Patterns to detect
PATTERNS=(
  '-----BEGIN (RSA|EC|DSA|OPENSSH) PRIVATE KEY-----'
  'AKIA[0-9A-Z]{16}'  # AWS Access Key pattern
  'sk-[0-9a-fA-F]{32}'  # OpenAI/Anthropic key pattern
  'ghp_[0-9a-zA-Z]{36}'  # GitHub personal token
  'xox[baprs]-[0-9a-zA-Z-]{10,}'  # Slack token
)

STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM)

if [ -z "$STAGED_FILES" ]; then
  exit 0
fi

for FILE in $STAGED_FILES; do
  # Skip binary files and large files
  if file "$FILE" | grep -q "binary"; then
    continue
  fi

  CONTENT=$(git diff --cached "$FILE" | grep '^\+' | grep -v '^+++' | sed 's/^+//')

  for PATTERN in "${PATTERNS[@]}"; do
    if echo "$CONTENT" | grep -qE "$PATTERN"; then
      echo "ERROR: Potential secret detected in $FILE"
      echo "Pattern matched: $PATTERN"
      echo "Commit aborted. Please remove the secret and try again."
      exit 1
    fi
  done
done

# Additional entropy check
for FILE in $STAGED_FILES; do
  if file "$FILE" | grep -q "text"; then
    HIGH_ENTROPY=$(git diff --cached "$FILE" | grep '^\+' | grep -v '^+++' | \
      sed 's/^+//' | grep -E '.{20,}' | \
      perl -MEncode -e 'undef $/; $str=<>; exit(0) if length($str)<20; \
      my %c; for(split//,$str){$c{$_}++} my $e=0; \
      for(values %c){my $p=$_/length($str); $e+=-$p*log($p)/log(2) if $p>0} \
      exit($e>4.5?1:0)')
    if [ $? -eq 1 ]; then
      echo "WARNING: High-entropy string found in $FILE. Please verify it is not a secret."
    fi
  fi
done

exit 0

To install this hook, save the script to .git/hooks/pre-commit in your repository and make it executable with chmod +x .git/hooks/pre-commit. Note that hooks in the .git/hooks directory are not shared across clones by default. For team-wide enforcement, consider using a tool like pre-commit framework or storing hooks in a tracked directory that developers symlink into their local .git/hooks.

Automated Scanning Tools

Several dedicated tools provide robust pre-commit secret scanning with more sophisticated detection algorithms, including entropy analysis, pattern matching, and integration with known secret formats. These tools are designed to be fast enough to run on every commit without disrupting workflow:

git-secrets by AWS Labs — Originally developed to prevent AWS credentials from being committed, it can be configured with custom patterns. Install it and set up scanning:

# Install git-secrets
git clone https://github.com/awslabs/git-secrets.git
cd git-secrets && sudo make install

# Navigate to your repository
cd /path/to/your-repo

# Register the AWS patterns and custom patterns
git secrets --register-aws
git secrets --add 'my-custom-pattern-[a-z]{8}'
git secrets --add '-----BEGIN PRIVATE KEY-----'

# Install the pre-commit hook
git secrets --install ~/.git-templates/secrets
git config --global init.templateDir ~/.git-templates/secrets

# Scan the entire history
git secrets --scan-history

Gitleaks — A fast, configurable SAST tool for detecting secrets in Git repositories. It supports over 100 secret types out of the box and can be run as a pre-commit hook, in CI/CD pipelines, or as a one-time scan. Install and configure it:

# Install Gitleaks (macOS with Homebrew)
brew install gitleaks

# Or download binary from GitHub releases for Linux/Windows
# https://github.com/gitleaks/gitleaks/releases

# Generate a baseline configuration file
gitleaks detect --source . --verbose --redact > gitleaks-report.json

# Run as a pre-commit check
gitleaks protect --staged --verbose

# Create a .gitleaks.toml configuration to customize rules
cat > .gitleaks.toml << 'EOF'
# Allow specific false positives
[allowlist]
  description = "Global allow list"
  paths = [
    "test/fixtures/",
    "docs/examples/"
  ]
  regexes = [
    "placeholder-token-123",
    "EXAMPLE_API_KEY"
  ]

# Custom rule for company-specific patterns
[[rules]]
  id = "company-internal-token"
  description = "Detects CompanyX internal tokens"
  regex = '''CX_TOKEN_[A-Z0-9]{32}'''
  keywords = ["CX_TOKEN"]
EOF

# Add to pre-commit workflow using pre-commit framework
# In .pre-commit-config.yaml:
# repos:
#   - repo: https://github.com/gitleaks/gitleaks
#     rev: v8.18.0
#     hooks:
#       - id: gitleaks

truffleHog — Searches through Git history for high-entropy strings and secrets. It uses both regex patterns and entropy analysis to identify potential credentials. It can scan live commits and entire histories:

# Install truffleHog
pip install trufflehog

# Scan current state of repository
trufflehog filesystem .

# Scan entire Git history
trufflehog git file:///path/to/your-repo --since-commit HEAD~50

# Scan with JSON output for CI integration
trufflehog git file://. --json > secrets-output.json

Client-Side Git Configuration

Global Gitignore and Hooks Template

To ensure secret prevention across all repositories you work with, configure global Git settings. A global .gitignore applies to every repository on your machine:

# Create a global gitignore
echo '.env' >> ~/.gitignore_global
echo '*.pem' >> ~/.gitignore_global
echo 'credentials.*' >> ~/.gitignore_global

# Configure Git to use it globally
git config --global core.excludesfile ~/.gitignore_global

# Verify the setting
git config --global core.excludesfile

Similarly, you can set up a global hooks template so every new repository initializes with pre-commit secret scanning:

# Set up a global hooks directory
mkdir -p ~/.git-templates/hooks

# Create the pre-commit hook in the template
cat > ~/.git-templates/hooks/pre-commit << 'HOOK'
#!/bin/bash
# Global pre-commit secret scan
if command -v gitleaks &> /dev/null; then
  gitleaks protect --staged --verbose --no-banner
  if [ $? -ne 0 ]; then
    echo "Gitleaks detected secrets. Commit aborted."
    exit 1
  fi
fi
HOOK

chmod +x ~/.git-templates/hooks/pre-commit

# Configure Git to use the template for new repos
git config --global init.templateDir ~/.git-templates

Environment Variables Over Hard-Coding

One of the most effective prevention techniques is to never store secrets in source code at all. Use environment variables and configuration management patterns to keep secrets out of the codebase entirely. Here is a practical approach for a Node.js application:

// BAD: Hard-coded secret
const stripe = require('stripe')('sk_live_4eC39HqLyjWDarjtT1zdp7dc');

// GOOD: Load from environment variable
const stripe = require('stripe')(process.env.STRIPE_SECRET_KEY);

// Provide a clear error if the variable is missing
if (!process.env.STRIPE_SECRET_KEY) {
  throw new Error('STRIPE_SECRET_KEY environment variable is required');
}

For local development, use a .env.example file (committed to the repository) that documents required variables without real values, while the actual .env file remains gitignored:

# .env.example - committed to repository
# Copy this file to .env and fill in real values
# Never commit .env to Git!

STRIPE_SECRET_KEY=sk_live_xxxxxxxxxxxx
DATABASE_URL=postgres://user:password@localhost:5432/mydb
JWT_SECRET=your-secret-here

# .env - NEVER committed, listed in .gitignore
STRIPE_SECRET_KEY=sk_live_4eC39HqLyjWDarjtT1zdp7dc
DATABASE_URL=postgres://produser:realpass@db.example.com:5432/proddb
JWT_SECRET=9a8b7c6d5e4f3a2b1c0d9e8f7a6b5c4d

CI/CD Pipeline Integration

Prevention at the developer's machine is essential, but defense in depth demands secret scanning in CI/CD pipelines as a second barrier. Even if a developer bypasses local hooks, the CI pipeline should reject any commit containing secrets. Here is an example GitHub Actions workflow:

# .github/workflows/secret-scan.yml
name: Secret Scan

on:
  pull_request:
    branches: [main, develop]
  push:
    branches: [main]

jobs:
  gitleaks-scan:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0  # Full history for comprehensive scan

      - name: Run Gitleaks
        uses: gitleaks/gitleaks-action@v2
        with:
          config: .gitleaks.toml
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Upload scan results
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: gitleaks-report
          path: gitleaks-report.json

For GitLab CI/CD, a similar job can be configured in .gitlab-ci.yml:

# .gitlab-ci.yml
secret-scan:
  stage: test
  image: zricethezav/gitleaks:latest
  script:
    - gitleaks detect --source $CI_PROJECT_DIR --verbose --redact
  artifacts:
    reports:
      secret_detection: gitleaks-report.json
  rules:
    - if: $CI_PIPELINE_SOURCE == "push" || $CI_PIPELINE_SOURCE == "merge_request_event"

Handling Accidental Leaks: Remediation

If a secret is committed despite all preventive measures, immediate action is required. The remediation process has two critical phases performed simultaneously: rotating the secret and purging Git history.

Step 1: Rotate the Secret Immediately

This is the most urgent step. Revoke and regenerate the leaked credential through the service provider's console or API. A leaked AWS access key must be deactivated in the IAM console immediately. A leaked GitHub token must be revoked in GitHub settings. Do not wait to clean the Git history first—every second the secret remains valid increases the window of exploitation.

Step 2: Purge Git History

After rotation, remove the secret from the repository's commit history. For repositories that have not been cloned by others, rewriting history is sufficient. Use git filter-branch or the faster BFG Repo-Cleaner:

# Using git filter-branch to remove a file from all commits
git filter-branch --force --index-filter \
  "git rm --cached --ignore-unmatch path/to/secret-file.env" \
  --prune-empty --tag-name-filter cat -- --all

# Force push the rewritten history
git push origin --force --all
git push origin --force --tags

# Instruct all collaborators to rebase their work
# from the cleaned branch, NOT merge

For a faster and simpler approach, use BFG Repo-Cleaner which is specifically designed for removing sensitive data:

# Download BFG (requires Java)
# https://rtyley.github.io/bfg-repo-cleaner/

# Clone a mirror of your repository
git clone --mirror git@github.com:user/repo.git
cd repo.git

# Remove a specific file from all history
java -jar ~/bfg.jar --delete-files secret-file.env repo.git

# Remove all files matching a pattern
java -jar ~/bfg.jar --delete-files '*.pem' repo.git

# Replace specific strings across all files
java -jar ~/bfg.jar --replace-text passwords.txt repo.git
# Where passwords.txt contains lines like:
# sk_live_4eC39HqLyjWDarjtT1zdp7dc==>PLACEHOLDER_REPLACED_KEY

# Clean up and push
cd repo.git
git reflog expire --expire=now --all
git gc --prune=now --aggressive
git push origin --force --all
git push origin --force --tags

Step 3: Notify and Clean Up

If the repository was public or accessible to a wide audience, assume the secret was compromised. Notify your security team, conduct an incident review, and monitor for unauthorized activity using the affected credentials. All team members must rebase their local clones from the cleaned remote branch—merging from the old history will reintroduce the leaked commits.

Best Practices Summary

Preventing secret leaks requires a layered defense strategy combining developer education, tooling, and process enforcement. Here are the essential practices to integrate into your development workflow:

Conclusion

Secret leaks in Git repositories represent one of the most preventable yet most damaging security vulnerabilities in modern software development. The tools and practices available today—from simple .gitignore configuration to sophisticated pre-commit scanning tools like Gitleaks and git-secrets—make it entirely feasible to catch secrets before they enter the permanent version history. The key is implementing a defense-in-depth strategy: prevent leaks at the developer workstation with hooks, block them in CI/CD pipelines with automated scanning, and never store secrets in source code by adopting environment variable patterns and secret management services. When a leak does occur, the response must be swift—rotate the secret immediately and purge it from Git history using tools like BFG Repo-Cleaner. By weaving these practices into your development workflow and fostering a security-conscious team culture, you can dramatically reduce the risk of credential exposure and protect your infrastructure from the cascading consequences of secret leaks.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles