← Back to DevBytes
Git July 10, 2026 12 min read

GitLab CI with GitLab Runner Setup

Understanding GitLab CI/CD and GitLab Runner

GitLab CI/CD is a built-in continuous integration and delivery system that automates the testing, building, and deployment of your code. At its core sits GitLab Runner — a lightweight agent that executes the jobs defined in your pipeline configuration. Think of GitLab as the orchestrator that tells Runner what to do, and Runner as the worker that actually runs the scripts, builds Docker images, or deploys to production.

When you push code to GitLab, the platform looks for a .gitlab-ci.yml file in your repository root. This file defines a pipeline — a sequence of stages and jobs. GitLab then dispatches these jobs to available Runners. A Runner can run on your local machine, a dedicated server, inside a Docker container, or on a Kubernetes cluster. This separation of concerns means your CI infrastructure can scale independently from your GitLab instance.

Key Terminology

Why Setting Up Your Own Runner Matters

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

GitLab.com provides shared Runners for all users, but they come with limitations: limited monthly minutes on free tiers, no control over the execution environment, and potential queue delays during peak usage. By deploying your own GitLab Runner, you gain:

For teams shipping production software, a dedicated Runner is not optional — it's the foundation of a reliable delivery pipeline.

Architecture Overview

The GitLab Runner architecture follows a simple, elegant pattern. The Runner agent registers with a GitLab instance (self-hosted or GitLab.com) using a registration token. Once registered, it continuously polls for pending jobs. When a job is available, the Runner spawns an isolated environment using its configured executor, runs the job scripts, and reports results back to GitLab.

The Runner itself is stateless — all project-specific configuration lives in the repository's .gitlab-ci.yml file. This means you can reuse the same Runner across hundreds of projects, each with their own pipeline definitions.

Installing GitLab Runner

Option 1: Native Package Installation (Linux)

This is the most common setup for production environments. The Runner runs as a systemd service directly on the host OS.

# Add the official GitLab Runner repository
curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash

# Install the package
sudo apt-get install gitlab-runner

# Verify installation
gitlab-runner --version
gitlab-runner status

For RHEL/CentOS systems, use the equivalent RPM repository script:

curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.rpm.sh" | sudo bash
sudo yum install gitlab-runner

Option 2: Docker Deployment

Running the GitLab Runner itself inside a Docker container gives you portability and simplifies upgrades. This is ideal when you want the Runner to use the Docker executor for jobs (Docker-in-Docker pattern).

# Create a persistent configuration volume
docker volume create gitlab-runner-config

# Start the Runner container
docker run -d --name gitlab-runner --restart always \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v gitlab-runner-config:/etc/gitlab-runner \
  gitlab/gitlab-runner:latest

The mounted Docker socket allows the Runner container to spawn sibling containers for each job — this is significantly faster and more reliable than true Docker-in-Docker (DinD).

Option 3: Kubernetes Deployment

For teams operating at scale, deploying Runner on Kubernetes provides elastic capacity. Each pipeline job runs in its own ephemeral pod, and you can leverage cluster auto-scaling to handle build spikes.

# Using Helm (recommended approach)
helm repo add gitlab https://charts.gitlab.io
helm repo update

# Install the GitLab Runner chart
helm install gitlab-runner gitlab/gitlab-runner \
  --set gitlabUrl=https://gitlab.example.com \
  --set runnerRegistrationToken="YOUR_REGISTRATION_TOKEN" \
  --set rbac.create=true \
  --set runners.privileged=true \
  --namespace gitlab-runner \
  --create-namespace

The Kubernetes executor dynamically creates pods with build and helper containers. The helper container handles cloning the repository, downloading artifacts, and uploading results, while the build container runs your actual job scripts.

Registering a Runner

Registration links a Runner instance to your GitLab project or group. You'll need a registration token, which you can find in your project's Settings → CI/CD → Runners section, or at the group/admin level for shared Runners.

# Interactive registration (asks for GitLab URL, token, description, tags, executor)
sudo gitlab-runner register

# Non-interactive registration with all parameters
sudo gitlab-runner register \
  --non-interactive \
  --url "https://gitlab.example.com" \
  --registration-token "PROJECT_OR_GROUP_TOKEN" \
  --description "production-docker-runner" \
  --tag-list "docker,linux,production" \
  --executor "docker" \
  --docker-image "alpine:latest" \
  --docker-volumes "/var/run/docker.sock:/var/run/docker.sock" \
  --run-untagged=true \
  --locked=false

The --tag-list parameter is critical — jobs in your .gitlab-ci.yml specify tags to route themselves to appropriate Runners. For example, a job tagged docker will only run on Runners that have docker in their tag list.

Configuring the Runner (config.toml)

After registration, GitLab Runner stores its configuration in /etc/gitlab-runner/config.toml. Understanding this file lets you fine-tune Runner behavior beyond what registration offers.

concurrent = 4
check_interval = 0
log_level = "info"

[[runners]]
  name = "production-docker-runner"
  url = "https://gitlab.example.com"
  id = 42
  token = "encrypted_runner_token"
  executor = "docker"
  limit = 2
  output_limit = 4096
  
  [runners.docker]
    image = "docker:20.10.16"
    privileged = true
    volumes = [
      "/var/run/docker.sock:/var/run/docker.sock",
      "/cache:/cache"
    ]
    shm_size = 256000000
    extra_hosts = ["internal.registry.local:192.168.1.100"]
    
  [runners.cache]
    Type = "s3"
    Shared = true
    [runners.cache.s3]
      ServerAddress = "s3.amazonaws.com"
      AccessKey = "AKIA..."
      SecretKey = "..."
      BucketName = "gitlab-runner-cache"
      BucketLocation = "us-east-1"

Key configuration options:

Writing Your First .gitlab-ci.yml Pipeline

Now that your Runner is installed and registered, let's create a pipeline configuration in your repository. Create a file named .gitlab-ci.yml at the project root.

Basic Three-Stage Pipeline

stages:
  - build
  - test
  - deploy

variables:
  DOCKER_IMAGE: registry.example.com/myapp
  DOCKER_TAG: ${CI_COMMIT_SHORT_SHA}

build-job:
  stage: build
  tags:
    - docker
  script:
    - echo "Building Docker image..."
    - docker build -t ${DOCKER_IMAGE}:${DOCKER_TAG} .
    - docker push ${DOCKER_IMAGE}:${DOCKER_TAG}
  artifacts:
    paths:
      - dist/
    expire_in: 1 hour

unit-test:
  stage: test
  tags:
    - docker
  script:
    - docker pull ${DOCKER_IMAGE}:${DOCKER_TAG}
    - docker run --rm ${DOCKER_IMAGE}:${DOCKER_TAG} npm test
  dependencies:
    - build-job

lint:
  stage: test
  tags:
    - docker
  script:
    - npm install
    - npm run lint
  cache:
    key: ${CI_COMMIT_REF_SLUG}
    paths:
      - node_modules/

deploy-staging:
  stage: deploy
  tags:
    - docker
  script:
    - echo "Deploying to staging..."
    - kubectl apply -f k8s/staging/
  environment:
    name: staging
    url: https://staging.example.com
  only:
    - develop

deploy-production:
  stage: deploy
  tags:
    - docker
  script:
    - echo "Deploying to production..."
    - kubectl apply -f k8s/production/
  environment:
    name: production
    url: https://app.example.com
  when: manual
  only:
    - main

This pipeline demonstrates several essential patterns. The tags directive ensures jobs run on your dedicated Docker Runner. The artifacts block passes build output between stages. The cache block persists node_modules across pipeline runs using a branch-specific key. The environment block integrates with GitLab's Environments dashboard for tracking deployments.

Using CI/CD Variables and Secrets

Never hardcode credentials in your pipeline file. Instead, set variables in GitLab's UI (Settings → CI/CD → Variables) and reference them securely.

deploy-production:
  stage: deploy
  tags:
    - docker
  script:
    # These variables are injected from GitLab's protected variables
    - aws configure set aws_access_key_id ${AWS_ACCESS_KEY_ID}
    - aws configure set aws_secret_access_key ${AWS_SECRET_ACCESS_KEY}
    - aws s3 sync dist/ s3://my-production-bucket/ --delete
  only:
    - main
  # Protected variables are only available on protected branches/tags
  variables:
    AWS_ACCESS_KEY_ID:
      description: "AWS access key for production S3"
    AWS_SECRET_ACCESS_KEY:
      description: "AWS secret key (masked in logs)"

Matrix Builds for Multi-Platform Testing

GitLab supports parallel matrix jobs — perfect for testing against multiple language versions or operating systems.

test-matrix:
  stage: test
  tags:
    - docker
  parallel:
    matrix:
      - NODE_VERSION: ["16", "18", "20"]
        DB: ["postgres", "mysql"]
  image: node:${NODE_VERSION}-alpine
  services:
    - ${DB}:latest
  script:
    - npm install
    - npm test
  # Exclude combinations that don't make sense
  exclude:
    - NODE_VERSION: "16"
      DB: "mysql"

This generates 6 parallel jobs (3 Node versions × 2 databases, minus the excluded combination). Each runs in its own container with the appropriate service linked.

Advanced Runner Configuration Patterns

Auto-Scaling Docker Runners with Docker Machine

For environments with variable build loads, auto-scaling Runners spin up cloud instances on demand. This uses the Docker Machine executor (legacy but still widely used) or the newer Docker Autoscaler executor.

# config.toml entry for an auto-scaling AWS runner
[[runners]]
  name = "autoscale-runner"
  url = "https://gitlab.example.com"
  token = "runner_token"
  executor = "docker+machine"
  limit = 10
  
  [runners.docker]
    image = "docker:20.10.16"
    privileged = true
    
  [runners.machine]
    IdleCount = 2
    IdleTime = 1800
    MaxGrowthRate = 10
    MachineDriver = "amazonec2"
    MachineName = "gitlab-runner-%s"
    MachineOptions = [
      "amazonec2-region=us-east-1",
      "amazonec2-instance-type=m5.large",
      "amazonec2-root-size=50",
      "amazonec2-iam-instance-profile=gitlab-runner",
      "amazonec2-private-address=true",
      "amazonec2-tag=gitlab-runner,managed"
    ]
    OffPeakPeriods = ["* * 0-6,22-23 * * mon-fri", "* * * * * sat,sun"]
    OffPeakIdleCount = 0
    OffPeakIdleTime = 600

This configuration maintains 2 warm instances during business hours, scales to 0 overnight and on weekends, and caps growth at 10 new instances per cycle to prevent runaway costs.

Distributed Caching with S3

When multiple Runners share a project, a distributed cache ensures all Runners benefit from previously downloaded dependencies.

# In your .gitlab-ci.yml
cache:
  key: ${CI_COMMIT_REF_SLUG}-${CI_JOB_NAME}
  paths:
    - .gradle/wrapper
    - .gradle/caches
    - node_modules/
    - vendor/bundle
  policy: pull-push

Combine this with the S3 cache configuration in config.toml (shown earlier). The pull-push policy ensures the job both downloads the latest cache before execution and uploads updated cache after completion.

Debugging and Troubleshooting Runners

Common Issues and Solutions

Useful Debugging Commands

# List all registered runners and their status
sudo gitlab-runner list

# Verify a specific runner's connectivity
sudo gitlab-runner verify --name "production-docker-runner"

# Run a job locally for debugging (single job from .gitlab-ci.yml)
gitlab-runner exec docker build-job

# View runner logs in real-time
sudo gitlab-runner run --working-dir /home/gitlab-runner --config /etc/gitlab-runner/config.toml

# Unregister and re-register a problematic runner
sudo gitlab-runner unregister --name "production-docker-runner"
sudo gitlab-runner register --non-interactive --url "..." --registration-token "..." --executor "docker"

The gitlab-runner exec command is invaluable for debugging pipeline failures without pushing to GitLab repeatedly. It runs a single job locally using the same executor and environment the Runner would use in production.

Security Best Practices

Performance Optimization

# Example: parallel test splitting in .gitlab-ci.yml
test-parallel:
  stage: test
  tags:
    - docker
  parallel: 4
  script:
    # CI_NODE_INDEX ranges from 1 to parallel count
    - npm install
    - npx jest --ci --shard=${CI_NODE_INDEX}/${CI_NODE_TOTAL}
  artifacts:
    reports:
      junit: junit.xml
    paths:
      - coverage/

Monitoring and Maintenance

A healthy CI/CD infrastructure requires ongoing attention. Set up monitoring for your Runners:

# Prometheus metrics endpoint (built into GitLab Runner)
# Access at: http://runner-host:9252/metrics

# Key metrics to monitor:
# - gitlab_runner_jobs_completed_total
# - gitlab_runner_jobs_failed_total  
# - gitlab_runner_jobs_duration_seconds
# - gitlab_runner_concurrent_jobs
# - gitlab_runner_errors_total

Regular maintenance tasks include pruning Docker images weekly (docker system prune -f), updating the Runner binary, reviewing job queue times in GitLab's CI/CD analytics, and capacity planning based on pipeline frequency trends.

Conclusion

GitLab Runner is the engine that transforms your .gitlab-ci.yml definitions into actual executed pipelines. By deploying dedicated Runners — whether on bare metal, Docker hosts, or Kubernetes clusters — you gain complete control over your CI/CD environment. The setup process is straightforward: install the Runner package, register it with a token, configure your executor, and start writing pipeline jobs.

The patterns covered here — from basic three-stage pipelines to matrix builds, distributed caching, auto-scaling infrastructure, and security hardening — form a solid foundation for any team's delivery workflow. Remember that CI/CD is not a "set it and forget it" system; it requires monitoring, maintenance, and continuous refinement. But the investment pays dividends in faster feedback loops, fewer production incidents, and a developer experience that encourages frequent, confident deployments. Start with a single dedicated Runner for your team's most critical project, iterate on the pipeline configuration, and expand your Runner fleet as your needs grow.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles