← Back to DevBytes

Karpenter Node Autoscaling: Complete Implementation Guide

What is Karpenter?

Karpenter is an open-source, high-performance Kubernetes node autoscaler that automatically provisions and deprovisions nodes in response to changing application demands. Originally developed by AWS for Amazon EKS, Karpenter has evolved into a multi-provider project under the Kubernetes Autoscaling SIG. Unlike the traditional Cluster Autoscaler, Karpenter operates at the pod-level granularity, making scaling decisions in seconds rather than minutes.

Karpenter works by directly watching for pods that are unschedulable due to resource constraints, then dynamically launching the optimal compute instance type and size for those pods. It continuously monitors node utilization and consolidates workloads onto fewer nodes when possible, reducing waste and driving infrastructure costs down. Think of it as an intelligent, just-in-time capacity provisioner that eliminates the need for pre-defined node groups and static scaling configurations.

Core Concepts

Why Karpenter Matters

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

The traditional Cluster Autoscaler relies on node groups with predefined instance types and sizes. This model introduces several pain points: you must forecast capacity needs, manually tune group sizes, and accept that scaling actions take minutes due to cloud provider API polling intervals. Workloads with diverse resource requirements often lead to stranded capacity — large nodes running small pods, or pods stuck pending because no single node group matches their specific needs.

Karpenter solves these problems by making provisioning decisions at the pod level, in real time. Here's why teams adopt it:

Immediate Scaling Response

Instead of waiting for a cluster-level controller loop to notice pending pods (typically 30 seconds to several minutes), Karpenter intercepts pod scheduling events directly via a webhook. Combined with direct cloud API calls for instance launch, new nodes can be ready in as little as 30–60 seconds.

Dynamic Right-Sizing

Karpenter evaluates the exact resource requests of pending pods and selects the most cost-effective instance type that satisfies those constraints. If a pod needs 3 vCPUs and 7GiB of memory, Karpenter won't launch a 16-core behemoth — it picks an instance that fits precisely, often from hundreds of possible SKUs across multiple families.

Spot Instance Optimization

Karpenter natively integrates with spot capacity pools and gracefully handles spot interruptions. It can prioritize spot instances while falling back to on-demand when spot capacity is scarce, and it automatically replaces nodes when a spot termination notice arrives, often before the workload is disrupted.

Consolidation and Cost Efficiency

Beyond scaling up, Karpenter continuously evaluates whether the cluster is running efficiently. If workloads can be packed onto fewer or cheaper nodes, Karpenter will cordon, drain, and replace those nodes automatically. This "defragmentation" behavior often yields 20–40% cost reductions compared to static node groups.

How Karpenter Works: Architecture Deep Dive

Karpenter runs as a controller inside your Kubernetes cluster. It consists of several core controllers that reconcile different resource types. Understanding this architecture helps you troubleshoot and tune the system effectively.

The Scheduling Hook

Karpenter registers a mutating webhook with the Kubernetes scheduler. When a pod enters the scheduling queue and no existing node satisfies its requirements, the scheduler emits an event. Karpenter intercepts this, evaluates the pod's constraints, and creates a NodeClaim resource. This tight integration avoids the polling delay inherent in the Cluster Autoscaler model.

NodeClaim Lifecycle

A NodeClaim is a custom resource that represents a provisioning intent. Karpenter's nodeclaim controller reconciles NodeClaims by calling the cloud provider's instance launch API. Once the instance reports healthy via a readiness check, Karpenter binds the NodeClaim to a Kubernetes Node object. The lifecycle then transitions to active management — health monitoring, drift detection, and eventual consolidation or termination.

Provider Interface

Karpenter abstracts cloud-specific logic behind a provider interface. The official AWS provider ships with extensive support for EC2 instance types, launch templates, subnet selection, and security group automation. Community providers exist for Azure, GCP, and on-premises environments. This abstraction ensures Karpenter's core scheduling logic remains provider-agnostic.

Complete Implementation Guide

Let's walk through a production-ready Karpenter deployment on Amazon EKS, covering every step from initial setup to advanced configuration. You'll need an existing EKS cluster, kubectl, helm, and the AWS CLI configured.

Step 1: Install Karpenter via Helm

Add the Karpenter Helm repository and install the controller. The Helm chart creates the CRDs, service accounts, and the core controller deployment.

# Add the Karpenter Helm repository
helm repo add karpenter https://charts.karpenter.sh
helm repo update

# Install Karpenter in the karpenter namespace
helm upgrade --install karpenter karpenter/karpenter \
  --namespace karpenter \
  --create-namespace \
  --set serviceAccount.annotations.eks.amazonaws.com/role-arn="arn:aws:iam::${AWS_ACCOUNT_ID}:role/KarpenterControllerRole" \
  --set settings.clusterName="${CLUSTER_NAME}" \
  --set settings.clusterEndpoint="${CLUSTER_ENDPOINT}" \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --wait

The installation requires an IAM role for the Karpenter controller. Create this using the provided policy that grants permissions for EC2 instance lifecycle management, IAM profile passthrough, and SQS queue interaction for interruption handling.

Step 2: Configure IAM Permissions

Karpenter needs broad EC2 permissions to discover instance types, manage launch templates, and handle instance lifecycle. Create an IAM role with the following trust policy and attach the required permissions.

# Create the IAM policy document
cat > karpenter-controller-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:DescribeInstanceTypes",
        "ec2:DescribeInstanceTypeOfferings",
        "ec2:DescribeAvailabilityZones",
        "ec2:DescribeSecurityGroups",
        "ec2:DescribeSubnets",
        "ec2:DescribeLaunchTemplates",
        "ec2:DescribeLaunchTemplateVersions",
        "ec2:RunInstances",
        "ec2:TerminateInstances",
        "ec2:CreateLaunchTemplate",
        "ec2:DeleteLaunchTemplate",
        "ec2:CreateTags",
        "ec2:DescribeImages",
        "iam:PassRole",
        "ssm:GetParameter",
        "sqs:ReceiveMessage",
        "sqs:DeleteMessage",
        "sqs:GetQueueUrl",
        "sqs:GetQueueAttributes"
      ],
      "Resource": "*"
    }
  ]
}
EOF

# Create the IAM policy
aws iam create-policy \
  --policy-name KarpenterControllerPolicy \
  --policy-document file://karpenter-controller-policy.json

# Attach the policy to the Karpenter controller role
aws iam put-role-policy \
  --role-name KarpenterControllerRole \
  --policy-name KarpenterControllerPolicy \
  --policy-document file://karpenter-controller-policy.json

Step 3: Define Your First NodePool

The NodePool is Karpenter's central configuration object. It defines which nodes Karpenter is allowed to provision. A well-designed NodePool balances cost, performance, and availability requirements. Start with a general-purpose NodePool that covers most workloads.

apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: default
spec:
  template:
    metadata:
      labels:
        node-type: karpenter
    spec:
      # Node taints for scheduling isolation
      taints:
        - key: node.karpenter.sh/capacity-type
          value: spot
          effect: NoSchedule
      # Node labels applied to the instance
      labels:
        node.karpenter.sh/capacity-type: spot
        node.karpenter.sh/owner: platform-team
      # Requirements constrain instance selection
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64", "arm64"]
        - key: kubernetes.io/os
          operator: In
          values: ["linux"]
        - key: node.karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: node.karpenter.sh/instance-category
          operator: In
          values: ["c", "m", "r", "t"]
        - key: node.karpenter.sh/instance-generation
          operator: Gt
          values: ["2"]
      # NodeClass reference (provider-specific)
      nodeClassRef:
        name: default
  # Limits to prevent runaway scaling
  limits:
    cpu: "1000"
    memory: "4000Gi"
  # Disruption settings
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 720h
  # Weighted preferences for cost optimization
  weight: 100

Each requirement uses an operator (In, Gt, Lt, Exists) and a list of values. The Gt: ["2"] on instance generation ensures only modern instance generations are used. The consolidation policy WhenUnderutilized tells Karpenter to replace nodes when packing efficiency drops below a threshold.

Step 4: Create the EC2NodeClass

The EC2NodeClass is the AWS-specific configuration object that specifies networking, security, and AMI details for provisioned nodes. This is where you define the infrastructure template.

apiVersion: karpenter.k8s.aws/v1beta1
kind: EC2NodeClass
metadata:
  name: default
spec:
  # AMI family discovery - automatically selects the latest EKS-optimized AMI
  amiFamily: Bottlerocket
  # Subnet selection via tags
  subnetSelectorTerms:
    - tags:
        kubernetes.io/cluster/${CLUSTER_NAME}: owned
        subnet-type: private
  # Security group selection
  securityGroupSelectorTerms:
    - tags:
        kubernetes.io/cluster/${CLUSTER_NAME}: owned
        aws:eks:cluster-name: ${CLUSTER_NAME}
  # Instance profile for the node
  instanceProfile: KarpenterNodeInstanceProfile
  # Detailed monitoring for CloudWatch
  detailedMonitoringDisabled: false
  # Metadata service configuration
  metadataOptions:
    httpEndpoint: enabled
    httpProtocolIPv6: disabled
    httpPutResponseHopLimit: 2
    httpTokens: required
  # Block device mappings for root volume
  blockDeviceMappings:
    - deviceName: /dev/xvda
      ebs:
        volumeSize: 80Gi
        volumeType: gp3
        iops: 3000
        throughput: 125
        deleteOnTermination: true

The subnetSelectorTerms and securityGroupSelectorTerms use tag-based discovery, so Karpenter automatically finds the right networking resources without hard-coded IDs. The AMI family Bottlerocket selects Amazon's purpose-built container operating system, which boots faster and has a smaller attack surface than standard Linux distributions.

Step 5: Test Provisioning with a Sample Workload

Deploy a workload that requests specific resources to trigger Karpenter provisioning. The pod's resource requests must be unsatisfiable by existing nodes for Karpenter to act.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-test
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx-test
  template:
    metadata:
      labels:
        app: nginx-test
    spec:
      # Node affinity to guide scheduling
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: node-type
                    operator: In
                    values: ["karpenter"]
      containers:
        - name: nginx
          image: nginx:1.25
          resources:
            requests:
              cpu: "2"
              memory: "4Gi"
            limits:
              cpu: "4"
              memory: "8Gi"

Watch the provisioning process with these commands:

# Watch NodeClaims being created
kubectl get nodeclaims --watch

# Watch nodes appearing
kubectl get nodes --watch

# Watch Karpenter controller logs
kubectl logs -f -n karpenter deployment/karpenter

You'll see Karpenter create a NodeClaim, call EC2, and within 60–90 seconds a new node joins the cluster. The pending pods then schedule onto it.

Step 6: Configure Consolidation and Disruption

Consolidation is Karpenter's cost optimization superpower. It evaluates nodes periodically and replaces them when doing so reduces overall cost without disrupting workloads. Configure disruption policies carefully to balance cost against stability.

apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: consolidated-pool
spec:
  template:
    spec:
      nodeClassRef:
        name: default
      requirements:
        - key: node.karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
  disruption:
    # Consolidation policy options:
    # WhenUnderutilized - replace when pods can fit on fewer nodes
    # WhenEmpty - only remove empty nodes
    consolidationPolicy: WhenUnderutilized
    # Maximum lifetime before node is replaced regardless of utilization
    expireAfter: 720h
    # Budget controls the rate of voluntary disruptions
    budgets:
      - nodes: "10%"
        schedule: "0 2 * * *"
        duration: 1h
      - nodes: "2"
        schedule: "* * * * *"
  limits:
    cpu: "500"
    memory: "2000Gi"

The budgets field limits how many nodes Karpenter can disrupt simultaneously. The cron-style schedule lets you define different disruption rates for maintenance windows versus peak traffic hours. For example, allow 10% disruption during a nightly maintenance window but only 2 nodes per operation during business hours.

Step 7: Implement Spot Best Practices with Interruption Handling

Karpenter natively handles spot instance interruptions. It subscribes to an SQS queue that receives EC2 spot termination notifications and scheduled maintenance events. When a termination notice arrives, Karpenter cordons and drains the affected node before the instance is reclaimed, then provisions a replacement if needed.

# Create the SQS queue for spot interruption notifications
aws sqs create-queue \
  --queue-name KarpenterInterruptionQueue \
  --region ${AWS_REGION}

# Configure EventBridge rule for EC2 spot interruptions
aws events put-rule \
  --name KarpenterSpotInterruptionRule \
  --event-pattern '{"source":["aws.ec2"],"detail-type":["EC2 Spot Instance Interruption Warning"]}'

# Connect EventBridge to SQS
aws events put-targets \
  --rule KarpenterSpotInterruptionRule \
  --targets "Id"="1","Arn"="arn:aws:sqs:${AWS_REGION}:${AWS_ACCOUNT_ID}:KarpenterInterruptionQueue"

# Grant SQS permissions to the Karpenter controller role
aws iam put-role-policy \
  --role-name KarpenterControllerRole \
  --policy-name KarpenterSQSInterruptionPolicy \
  --policy-document '{
    "Version": "2012-10-17",
    "Statement": [
      {
        "Effect": "Allow",
        "Action": [
          "sqs:ReceiveMessage",
          "sqs:DeleteMessage",
          "sqs:GetQueueUrl",
          "sqs:GetQueueAttributes"
        ],
        "Resource": "arn:aws:sqs:'${AWS_REGION}':'${AWS_ACCOUNT_ID}':KarpenterInterruptionQueue"
      }
    ]
  }'

To maximize spot usage while maintaining reliability, create separate NodePools with weighted preferences:

# Spot-optimized pool with high weight
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: spot-pool
spec:
  weight: 100
  template:
    metadata:
      labels:
        capacity-type: spot
    spec:
      requirements:
        - key: node.karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
      nodeClassRef:
        name: default
  disruption:
    consolidationPolicy: WhenUnderutilized
    expireAfter: 168h

---
# On-demand fallback pool with lower weight
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: ondemand-fallback
spec:
  weight: 10
  template:
    metadata:
      labels:
        capacity-type: on-demand
    spec:
      requirements:
        - key: node.karpenter.sh/capacity-type
          operator: In
          values: ["on-demand"]
        - key: node.karpenter.sh/instance-category
          operator: In
          values: ["c", "m"]
      nodeClassRef:
        name: default

When spot capacity is available, the higher-weighted spot pool provisions nodes. If spot capacity is constrained, Karpenter falls back to the on-demand pool automatically. This gives you spot economics with on-demand reliability.

Advanced Configuration Patterns

Multi-Architecture Clusters

Karpenter can provision both x86 and ARM64 (Graviton) instances simultaneously. Graviton processors offer up to 40% better price-performance for many workloads. Configure your NodePool to prefer ARM64 but fall back to x86:

apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: multi-arch
spec:
  template:
    spec:
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["arm64", "amd64"]
        - key: node.karpenter.sh/instance-category
          operator: In
          values: ["c", "m", "r"]
      nodeClassRef:
        name: default

Use pod-level node affinity or nodeSelector to pin specific workloads to ARM64. For workloads that benefit from Graviton (web servers, microservices, stateless applications), add:

nodeSelector:
  kubernetes.io/arch: arm64

GPU Workload Provisioning

Karpenter supports GPU instance provisioning for AI/ML workloads. Define a dedicated NodePool with GPU instance requirements and appropriate taints to prevent non-GPU workloads from landing on expensive GPU nodes:

apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: gpu-pool
spec:
  template:
    metadata:
      labels:
        workload-type: gpu
    spec:
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule
      requirements:
        - key: node.karpenter.sh/instance-category
          operator: In
          values: ["g", "p"]
        - key: node.karpenter.sh/instance-generation
          operator: Gt
          values: ["3"]
        - key: kubernetes.io/arch
          operator: In
          values: ["amd64"]
      nodeClassRef:
        name: gpu-node-class
  limits:
    cpu: "200"
  disruption:
    consolidationPolicy: WhenEmpty

The GPU NodeClass should specify an AMI with NVIDIA drivers installed (like the EKS GPU-optimized AMI) and potentially larger EBS volumes for model storage.

Zone-Aware Topology Spread

For high availability, Karpenter respects topology spread constraints and can distribute nodes across availability zones. Configure zone requirements in the NodePool and use pod topology spread constraints:

# NodePool with zone requirements
spec:
  requirements:
    - key: topology.kubernetes.io/zone
      operator: In
      values: ["us-east-1a", "us-east-1b", "us-east-1c"]

---
# Pod specification with topology spread
apiVersion: apps/v1
kind: Deployment
spec:
  template:
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: my-app
        - maxSkew: 1
          topologyKey: node-type
          whenUnsatisfiable: ScheduleAnyway
          labelSelector:
            matchLabels:
              app: my-app

Karpenter provisions nodes across the specified zones to satisfy the topology spread constraints. The combination of zone requirements and pod-level constraints ensures workloads remain resilient to zone failures.

Monitoring and Observability

Karpenter emits Prometheus metrics and structured logs. Set up monitoring to track provisioning latency, consolidation effectiveness, and error rates. Key metrics include:

# Prometheus metrics exposed by Karpenter
karpenter_nodes_created_total
karpenter_nodes_terminated_total
karpenter_nodeclaims_created_total
karpenter_nodeclaims_failed_total
karpenter_consolidation_attempts_total
karpenter_consolidation_replacements_total
karpenter_provisioning_duration_seconds
karpenter_interruption_actions_total

Configure alerts for provisioning failures, high consolidation churn, or approaching capacity limits:

# Example Prometheus alert rule
- alert: KarpenterHighProvisioningFailureRate
  expr: |
    rate(karpenter_nodeclaims_failed_total[5m]) > 0.1
  for: 5m
  labels:
    severity: warning
  annotations:
    summary: "Karpenter is failing to provision nodes"
    description: "NodeClaim failure rate exceeds 10% over 5 minutes. Check cloud provider capacity and IAM permissions."

- alert: KarpenterCapacityLimitApproaching
  expr: |
    karpenter_nodes_usage_ratio > 0.85
  for: 10m
  labels:
    severity: critical
  annotations:
    summary: "Karpenter approaching capacity limits"
    description: "NodePool utilization exceeds 85% of defined limits. Consider increasing limits or checking for stuck pods."

Best Practices

1. Start Simple, Then Refine

Begin with a single general-purpose NodePool covering common instance families (C, M, R). Observe provisioning patterns for a week before creating specialized pools. Over-segmentation early on leads to stranded capacity and complex troubleshooting.

2. Set Realistic Limits

Always configure limits on every NodePool. Without limits, a buggy deployment requesting impossible resources (like 500 vCPUs per pod) could trigger an infinite provisioning loop that exhausts your cloud account's capacity. Set limits slightly above your expected peak usage with a 20% buffer.

3. Use Consolidation Budgets

Unrestricted consolidation can cause excessive node churn during traffic spikes. Apply disruption budgets to limit the number of simultaneous node replacements. For production clusters, start with nodes: "20%" and adjust based on your workload's tolerance for rescheduling events.

4. Implement Pod Disruption Budgets

Karpenter respects PodDisruptionBudget objects. Define PDBs for critical deployments to prevent consolidation from violating availability guarantees. Without PDBs, Karpenter might drain a node hosting the last replica of a critical service.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: critical-app-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: critical-app

5. Tag Resources Consistently

Karpenter's tag-based resource discovery relies on consistent tagging across subnets, security groups, and launch templates. Create a tagging convention and enforce it through infrastructure-as-code pipelines. Missing tags cause provisioning failures that are hard to debug.

6. Monitor Spot Interruption Rates

Track how often spot instances are reclaimed. If interruption rates exceed 5% of your fleet per hour, consider increasing the on-demand fallback pool weight or switching instance families to those with more stable spot availability.

7. Version Pin Your AMI

While amiFamily: Bottlerocket auto-discovers the latest AMI, in production you should pin to a specific AMI version to prevent unexpected node image changes during a deployment window. Use amiSelectorTerms with explicit AMI IDs for controlled rollouts.

8. Separate System and Application NodePools

Create dedicated NodePools for system workloads (CoreDNS, ingress controllers, metrics servers) with on-demand capacity and smaller instance sizes. Keep application workloads in separate pools where spot and consolidation can operate more aggressively.

9. Test Failover Behavior

Regularly simulate spot interruptions, zone failures, and capacity constraints. Use tools like aws:ec2-spot-interruption EventBridge events injected manually or chaos engineering frameworks. Validate that critical workloads reschedule within acceptable timeframes.

10. Keep Karpenter Updated

Karpenter's release cadence is rapid — roughly monthly minor versions. Subscribe to the GitHub releases page and test upgrades in staging environments. Version drift beyond 2–3 months often introduces subtle compatibility issues with newer Kubernetes versions or cloud provider API changes.

Troubleshooting Common Issues

Nodes Stuck Pending

If pods remain pending despite Karpenter running, check NodeClaim status first:

kubectl get nodeclaims -o wide
kubectl describe nodeclaim <claim-name>

Common causes include subnet selectors not matching any subnets, security group tags missing, IAM permissions insufficient for instance profile passthrough, or instance capacity shortages in the requested availability zone.

Consolidation Not Triggering

Consolidation requires nodes to be underutilized below a threshold (typically 50% of allocatable resources). Check node utilization:

kubectl top nodes

If nodes are packed densely but consolidation isn't happening, verify the NodePool's consolidationPolicy is set to WhenUnderutilized and that no PodDisruptionBudgets are blocking the drain operation.

High Provisioning Latency

If new nodes take more than 2 minutes to become ready, investigate the AMI boot time, instance initialization scripts, and container image pull times. Bottlerocket AMIs typically boot in under 30 seconds. Pre-warm container images using a DaemonSet or configure image pull-through caches.

Conclusion

Karpenter represents a fundamental shift in Kubernetes node management — from static, human-configured node groups to dynamic, workload-driven provisioning. By operating at pod-level granularity, it eliminates the capacity planning guesswork that plagues traditional autoscaling approaches. The combination of just-in-time provisioning, continuous consolidation, and native spot instance handling delivers both operational simplicity and significant cost savings.

Implementing Karpenter requires thoughtful NodePool design, proper IAM configuration, and a monitoring framework that surfaces provisioning health. Start with the patterns described here — a general-purpose pool, spot with on-demand fallback, consolidation budgets, and PodDisruptionBudgets — then evolve your configuration as workload patterns emerge. The investment in learning Karpenter's abstractions pays dividends in cluster efficiency, developer velocity, and cloud infrastructure cost reduction.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles