Event Grid Best Practices: Cost, Security, and Performance

Introduction to Azure Event Grid

Azure Event Grid is a fully managed event-routing service that enables you to build reactive, event-driven applications using a publish-subscribe model. It acts as the central nervous system of your cloud architecture, ingesting events from various sources—such as Azure Blob Storage, Azure IoT Hub, custom applications, or third-party services—and reliably routing them to subscriber endpoints like Azure Functions, Webhooks, Event Hubs, or Service Bus queues.

Unlike traditional message brokers that focus on high-throughput streaming or command-style messaging, Event Grid is purpose-built for discrete event notifications. An event signals that something has happened—a blob was created, a resource was provisioned, a user signed in—and typically carries a lightweight payload describing what occurred. Event Grid guarantees delivery with retry policies, supports advanced filtering so subscribers receive only the events they care about, and scales dynamically to handle millions of events per second across diverse Azure and non-Azure environments.

Why Cost, Security, and Performance Matter

When you adopt Event Grid in production, three pillars demand deliberate attention: cost, security, and performance. Each directly impacts the reliability and efficiency of your event-driven workflows.

Cost can spiral unexpectedly if you overlook how Event Grid's billing model works. The service charges per operation—every ingress event, every delivery attempt, and every filtered-out event counts. Without careful architectural planning, a high-volume source combined with broad fan-out subscriptions can multiply your bill dramatically. Understanding the metering granularity and optimizing event flow is essential to keep costs predictable.

Security is paramount because Event Grid often sits at the intersection of sensitive data flows and cross-service communication. Improperly configured access controls can expose internal events to unauthorized consumers, while weak validation on webhook endpoints can open vectors for injection attacks or data leakage. You must treat Event Grid topics and subscriptions as protected assets within your zero-trust perimeter.

Performance dictates whether your system meets latency SLAs and handles spikes gracefully. Event Grid offers a default throughput ceiling per topic, and exceeding it triggers throttling with 429 responses. Subscriber endpoints must be provisioned to absorb bursts, and retry back-off logic must be tuned to avoid cascading failures. Performance tuning ensures your pipeline remains resilient under load without dropping events or introducing unacceptable delays.

Understanding Event Grid's Billing and Metering Model

Before diving into best practices, it helps to understand exactly how Event Grid charges you. The service meters the following operations:

• Incoming events: Each event published to a custom topic or system topic counts as one billable operation if it exceeds 64 KB in size (events under 64 KB are billed as one operation; larger events are prorated).
• Delivery attempts: Every attempt to deliver an event to a subscriber endpoint is billable, including retries. If an event is delivered successfully on the first try, you pay for one delivery operation. If it fails and retries three times before succeeding or being dead-lettered, you pay for all four attempts.
• Filtered events: Even if a subscription filter discards an event before delivery, the filtering evaluation itself is metered. This means broad subscription filters that catch many events but deliver few still incur costs for the filtering operations.
• Dead-letter events: When events are sent to a dead-letter container (blob storage), the write operation to that storage account incurs separate Storage charges on top of Event Grid billing.

There is no base charge for having a topic or domain provisioned—you pay only for usage. However, that usage can accumulate rapidly, so cost-conscious design starts from day one.

Cost Optimization Best Practices

1. Consolidate Topics with Domains Instead of Many Isolated Topics

If your architecture requires multiple logical channels for different event types, avoid creating dozens of separate custom topics. Instead, use an Event Grid Domain, which lets you create many topic endpoints under a single domain resource. Domains share the underlying infrastructure and simplify management, but more importantly they reduce the overhead of managing permissions and endpoints across scattered topics. From a cost perspective, domains themselves do not alter per-event pricing, but they reduce the operational complexity that often leads to over-provisioning of redundant topics and unnecessary event duplication.

# Create an Event Grid Domain via Azure CLI
az eventgrid domain create \
  --name "orders-domain" \
  --resource-group "rg-events-prod" \
  --location "westus2" \
  --sku "Basic"

# Create a topic space within the domain (logical topic)
az eventgrid domain topic create \
  --domain-name "orders-domain" \
  --name "order-created" \
  --resource-group "rg-events-prod"

2. Design Precise Subscription Filters

Every event that passes through a filter evaluation incurs a cost. If you create a subscription with a broad filter like subjectBeginsWith: "/" and then discard most events inside your handler, you pay for the filtering of every single event even though your handler ignores them. Instead, push filtering logic into the subscription definition itself so that Event Grid only routes relevant events to your endpoint.

Use advanced filtering with operators like contains, in, numberIn, and boolEquals to narrow events at the routing layer, before they ever reach your subscriber. This reduces both delivery costs and unnecessary compute consumption in your handler.

# ARM template snippet: subscription with precise advanced filtering
{
  "type": "Microsoft.EventGrid/systemTopics/eventSubscriptions",
  "name": "[concat(parameters('storageAccountName'), '/blob-created-subscription')]",
  "properties": {
    "destination": {
      "endpointType": "WebHook",
      "properties": {
        "endpointUrl": "[parameters('functionAppEndpoint')]"
      }
    },
    "filter": {
      "subjectBeginsWith": "/blobServices/default/containers/invoices/",
      "subjectEndsWith": ".pdf",
      "includedEventTypes": ["Microsoft.Storage.BlobCreated"],
      "advancedFilters": [
        {
          "operatorType": "NumberLessThan",
          "key": "data.api",
          "values": [10],
          "operator": "LessThan"
        }
      ]
    }
  }
}

3. Batch Events Where Possible

Event Grid supports publishing arrays of events in a single API call. Batching reduces the number of HTTP operations you perform and, while the per-event metering still applies, it minimizes the overhead of connection setup and reduces the likelihood of hitting rate limits on the publisher side. Always batch events when you control the producer.

// Publishing a batch of events to a custom topic using .NET SDK
var events = new List
{
    new EventGridEvent(
        "order-123",
        "OrderCreated",
        "v1",
        new { orderId = "123", total = 99.95m }),
    new EventGridEvent(
        "order-124",
        "OrderCreated", 
        "v1",
        new { orderId = "124", total = 149.99m }),
    new EventGridEvent(
        "order-125",
        "OrderCreated",
        "v1",
        new { orderId = "125", total = 49.50m })
};

var topicEndpoint = new Uri("https://orders-topic.westus2-1.eventgrid.azure.net/api/events");
var credential = new AzureKeyCredential("your-access-key");
var client = new EventGridPublisherClient(topicEndpoint, credential);

// Send all events in a single batch
await client.SendEventsAsync(events);
Console.WriteLine($"Published {events.Count} events in one batch operation.");

4. Set Appropriate Retry Policies to Avoid Delivery Multiplication

Event Grid's default retry policy attempts delivery up to 24 times over 24 hours with exponential back-off. If your subscriber endpoint is unreliable, a single event can generate many billable delivery attempts before being dead-lettered. Configure a tighter retry policy for subscriptions where fast failure is acceptable, or ensure your endpoint is robust enough to succeed on the first attempt.

# Create a subscription with a custom retry policy (max 5 attempts, every 30 seconds)
az eventgrid event-subscription create \
  --name "low-latency-subscription" \
  --source-resource-id "/subscriptions/.../resourceGroups/rg-events/providers/Microsoft.EventGrid/topics/orders-topic" \
  --endpoint-type "webhook" \
  --endpoint "https://myapp.azurewebsites.net/api/events" \
  --max-delivery-attempts 5 \
  --event-delivery-schema "EventGridSchema" \
  --retry-policy-max-delivery-attempts 5 \
  --retry-policy-event-time-to-live "PT10M"

5. Monitor and Set Alerts on Event Grid Metrics

Azure exposes metrics like PublishSuccessCount, DeliverySuccessCount, and M matchedEvents through Azure Monitor. Track these daily to detect unexpected volume spikes. A sudden surge in matched events (those evaluated against filters) or delivery attempts can signal a misconfiguration or a noisy publisher that needs throttling at the source.

# Query Event Grid metrics via Azure CLI for the last hour
az monitor metrics list \
  --resource "/subscriptions/{sub}/resourceGroups/rg-events/providers/Microsoft.EventGrid/topics/orders-topic" \
  --metric "PublishSuccessCount" \
  --interval "PT5M" \
  --start-time "$(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ)" \
  --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --output table

Security Best Practices

1. Use Managed Identity for Delivery Instead of Shared Access Keys

When Event Grid delivers events to Azure Functions, Service Bus, or Event Hubs, avoid embedding connection strings or access keys directly in the subscription definition. Instead, configure the subscription to use a system-assigned or user-assigned managed identity. Event Grid obtains a token from Azure AD at delivery time, eliminating the risk of key leakage and simplifying credential rotation.

# Create a system-assigned managed identity for the Event Grid System Topic
az eventgrid system-topic identity assign \
  --name "storage-account-topic" \
  --resource-group "rg-events-prod" \
  --identity-type "SystemAssigned"

# Grant the identity "Azure Service Bus Data Sender" role on the target Service Bus
az role assignment create \
  --assignee "00000000-0000-0000-0000-000000000000" \  # Replace with actual principal ID
  --role "Azure Service Bus Data Sender" \
  --scope "/subscriptions/{sub}/resourceGroups/rg-events/providers/Microsoft.ServiceBus/namespaces/mynamespace"

# Create subscription with managed identity delivery
az eventgrid event-subscription create \
  --name "servicebus-subscription" \
  --source-resource-id "/subscriptions/{sub}/resourceGroups/rg-events/providers/Microsoft.EventGrid/systemTopics/storage-account-topic" \
  --endpoint-type "ServiceBusQueue" \
  --endpoint "/subscriptions/{sub}/resourceGroups/rg-events/providers/Microsoft.ServiceBus/namespaces/mynamespace/queues/myqueue" \
  --delivery-identity "system"

2. Validate Webhook Endpoints with Handshake Events

For webhook subscribers, Event Grid sends a validation handshake event when the subscription is first created. Your endpoint must respond with the validation code to prove ownership and prevent malicious subscription hijacking. Always implement the handshake correctly—this is not optional; Event Grid will not deliver production events until validation succeeds.

// ASP.NET Core webhook handler with proper handshake validation
[HttpPost("api/events")]
public async Task ReceiveEvents(
    [FromHeader(Name = "aeg-event-type")] string eventType)
{
    if (string.Equals(eventType, "SubscriptionValidation", 
        StringComparison.OrdinalIgnoreCase))
    {
        using var reader = new StreamReader(Request.Body);
        var body = await reader.ReadToEndAsync();
        var validationEvent = JsonSerializer.Deserialize(body)?[0];
        var validationCode = validationEvent?.Data?.GetProperty("validationCode")
            .GetString();

        return new OkObjectResult(new 
        { 
            validationResponse = validationCode 
        });
    }

    // Process regular events
    // ... event handling logic ...

    return new OkResult();
}

3. Apply IP Restrictions and Private Endpoints

Lock down your Event Grid topics so that only known networks can publish events. Use IP firewall rules to restrict inbound publication traffic to your corporate CIDR ranges, or deploy Event Grid behind a Private Endpoint so that traffic traverses your virtual network rather than the public internet. This is especially critical for custom topics handling sensitive business events.

# Create a private endpoint for an Event Grid topic
az network private-endpoint create \
  --name "eventgrid-private-endpoint" \
  --resource-group "rg-events-prod" \
  --location "westus2" \
  --connection-name "eventgrid-connection" \
  --private-connection-resource-id "/subscriptions/{sub}/resourceGroups/rg-events/providers/Microsoft.EventGrid/topics/orders-topic" \
  --group-id "topic" \
  --vnet-name "vnet-prod" \
  --subnet "private-endpoints-subnet"

# Apply IP firewall rules to the topic (allowing only specific ranges)
az eventgrid topic update \
  --name "orders-topic" \
  --resource-group "rg-events-prod" \
  --public-network-access "disabled" \
  --inbound-ip-rules "[{'IpMask':'10.100.0.0/16','Action':'Allow'},{'IpMask':'192.168.1.0/24','Action':'Allow'}]"

4. Use SAS Tokens or RBAC for Publishing Access

Never hard-code the primary access key of your Event Grid topic in application configuration. Instead, generate short-lived Shared Access Signature (SAS) tokens scoped to a specific topic and with a limited validity window. Alternatively, assign RBAC roles like EventGridDataSender to your publisher identities (Azure Functions, AKS pods, etc.) so they authenticate with Azure AD rather than raw keys.

// Generate a SAS token for an Event Grid topic programmatically
using System.Security.Cryptography;
using System.Text;

public static string GenerateEventGridSasToken(
    string topicEndpoint, 
    string accessKey, 
    DateTime expiryUtc)
{
    var resourcePath = new Uri(topicEndpoint).PathAndQuery;
    var expirySeconds = new DateTimeOffset(expiryUtc).ToUnixTimeSeconds();
    var stringToSign = $"{resourcePath}\n{expirySeconds}";
    
    using var hmac = new HMACSHA256(Convert.FromBase64String(accessKey));
    var hash = hmac.ComputeHash(Encoding.UTF8.GetBytes(stringToSign));
    var signature = Convert.ToBase64String(hash);
    
    return $"sr={Uri.EscapeDataString(resourcePath)}&se={expirySeconds}&sig={Uri.EscapeDataString(signature)}";
}

// Usage: publish with SAS token in the Authorization header
var sasToken = GenerateEventGridSasToken(
    "https://orders-topic.westus2-1.eventgrid.azure.net/api/events",
    "your-access-key",
    DateTime.UtcNow.AddHours(1));

var client = new EventGridPublisherClient(
    new Uri("https://orders-topic.westus2-1.eventgrid.azure.net/api/events"),
    new AzureSasCredential(sasToken));

5. Enable Dead-Letter with Encrypted Storage

Dead-lettering captures events that cannot be delivered after exhausting retries, preserving them for forensic analysis. Always point the dead-letter destination to a storage account that has encryption enabled (either Microsoft-managed or customer-managed keys) and apply strict access policies so only authorized operators can inspect failed events, which may contain sensitive payloads.

# Create a subscription with dead-lettering to an encrypted blob storage account
az eventgrid event-subscription create \
  --name "dlq-enabled-subscription" \
  --source-resource-id "/subscriptions/{sub}/resourceGroups/rg-events/providers/Microsoft.EventGrid/topics/orders-topic" \
  --endpoint-type "webhook" \
  --endpoint "https://myhandler.azurewebsites.net/api/events" \
  --deadletter-endpoint "/subscriptions/{sub}/resourceGroups/rg-events/providers/Microsoft.Storage/storageAccounts/eventgriddlqstorage/blobServices/default/containers/deadletters" \
  --max-delivery-attempts 10

Performance Best Practices

1. Understand and Design Around Throughput Limits

Event Grid topics have a default ingress rate of 5,000 events per second per topic, and egress (delivery) rates scale proportionally. If you anticipate higher throughput, request a quota increase through Azure Support well before go-live. For bursty workloads, implement client-side buffering and exponential back-off when you receive HTTP 429 (Too Many Requests) responses from the Event Grid API.

// Resilient publisher with exponential back-off for 429 handling
public async Task PublishWithRetryAsync(
    EventGridPublisherClient client, 
    IEnumerable events, 
    int maxRetries = 5)
{
    var retryCount = 0;
    var delay = TimeSpan.FromMilliseconds(500);

    while (retryCount <= maxRetries)
    {
        try
        {
            await client.SendEventsAsync(events);
            return; // Success
        }
        catch (RequestFailedException ex) when (ex.Status == 429)
        {
            retryCount++;
            if (retryCount > maxRetries)
                throw;

            Console.WriteLine($"Throttled. Retry {retryCount} after {delay.TotalSeconds}s");
            await Task.Delay(delay);
            delay = TimeSpan.FromSeconds(Math.Pow(2, retryCount)); // Exponential back-off
        }
    }
}

2. Choose the Right Subscriber Endpoint Type for Latency Sensitivity

Different endpoint types offer different latency profiles. Azure Functions with Event Grid triggers provide the lowest end-to-end latency because the Function runtime is optimized for Event Grid's delivery protocol. Webhook endpoints introduce variable network latency. Service Bus and Event Hubs add buffering that increases latency slightly but improves resilience for downstream processors that need to consume at their own pace. Match the endpoint type to your latency SLA.

# Azure Function with Event Grid trigger (C# isolated process)
public class OrderCreatedHandler
{
    [Function("ProcessOrderCreated")]
    public void Run(
        [EventGridTrigger] EventGridEvent[] events,
        FunctionContext context)
    {
        foreach (var eventGridEvent in events)
        {
            context.Logger.LogInformation(
                "Processing order {orderId} with latency measured from enqueue time",
                eventGridEvent.Data?.GetProperty("orderId").GetString());
            
            // Business logic here—this runs with minimal cold-start overhead
            // because Event Grid trigger uses the Functions scaling controller
        }
    }
}

3. Scale Out Subscribers to Prevent Backpressure

If your subscriber is a webhook running on a fixed-capacity App Service Plan, a sudden burst of events can overwhelm it, causing HTTP 500 or 503 errors that trigger costly retries. Configure horizontal scaling—for Azure Functions, use the Consumption or Premium plan so instances scale out automatically based on event queue depth. For containerized subscribers on AKS, use KEDA (Kubernetes Event-Driven Autoscaling) to scale pods in response to Event Grid metrics.

# KEDA scaled job definition for Event Grid trigger on AKS
apiVersion: keda.sh/v1alpha1
kind: ScaledJob
metadata:
  name: eventgrid-consumer-scaler
spec:
  jobTargetRef:
    template:
      spec:
        containers:
        - name: event-processor
          image: myregistry.azurecr.io/event-processor:latest
  triggers:
  - type: azure-eventgrid
    metadata:
      topicEndpoint: "https://orders-topic.westus2-1.eventgrid.azure.net/api/events"
      subscriptionName: "keda-subscription"
      authenticationMode: "KeyBased"
      accessKey: "env:EVENTGRID_ACCESS_KEY"
      activationCount: "100"
      maxCount: "500"

4. Optimize Event Size and Schema

Event Grid charges extra for events larger than 64 KB. Keep event payloads lean by including only the data necessary for subscribers to act—typically an identifier and a few key properties. If subscribers need full context, have them retrieve the complete record from a source system (like Cosmos DB or a REST API) rather than bloating the event. Use the CloudEvents v1.0 schema for interoperability and consistent metadata across services.

// Lean event payload following CloudEvents v1.0 schema
var cloudEvent = new CloudEvent(
    source: new Uri("/orders-service/europe-west"),
    type: "com.example.order.created.v1",
    subject: "orders/12345",
    data: JsonSerializer.Serialize(new 
    {
        orderId = "12345",
        customerId = "C789",
        totalAmount = 99.95,
        currency = "EUR"
    }),
    dataContentType: "application/json")
{
    Id = Guid.NewGuid().ToString(),
    Time = DateTimeOffset.UtcNow
};

// The subscriber uses orderId to fetch full order details from the Orders API
// rather than carrying every line item in the event payload
await client.SendEventAsync(cloudEvent);

5. Test with Chaos Engineering for Failure Modes

Simulate subscriber outages, network partitions, and throttling scenarios in your staging environment before production rollout. Verify that dead-lettering activates correctly, that retry back-off does not cascade, and that your monitoring alerts fire on delivery failure thresholds. A controlled chaos experiment reveals whether your performance assumptions hold under real-world stress.

# Chaos test script: simulate subscriber outage by stopping the Function App
az functionapp stop \
  --name "my-event-handler-function" \
  --resource-group "rg-events-prod"

# Publish a burst of 10,000 events to the topic
for i in $(seq 1 10000); do
  curl -s -X POST \
    -H "Content-Type: application/json" \
    -H "aeg-sas-key: $EVENTGRID_KEY" \
    -d "[{\"id\":\"$i\",\"subject\":\"test/$i\",\"eventType\":\"Chaos.Test\",\"data\":{\"iteration\":$i},\"eventTime\":\"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"}]" \
    "https://orders-topic.westus2-1.eventgrid.azure.net/api/events"
done

# After 10 minutes, restore the Function App and verify dead-letter container contents
az functionapp start \
  --name "my-event-handler-function" \
  --resource-group "rg-events-prod"

# Query dead-letter blob count to confirm events were preserved
az storage blob list \
  --account-name "eventgriddlqstorage" \
  --container-name "deadletters" \
  --query "length([])"

Conclusion

Azure Event Grid is a powerful backbone for event-driven architectures, but its true value emerges only when you operate it with disciplined attention to cost, security, and performance. By consolidating topics into domains, crafting precise subscription filters, batching events, and tightening retry policies, you keep your monthly bill predictable and aligned with actual business value. By enforcing managed identity authentication, validating webhook handshakes, locking down network access with private endpoints, and rotating SAS tokens, you build a security posture that withstands modern threats. And by designing around throughput limits, selecting the right subscriber endpoint types, scaling consumers dynamically, keeping payloads lean, and validating resilience through chaos testing, you ensure your event pipeline remains fast and reliable even under extreme load.

Start with these practices early in your design phase, instrument every topic and subscription with Azure Monitor alerts, and iterate based on observed traffic patterns. An optimized Event Grid deployment not only reduces operational toil but also unlocks the full potential of real-time, reactive cloud applications.