← Back to DevBytes

Go for System Programming: Step-by-Step Guide to

Go for System Programming: Step-by-Step Guide to Building Robust System Tools

System programming traditionally belongs to the realm of C and C++, where direct memory manipulation, low-level OS interaction, and high performance are paramount. However, Go has carved out a significant niche in this space by offering a compelling blend of simplicity, safety, and raw capability. This tutorial walks you through everything you need to know to start writing system-level software in Go—from basic file operations to advanced process management and signal handling.

What Is Go System Programming?

Go system programming refers to using the Go language to build software that interacts directly with the operating system kernel and hardware resources. This includes tasks such as managing files and directories at a low level, spawning and controlling processes, handling inter-process communication, working with network sockets, catching OS signals, and manipulating memory when necessary. Unlike application-level programming, system programming often requires working with syscalls, understanding kernel behavior, and caring deeply about resource cleanup and error handling. Go's standard library provides extensive support for these operations through packages like os, syscall, net, and os/signal, while the golang.org/x/sys extended library fills in platform-specific gaps.

Why Go Matters for System Programming

Several characteristics make Go uniquely suited for system-level work:

Setting Up Your Environment for System Programming

Before diving into code, ensure your Go environment is ready. You'll want Go 1.21 or later for the best system programming support. Install Go from the official website, then verify your installation:

go version
# go version go1.23.2 linux/amd64

Create a new directory for your system programming experiments and initialize a module:

mkdir go-sys-programming
cd go-sys-programming
go mod init sysprog

For many system programming tasks, you'll want the extended system package. Install it with:

go get golang.org/x/sys/unix

This package provides Unix-specific syscall constants, structures, and functions that the standard syscall package sometimes lacks or deprecates. Note that system programming code is inherently platform-specific. Most examples in this tutorial target Linux/Unix systems. Where Windows differences matter, they will be noted.

Working with Files and Directories at the System Level

The os package is your primary toolkit for file system operations. Beyond simple file reading and writing, system programming requires understanding file modes, ownership, locking, and directory traversal.

Creating, Reading, and Writing Files with Full Control

Here's how to create a file with specific permissions, write data, and then read it back using low-level file descriptors:

package main

import (
    "fmt"
    "os"
)

func main() {
    // Create a file with owner-only read/write permissions (0600)
    file, err := os.OpenFile("testfile.txt", os.O_CREATE|os.O_RDWR, 0600)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Failed to create file: %v\n", err)
        os.Exit(1)
    }
    defer file.Close()

    // Write data
    data := []byte("Hello from Go system programming!\n")
    bytesWritten, err := file.Write(data)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Write error: %v\n", err)
        os.Exit(1)
    }
    fmt.Printf("Wrote %d bytes\n", bytesWritten)

    // Seek back to beginning
    _, err = file.Seek(0, 0)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Seek error: %v\n", err)
        os.Exit(1)
    }

    // Read data back
    buf := make([]byte, 128)
    bytesRead, err := file.Read(buf)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Read error: %v\n", err)
        os.Exit(1)
    }
    fmt.Printf("Read %d bytes: %s", bytesRead, buf[:bytesRead])
}

Notice the use of os.OpenFile instead of os.Create—this gives you explicit control over the flags (os.O_CREATE, os.O_RDWR) and the file mode (0600). For system tools, this level of control is essential.

Recursive Directory Traversal

System tools often need to walk directory trees efficiently. Go's filepath.Walk function handles this elegantly:

package main

import (
    "fmt"
    "os"
    "path/filepath"
)

func main() {
    root := "." // Start from current directory
    err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
        if err != nil {
            fmt.Fprintf(os.Stderr, "Error accessing %s: %v\n", path, err)
            return nil // Continue walking despite the error
        }
        
        // Determine the type indicator
        typeIndicator := " "
        if info.IsDir() {
            typeIndicator = "d"
        } else if info.Mode()&os.ModeSymlink != 0 {
            typeIndicator = "L"
        }
        
        // Format permissions as a string
        perms := info.Mode().String()
        
        fmt.Printf("%s %s %8d %s\n", typeIndicator, perms, info.Size(), path)
        return nil
    })
    if err != nil {
        fmt.Fprintf(os.Stderr, "Walk error: %v\n", err)
        os.Exit(1)
    }
}

This example mimics a simplified ls -laR output, showing file type, permissions, size, and path. The callback approach allows you to handle errors per-entry without aborting the entire traversal.

Process Management

Process management is at the heart of system programming. Go allows you to spawn new processes, wait for their completion, pipe data between them, and even replace the current process image entirely.

Spawning and Waiting for Child Processes

The os/exec package provides a safe, high-level interface for running external commands:

package main

import (
    "fmt"
    "os"
    "os/exec"
)

func main() {
    // Construct the command
    cmd := exec.Command("ls", "-la", "/tmp")
    
    // Set environment variables for the child process
    cmd.Env = append(os.Environ(), "LC_ALL=C")
    
    // Capture stdout and stderr
    cmd.Stdout = os.Stdout
    cmd.Stderr = os.Stderr
    
    // Run and wait for completion
    err := cmd.Run()
    if err != nil {
        fmt.Fprintf(os.Stderr, "Command failed: %v\n", err)
        os.Exit(1)
    }
    
    // Get the exit code
    exitCode := cmd.ProcessState.ExitCode()
    fmt.Printf("Command exited with code: %d\n", exitCode)
}

Piping Data Between Processes

For more complex scenarios, you can chain processes together by connecting their standard streams programmatically, similar to shell pipes:

package main

import (
    "bytes"
    "fmt"
    "os"
    "os/exec"
)

func main() {
    // First command: echo "hello world"
    cmd1 := exec.Command("echo", "hello world")
    
    // Second command: tr 'a-z' 'A-Z' (uppercase conversion)
    cmd2 := exec.Command("tr", "a-z", "A-Z")
    
    // Create a pipe between cmd1's stdout and cmd2's stdin
    var buf bytes.Buffer
    cmd1.Stdout = &buf
    cmd2.Stdin = &buf
    
    // Capture final output
    cmd2.Stdout = os.Stdout
    
    // Run both commands sequentially
    if err := cmd1.Run(); err != nil {
        fmt.Fprintf(os.Stderr, "cmd1 failed: %v\n", err)
        os.Exit(1)
    }
    if err := cmd2.Run(); err != nil {
        fmt.Fprintf(os.Stderr, "cmd2 failed: %v\n", err)
        os.Exit(1)
    }
}

For true concurrent piping (where both processes run simultaneously), use io.Pipe or the cmd.StdinPipe / cmd.StdoutPipe methods and run the commands in separate goroutines. This prevents deadlocks when buffer sizes are exceeded.

Replacing the Current Process (Exec)

When you need to completely replace the current process with a new one—a classic syscall operation—use syscall.Exec from the golang.org/x/sys package or the standard library's os package on some platforms. On Unix systems:

package main

import (
    "fmt"
    "os"
    "syscall"
    "golang.org/x/sys/unix"
)

func main() {
    binary, err := exec.LookPath("ls")
    if err != nil {
        fmt.Fprintf(os.Stderr, "Could not find 'ls': %v\n", err)
        os.Exit(1)
    }
    
    args := []string{"ls", "-la", "/home"}
    env := os.Environ()
    
    // This replaces the current process with 'ls'
    // Any code after this call will never execute if successful
    err = unix.Exec(binary, args, env)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Exec failed: %v\n", err)
        os.Exit(1)
    }
}

This is the underlying mechanism behind Docker containers and init systems. The current Go process vanishes entirely and is replaced by the new executable, inheriting the same PID and file descriptors.

Interacting with the Operating System via Syscalls

Sometimes the os package isn't low-level enough. For direct syscall access, Go provides the syscall package (standard library) and the more comprehensive golang.org/x/sys/unix package. These let you invoke raw kernel system calls.

Getting System Information (uname)

package main

import (
    "fmt"
    "os"
    "golang.org/x/sys/unix"
)

func main() {
    var utsname unix.Utsname
    err := unix.Uname(&utsname)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Uname failed: %v\n", err)
        os.Exit(1)
    }
    
    // Convert fixed-size byte arrays to strings
    sysname := string(utsname.Sysname[:])
    nodename := string(utsname.Nodename[:])
    release := string(utsname.Release[:])
    version := string(utsname.Version[:])
    machine := string(utsname.Machine[:])
    
    fmt.Printf("System:    %s\n", sysname)
    fmt.Printf("Node:      %s\n", nodename)
    fmt.Printf("Release:   %s\n", release)
    fmt.Printf("Version:   %s\n", version)
    fmt.Printf("Machine:   %s\n", machine)
}

Retrieving Resource Usage Statistics

The syscall.Getrusage function provides detailed resource usage for the calling process:

package main

import (
    "fmt"
    "os"
    "syscall"
)

func main() {
    var usage syscall.Rusage
    err := syscall.Getrusage(syscall.RUSAGE_SELF, &usage)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Getrusage failed: %v\n", err)
        os.Exit(1)
    }
    
    fmt.Printf("User CPU time:   %d.%06d seconds\n", 
        usage.Utime.Sec, usage.Utime.Usec)
    fmt.Printf("System CPU time: %d.%06d seconds\n", 
        usage.Stime.Sec, usage.Stime.Usec)
    fmt.Printf("Max RSS:         %d kilobytes\n", usage.Maxrss)
    fmt.Printf("Page faults:     %d (major), %d (minor)\n", 
        usage.Majflt, usage.Minflt)
}

Networking and Sockets at the System Level

While Go's net package abstracts most networking needs beautifully, system programming sometimes requires raw socket operations—setting socket options, binding to specific interfaces, or working with Unix domain sockets for local IPC.

Creating a Raw Unix Domain Socket Server

package main

import (
    "fmt"
    "net"
    "os"
)

func main() {
    socketPath := "/tmp/go-sys-socket.sock"
    
    // Clean up any existing socket file
    os.Remove(socketPath)
    
    // Listen on a Unix domain socket
    listener, err := net.Listen("unix", socketPath)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Listen error: %v\n", err)
        os.Exit(1)
    }
    defer listener.Close()
    defer os.Remove(socketPath)
    
    fmt.Printf("Listening on %s\n", socketPath)
    
    for {
        conn, err := listener.Accept()
        if err != nil {
            fmt.Fprintf(os.Stderr, "Accept error: %v\n", err)
            continue
        }
        
        // Handle connection in a goroutine
        go handleUnixConn(conn)
    }
}

func handleUnixConn(conn net.Conn) {
    defer conn.Close()
    
    buf := make([]byte, 1024)
    n, err := conn.Read(buf)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Read error: %v\n", err)
        return
    }
    
    fmt.Printf("Received: %s\n", buf[:n])
    
    // Echo back
    conn.Write([]byte(fmt.Sprintf("Server received: %s", buf[:n])))
}

Setting Socket Options with syscall

For low-level socket option manipulation, you can extract the file descriptor from a net.Conn and use syscall.SetsockoptInt:

package main

import (
    "fmt"
    "net"
    "os"
    "syscall"
)

func main() {
    // Create a TCP listener
    listener, err := net.Listen("tcp", ":0") // port 0 = ephemeral
    if err != nil {
        fmt.Fprintf(os.Stderr, "Listen error: %v\n", err)
        os.Exit(1)
    }
    defer listener.Close()
    
    // Get the underlying file descriptor
    tcpListener := listener.(*net.TCPListener)
    listenerFile, err := tcpListener.File()
    if err != nil {
        fmt.Fprintf(os.Stderr, "Failed to get file: %v\n", err)
        os.Exit(1)
    }
    defer listenerFile.Close()
    
    fd := int(listenerFile.Fd())
    
    // Enable SO_REUSEADDR to allow quick rebinding
    err = syscall.SetsockoptInt(fd, syscall.SOL_SOCKET, syscall.SO_REUSEADDR, 1)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Setsockopt error: %v\n", err)
        os.Exit(1)
    }
    
    fmt.Printf("Socket options set successfully on fd %d\n", fd)
    
    // Accept connections as normal
    addr := listener.Addr()
    fmt.Printf("Listening on %s\n", addr.String())
}

Signal Handling

Robust system programs must handle OS signals gracefully—cleaning up resources, saving state, and shutting down cleanly when receiving SIGTERM or SIGINT. Go's os/signal package makes this straightforward.

Catching and Handling Signals

package main

import (
    "context"
    "fmt"
    "os"
    "os/signal"
    "syscall"
    "time"
)

func main() {
    // Create a context that cancels on signal
    ctx, cancel := context.WithCancel(context.Background())
    defer cancel()
    
    // Create a channel to receive signals
    sigChan := make(chan os.Signal, 1)
    
    // Register for SIGINT (Ctrl+C) and SIGTERM (kill command)
    signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
    
    // Start a goroutine that watches for signals
    go func() {
        sig := <-sigChan
        fmt.Printf("\nReceived signal: %v\n", sig)
        fmt.Println("Shutting down gracefully...")
        cancel() // Trigger context cancellation
    }()
    
    // Simulate work
    fmt.Println("Service running. Press Ctrl+C to stop.")
    fmt.Printf("PID: %d\n", os.Getpid())
    
    // Main work loop that respects context cancellation
    ticker := time.NewTicker(2 * time.Second)
    defer ticker.Stop()
    
    for {
        select {
        case <-ctx.Done():
            fmt.Println("Context cancelled, performing cleanup...")
            // Perform cleanup here: close files, flush buffers, etc.
            time.Sleep(500 * time.Millisecond) // Simulate cleanup
            fmt.Println("Cleanup complete. Goodbye.")
            return
        case <-ticker.C:
            fmt.Println("Doing work...")
        }
    }
}

Ignoring and Resetting Signals

You can also selectively ignore signals or reset them to default behavior:

package main

import (
    "fmt"
    "os"
    "os/signal"
    "syscall"
    "time"
)

func main() {
    // Ignore SIGHUP (terminal disconnect) entirely
    signal.Ignore(syscall.SIGHUP)
    
    // Catch SIGINT with a custom handler
    sigChan := make(chan os.Signal, 1)
    signal.Notify(sigChan, syscall.SIGINT)
    
    go func() {
        <-sigChan
        fmt.Println("SIGINT received, but we're resilient!")
        // Reset SIGINT to default behavior for the next occurrence
        signal.Reset(syscall.SIGINT)
    }()
    
    fmt.Println("Running with modified signal dispositions...")
    fmt.Printf("PID: %d — Try sending SIGHUP and SIGINT\n", os.Getpid())
    
    // Run for 30 seconds
    time.Sleep(30 * time.Second)
    fmt.Println("Exiting normally.")
}

Memory Management and Unsafe Operations

System programming occasionally requires stepping outside Go's safe memory model. The unsafe package allows pointer arithmetic and type reinterpretation, which is necessary when constructing complex structures for syscalls or interacting with memory-mapped devices.

Using unsafe for Syscall Structure Construction

package main

import (
    "fmt"
    "os"
    "unsafe"
    "golang.org/x/sys/unix"
)

func main() {
    // Allocate a buffer for a complex ioctl structure
    // This demonstrates the pattern, not a specific working ioctl
    
    // Create a byte buffer
    buf := make([]byte, 4096)
    
    // Get a pointer to the buffer's underlying array
    // This is an unsafe operation that bypasses Go's type safety
    ptr := unsafe.Pointer(&buf[0])
    
    // In real system programming, you'd cast this to a C struct pointer
    // and pass it to a syscall like ioctl()
    _ = ptr
    
    // Safer alternative: use the unix package's typed helpers
    var stat unix.Stat_t
    err := unix.Stat(".", &stat)
    if err != nil {
        fmt.Fprintf(os.Stderr, "Stat error: %v\n", err)
        os.Exit(1)
    }
    
    fmt.Printf("Inode: %d\n", stat.Ino)
    fmt.Printf("Blocks: %d\n", stat.Blocks)
    fmt.Printf("Size of Stat_t struct: %d bytes\n", unsafe.Sizeof(stat))
}

Use unsafe sparingly and only when the standard or extended library doesn't provide a safe wrapper. Most common syscalls already have safe Go interfaces in golang.org/x/sys/unix.

Building a Complete System Tool: A File Watcher with Signal Handling

Let's combine everything into a practical, complete system tool—a file watcher that monitors a directory for changes using the inotify subsystem (on Linux) and gracefully shuts down on signals:

package main

import (
    "fmt"
    "os"
    "os/signal"
    "path/filepath"
    "syscall"
    "golang.org/x/sys/unix"
)

func main() {
    if len(os.Args) < 2 {
        fmt.Fprintf(os.Stderr, "Usage: %s \n", os.Args[0])
        os.Exit(1)
    }
    
    watchDir := os.Args[1]
    
    // Validate the directory exists
    dirInfo, err := os.Stat(watchDir)
    if err != nil || !dirInfo.IsDir() {
        fmt.Fprintf(os.Stderr, "Error: %s is not a valid directory\n", watchDir)
        os.Exit(1)
    }
    
    // Create an inotify instance
    inotifyFd, err := unix.InotifyInit1(unix.IN_CLOEXEC)
    if err != nil {
        fmt.Fprintf(os.Stderr, "InotifyInit1 error: %v\n", err)
        os.Exit(1)
    }
    defer unix.Close(inotifyFd)
    
    // Add a watch on the directory
    watchDescriptor, err := unix.InotifyAddWatch(inotifyFd, watchDir, 
        unix.IN_CREATE|unix.IN_DELETE|unix.IN_MODIFY|unix.IN_MOVED_FROM|unix.IN_MOVED_TO)
    if err != nil {
        fmt.Fprintf(os.Stderr, "InotifyAddWatch error: %v\n", err)
        os.Exit(1)
    }
    
    fmt.Printf("Watching directory: %s (Press Ctrl+C to stop)\n", watchDir)
    fmt.Printf("PID: %d\n", os.Getpid())
    
    // Set up signal handling
    sigChan := make(chan os.Signal, 1)
    signal.Notify(sigChan, syscall.SIGINT, syscall.SIGTERM)
    
    // Buffer for reading inotify events
    eventBuf := make([]byte, 4096)
    
    // Main event loop
    for {
        // Use select to handle both signals and inotify events
        // We need to make the inotify read non-blocking or use a goroutine
        // For simplicity, we'll poll with a short timeout using a goroutine
        
        done := make(chan bool, 1)
        eventCh := make(chan string, 10)
        
        // Goroutine to read inotify events
        go func() {
            n, err := unix.Read(inotifyFd, eventBuf)
            if err != nil {
                if err == syscall.EINTR {
                    // Interrupted by signal, try again
                    done <- false
                    return
                }
                fmt.Fprintf(os.Stderr, "Read error: %v\n", err)
                done <- true
                return
            }
            
            if n > 0 {
                events, parseErr := parseInotifyEvents(eventBuf[:n], watchDescriptor)
                if parseErr != nil {
                    fmt.Fprintf(os.Stderr, "Parse error: %v\n", parseErr)
                }
                for _, evt := range events {
                    eventCh <- evt
                }
            }
            done <- false
        }()
        
        select {
        case sig := <-sigChan:
            fmt.Printf("\nReceived signal: %v\n", sig)
            fmt.Println("Removing watch and cleaning up...")
            unix.InotifyRmWatch(inotifyFd, uint32(watchDescriptor))
            fmt.Println("Cleanup complete. Goodbye.")
            return
            
        case evt := <-eventCh:
            fmt.Printf("[%s]\n", evt)
            
        case <-done:
            // Read goroutine completed (or errored)
        }
    }
}

// parseInotifyEvents converts raw bytes into human-readable event descriptions
func parseInotifyEvents(buf []byte, wd int) ([]string, error) {
    var events []string
    offset := 0
    
    for offset < len(buf) {
        if offset+unix.SizeofInotifyEvent > len(buf) {
            break
        }
        
        // Interpret raw bytes as an InotifyEvent structure
        event := (*unix.InotifyEvent)(unsafe.Pointer(&buf[offset]))
        
        // Calculate offset to the name (if present)
        nameOffset := offset + unix.SizeofInotifyEvent
        var name string
        if event.Len > 0 {
            nameBytes := buf[nameOffset : nameOffset+int(event.Len)]
            // The name is null-terminated
            for i, b := range nameBytes {
                if b == 0 {
                    name = string(nameBytes[:i])
                    break
                }
            }
        }
        
        // Build event description
        desc := ""
        if event.Mask&unix.IN_CREATE != 0 {
            desc = "CREATED"
        } else if event.Mask&unix.IN_DELETE != 0 {
            desc = "DELETED"
        } else if event.Mask&unix.IN_MODIFY != 0 {
            desc = "MODIFIED"
        } else if event.Mask&unix.IN_MOVED_FROM != 0 {
            desc = "MOVED_FROM"
        } else if event.Mask&unix.IN_MOVED_TO != 0 {
            desc = "MOVED_TO"
        }
        
        if desc != "" && name != "" {
            events = append(events, fmt.Sprintf("%s %s", desc, name))
        }
        
        // Advance to the next event
        offset += unix.SizeofInotifyEvent + int(event.Len)
    }
    
    return events, nil
}

This file watcher demonstrates real system programming patterns: direct kernel subsystem interaction (inotify), unsafe pointer casting for binary structure parsing, signal handling for graceful shutdown, and proper resource cleanup. To use it, save it as watcher.go and run:

go run watcher.go /tmp

Then create, modify, or delete files in the watched directory from another terminal to see events stream in real time.

Best Practices for Go System Programming

Advanced Topics to Explore

Once you're comfortable with the fundamentals covered here, several deeper areas await. Seccomp and sandboxing allow you to restrict your process's own syscall surface using libseccomp bindings. Namespaces and cgroups—the building blocks of containers—can be manipulated directly from Go using the unix package's namespace syscalls. BPF (Berkeley Packet Filter) tracing and network filtering is accessible through packages like github.com/cilium/ebpf, enabling deep observability. Memory-mapped I/O via syscall.Mmap offers extreme performance for file access patterns that benefit from it. Each of these topics builds on the foundation you've established here.

Conclusion

Go occupies a sweet spot for system programming—it offers the safety and productivity of a modern language while providing direct access to the kernel interfaces that system software requires. Through its comprehensive standard library, the extended golang.org/x/sys package, and the judicious use of unsafe when necessary, you can build everything from simple file watchers to complex container runtimes entirely in Go. The language's concurrency model, static compilation, and cross-platform support make it an increasingly popular choice for infrastructure teams building the next generation of system tools. Start with the patterns in this guide, practice the examples, and you'll be well-equipped to tackle real-world system programming challenges with confidence.

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles