Implement Structured Logging for Production

Structured logging in production microservices and modular monoliths requires moving beyond trivial fmt.Println statements or naive global logger instances. Real-world applications demand explicit dependency injection, thread-safe request isolation, consistent correlation IDs across distributed tiers, and type-safe contextual value propagation. This guide explores the architectural refactoring required to transform basic Go slog implementations into decoupled, production-ready logging pipelines.

1. The Pitfalls of Global Loggers in Production Systems

Using a global logger instance (e.g., slog.SetDefault() coupled with direct package-level calls like slog.Info() or slog.Error()) is common in small prototypes. However, in production-grade software, global state introduces three critical architectural liabilities:

Problem 1: Hidden Dependencies

When a handler, service, or repository invokes a global logger directly, its dependencies remain hidden from its public interface. Looking at a constructor like:

func NewListingHandler(db *sql.DB) *ListingHandler

A consumer assumes the handler only depends on the database. If logging calls are buried within its methods, logging becomes an undeclared side-effect. Idiomatic Go emphasizes transparency: components should declare every dependency they require to operate.

Problem 2: Impaired Testing and Concurrency Hazards

Unit and integration tests run in parallel (t.Parallel()). If components rely on a shared global logger:

• Tests cannot safely mock, redirect, or swap the logger without inducing data races on shared memory.
• If Test A redirects global log output to an in-memory buffer to verify error logging while Test B alters the global log level to DEBUG, both tests cross-contaminate their configurations.
• Swapping a real disk or network logger for a test buffer requires explicit, localized control.

Problem 3: Inability to Apply Per-Component Configurations

Large systems frequently require distinct logging policies across boundaries:

• A high-throughput ingest pipeline might run strictly at WARN or ERROR levels with source tracing disabled to preserve CPU cycles.
• A critical payment gateway component within the same application might require DEBUG logging with detailed metadata.
• With a monolithic global logger, granular configuration per component is difficult to achieve.

2. Decoupling Loggers via Explicit Dependency Injection

To eliminate global state, pass the logger explicitly to domain components through struct constructors. This mirrors how database pools and network clients are managed.

Main Entrypoint (main.go)
   |
   | creates *slog.Logger instance
   v
ListingsHandler Constructor
   |
   | assigns logger to struct field
   v
Handler Methods (lh.logger.Info / lh.logger.Error)

Refactoring the Handler Struct and Constructor

Instead of referencing slog globally, embed the logger instance into the receiver struct:

package handlers

import (
	"database/sql"
	"log/slog"
)

type ListingsHandler struct {
	db     *sql.DB
	logger *slog.Logger
}

// NewListingsHandler explicitly declares all external dependencies.
func NewListingsHandler(db *sql.DB, logger *slog.Logger) *ListingsHandler {
	return &ListingsHandler{
		db:     db,
		logger: logger,
	}
}

Method Receiver Logging

Within handler methods, invoke the injected instance using structured key-value attributes:

func (lh *ListingsHandler) List(w http.ResponseWriter, r *http.Request) {
	rows, err := lh.db.QueryContext(r.Context(), "SELECT id, title FROM listings")
	if err != nil {
		lh.logger.Error("listings query error", "err", err.Error())
		http.Error(w, "internal server error", http.StatusInternalServerError)
		return
	}
	defer rows.Close()

	// Business logic and scanning omitted for brevity
	lh.logger.Info("listings fetched", "count", 42)
}

3. Distributed Tracing: The Need for Request IDs

In multi-tenant or concurrent web servers, logs from hundreds of simultaneous requests interleave chronologically. When an end user reports an issue, isolating their specific failure from millions of lines of application logs is impossible without a unique correlation handle.

Client (Browser / App)
   |
   | 1. HTTP Request (optional: X-Request-ID)
   v
API Gateway / Middleware
   |
   | 2. Generate or Preserve Unique Request ID (UUID)
   | 3. Set X-Request-ID on Response Header
   v
Application Core / Handlers
   |
   | 4. Attach Request ID to every log line
   v
Log Storage / Monitoring System (Render, Datadog, CloudWatch)
   ^
   | 5. Support team searches using exact UUID
Support Engineer

The X-Request-ID Convention

Industry leaders (e.g., Stripe, GitHub) standardize on the X-Request-ID HTTP header:

1. Inbound Propagation: If a front-end client or upstream API gateway generates a correlation token, the service preserves it.
2. Fallback Generation: If the request lacks an ID, the server generates a cryptographically secure, universally unique identifier ($UUID$).
3. Outbound Mirroring: The ID is returned in the HTTP response headers. If an error occurs (such as an HTTP 500), the UI can expose this ID to the user, allowing customer support to locate the exact log trail immediately.

4. Go HTTP Middleware Fundamentals

To avoid duplicating extraction, generation, and header-injection logic across every route handler, Go applications utilize HTTP middleware.

Conceptual Model

An HTTP middleware in Go is an adapter that wraps an http.Handler, executes pre-processing logic, delegates execution down the pipeline, and performs optional post-processing.

The http.Handler Interface vs. http.HandlerFunc

The standard library defines an HTTP handler via an interface:

type Handler interface {
    ServeHTTP(ResponseWriter, *Request)
}

An ordinary function with the signature func(http.ResponseWriter, *http.Request) does not implicitly implement this interface unless cast to http.HandlerFunc. The standard adapter type is defined as:

type HandlerFunc func(ResponseWriter, *Request)

func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) {
    f(w, r)
}

Wrapping an anonymous function with http.HandlerFunc(func(w, r) { ... }) adapts regular functions into compliant http.Handler types.

5. Propagating Values with context.Context

The context.Context type in Go serves three distinct concurrency and lifecycle purposes:

While timeouts and cancellations control execution duration, context.WithValue attaches immutable, request-scoped metadata across architectural layers (middleware $\rightarrow$ handlers $\rightarrow$ service layers $\rightarrow$ database access objects).

Updating the Context in Middleware

Because http.Request contains an immutable context, injecting new values requires creating a shallow copy of the request containing the derived context:

ctx := context.WithValue(r.Context(), key, value)
r = r.WithContext(ctx)
next.ServeHTTP(w, r)

6. Context Key Collisions and the Custom Type Pattern

A dangerous pitfall when using context.WithValue is using primitive data types (such as string or int) as context keys.

The Collision Anti-Pattern

Consider two packages independently storing context values using the string key "request_id":

// Package middleware
ctx = context.WithValue(ctx, "request_id", "uuid-middleware-1234")

// External third-party package or another internal module
ctx = context.WithValue(ctx, "request_id", 99999)

Because both packages rely on the standard string type, the second invocation overwrites the first. Worse, if package A attempts a type assertion ctx.Value("request_id").(string), the application will panic at runtime due to receiving an int.

The standard Go documentation specifies:

"The provided key must be comparable and should not be of type string or any other built-in type to avoid collisions between packages using context."

The Unexported Custom Type Solution

To guarantee complete isolation, define an unexported custom type backed by an integer, initialized via iota:

Even if another package defines type ctxKey int with the exact same name and underlying integer, Go's type system treats them as distinct identifiers. Neither package can overwrite or access the other's context entry directly.

7. Step-by-Step Implementation

Step 1: Install the UUID Dependency

Use Google's industry-standard RFC 4122 compliant UUID package:

go get github.com/google/uuid

Step 2: Implement the Request ID Middleware

Create internal/middleware/request_id.go incorporating the custom key pattern, the middleware wrapper, and an exported accessor function:

package middleware

import (
	"context"
	"net/http"

	"github.com/google/uuid"
)

// HeaderRequestID defines the standard HTTP header key for correlation tracking.
const HeaderRequestID = "X-Request-ID"

// ctxKey is an unexported type preventing cross-package context key collisions.
type ctxKey int

// requestIDKey is the specific private key used to map request IDs in the context.
const requestIDKey ctxKey = iota

// RequestID extracts an incoming X-Request-ID or generates a fresh UUIDv4.
// It sets the header on the response, embeds the ID in the request context,
// and invokes the next handler.
func RequestID(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
		reqID := r.Header.Get(HeaderRequestID)
		if reqID == "" {
			reqID = uuid.NewString()
		}

		// Set outbound response header for client transparency
		w.Header().Set(HeaderRequestID, reqID)

		// Attach ID to context using private, type-safe key
		ctx := context.WithValue(r.Context(), requestIDKey, reqID)
		r = r.WithContext(ctx)

		// Hand execution to the next handler in the chain
		next.ServeHTTP(w, r)
	})
}

// RequestIDFromContext retrieves the request ID stored in ctx.
// Returns an empty string if no request ID is configured.
func RequestIDFromContext(ctx context.Context) string {
	if val, ok := ctx.Value(requestIDKey).(string); ok {
		return val
	}
	return ""
}

Step 3: Wrap the Multiplexer in cmd/api/main.go

Wrap the primary application router (http.ServeMux) with the middleware. This ensures that every route registered on the multiplexer executes the correlation logic before hitting handler logic:

package main

import (
	"log/slog"
	"net/http"
	"os"

	"myapp/internal/handlers"
	"myapp/internal/middleware"
)

func main() {
	// 1. Initialize production JSON logger
	logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
		Level: slog.LevelInfo,
	}))

	// 2. Initialize router and route-specific dependencies
	mux := http.NewServeMux()
	listingHandler := handlers.NewListingsHandler(nil, logger)

	mux.HandleFunc("DELETE /listings/{id}", listingHandler.Delete)

	// 3. Wrap router globally with middleware pipeline
	wrappedMux := middleware.RequestID(mux)

	logger.Info("server starting on port :8080")
	if err := http.ListenAndServe(":8080", wrappedMux); err != nil {
		logger.Error("server failed", "err", err)
		os.Exit(1)
	}
}

Step 4: Extract and Log the Request ID in the Domain Handler

Within the handler methods, extract the contextual identifier using the public accessor and attach it to structured log statements:

package handlers

import (
	"net/http"

	"myapp/internal/middleware"
)

func (lh *ListingsHandler) Delete(w http.ResponseWriter, r *http.Request) {
	ctx := r.Context()
	reqID := middleware.RequestIDFromContext(ctx)

	// Simulating domain logic failure
	err := simulateFailure()
	if err != nil {
		lh.logger.Error("failed to delete listing",
			"error", err.Error(),
			"request_id", reqID,
		)

		http.Error(w, "internal server error", http.StatusInternalServerError)
		return
	}

	w.WriteHeader(http.StatusNoContent)
}

func simulateFailure() error {
	return sql.ErrConnDone
}

8. Verification and Production Debugging Workflow

When properly deployed to production, correlation logging creates an auditable link between end-user experiences and server internals.

The Client Perspective

When a client sends a request that triggers an error, inspect the raw HTTP response headers:

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain; charset=utf-8
X-Request-ID: c54be22b-8a7e-49b8-b80c-e2f7528e578a
Date: Mon, 15 Jan 2024 10:15:30 GMT
Content-Length: 22

internal server error

The user or frontend logging system captures the X-Request-ID token: c54be22b-8a7e-49b8-b80c-e2f7528e578a.

The Engineering Triage Perspective

An engineer troubleshooting the user's issue searches the log aggregation platform (e.g., Render, Datadog, or Grafana Loki) using the correlation ID:

{
  "time": "2024-01-15T10:15:30.123456Z",
  "level": "ERROR",
  "msg": "failed to delete listing",
  "error": "sql: connection already that was closed",
  "request_id": "c54be22b-8a7e-49b8-b80c-e2f7528e578a"
}

This diagnostic trace isolates the failure to an exact database pool disconnect without requiring full user sessions, broad timestamps, or invasive debugging in production.

Note on Upcoming Architectural Improvements: Notice that while the logger emits structured JSON, the HTTP error handler returns raw text/plain using http.Error(). A resilient API must never return unformatted strings on failures. Standardizing all error responses into a consistent JSON envelope is a necessary next step.

9. Structural and Architectural Comparisons

Global vs. Injected Logging

Core Mechanics of context.Context

10. Summary and Key Takeaways

• Eliminate Global State: Never rely on package-level logging singletons (slog.Info) in production applications. Inject *slog.Logger directly into struct receivers via constructors.
• Surface Hidden Dependencies: Explicit constructor parameters make domain components transparent, modular, and easily testable.
• Isolate Parallel Unit Tests: Injecting distinct logger instances prevents cross-test contamination and race conditions when mutating log sinks or log thresholds.
• Leverage Standard Correlation Headers: Always read and populate the X-Request-ID header to correlate client actions with server-side traces.
• Generate UUIDs Conservatively: Accept client-provided correlation IDs from trusted API gateways; fallback to generating RFC 4122 compliant UUIDs via google/uuid when missing.
• Wrap Handlers via Middleware: Centralize request ID processing using the standard func(http.Handler) http.Handler Go middleware pattern rather than repeating generation logic inside individual handlers.
• Understand http.HandlerFunc: Use the standard adapter type http.HandlerFunc to cast plain functions matching the (ResponseWriter, *Request) signature into standard http.Handler interfaces.
• Prevent Context Key Collisions: Never use primitive types (such as string or int) inside context.WithValue. Define an unexported custom type (e.g., type ctxKey int) with iota to enforce strict type-level isolation across package boundaries.
• Encapsulate Value Extraction: Keep context keys strictly unexported. Provide an exported accessor function (e.g., RequestIDFromContext(ctx)) to retrieve contextual data safely.