runtime/rants

Deep divePublished 4 min read

RoundTripper in Go: The secret behind http.Client nobody explained

onGo posts

Part 1 of net/http under the hood

Understand how Go's RoundTripper works, the low-level mechanism behind every HTTP request. Learn to customize it for logging, retry, headers, and more.

In this post

  • What the RoundTripper interface is, and where http.Client plugs it in
  • The two layers of the HTTP client: http.Client on top, RoundTripper underneath
  • The eleven steps one request takes through your code, the Client and the RoundTripper
  • Wrapping DefaultTransport with the decorator pattern, and chaining several wrappers
  • Why RoundTrip has no context parameter, and how to read the context from the request
  • A complete observable transport with correlation IDs, logs and latency, plus retry and cache sketches
  • The rules every RoundTripper implementation must follow

You use http.Client every day. You call client.Get(), receive your response, and move on with your life. But what happens underneath? Why do you sometimes need to customize the “Transport”? What the hell is a RoundTripper?

In this post I open the hood of Go’s HTTP client: the RoundTripper interface, the two layers of the client, the path one request takes through them, and how to wrap the transport to add logging, retry, headers and more.

What is the RoundTripper?

The RoundTripper is an interface from the net/http package that represents the ability to execute a single HTTP transaction: send a request and receive a response.

type RoundTripper interface {
RoundTrip(*Request) (*Response, error)
}

That’s it. A function that receives a *Request and returns a *Response.

It’s the low-level mechanism that http.Client uses to make requests. When you call client.Get() or client.Do(), it internally delegates to the RoundTripper configured in the client’s Transport field.

client := &http.Client{
Transport: myRoundTripper, // ← Configured here
}

If you don’t configure anything, Go uses http.DefaultTransport, which is a very complete implementation that handles connection pooling, TLS, timeouts, and more.

The mental model: two layers

Go’s HTTP client has two clearly separated layers.

┌─────────────────────────────────────────────────────────────┐
│ YOUR CODE │
│ client.Get(url) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ http.Client │
│ (HIGH LEVEL) │
│ │
│ • Manages cookies (CookieJar) │
│ • Follows redirects automatically │
│ • Applies global timeouts │
│ • Handles redirect policy │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ RoundTripper │
│ (LOW LEVEL) │
│ │
│ • Opens TCP connections │
│ • Performs TLS handshake for HTTPS │
│ • Sends request bytes │
│ • Reads response bytes │
│ • Manages connection pool │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ NETWORK │
│ (TCP/IP Stack) │
└─────────────────────────────────────────────────────────────┘

The http.Client is the orchestra conductor: it decides what to do with redirects, when to apply cookies, when to timeout. But when it’s time to send bytes over the network, it delegates everything to the RoundTripper.

The complete journey of an HTTP request

Let’s go step by step. What exactly happens when you run client.Get("https://api.example.com/users")?

YOUR CODE http.Client RoundTripper
───────── ─────────── ────────────
client.Get(url)
│
└──────────────────→ 1. Creates *http.Request
2. Determines Transport
(if nil, uses DefaultTransport)
3. Enters redirect loop
│
└────────────────────→ 4. transport.RoundTrip(req)
│
5. Connection in pool?
└─ Yes: reuse it
└─ No: TCP connect
│
6. If HTTPS:
TLS handshake
│
7. Writes request
to connection
│
8. Reads response
│
9. Returns connection
to pool (Keep-Alive)
│
←────────────────────────────────┘
10. Is it redirect (301, 302...)?
└─ Yes: back to step 3
└─ No: continue
11. Process cookies if CookieJar exists
│
←───────────────────────────┘
Receives *http.Response

The RoundTripper does the dirty work: connections, TLS, bytes. The Client just coordinates.

Why customize the RoundTripper?

Think of the RoundTripper as middleware for the HTTP client. Every request leaving your application passes through it. This gives you a perfect interception point for:

Use caseWhat you do in the RoundTripper
LoggingLog URL, method, status code, latency
Automatic retryIf it fails, retry N times
Global headersAdd Authorization, X-Request-ID
CachingCache responses, return from cache
Rate limitingLimit requests per second
MetricsExport latencies to Prometheus
Circuit breakerCut off if service is down
TestingMock responses without a real server

Your business code doesn’t change. It keeps doing client.Get() as always.

The decorator pattern: wrap, don’t replace

Here comes the important part. When you customize the RoundTripper, you DON’T replace DefaultTransport. You wrap it.

logging.go
type loggingTransport struct {
base http.RoundTripper // ← The original transport
}
func (t *loggingTransport) RoundTrip(req *http.Request) (*http.Response, error) {
// Your logic BEFORE the request
log.Printf("→ %s %s", req.Method, req.URL)
start := time.Now()
// Delegate to the original transport
resp, err := t.base.RoundTrip(req)
// Your logic AFTER the response
if err != nil {
log.Printf("✗ Error: %v", err)
} else {
log.Printf("← %d %s (%v)", resp.StatusCode, req.URL.Path, time.Since(start))
}
return resp, err
}
// Usage:
client := &http.Client{
Transport: &loggingTransport{
base: http.DefaultTransport, // ← Wrap, don't replace
},
}

This is the decorator pattern in action. Your loggingTransport adds functionality (logging) without modifying the original behavior (making the HTTP request).

Chaining decorators

You can chain multiple decorators:

// Each layer adds functionality
client := &http.Client{
Transport: &retryTransport{
base: &loggingTransport{
base: &metricsTransport{
base: http.DefaultTransport,
},
},
},
}

The request goes through: retry → logging → metrics → DefaultTransport → network.

Context in the RoundTripper

Why RoundTrip has no context parameter

If you come from other languages, you might wonder: “Why doesn’t RoundTrip receive a context.Context as its first parameter?”

// The current interface
RoundTrip(*Request) (*Response, error)
// What you'd expect
RoundTrip(context.Context, *Request) (*Response, error)

The answer is backward compatibility. The RoundTripper interface existed before context.Context arrived in Go (Go 1.7, 2016). Changing it would break the entire ecosystem.

The context goes inside the Request

The solution: the context goes inside the Request.

func (t *myTransport) RoundTrip(req *http.Request) (*http.Response, error) {
ctx := req.Context() // ← This is how you access the context
// You can use it for timeouts, cancellation, values
select {
case <-ctx.Done():
return nil, ctx.Err()
default:
// Continue with the request
}
return t.base.RoundTrip(req)
}

The http.Client takes care of putting the context in the request when you use NewRequestWithContext or when it applies its global timeout.

Seeing it for real: an observable transport

Let’s build a RoundTripper that does three things:

  1. Adds a correlation header (X-Correlation-ID) for tracing
  2. Logs the request and response
  3. Measures latency and exposes it as a metric
main.go
package main
import (
"log"
"net/http"
"time"
"github.com/google/uuid"
)
// observableTransport adds observability to all requests
type observableTransport struct {
base http.RoundTripper
}
func (t *observableTransport) RoundTrip(req *http.Request) (*http.Response, error) {
// 1. Generate or propagate correlation ID
correlationID := req.Header.Get("X-Correlation-ID")
if correlationID == "" {
correlationID = uuid.New().String()
req.Header.Set("X-Correlation-ID", correlationID)
}
// 2. Log outgoing request
log.Printf("[%s] → %s %s", correlationID, req.Method, req.URL)
// 3. Measure latency
start := time.Now()
// Execute the actual request
resp, err := t.base.RoundTrip(req)
// Calculate duration
duration := time.Since(start)
// 4. Log result
if err != nil {
log.Printf("[%s] ✗ Error after %v: %v", correlationID, duration, err)
// Here you could increment an error counter
return nil, err
}
log.Printf("[%s] ← %d %s (%v)",
correlationID,
resp.StatusCode,
http.StatusText(resp.StatusCode),
duration,
)
// 5. Export metric (example with channel, in prod you'd use prometheus)
// metrics.RecordLatency("http_client", req.URL.Host, duration)
return resp, nil
}
// NewObservableClient creates an HTTP client with built-in observability
func NewObservableClient(timeout time.Duration) *http.Client {
return &http.Client{
Timeout: timeout,
Transport: &observableTransport{
base: http.DefaultTransport,
},
}
}
func main() {
client := NewObservableClient(10 * time.Second)
// All requests now have automatic logging and correlation ID
resp, err := client.Get("https://httpbin.org/get")
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
log.Printf("Response status: %s", resp.Status)
}

Output:

[550e8400-e29b-41d4-a716-446655440000] → GET https://httpbin.org/get
[550e8400-e29b-41d4-a716-446655440000] ← 200 OK (234.567ms)
Response status: 200 OK

Without changing a single line of your business code, you now have:

  • End-to-end traceability with correlation IDs
  • Logs for every request leaving your application
  • Latency measurement ready for Prometheus

Advanced use cases

Retry with exponential backoff

retry.go
type retryTransport struct {
base http.RoundTripper
maxRetries int
}
func (t *retryTransport) RoundTrip(req *http.Request) (*http.Response, error) {
var resp *http.Response
var err error
for attempt := 0; attempt <= t.maxRetries; attempt++ {
// Clone the request for retry (body may have been consumed)
reqCopy := req.Clone(req.Context())
resp, err = t.base.RoundTrip(reqCopy)
// If successful or not retriable, exit
if err == nil && resp.StatusCode < 500 {
return resp, nil
}
// Close response body if there was a server error
if resp != nil {
resp.Body.Close()
}
// Exponential backoff: 100ms, 200ms, 400ms...
if attempt < t.maxRetries {
backoff := time.Duration(100<<attempt) * time.Millisecond
time.Sleep(backoff)
}
}
return resp, err
}

Simple in-memory cache

cache.go
type cachingTransport struct {
base http.RoundTripper
cache map[string]*http.Response
mu sync.RWMutex
}
func (t *cachingTransport) RoundTrip(req *http.Request) (*http.Response, error) {
// Only cache GETs
if req.Method != http.MethodGet {
return t.base.RoundTrip(req)
}
key := req.URL.String()
// Look in cache
t.mu.RLock()
if cached, ok := t.cache[key]; ok {
t.mu.RUnlock()
return cached, nil
}
t.mu.RUnlock()
// Make actual request
resp, err := t.base.RoundTrip(req)
if err != nil {
return nil, err
}
// Store in cache (simplified, in prod consider TTL, size, etc.)
t.mu.Lock()
t.cache[key] = resp
t.mu.Unlock()
return resp, nil
}

RoundTripper rules

Go’s documentation specifies rules that your implementation must follow:

  1. Don’t modify the Request (except closing the Body)

    • If you need to modify headers, clone first: req.Clone(ctx)
  2. Always close the Request Body if not nil

    • The RoundTripper is responsible for closing it
  3. Return the Response Body unconsumed

    • The caller is who must read and close the body
  4. Handle context correctly

    • Respect req.Context().Done() for cancellation

TL;DR

  • RoundTripper is the low-level interface that executes HTTP transactions
  • http.Client handles cookies and redirects, RoundTripper handles the connection
  • Never replace http.DefaultTransport, wrap it with the decorator pattern
  • Use cases: logging, retry, global headers, caching, metrics
  • The context.Context goes inside the *Request, not as a separate parameter
  • Chain multiple transports for behavior composition
  • Remember: the Response Body can only be read once

Materials used to write this article

More of what I read in my software-engineering-guide.

About the author

Joel Teodoro

Joel Teodoro

Backend engineer working with Go in production. I build small systems to understand how big ones work, and draw them here.