Deep divePublished 4 min read
RoundTripper in Go: The secret behind http.Client nobody explained
onGo posts
Part 1 of net/http under the hood
Understand how Go's RoundTripper works, the low-level mechanism behind every HTTP request. Learn to customize it for logging, retry, headers, and more.
In this post
- What the RoundTripper interface is, and where http.Client plugs it in
- The two layers of the HTTP client: http.Client on top, RoundTripper underneath
- The eleven steps one request takes through your code, the Client and the RoundTripper
- Wrapping DefaultTransport with the decorator pattern, and chaining several wrappers
- Why RoundTrip has no context parameter, and how to read the context from the request
- A complete observable transport with correlation IDs, logs and latency, plus retry and cache sketches
- The rules every RoundTripper implementation must follow
On this page
- What is the RoundTripper?
- The mental model: two layers
- The complete journey of an HTTP request
- Why customize the RoundTripper?
- The decorator pattern: wrap, don’t replace
- Chaining decorators
- Context in the RoundTripper
- Why RoundTrip has no context parameter
- The context goes inside the Request
- Seeing it for real: an observable transport
- Advanced use cases
- Retry with exponential backoff
- Simple in-memory cache
- RoundTripper rules
You use http.Client every day. You call client.Get(), receive your response, and move on with your life. But what happens underneath? Why do you sometimes need to customize the “Transport”? What the hell is a RoundTripper?
In this post I open the hood of Go’s HTTP client: the RoundTripper interface, the two layers of the client, the path one request takes through them, and how to wrap the transport to add logging, retry, headers and more.
What is the RoundTripper?
The RoundTripper is an interface from the net/http package that represents the ability to execute a single HTTP transaction: send a request and receive a response.
type RoundTripper interface { RoundTrip(*Request) (*Response, error)}That’s it. A function that receives a *Request and returns a *Response.
It’s the low-level mechanism that http.Client uses to make requests. When you call client.Get() or client.Do(), it internally delegates to the RoundTripper configured in the client’s Transport field.
client := &http.Client{ Transport: myRoundTripper, // ← Configured here}If you don’t configure anything, Go uses http.DefaultTransport, which is a very complete implementation that handles connection pooling, TLS, timeouts, and more.
The mental model: two layers
Go’s HTTP client has two clearly separated layers.
┌─────────────────────────────────────────────────────────────┐│ YOUR CODE ││ client.Get(url) │└─────────────────────────────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────────┐│ http.Client ││ (HIGH LEVEL) ││ ││ • Manages cookies (CookieJar) ││ • Follows redirects automatically ││ • Applies global timeouts ││ • Handles redirect policy │└─────────────────────────────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────────┐│ RoundTripper ││ (LOW LEVEL) ││ ││ • Opens TCP connections ││ • Performs TLS handshake for HTTPS ││ • Sends request bytes ││ • Reads response bytes ││ • Manages connection pool │└─────────────────────────────────────────────────────────────┘ │ ▼┌─────────────────────────────────────────────────────────────┐│ NETWORK ││ (TCP/IP Stack) │└─────────────────────────────────────────────────────────────┘The http.Client is the orchestra conductor: it decides what to do with redirects, when to apply cookies, when to timeout. But when it’s time to send bytes over the network, it delegates everything to the RoundTripper.
The complete journey of an HTTP request
Let’s go step by step. What exactly happens when you run client.Get("https://api.example.com/users")?
YOUR CODE http.Client RoundTripper───────── ─────────── ────────────
client.Get(url) │ └──────────────────→ 1. Creates *http.Request 2. Determines Transport (if nil, uses DefaultTransport) 3. Enters redirect loop │ └────────────────────→ 4. transport.RoundTrip(req) │ 5. Connection in pool? └─ Yes: reuse it └─ No: TCP connect │ 6. If HTTPS: TLS handshake │ 7. Writes request to connection │ 8. Reads response │ 9. Returns connection to pool (Keep-Alive) │ ←────────────────────────────────┘ 10. Is it redirect (301, 302...)? └─ Yes: back to step 3 └─ No: continue 11. Process cookies if CookieJar exists │ ←───────────────────────────┘Receives *http.ResponseThe RoundTripper does the dirty work: connections, TLS, bytes. The Client just coordinates.
Why customize the RoundTripper?
Think of the RoundTripper as middleware for the HTTP client. Every request leaving your application passes through it. This gives you a perfect interception point for:
| Use case | What you do in the RoundTripper |
|---|---|
| Logging | Log URL, method, status code, latency |
| Automatic retry | If it fails, retry N times |
| Global headers | Add Authorization, X-Request-ID |
| Caching | Cache responses, return from cache |
| Rate limiting | Limit requests per second |
| Metrics | Export latencies to Prometheus |
| Circuit breaker | Cut off if service is down |
| Testing | Mock responses without a real server |
Your business code doesn’t change. It keeps doing client.Get() as always.
The decorator pattern: wrap, don’t replace
Here comes the important part. When you customize the RoundTripper, you DON’T replace DefaultTransport. You wrap it.
type loggingTransport struct { base http.RoundTripper // ← The original transport}
func (t *loggingTransport) RoundTrip(req *http.Request) (*http.Response, error) { // Your logic BEFORE the request log.Printf("→ %s %s", req.Method, req.URL) start := time.Now()
// Delegate to the original transport resp, err := t.base.RoundTrip(req)
// Your logic AFTER the response if err != nil { log.Printf("✗ Error: %v", err) } else { log.Printf("← %d %s (%v)", resp.StatusCode, req.URL.Path, time.Since(start)) }
return resp, err}
// Usage:client := &http.Client{ Transport: &loggingTransport{ base: http.DefaultTransport, // ← Wrap, don't replace },}This is the decorator pattern in action. Your loggingTransport adds functionality (logging) without modifying the original behavior (making the HTTP request).
Chaining decorators
You can chain multiple decorators:
// Each layer adds functionalityclient := &http.Client{ Transport: &retryTransport{ base: &loggingTransport{ base: &metricsTransport{ base: http.DefaultTransport, }, }, },}The request goes through: retry → logging → metrics → DefaultTransport → network.
Context in the RoundTripper
Why RoundTrip has no context parameter
If you come from other languages, you might wonder: “Why doesn’t RoundTrip receive a context.Context as its first parameter?”
// The current interfaceRoundTrip(*Request) (*Response, error)
// What you'd expectRoundTrip(context.Context, *Request) (*Response, error)The answer is backward compatibility. The RoundTripper interface existed before context.Context arrived in Go (Go 1.7, 2016). Changing it would break the entire ecosystem.
The context goes inside the Request
The solution: the context goes inside the Request.
func (t *myTransport) RoundTrip(req *http.Request) (*http.Response, error) { ctx := req.Context() // ← This is how you access the context
// You can use it for timeouts, cancellation, values select { case <-ctx.Done(): return nil, ctx.Err() default: // Continue with the request }
return t.base.RoundTrip(req)}The http.Client takes care of putting the context in the request when you use NewRequestWithContext or when it applies its global timeout.
Seeing it for real: an observable transport
Let’s build a RoundTripper that does three things:
- Adds a correlation header (X-Correlation-ID) for tracing
- Logs the request and response
- Measures latency and exposes it as a metric
package main
import ( "log" "net/http" "time"
"github.com/google/uuid")
// observableTransport adds observability to all requeststype observableTransport struct { base http.RoundTripper}
func (t *observableTransport) RoundTrip(req *http.Request) (*http.Response, error) { // 1. Generate or propagate correlation ID correlationID := req.Header.Get("X-Correlation-ID") if correlationID == "" { correlationID = uuid.New().String() req.Header.Set("X-Correlation-ID", correlationID) }
// 2. Log outgoing request log.Printf("[%s] → %s %s", correlationID, req.Method, req.URL)
// 3. Measure latency start := time.Now()
// Execute the actual request resp, err := t.base.RoundTrip(req)
// Calculate duration duration := time.Since(start)
// 4. Log result if err != nil { log.Printf("[%s] ✗ Error after %v: %v", correlationID, duration, err) // Here you could increment an error counter return nil, err }
log.Printf("[%s] ← %d %s (%v)", correlationID, resp.StatusCode, http.StatusText(resp.StatusCode), duration, )
// 5. Export metric (example with channel, in prod you'd use prometheus) // metrics.RecordLatency("http_client", req.URL.Host, duration)
return resp, nil}
// NewObservableClient creates an HTTP client with built-in observabilityfunc NewObservableClient(timeout time.Duration) *http.Client { return &http.Client{ Timeout: timeout, Transport: &observableTransport{ base: http.DefaultTransport, }, }}
func main() { client := NewObservableClient(10 * time.Second)
// All requests now have automatic logging and correlation ID resp, err := client.Get("https://httpbin.org/get") if err != nil { log.Fatal(err) } defer resp.Body.Close()
log.Printf("Response status: %s", resp.Status)}Output:
[550e8400-e29b-41d4-a716-446655440000] → GET https://httpbin.org/get[550e8400-e29b-41d4-a716-446655440000] ← 200 OK (234.567ms)Response status: 200 OKWithout changing a single line of your business code, you now have:
- End-to-end traceability with correlation IDs
- Logs for every request leaving your application
- Latency measurement ready for Prometheus
Advanced use cases
Retry with exponential backoff
type retryTransport struct { base http.RoundTripper maxRetries int}
func (t *retryTransport) RoundTrip(req *http.Request) (*http.Response, error) { var resp *http.Response var err error
for attempt := 0; attempt <= t.maxRetries; attempt++ { // Clone the request for retry (body may have been consumed) reqCopy := req.Clone(req.Context())
resp, err = t.base.RoundTrip(reqCopy)
// If successful or not retriable, exit if err == nil && resp.StatusCode < 500 { return resp, nil }
// Close response body if there was a server error if resp != nil { resp.Body.Close() }
// Exponential backoff: 100ms, 200ms, 400ms... if attempt < t.maxRetries { backoff := time.Duration(100<<attempt) * time.Millisecond time.Sleep(backoff) } }
return resp, err}Simple in-memory cache
type cachingTransport struct { base http.RoundTripper cache map[string]*http.Response mu sync.RWMutex}
func (t *cachingTransport) RoundTrip(req *http.Request) (*http.Response, error) { // Only cache GETs if req.Method != http.MethodGet { return t.base.RoundTrip(req) }
key := req.URL.String()
// Look in cache t.mu.RLock() if cached, ok := t.cache[key]; ok { t.mu.RUnlock() return cached, nil } t.mu.RUnlock()
// Make actual request resp, err := t.base.RoundTrip(req) if err != nil { return nil, err }
// Store in cache (simplified, in prod consider TTL, size, etc.) t.mu.Lock() t.cache[key] = resp t.mu.Unlock()
return resp, nil}RoundTripper rules
Go’s documentation specifies rules that your implementation must follow:
-
Don’t modify the Request (except closing the Body)
- If you need to modify headers, clone first:
req.Clone(ctx)
- If you need to modify headers, clone first:
-
Always close the Request Body if not nil
- The RoundTripper is responsible for closing it
-
Return the Response Body unconsumed
- The caller is who must read and close the body
-
Handle context correctly
- Respect
req.Context().Done()for cancellation
- Respect
TL;DR
RoundTripperis the low-level interface that executes HTTP transactionshttp.Clienthandles cookies and redirects,RoundTripperhandles the connection- Never replace
http.DefaultTransport, wrap it with the decorator pattern - Use cases: logging, retry, global headers, caching, metrics
- The
context.Contextgoes inside the*Request, not as a separate parameter - Chain multiple transports for behavior composition
- Remember: the Response
Bodycan only be read once
Materials used to write this article
- Package net/http documentation: RoundTripper — Cited as "Go's documentation" in RoundTripper rules
More of what I read in my software-engineering-guide.
Related posts
Deep dive
Go's HTTP Connection Pool: The Key Is the Server, Not the Endpoint
I thought I understood net/http's connection pool. Then I found out the pool key ignores the path entirely, and that my own Transport config had silently turned off HTTP/2. Three discoveries, measured with httptrace.
onGo postsLinux & Networking posts
8 min read · Part 2 of net/http under the hood
Build log
A dispatcher for delivery riders: one routing problem a minute
Every minute, every open order against every rider on duty, solved as one vehicle routing problem with VROOM. How the round works, why the solver's answer is only a proposal, and why the legs are diffed instead of rebuilt.
17 min read
Build log
A Spotify-style search over six verticals and four languages with Elasticsearch
One search box, six indexes, one round trip. How I built a multi-vertical search in Go on Elasticsearch, why it rebuilds its indexes instead of syncing them, and how I would sync them at a million documents per vertical.
16 min read