RoundTripper in Go: The secret behind http.Client nobody explained
RoundTripper in Go: The secret behind http.Client nobody explained
You use http.Client every day. You call client.Get(), receive your response, and move on with your life. But what happens underneath? Why do you sometimes need to customize the "Transport"? What the hell is a RoundTripper?
Today we're opening the hood of Go's HTTP client.
What is the RoundTripper?
The RoundTripper is an interface from the net/http package that represents the ability to execute a single HTTP transaction: send a request and receive a response.
type RoundTripper interface {
RoundTrip(*Request) (*Response, error)
}That's it. A function that receives a *Request and returns a *Response. Simple, elegant, powerful.
It's the low-level mechanism that http.Client uses to make requests. When you call client.Get() or client.Do(), it internally delegates to the RoundTripper configured in the client's Transport field.
client := &http.Client{
Transport: myRoundTripper, // ← Configured here
}If you don't configure anything, Go uses http.DefaultTransport, which is a very complete implementation that handles connection pooling, TLS, timeouts, and more.
Architecture: The two layers of the HTTP client
Here's the key insight: Go's HTTP client has two clearly separated layers.
┌─────────────────────────────────────────────────────────────┐
│ YOUR CODE │
│ client.Get(url) │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ http.Client │
│ (HIGH LEVEL) │
│ │
│ • Manages cookies (CookieJar) │
│ • Follows redirects automatically │
│ • Applies global timeouts │
│ • Handles redirect policy │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ RoundTripper │
│ (LOW LEVEL) │
│ │
│ • Opens TCP connections │
│ • Performs TLS handshake for HTTPS │
│ • Sends request bytes │
│ • Reads response bytes │
│ • Manages connection pool │
└─────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────┐
│ NETWORK │
│ (TCP/IP Stack) │
└─────────────────────────────────────────────────────────────┘The http.Client is the orchestra conductor: it decides what to do with redirects, when to apply cookies, when to timeout. But when it's time to send bytes over the network, it delegates everything to the RoundTripper.
This two-layer architecture lets you modify low-level behavior (add headers, logging, retry) without touching the high-level logic (cookies, redirects). It's the single responsibility principle applied to the HTTP client.
The complete journey of an HTTP request
Let's go step by step. What exactly happens when you run client.Get("https://api.example.com/users")?
YOUR CODE http.Client RoundTripper
───────── ─────────── ────────────
client.Get(url)
│
└──────────────────→ 1. Creates *http.Request
2. Determines Transport
(if nil, uses DefaultTransport)
3. Enters redirect loop
│
└────────────────────→ 4. transport.RoundTrip(req)
│
5. Connection in pool?
└─ Yes: reuse it
└─ No: TCP connect
│
6. If HTTPS:
TLS handshake
│
7. Writes request
to connection
│
8. Reads response
│
9. Returns connection
to pool (Keep-Alive)
│
←────────────────────────────────┘
10. Is it redirect (301, 302...)?
└─ Yes: back to step 3
└─ No: continue
11. Process cookies if CookieJar exists
│
←───────────────────────────┘
Receives *http.ResponseThe RoundTripper does the dirty work: connections, TLS, bytes. The Client just coordinates.
Why customize the RoundTripper?
Think of the RoundTripper as middleware for the HTTP client. Every request leaving your application passes through it. This gives you a perfect interception point for:
| Use case | What you do in the RoundTripper |
|---|---|
| Logging | Log URL, method, status code, latency |
| Automatic retry | If it fails, retry N times |
| Global headers | Add Authorization, X-Request-ID |
| Caching | Cache responses, return from cache |
| Rate limiting | Limit requests per second |
| Metrics | Export latencies to Prometheus |
| Circuit breaker | Cut off if service is down |
| Testing | Mock responses without a real server |
The beauty is that your business code doesn't change. It keeps doing client.Get() as always, but magic happens underneath.
The decorator pattern: How to customize without breaking anything
Here comes the important part. When you customize the RoundTripper, you DON'T replace DefaultTransport. You wrap it.
type loggingTransport struct {
base http.RoundTripper // ← The original transport
}
func (t *loggingTransport) RoundTrip(req *http.Request) (*http.Response, error) {
// Your logic BEFORE the request
log.Printf("→ %s %s", req.Method, req.URL)
start := time.Now()
// Delegate to the original transport
resp, err := t.base.RoundTrip(req)
// Your logic AFTER the response
if err != nil {
log.Printf("✗ Error: %v", err)
} else {
log.Printf("← %d %s (%v)", resp.StatusCode, req.URL.Path, time.Since(start))
}
return resp, err
}
// Usage:
client := &http.Client{
Transport: &loggingTransport{
base: http.DefaultTransport, // ← Wrap, don't replace
},
}The http.DefaultTransport handles connection pooling, TLS certificates, proxy, timeouts, and dozens of edge cases. If you replace it with your own implementation, you'll lose all of that. Always delegate to it.
This is the decorator pattern in action. Your loggingTransport adds functionality (logging) without modifying the original behavior (making the HTTP request).
You can chain multiple decorators:
// Each layer adds functionality
client := &http.Client{
Transport: &retryTransport{
base: &loggingTransport{
base: &metricsTransport{
base: http.DefaultTransport,
},
},
},
}The request goes through: retry → logging → metrics → DefaultTransport → network.
Context in the RoundTripper
If you come from other languages, you might wonder: "Why doesn't RoundTrip receive a context.Context as its first parameter?"
// The current interface
RoundTrip(*Request) (*Response, error)
// What you'd expect
RoundTrip(context.Context, *Request) (*Response, error)The answer is backward compatibility. The RoundTripper interface existed before context.Context arrived in Go (Go 1.7, 2016). Changing it would break the entire ecosystem.
The solution: the context goes inside the Request.
func (t *myTransport) RoundTrip(req *http.Request) (*http.Response, error) {
ctx := req.Context() // ← This is how you access the context
// You can use it for timeouts, cancellation, values
select {
case <-ctx.Done():
return nil, ctx.Err()
default:
// Continue with the request
}
return t.base.RoundTrip(req)
}The http.Client takes care of putting the context in the request when you use NewRequestWithContext or when it applies its global timeout.
Instead of http.NewRequest(), use http.NewRequestWithContext(ctx, method, url, body). This way your RoundTripper will always have access to the context for cancellation and timeouts.
Complete practical example
Let's build a RoundTripper that does three things:
- Adds a correlation header (X-Correlation-ID) for tracing
- Logs the request and response
- Measures latency and exposes it as a metric
package main
import (
"log"
"net/http"
"time"
"github.com/google/uuid"
)
// observableTransport adds observability to all requests
type observableTransport struct {
base http.RoundTripper
}
func (t *observableTransport) RoundTrip(req *http.Request) (*http.Response, error) {
// 1. Generate or propagate correlation ID
correlationID := req.Header.Get("X-Correlation-ID")
if correlationID == "" {
correlationID = uuid.New().String()
req.Header.Set("X-Correlation-ID", correlationID)
}
// 2. Log outgoing request
log.Printf("[%s] → %s %s", correlationID, req.Method, req.URL)
// 3. Measure latency
start := time.Now()
// Execute the actual request
resp, err := t.base.RoundTrip(req)
// Calculate duration
duration := time.Since(start)
// 4. Log result
if err != nil {
log.Printf("[%s] ✗ Error after %v: %v", correlationID, duration, err)
// Here you could increment an error counter
return nil, err
}
log.Printf("[%s] ← %d %s (%v)",
correlationID,
resp.StatusCode,
http.StatusText(resp.StatusCode),
duration,
)
// 5. Export metric (example with channel, in prod you'd use prometheus)
// metrics.RecordLatency("http_client", req.URL.Host, duration)
return resp, nil
}
// NewObservableClient creates an HTTP client with built-in observability
func NewObservableClient(timeout time.Duration) *http.Client {
return &http.Client{
Timeout: timeout,
Transport: &observableTransport{
base: http.DefaultTransport,
},
}
}
func main() {
client := NewObservableClient(10 * time.Second)
// All requests now have automatic logging and correlation ID
resp, err := client.Get("https://httpbin.org/get")
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
log.Printf("Response status: %s", resp.Status)
}Output:
[550e8400-e29b-41d4-a716-446655440000] → GET https://httpbin.org/get
[550e8400-e29b-41d4-a716-446655440000] ← 200 OK (234.567ms)
Response status: 200 OKWithout changing a single line of your business code, you now have:
- End-to-end traceability with correlation IDs
- Logs for every request leaving your application
- Latency measurement ready for Prometheus
Advanced use cases
Retry with exponential backoff
type retryTransport struct {
base http.RoundTripper
maxRetries int
}
func (t *retryTransport) RoundTrip(req *http.Request) (*http.Response, error) {
var resp *http.Response
var err error
for attempt := 0; attempt <= t.maxRetries; attempt++ {
// Clone the request for retry (body may have been consumed)
reqCopy := req.Clone(req.Context())
resp, err = t.base.RoundTrip(reqCopy)
// If successful or not retriable, exit
if err == nil && resp.StatusCode < 500 {
return resp, nil
}
// Close response body if there was a server error
if resp != nil {
resp.Body.Close()
}
// Exponential backoff: 100ms, 200ms, 400ms...
if attempt < t.maxRetries {
backoff := time.Duration(100<<attempt) * time.Millisecond
time.Sleep(backoff)
}
}
return resp, err
}Simple in-memory cache
type cachingTransport struct {
base http.RoundTripper
cache map[string]*http.Response
mu sync.RWMutex
}
func (t *cachingTransport) RoundTrip(req *http.Request) (*http.Response, error) {
// Only cache GETs
if req.Method != http.MethodGet {
return t.base.RoundTrip(req)
}
key := req.URL.String()
// Look in cache
t.mu.RLock()
if cached, ok := t.cache[key]; ok {
t.mu.RUnlock()
return cached, nil
}
t.mu.RUnlock()
// Make actual request
resp, err := t.base.RoundTrip(req)
if err != nil {
return nil, err
}
// Store in cache (simplified, in prod consider TTL, size, etc.)
t.mu.Lock()
t.cache[key] = resp
t.mu.Unlock()
return resp, nil
}The resp.Body is an io.ReadCloser that can only be read once. If you cache the response, you need to read the entire body, store it, and create a new io.ReadCloser for each read. This also applies to retry.
RoundTripper rules
Go's documentation specifies rules that your implementation must follow:
-
Don't modify the Request (except closing the Body)
- If you need to modify headers, clone first:
req.Clone(ctx)
- If you need to modify headers, clone first:
-
Always close the Request Body if not nil
- The RoundTripper is responsible for closing it
-
Return the Response Body unconsumed
- The caller is who must read and close the body
-
Handle context correctly
- Respect
req.Context().Done()for cancellation
- Respect
TL;DR
-
RoundTripperis the low-level interface that executes HTTP transactions -
http.Clienthandles cookies and redirects,RoundTripperhandles the connection - Never replace
http.DefaultTransport, wrap it with the decorator pattern - Use cases: logging, retry, global headers, caching, metrics
- The
context.Contextgoes inside the*Request, not as a separate parameter - Chain multiple transports for behavior composition
- Remember: the Response
Bodycan only be read once