Server-Side Request Flow in Depth: Proxies, Middleware, Auth, Rate Limiting, and Observability

Part 1 of this series treated "the application server handles the request" as a single step. In a Node.js or Fastify application, the handler is usually the middle of the story, not the beginning. Proxies, request parsing, authentication, authorization, rate limiting, database calls, and logging all influence what the handler eventually sees.

This is Part 5 of the How the Web Works series. It covers what happens between "a request arrives at your infrastructure" and "a response is ready to send back."

---

Quick Answer

A request rarely goes straight from the internet to your application code. It typically flows through several layers, each responsible for one concern:

Internet
   ↓
Load balancer / reverse proxy   (TLS termination, routing, rate limiting)
   ↓
Middleware pipeline              (logging, auth, parsing, CORS)
   ↓
Router                           (matches path + method to a handler)
   ↓
Handler / controller             (your actual business logic)
   ↓
Service layer                    (domain logic, calls to the database)
   ↓
Database / external APIs
   ↓
Response flows back up through every layer in reverse

Many "the server did something weird" bugs come from one layer in this chain. The useful skill is knowing which layer owns which responsibility so you can isolate it quickly.

---

Where This Layer Sits in the Bigger Picture

By the time a request reaches this part of the pipeline, the client has already selected an endpoint, established or reused a transport connection, and formed an HTTP message. TLS may terminate at the edge rather than in the application process. This part covers the server-side machinery that validates, routes, processes, and answers that HTTP request.

---

Reverse Proxies and Load Balancers

Almost no production server receives internet traffic directly on its application process. Instead, a reverse proxy or load balancer sits in front, and does several jobs before your application code ever runs:

• TLS termination: decrypting HTTPS at an edge proxy. The next hop may use plain HTTP on a tightly controlled private network, but zero-trust, compliance-sensitive, and multi-tenant environments often re-encrypt traffic or use mTLS internally.
• Load balancing: distributing requests across multiple application server instances, using a strategy like round-robin, least-connections, or IP hashing for session affinity.
• Path-based or host-based routing: sending /api/* to one service and / to another, or routing based on the Host header for multi-tenant setups.
• Basic rate limiting and request filtering: rejecting obviously malicious or excessive traffic before it reaches application servers, saving compute for legitimate requests.

                     ┌─────────────┐
Internet ──────────► │ Load balancer│
                     └──────┬──────┘
                    ┌────────┼────────┐
                    ▼        ▼        ▼
              App server  App server  App server
                (1)         (2)         (3)

A subtle failure mode appears when a process passes a shallow liveness check but cannot serve real traffic within its latency budget, for example because its database pool is exhausted. Separate liveness (should the process be restarted?) from readiness (should it receive traffic?). Readiness checks should cover essential serving capability without turning every dependency disturbance into a fleet-wide cascade.

---

Middleware Pipelines: The Onion Model

Most modern server frameworks process a request through a chain of middleware. Each middleware function handles one cross-cutting concern, runs in sequence, and can either pass the request along or stop it early.

Request  →  [Logging] → [CORS] → [Auth] → [Body parsing] → [Handler]
Response ←  [Logging] ← [CORS] ← [Auth] ← [Body parsing] ← [Handler]

This is often called the "onion model" because each middleware wraps the ones inside it. Code before calling next() runs on the way in; code after next() runs on the way out.

async function loggingMiddleware(req, res, next) {
  const start = Date.now()
  await next()                      // everything inside runs here
  console.log(`${req.method} ${req.path}: ${Date.now() - start}ms`)
}

A common middleware ordering bug is trusting identity-dependent data before authentication, for example keying a limit by an unverified user ID or logging sensitive raw bodies before redaction. Parsing with strict size limits, assigning a request ID, or applying coarse IP limits before authentication can be legitimate. Middleware order is not cosmetic: each step should consume only data that earlier steps have established as trustworthy.

---

Routing: Matching a Request to a Handler

The router's job is narrow but critical: given a method and a path, find the one handler responsible for it.

GET  /users/42        →  matches  GET  /users/:id     →  getUserHandler
POST /users            →  matches  POST /users          →  createUserHandler
GET  /users/42/orders  →  matches  GET  /users/:id/orders → getUserOrdersHandler

Two details cause more routing bugs than anything else:

• Route ordering with overlapping patterns. A wildcard or catch-all route registered before a more specific one can silently swallow requests meant for the specific route, especially in frameworks that match routes in registration order rather than by specificity.
• Trailing slash and case-sensitivity mismatches. /Users/42 vs /users/42, or /users/42 vs /users/42/, may or may not be treated as the same route depending on framework configuration. This is a frequent source of "it works locally but 404s in production" when a proxy normalizes URLs differently than the app framework.

---

Authentication vs Authorization

These two get used almost interchangeably in casual conversation, but they answer different questions. Mixing them up is a common source of security bugs.

Authentication:  verify the token/session → attach the user's identity to the request
Authorization:   given that identity, check policy → allow or reject the specific action

A concrete, common bug: an API correctly authenticates a user (confirms the token is valid and identifies user #42), but then trusts a userId sent in the request to decide whose data to return or modify. Unless an authorization rule explicitly permits acting on behalf of another user, the server should derive ownership from the authenticated principal or verify access to the requested resource. Missing or invalid credentials normally produce 401; a known identity that lacks permission normally produces 403.

---

Rate Limiting Strategies

Rate limiting protects a server from being overwhelmed by legitimate traffic spikes, misbehaving clients, or deliberate abuse. Common algorithms include:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
RateLimit-Policy: "api";q=100;w=60
RateLimit: "api";r=0;t=30

Retry-After is standardized. The consolidated RateLimit and RateLimit-Policy fields shown above are standardized in RFC 9331, although production APIs may still expose vendor-specific fields such as X-RateLimit-Remaining. Document whichever contract your clients must consume.

Where to key the limit matters as much as the algorithm. Rate limiting purely by IP address breaks down behind NAT, where many legitimate users share one IP, and can be bypassed by clients rotating IPs. Keying by authenticated user ID is stronger where authentication exists, but unauthenticated endpoints such as a public login route still need an IP-based fallback.

---

Request Validation and Parsing

Before business logic runs, the request body needs parsing, such as JSON, form data, or multipart uploads, and validation against an expected shape. Many security bugs start when malformed input reaches business logic or a database query unchecked.

Enforce size/type limits → Parse → Validate shape, types, ranges, and invariants
                         → Normalize where appropriate → Business logic

Avoid treating generic "sanitization" as a universal security step. Use parameterized queries for database input and context-specific escaping or encoding when data is rendered into HTML, URLs, shell commands, or other interpreters.

Validate, don't just check truthiness. Confirming a field merely exists (if (req.body.email)) is not the same as confirming it is a valid, well-formed email string within expected length limits. Many injection and malformed-data bugs live in that gap.

---

Business Logic and the Service Layer

This is the part specific to your product. It decides what an order creation means, what counts as a valid state transition, and what business rules apply. A useful pattern is to keep this layer free of HTTP-specific concerns, with no direct access to req/res objects, so it can be tested and reused independently of a specific route or protocol. This is often called a service layer or use-case layer.

Handler (HTTP-aware)  →  Service (HTTP-agnostic business logic)  →  Repository (data access)

This separation makes it possible to add another interface later, such as a CLI tool, a background job, or a gRPC endpoint, that reuses the same business logic.

---

Talking to a Database Safely

A few patterns matter enough here to call out specifically, because getting them wrong shows up as production incidents, not just code smell:

• Connection pooling: reusing a fixed set of database connections across requests instead of opening a new one per request. A pool that is too small under load causes requests to queue waiting for a free connection, often surfacing as slow responses rather than obvious errors.
• Parameterized queries: passing user input as bound parameters instead of string-concatenating it directly into a query. This is the basic defense against SQL injection.
• Transactions for multi-step writes: if an operation needs to update two related tables, such as decrementing inventory and creating an order, both writes should happen inside a single transaction. Otherwise a crash between them can leave data partially updated.

-- Vulnerable to SQL injection
SELECT * FROM users WHERE email = '${userInput}'

-- Safe: parameterized
SELECT * FROM users WHERE email = $1

---

Building the Response

The final response needs the same care as the request that produced it: the right status code, correct headers such as Content-Type and Cache-Control where relevant, and a body shape consistent with the API contract. Consistent error response shapes make client integration much easier.

{
  "error": {
    "code": "VALIDATION_FAILED",
    "message": "Email is required",
    "field": "email"
  }
}

A consistent error envelope for application-generated errors lets client code handle failures generically. Infrastructure components and protocol-generated responses may still use a different shape, so clients should also tolerate an absent or non-JSON body.

---

Observability: Logs, Metrics, and Traces

Once a request flows through several layers, diagnosing a problem after the fact requires more than "check the logs." Logs, metrics, and traces answer different questions:

A trace becomes essential once a request touches more than one service, because logs and metrics alone can tell you that something was slow, but not necessarily where. A request that took 2 seconds total might have spent 1.9 seconds waiting on a downstream API call, and a trace makes that visible.

Correlation IDs are unique identifiers generated at the edge and threaded through every log line and downstream call for a single request. They make it possible to reconstruct one request across a distributed system after the fact.

---

Error Handling and Status Code Discipline

A frequent production anti-pattern is catching every error and returning 500. A malformed request can be 400, missing or invalid authentication is usually 401, insufficient permission is 403, and a missing resource is 404. Map expected domain and protocol failures deliberately; reserve 500 for unexpected server failures, while avoiding leakage of sensitive internal details.

Bad:   try { ... } catch (e) { res.status(500).send('error') }

Better: distinguish known error types (validation, not-found, forbidden)
        from truly unexpected failures, and only the latter is a real 500

---

Common Server-Side Failure Modes

---

Practical Debugging Checklist

1. Isolate which layer is responsible

Is it the proxy/load balancer?  → check its own access logs and health check status
Is it middleware?                → check middleware ordering and what runs before auth
Is it routing?                   → confirm the exact matched route, not just the intended one
Is it the handler/service?       → check business logic and validation
Is it the database?              → check connection pool metrics and slow query logs

2. Check for a correlation ID

If one exists, use it to pull every log line for that specific request across every service it touched. This removes a lot of guesswork in distributed systems.

3. Check auth vs authorization separately

Confirm the identity being attached to the request first (authentication), then separately confirm the policy check for the specific action being performed (authorization). Do not assume one implies the other.

4. Check rate limit headers on a 429

curl -I https://api.suriyaprakash.in/orders

Inspect Retry-After plus the API's documented rate-limit fields. These may be the emerging RateLimit fields or vendor-specific X-RateLimit-* fields. Response headers reveal the quota state, but the server configuration or logs are still needed to confirm the partition key.

5. Check the database connection pool under load

A saturated pool shows up as growing response times under load, not always outright errors. It is easy to misattribute this to "the code is slow" when requests are queued waiting for a connection.

---

A Request Review Checklist

The short explanation:

The server receives the request, runs it through middleware, routes it to a handler,
and returns a response.

When reviewing a backend request path, I would ask:

• The distinction between what a reverse proxy/load balancer owns versus what application middleware owns
• Why middleware ordering is a correctness property, not just style
• The concrete difference between an authentication bug and an authorization bug, with an example
• Why rate limiting's keying strategy matters as much as its algorithm
• Why tracing (not just logs and metrics) becomes necessary once a request spans multiple services

---

FAQ

What's the actual difference between a load balancer and a reverse proxy?

They overlap heavily and modern tools often do both, but conceptually a reverse proxy sits in front of one or more backend servers handling routing/TLS termination, while a load balancer specifically focuses on distributing traffic across multiple instances of the same service. Most production setups use a single tool doing both jobs at once.

Why is middleware order important?

Because each middleware can read, modify, or reject the request before passing it along. Putting authentication after middleware that already trusts request data, such as a rate limiter keyed on an unverified user ID, creates a real security gap.

What's a concrete example of an authorization bug that isn't an authentication bug?

A user logs in correctly, so authentication succeeds, but the API trusts a userId field from the request body instead of the authenticated identity attached to the token. A correctly authenticated user can then access or modify another user's data by changing that field.

Why does rate limiting by IP address alone cause problems?

Many legitimate users can share one public IP behind NAT or a corporate network, so an IP-only policy can throttle unrelated users together. Attackers may also distribute traffic across addresses. IP remains a useful signal for unauthenticated endpoints, but it is usually combined with account, device, route, risk, and global-capacity limits.

Why do I need distributed tracing if I already have logs and metrics?

Logs tell you what happened in detail for one event, and metrics tell you aggregate trends. Neither one, by itself, reliably shows you where time was spent in a multi-service request. That is what a trace is for.

Should every error return a 500 status code if I'm not sure what went wrong?

No. A validation failure, a missing resource, or an authorization failure is usually a client-side (4xx) error and should be returned as such. Collapsing everything into 500 erases a useful signal for API consumers and server monitoring.

Why would a "healthy" server instance still cause slow responses?

If a load balancer's health check only confirms the process is running, rather than confirming it can serve a request within an acceptable time, an instance stuck on something like a saturated database connection pool can keep passing health checks while still responding slowly to real traffic.

---

Glossary

---

Final Mental Model

Request arrives at the edge
      ↓
Load balancer / reverse proxy (TLS termination, routing, coarse rate limiting)
      ↓
Middleware pipeline (logging, CORS, auth, parsing), where order matters
      ↓
Router matches method + path to a handler
      ↓
Handler validates input, delegates to the service layer
      ↓
Service layer applies business rules, talks to the database inside transactions where needed
      ↓
Response built with a correct status code and consistent error shape
      ↓
Response flows back up through every layer in reverse
      ↓
Logs, metrics, and traces (tied together by a correlation ID) make all of this debuggable after the fact

With this model, "the API is being weird" turns into a specific question: which layer owns the behavior, proxy, middleware, routing, handler, service, or database?

---