One gateway, eight backends, and no MCP session in sight

I built an internal MCP gateway at KnowBe4 that fronts eight backend MCP servers behind one public connector — seven distinct services, really, since two of those eight declared backends are two logical namespaces on the same physical service. Every employee, whether they're on claude.ai, Claude Code, or an internal agent coordinator, configures exactly one MCP endpoint. The gateway fans out tools/list to all eight, merges the results, and routes every tools/call to the right one. That's the whole pitch: one connector instead of eight, adopted once instead of eight times.

The implementation detail that actually mattered, though, is what the gateway talks to those backends with. It's not the official MCP SDK client. It's node:http and node:https, hand-rolled. That looks like reinventing a wheel the SDK already built, and I want to explain why it isn't.

What the SDK client assumes

The MCP SDK's client transport is built around a session. You call initialize, the server hands back session state, and every subsequent request on that connection is understood to be part of the same conversation with the same server. That's a reasonable model for a single client talking to a single server over a long-lived connection. It's also a model the gateway can't afford at either end of its own request path.

At the public edge, the gateway runs at least three tasks behind a load balancer with no session affinity, specifically so a task recycling or an AZ event doesn't take the whole service down. A client that initializes against task A and gets routed to task B on its next call hits a server that never saw the initialize handshake. The failure isn't subtle: -32000 Server not initialized. It reads exactly like an outage, because from the client's point of view, it is one.

At the backend boundary, the same problem shows up in the other direction. If the gateway held a real MCP session per backend, it would need to pin a client connection to a specific upstream instance, track that session's lifecycle, and re-initialize it if the backend restarted or the gateway itself scaled. Multiply that by eight backends and however many gateway tasks are running concurrently, and you've built a stateful mesh in the one component whose entire job is to be a thin, replaceable front door.

One SDK, two different answers

The two ends of the gateway didn't land on the same answer for the same reason, and it's worth being precise about which is which. At the edge, the gateway is an MCP server, and it still uses the SDK's own Server and StreamableHTTPServerTransport, just statelessly, one per request, instead of pinning a long-lived session to a task. That's not a rejection of the SDK. It's the SDK's own transport, used in a mode that doesn't assume the load balancer will keep routing a client back to the same task.

At the backend boundary, the gateway is an MCP client, and there the SDK client genuinely was rejected, for the structural reason above: it assumes and imposes a session lifecycle, and pinning a session per backend across however many gateway tasks are running concurrently would have built a stateful mesh in the one component whose entire job is to be a thin, replaceable front door. Raw node:http/node:https calls don't carry that assumption at all.

Same constraint, no durable session, produced two different fixes at the two ends: stateless reuse of the SDK's own server transport at the edge, and hand-rolled HTTP in place of the SDK's client at the backend boundary.

The backend side isn't dogmatically stateless either. Most backends handle a sessionless call fine. A handful genuinely need session state and get promoted to a real handshake only when a sessionless call actually 400s. Stateless is the default, not a rule enforced past the point it stops making sense.

What raw HTTP buys back

Once you're not routing everything through the SDK's session-shaped API, you get direct control over the parts of the request path that the SDK client abstracts away: connection keep-alive behavior, response byte caps so one runaway backend can't blow up gateway memory, a per-call timeout applied as one global value across every backend today, forwarding the caller's bearer token on every single call rather than once at session setup, negotiating MCP protocol version per request, and handling both SSE and plain JSON response bodies depending on what a given backend returns. None of that is exotic. It's the kind of control you'd want from any HTTP proxy, and the SDK client isn't built to be a proxy. It's built to be an application talking to one server it has an ongoing relationship with. The gateway has eight of those relationships, all of them arm's length, all of them re-established on every call.

Flat namespace, one separator

Aggregating eight backends' tool lists into one creates an obvious second problem: name collisions. Two backends can each register a tool called search or get_details, and a merged tool list has no way to tell a model which is which unless something disambiguates them first.

The gateway prefixes every tool with its origin backend, joined by a double underscore: serverName__remoteName. Every one of the gateway's declared backend names carries a kaia_ prefix of its own, so a tool called create_issue on a backend named kaia_org shows up as kaia_org__create_issue, never as a bare create_issue or under an unprefixed name. The same tool name from a different backend would show up under a different prefix entirely, so collisions can't happen by construction. Splitting the name back apart at dispatch time is a single string split on the first __, which keeps the parsing trivial and keeps server names free to contain underscores of their own.

The one sharp edge is a name that's a prefix of another name. If one backend is called search and another is called search_internal, splitting on the first __ becomes ambiguous the moment both register a tool with the same remote name. The gateway closes that at startup: it rejects any backend configuration where one server name is a prefix of another, before the service ever accepts traffic. That's a boot-time check, not a runtime one, because a naming collision is a configuration mistake, and configuration mistakes should fail loud and early rather than surface as a misrouted tool call three weeks later.

What the next backend costs

Adding a ninth backend to the gateway today means one entry in config.ts and, if its name collides with an existing one, a boot-time failure that says so before any traffic gets routed through it. It doesn't mean touching the SDK, the session logic, or the namespace prefixing at all. That's the actual test of whether the raw-HTTP decision paid for itself: not that it was the more interesting choice, but that the ninth backend is cheaper to add than the first eight were to design around.