When LLM agents interact with APIs, the traditional interaction model is conversational and turn-based. If an agent needs to calculate portfolio risk by joining customer positions with market tickers, it makes sequential tool calls:
- Turn 1:
fetch_portfolio(accountId)returns 400 positions over HTTP. - Turn 2: The LLM reads all 400 records and requests
fetch_market_prices(symbols). - Turn 3: The LLM reads the pricing payload and calculates the portfolio risk score.
Every step in this chain incurs network transfer latency, token serialization costs, and an entire LLM inference cycle. For multi-step workflows, total execution time often exceeds 15 to 30 seconds.
In software engineering, we solved this decades ago: you do not ask a database to send raw tables over the network so your client can do the filtering. You write a query or stored procedure that runs inside the execution engine.
Code Mode applies this exact principle to Model Context Protocol (MCP) servers. Instead of repeatedly invoking tools over the wire, the LLM writes a concise JavaScript program that runs directly on the server, orchestrating multiple tools in a single inference turn.
In this deep dive, we explore how NitroStack implements Code Mode using an isolated QuickJS WebAssembly worker pool, how bidirectional IPC bridges guest code with native TypeScript controllers, and how we solved memory leak containment across recycled worker threads.
text┌─────────────────────────────────────────────────────────────────────────────┐ │ CONVERSATIONAL MCP VS CODE MODE IN WASM │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ Traditional Conversational MCP (3 Turns, 3 Round-Trips): │ │ Agent ───(call: get_orders)───► Server ───► returns 500 rows ───► Agent │ │ Agent ───(call: filter_open)──► Server ───► returns 40 rows ───► Agent │ │ Agent ───(call: notify_ops)───► Server ───► returns success ───► Agent │ │ Total: 3 LLM completions, high token consumption, ~18s latency │ │ │ │ NitroStack Code Mode (1 Turn, Server-Side QuickJS WASM Sandbox): │ │ Agent ───(execute: script.js)─► QuickJS WASM Worker Pool │ │ │ │ │ ├── callTool('get_orders') │ │ ├── array.filter(...) locally in WASM │ │ └── callTool('notify_ops') │ │ │ │ │ Agent ◄──(returns summary)─────────┴──────────────────────────────────────┘ │ Total: 1 LLM completion, minimal token footprint, <1.2s execution │ │ │ └─────────────────────────────────────────────────────────────────────────────┘
1. Why QuickJS in WebAssembly?
Executing arbitrary, LLM-generated code on a production backend introduces immediate security and operational hazards. Running code directly in Node.js via eval() or standard vm modules is unsafe because vm does not provide an isolated security boundary. Containerized execution engines (like microVMs or Docker containers) offer strong security, but their cold-start latency (100ms to 800ms) is too slow for interactive agent loops.
We chose QuickJS compiled to WebAssembly (WASM).
QuickJS is Fabrice Bellard's lightweight, fully standards-compliant ECMAScript engine written in C. Compiling QuickJS to a standalone WebAssembly module delivers three distinct advantages:
- Sub-Millisecond Cold Starts: Instantiating a new QuickJS runtime within an existing WebAssembly instance takes less than 2 milliseconds.
- Deterministic Sandboxing: The WebAssembly linear memory space provides absolute boundary isolation from the host operating system. The guest script has zero access to host memory, filesystem sockets, or environment variables unless explicitly exposed through host imports.
- Cross-Platform Reproducibility: The WASM binary runs identically across macOS development workstations, Linux Kubernetes clusters, and serverless edge runtimes.
2. The Worker Pool and IPC Architecture
While QuickJS is lightweight, spawning a new WebAssembly instance and recompiling JavaScript code for every single tool invocation still creates CPU pressure.
To achieve sustained high throughput, NitroStack implements a Worker Pool architecture with dedicated worker threads running QuickJS instances:
text┌─────────────────────────────────────────────────────────────────────────────┐ │ QUICKJS WORKER POOL & HOST IPC BRIDGE │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ @nitrostack/core Runtime Host │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ CodeModeTransform Runner │ │ │ │ ├── Worker Pool Dispatcher (size: 4 workers) │ │ │ │ └── AsyncLocalStorage Pipeline Bypass │ │ │ └──────────────────────────────────┬──────────────────────────────────┘ │ │ │ Bidirectional MessagePort / IPC │ │ ▼ │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ Worker Thread (Isolated WebAssembly Runtime) │ │ │ │ ┌─────────────────────────────────────────────────────────────────┐ │ │ │ │ │ QuickJS Engine (C compiled to WASM) │ │ │ │ │ │ │ │ │ │ │ │ Global Sandbox Scope: │ │ │ │ │ │ • callTool(name, args) ◄─── Async bridge to host controller │ │ │ │ │ │ • console.log(...) ◄─── Captured execution logs │ │ │ │ │ │ • Standard ES2023 primitives (JSON, Math, Array, Object) │ │ │ │ │ └─────────────────────────────────┬───────────────────────────────┘ │ │ │ │ │ │ │ │ │ Watchdog Timer: 15,000ms │ Memory Cap: 128 MB │ │ │ └───────────────────────────────────┴─────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────────────┘
Exposing Synthetic Discovery Tools
When CodeModeTransform is enabled in your @McpApp configuration, it intercepts the tools/list handler and presents three synthetic tools to the LLM:
search: Accepts a keyword query or regex string and returns matching tool signatures.get_schema: Returns the full Zod or JSON Schema definition for a specified tool.execute: Accepts acodestring and executes it inside the QuickJS sandbox.
typescriptimport { McpApp, Module, CodeModeTransform } from '@nitrostack/core'; import { CustomerController } from './customer.controller'; import { BillingController } from './billing.controller'; @McpApp({ module: AppModule, server: { name: 'finance-service', version: '1.1.0' }, transforms: [ new CodeModeTransform({ timeoutMs: 15000, // Terminate scripts exceeding 15 seconds memoryLimitMb: 128, // Hard cap on QuickJS heap memory workerPoolSize: 4, // Dedicated pool of isolated workers }), ], }) @Module({ name: 'finance-module', controllers: [CustomerController, BillingController], }) export class AppModule {}
3. Bidirectional IPC: Bridging callTool()
The primary power of Code Mode lies in the guest environment's ability to call back into native TypeScript controllers via callTool().
When the guest script calls:
javascriptconst invoices = await callTool('get_unpaid_invoices', { customerId: 1042 });
The underlying bridge executes through the following stages:
- Guest Invocation: The QuickJS C bridge intercepts
callTool, serializes the arguments, and emits an asynchronous message across the WebAssembly boundary to the worker thread. - Worker to Host IPC: The worker forwards the invocation request to the main Node.js process using structured cloning.
- Internal Controller Execution: The host executes the native
@Toolcontroller method. Because NitroStack wraps this call in theAsyncLocalStoragebypass context, the invocation executes directly without re-triggering the transform pipeline. - Result Return: The controller's response is serialized, transferred back to the worker, and injected back into the waiting QuickJS Promise resolution.
This entire sequence completes in sub-millisecond round-trip times because all communication occurs locally within the same host process.
4. Systems Resiliency: Watchdogs and Memory Leak Containment
Allowing agents to generate and execute code introduces two major operational hazards: infinite execution loops and progressive memory exhaustion.
Watchdog Interruption Handling
If an agent generates code containing an unintentional loop (while(true) {}), it could permanently starve the worker thread of CPU resources.
QuickJS provides an execution interrupt handler: JS_SetInterruptHandler(). In NitroStack's worker runtime, we configure a dedicated watchdog timer:
typescript// Conceptual implementation inside worker runtime let startTime = Date.now(); const timeoutLimit = config.timeoutMs || 15000; runtime.setInterruptHandler(() => { if (Date.now() - startTime > timeoutLimit) { // Forcefully interrupts execution and unwinds call stack return 1; } return 0; });
If the execution duration exceeds timeoutMs, the engine throws an uncatchable TimeoutError, halts the script, and emits a structured error response back to the client.
Containing Heap Growth Across Reused Workers
One of the most subtle engineering challenges in building long-running worker pools is gradual heap expansion across recycled worker threads.
In early prototypes, when workers were reused across sequential executions, global variables or uncleared QuickJS context handles caused memory to grow over time. Under continuous load, a 4-worker pool would steadily climb toward process memory limits.
We implemented a two-tier containment strategy:
- Per-Execution Context Recycling: While the WebAssembly module instance remains warm in memory, the
JSContextis explicitly destroyed (JS_FreeContext) and recreated (JS_NewContext) on every execution turn. This guarantees that all globals, closures, and object graphs created during a run are reclaimed. - Memory Ceiling Guards (
memoryLimitMb): QuickJS exposes runtime memory bounds viaJS_SetMemoryLimit(). If guest code attempts to allocate arrays or strings exceeding the configured threshold (e.g., 128 MB), the allocator immediately fails with anOutOfMemoryexception, shielding the host container from Linux OOM kills. - Heap Health Telemetry: If a worker's base memory footprint exceeds a safety delta between executions, the pool supervisor automatically terminates the worker and spawns a pristine replacement thread.
text┌─────────────────────────────────────────────────────────────────────────────┐ │ WORKER RECYCLING & HEAP CONTAINMENT LIFECYCLE │ ├─────────────────────────────────────────────────────────────────────────────┤ │ │ │ Worker Idle (Pristine QuickJS Runtime) │ │ │ │ │ ▼ │ │ New Execution Turn ──► Create JSContext ──► Run Script (128 MB Cap) │ │ │ │ │ ▼ │ │ Script Finish or Timeout ◄────────────────────────┘ │ │ │ │ │ ▼ │ │ JS_FreeContext() ──► Full GC Pass ──► Inspect Base Heap Size │ │ │ │ │ ├── If Heap Delta ≤ Baseline ──► Return Worker to Idle Pool │ │ │ │ │ └── If Heap Delta > Baseline ──► Destroy Worker & Spawn Fresh Thread │ │ │ └─────────────────────────────────────────────────────────────────────────────┘
5. Destructive Guardrails and Execution Safety
Allowing an LLM to generate scripts raises legitimate questions about safety. What prevents an agent from invoking destructive tools in an uncontrolled loop?
NitroStack provides defense-in-depth through three complementary mechanisms:
- Schema-Enforced Tool Filtering: Transforms can filter which tools are visible to the QuickJS execution context. Tools marked with administrative or destructive metadata can be excluded from Code Mode entirely.
- Dynamic Authorization Inheritance: Every
callTool()invocation triggered from inside QuickJS inherits the original caller's authenticatedExecutionContext. If a tool is protected by@OAuthGuardor requires specific JWT scopes, the authorization layer rejects unauthorized calls with standard protocol errors. - Execution Logging and Auditing: Every script executed by
CodeModeTransform, along with captured console output and tool call traces, is emitted to standard OpenTelemetry tracing collectors.
6. Real-World Benchmark: Latency and Token Economics
To validate the real-world impact of Code Mode, we benchmarked a multi-step data reconciliation scenario comparing traditional conversational MCP against NitroStack Code Mode:
Scenario: Retrieve 250 enterprise user records, filter for inactive accounts with pending balances, and schedule a reminder task.
text┌─────────────────────────────────────────────────────────────────────────────┐ │ CONVERSATIONAL MCP VS CODE MODE BENCHMARK │ ├───────────────────────────────┬──────────────────────┬──────────────────────┤ │ Metric │ Traditional MCP │ NitroStack Code Mode │ ├───────────────────────────────┼──────────────────────┼──────────────────────┤ │ Total Execution Time (Latency)│ 16.4 seconds │ 1.18 seconds │ │ LLM Inference Turns │ 3 completions │ 1 completion │ │ Total Prompt / Return Tokens │ 18,400 tokens │ 1,840 tokens │ │ Network Round-Trips │ 3 over HTTP │ 1 over HTTP │ │ Token Cost Savings │ Baseline (100%) │ 90.0% Reduction │ └───────────────────────────────┴──────────────────────┴──────────────────────┘
By shifting intermediate data processing from the LLM prompt down to the server-side WebAssembly runtime, Code Mode achieves a 13.8x latency improvement and cuts token consumption by 90%.
Summary and Next Steps
Code Mode represents a fundamental transition in how autonomous agents interact with services. By pairing Fabrice Bellard's QuickJS engine with the safety of WebAssembly, NitroStack turns MCP servers into high-performance execution runtimes that slash latency and protect token budgets.
In the next article in this series, we will explore the input side of the context window problem: how BM25 Progressive Tool Discovery prunes 80+ tool catalogs down to lightweight meta-tools with zero developer configuration.
About the Author
