What if Lambda and EC2 had a baby? Anatomy of an agent runtime

We've learned that users rarely care about infrastructure until it breaks or gets too slow.

We constantly evaluate, benchmark, and optimize our platform. Our docs and code samples use names like "Mark 2.0" and "Mark 3.0".

Each "Mark" is an infrastructure generation that we designed, tested, rolled out to production, and eventually deprecated in favor of something better.

AI agents that execute code are a new kind of workload. They need a serverless function's fast start and the persistence of a full server. Standard cloud infrastructure forces you to pick one. We decided to build both.

Carbon, also called Mark 3.1, is that runtime in private preview, built on bare metal and microVMs.

Key takeaways

• Mark 3.0 is the default, with Firecracker microVMs that resume from standby in under 25ms.
• Carbon, also called Mark 3.1, is in private preview and rolls out by region and workspace.
• Agent Drive, firewalling, and dedicated egress stay on Mark 3.0 for now.
• Standby has no compute charge, so idle cost is snapshot and volume storage.

What an agent runtime does

An agent runtime is the execution layer that runs an agent's actions in production. The framework decides what the agent does next, and the runtime supplies the place where that work happens. It hosts processes and files and keeps state between steps and sessions. It also isolates untrusted code, controls network access, and manages each environment from creation to teardown.

You'll often see the runtime blurred with the layers around it. Here's how they split:

The sandbox decides whether an exploit in AI-generated code stays contained. The runtime around it decides whether the agent's work survives an idle period or a move to another host.

The same split now shows up in the frameworks your team already uses. In April 2026, OpenAI added native sandbox execution to the Agents SDK. Built-in providers include Blaxel, Cloudflare, Daytona, E2B, Modal, Runloop, and Vercel. Blaxel is a first-class sandbox provider in the OpenAI Agents SDK. For your team, picking a runtime becomes an execution-layer decision rather than a framework rewrite.

Mark 1.0, October 2024: Self-managed Kubernetes with Knative Serving

When we first started Blaxel, our goal was to offer serverless infrastructure for AI models. We focused on AI inference workloads. Users could offload model requests to remote compute through our Global Agentic Gateway. This first-generation infrastructure (Mark 1.0) ran on Kubernetes with Knative Custom Resource Definitions (CRDs).

Knative is a Kubernetes extension for serverless workloads. Its CRDs cover the key parts of a serverless platform: services, routes, configurations, and revisions. It has three primary components. Knative Serving handles auto-scaling, Knative Eventing manages events, and Knative Functions supports serverless function development. Within clusters, Mark 1.0 relied on Knative Serving's auto-scaling. When requests for a model spiked, Knative Serving added pods to meet demand and scaled back down later.

That covered scaling inside a cluster. We still needed to manage traffic between clusters and keep availability high. So we built a custom Blaxel Inference gateway to offload inference requests from the customer's cluster to ours. It routed based on inference metrics, HTTP status, and health checks. The gateway sat in front of the cluster load balancer and rerouted requests to our closest managed clusters when thresholds were exceeded. Integrated heartbeat monitoring redirected traffic as soon as a threshold was crossed. The gateway also spun up new instances of the customer's model on the target cluster.

Serving inference alone wasn't enough, though. The economics of being only an overflow option were hard to sustain (but that's another story). Customer interviews showed us that infrastructure was still built for the SaaS era. Very few specialized options existed for non-inference workloads like agents, tool servers, or batch jobs.

Mark 1.0 was reasonably stable. Its cold starts were acceptable for models, because model loading often takes far longer than an operating system boot. Lightweight workloads were a different story. Tool calls, sub-agent orchestration, and external model routing couldn't absorb those cold starts.

Mark 2.0, November 2024: Managed Kubernetes with managed Knative aka Google Cloud Run

In January 2025, we decommissioned Mark 1.0 in favor of Mark 2.0, which ran containers on a managed cloud. We benchmarked Google Cloud Run, Cloudflare Workers, and AWS Lambda, then picked Google Cloud Run for two reasons. First, it was compatible with Knative APIs, so we could migrate quickly. Second, it gave us the most flexibility in workload type and run time. AWS Lambda functions, for example, are limited to 15 minutes, a hard ceiling for long agent sessions. Cloud Run wasn't the best on cold starts. Cloudflare Workers were interesting, but WebAssembly's limitations were too numerous to support AI use cases.

Google Cloud's APIs and Terraform provider also had built-in multi-region automation. That let us replicate our infrastructure in more than a dozen global regions with little extra work. We built our own orchestrator to decide where to deploy workloads. It also applied deployment policies (for example, no deployment in the US or EU). The orchestrator handled autoscaling as well and routed requests to the closest region, which we cover in our networking layer post.

Mark 2.0 also suffered from slow cold starts, but it offered two benefits. Containers used the host kernel for system calls. And once warmed up, deployments answered later requests almost instantly.

Mark 2.0 suited models and batch jobs. It didn't work well for agent workloads, tool calls, or sandboxes, for two main reasons:

• Slow cold starts: On Mark 2.0, container cold starts typically took between 2 and 10 seconds, which slowed agent responses. Those delays slowed agent responses and hurt the end-user experience.
• Weak isolation for untrusted code: Containers are the standard for running your own trusted software. Agents that execute AI-generated code need more, because containers share the host kernel. Prompt injection lets malicious users retrieve the agent's environment variables and credentials. Attackers then use those credentials directly in the agent container. Container escape exploits give attackers direct access to host resources and data.

Enter Mark 3.0.

Mark 3.0, March 2025: Unikernels and microVMs

We needed fast, sandboxed execution of arbitrary code that could also scale horizontally. So in March 2025, we started prototyping unikernel-based machines with the Unikraft open source project.

Unikernels are fast, single-purpose kernels. They contain only the libraries their purpose requires, and they can run directly under a hypervisor. Agentic workloads typically involve many short-lived agents, each needing an isolated environment and a small burst of compute. Small footprints and fast boots fit that pattern well.

Unikernels gave us the boot times and isolation we needed. Then we hit a speed bump: missing shared libraries, a side effect of unikernels' static linking. For example, we needed libssl in our microVMs, which relies on operating-system entropy sources to seed OpenSSL’s cryptographic random-number generator. Those modules and libraries didn't exist by default in the unikernel. We had to import them manually during build, and that approach did not cover applications with different dependencies. We eventually replaced the unikernel with a stripped-down, highly optimized full kernel. As a side bonus, that kernel also cut our cold start times.

That shift from pure containers to a microVM-based hypervisor defined Mark 3.0. In April 2025, during Y Combinator, we launched sandboxes that stay in standby on this infrastructure. Our stack ran self-managed Unikraft nodes with Firecracker microVMs. A custom Virtual Kubelet, built for sandboxes, reduced the latency Kubernetes adds to cluster orchestration. A custom cluster gateway based on the Pingora framework handled routing. Our build process downloaded and copied four artifacts. They were a custom kernel image, a custom Enhanced Read-Only File System (EROFS) image, a boot wrapper for custom boot behavior, and a metadata file.

This infrastructure was fast and built for agent workloads. Sandboxes resumed from standby in under 25ms. For a coding agent that pauses between user prompts, the sandbox can sleep without the user noticing the wake-up. Sandboxes offered hardware-enforced isolation and a filesystem that persisted across standby, but was erased when the sandbox was terminated. They supported both REST and Model Context Protocol (MCP) for in-sandbox operations. The platform managed the full sandbox lifecycle automatically, including snapshotting and resume from standby.

Node density improved in two ways. Spatially, our optimized images let us run more sandboxes per node while keeping strong microVM isolation. Temporally, fast resume let us turn idle sandboxes off aggressively, freeing capacity for more customers. Both gains reduced the physical nodes needed for the same number of sandboxes. Customers won too, since they paid compute costs only while a sandbox was actively processing a request.

The downside was a bare-metal requirement. We had to avoid the overhead and compatibility issues of nested virtualization (VMs running inside VMs). That added new constraints, along with chances to tune software and hardware together.

Mark 3.0 worked well for our launch and still runs in production. We wanted to improve it in three areas:

• Multi-tenancy and scheduling: We wanted native multi-tenancy management and cluster scheduling. Both were critical for horizontal scaling.
• Networking: We wanted a complete virtual private cloud (VPC) experience, with sandboxes on different hosts able to talk to each other. We wanted faster, more scalable networking without standard TAP virtual network devices. We also wanted per-sandbox dedicated IPs and granular network policies between workloads.
• Storage: We wanted faster filesystem performance for package managers like npm and pnpm. We also wanted more flexible capacity allocation, including moving storage-dependent workloads like databases between nodes.

In June 2025, we started work on Mark 3.1 and kept Mark 3.0 in production. We deprecated Mark 2.0 in Q4 2025 and decommissioned it in Q2 2026.

Carbon (Mark 3.1), in private preview: MicroVMs with cluster orchestration and VPPs

Carbon, also called Mark 3.1, is in private preview and rolls out by region and workspace. It keeps the Mark 3.0 microVM foundation and adds cluster orchestration, networking, and manual snapshots. Mark 3.0 remains the default for sandboxes and batch jobs.

Here are the key technical improvements in Mark 3.1.

Storage and filesystem

Snapshots can now move between hosts. CPU templates make sure a snapshot can resume later on another host with the same CPU architecture. The two hosts' CPU feature sets can differ. For example, a snapshot can move from AMD EPYC 9654 (Zen 4) to AMD EPYC 9754 (Zen 4c).

The default filesystem still lives in RAM, and we snapshot memory state and filesystem together. That gives the best performance and full consistency when a sandbox resumes from standby.

These three storage tiers are on Mark 3.0. Agent Drive is not on Carbon yet.

Volumes survive sandbox destruction and recreation. Agent Drive stays on Mark 3.0 for now, in private preview and only in us-was-1. Both Volumes and Agent Drive are slower than the sandbox filesystem, so keep hot working data local.

We're also evaluating NVMe over Fabrics (NVMe-oF) as a transport, with a custom disk controller, for future block storage.

Networking

Mark 3.1 moves entirely to IPv6 with VPP (Vector Packet Processing). Each sandbox appears as a distinct compute instance with a real IPv6 address, and VPP routes packets to it correctly. We also changed the virtual interface type, which makes creating a VM and configuring its network much faster.

With this, you can create private networks between sandboxes. For example, an application in one sandbox can call API servers and databases in other sandboxes inside your private network. Per-sandbox addresses also remove rate limiting caused by shared IP addresses and lay the groundwork for a true VPC experience. Dedicated egress gateways are in private preview on Mark 3.0, and firewalling stays there too. Proxy secrets injection is in public preview. It adds secrets on the way out, so sandbox code never sees raw API keys.

Forking a sandbox is part of Carbon, so it is in private preview. A forked snapshot keeps the image, memory size, ports, generation, region, and volumes. A coding agent can then try several fixes in parallel from one warm state.

Scalability

Mark 3.1 has a new virtual machine monitor (VMM) that helps our customized Kubernetes scheduler spread workloads across nodes more efficiently. The VMM handles Firecracker deployment on each node. Virtual Kubelet translates between the Kubernetes control plane and the VMM. To speed things up further, we built a custom scheduler. We also inspected Kubernetes' deployment controllers and removed every component that slowed them down. New sandboxes reach a node with less scheduling delay.

We improved the build process. The kernel image now sits directly on the host, so there's no download step. The build also uses a custom EROFS image, a shared boot library on the host, and a metadata file. With no kernel download, sandbox creation skips a network step.

What happens when a sandbox goes idle

For agents executing code, idle periods test whether a runtime preserves state without charging for compute. Blaxel is the infrastructure foundation for these autonomous agents. Its sandboxes move through these states:

• Active to standby: A sandbox begins returning to standby after 15 seconds of network inactivity. Standby carries no compute charge; you pay only for snapshot and volume storage.
• Standby duration: Sandboxes can stay in standby indefinitely. Configured expiry policies can cap persistence for automatic cleanup.
• Resume: A sandbox wakes with memory, files, and running processes intact.
• Archive: Archiving drops memory, and processes restart automatically on restore. A snapshot can be forked into other sandboxes, while an archive only restores the sandbox it came from.
• Durability: Standby keeps the session ready to resume. For data that must last for months, write it to a Volume.

With coding agents and pull request (PR) review agents, users return hours later to the same repository checkout. They don't pay compute for the hours in between. Match each piece of agent state to the tier that fits how much losing it would cost you.

Mark 3.1 performance

Resume from standby on Mark 3.0 stays under 25ms, with memory and files intact. Carbon's own resume and bandwidth numbers are not public yet.

Software and networking aren't the only gains. Mark 3.1 runs on better hardware, and we measured a clear CPU performance gain. We also have more room to tune software to the hardware we choose. One example is CPU features for cryptography. Another is hardware-accelerated networking that offloads packet processing to SmartNICs with Data Plane Development Kit (DPDK).

Carbon is in private preview and rolls out by region and workspace. Batch jobs still run on Mark 3.0, which remains the default. If you'd like to know more, we'd be happy to answer questions. Join our Discord and drop us a comment! And if hacking on AI-native infrastructure is your jam... we're hiring!

Choosing an agent runtime for code-executing agents

An agent runtime determines whether your agents' code runs isolated, how fast it starts, and what survives between sessions. From Mark 2.0 onward, each generation was our attempt to get all three right at once. If your agents execute code, check candidate runtimes on:

• Isolation boundary (microVM vs. shared kernel)
• Resume time from standby
• How long a sandbox stays in standby, including any expiry policy you set
• Egress and secrets control
• Pricing while idle

For coding agents and PR review agents, Blaxel is one option to evaluate alongside Modal, E2B, and Daytona. Blaxel provides Firecracker microVM sandboxes that can stay in standby with no compute charge, Agent Drive (in private preview) for sharing context across sessions, and Batch Jobs for parallel fan-out work. Use the Blaxel documentation to check these capabilities against your requirements.

FAQs about agent runtimes

What is an agent runtime?

It's the infrastructure that executes an agent's work in production. The runtime provides compute, isolation, state, network controls, and lifecycle management. A framework like LangGraph, CrewAI, or the OpenAI Agents SDK defines the agent's logic and runs on top of it.

Is a standard serverless runtime enough for agents that execute code?

For short, stateless tasks, it often is. Agents that run AI-generated code across long sessions need hardware-enforced isolation plus state that survives idle periods. We moved from Cloud Run containers to Firecracker microVMs.

Do I need microVMs, or are containers fine?

Containers work well for your own trusted software. For AI-generated or untrusted code, microVMs provide hardware-enforced isolation that containers can't match, because containers share the host kernel. gVisor improves on containers by intercepting system calls in user space, but microVMs still give you a hardware boundary.

What are persistent sandboxes?

Persistent sandboxes keep their filesystem and memory across idle periods rather than being destroyed after each task. Blaxel sandboxes sit in standby with no compute charge and resume with memory and files intact. Your code-executing agent picks up exactly where it stopped.