A few weeks ago, I spoke at the inaugural MCP Dev Summit Mumbai, part of a global series of events by the Agentic AI Foundation. The talk covered three challenges that come up when building and running multi-agent systems at scale: integration complexity, code isolation, and state preservation. This post focuses on the third challenge, and why we at Blaxel think the answer is a shared filesystem.
The common approach to coordinating agents working the same task is to pass serialized JSON between them: state bundled into messages and transferred from one agent to the next. This works fine for simple workflows, but both durability and complexity become harder to manage as swarms grow.
In my talk, I examined the feasibility of replacing this JSON messaging approach with filesystem-based storage. Under this approach, agents read and write their results as simple Markdown files to a shared filesystem.
This lets the state persist beyond any single message and also removes size constraints when dealing with large or binary artifacts. An additional - and often underrated - benefit is that it significantly simplifies debugging, because you can always go back and look at the files an agent left behind to figure out where things went wrong.
Blaxel's Agent Drive is designed to support these types of agentic workflows:
- It's a distributed filesystem that can be mounted to multiple sandboxes or agents simultaneously, giving every agent a common workspace.
- It supports concurrent read-write access, so agents can write outputs and pick up inputs without hitting conflicts.
- Replication is built in, so generated artifacts can be persisted, reused, and audited.
- It has granular access control, enabling the same Agent Drive to be mounted read-write for one agent and read-only for others
After walking through this filesystem-based approach (plus discussing solutions to the other challenges), I demonstrated it live: a small swarm of agents working on a task, coordinating with each other through files written to an Agent Drive.
The demo went well and once I was off-stage, numerous developers came up to me with follow-up questions, both about Agent Drive and Blaxel's other platform features more broadly. We talked about scalability, data protection, performance, and applicability to different use cases.
Beyond the talk itself, this was a great event. It was extremely well organized, it was packed with excellent technical talks, and I made connections with many people building in the agentic AI space in India. Here’s to the next one!
If you're building multi-agent workflows and Agent Drive sounds like it might be a good fit for you, it's currently in private preview. Join the waitlist to get access.
