To The Moon

I think we’re starting to see the core pillars of “agent cloud infrastructure” emerge, and it’s starting to look a whole lot bigger than what traditional cloud infrastructure became.

If you started a SaaS company in the earlier days of the public cloud (say ~2012), the public cloud guys ( $Amazon.com(AMZN)$ AWS, $Microsoft(MSFT)$ Azure, $Alphabet(GOOGL)$ GCP) probably sold you a pretty simple bundle.

Some virtual servers (EC2), a place to put files (S3, ie storage), a managed database (RDS) and some kind of load balancer to spread traffic across your servers (and of course all the bells and whistles of autoscaling, backup, recover, etc). Later on as the cloud got more mature we got containers and services like Lambda, but the design center didn’t change that much - you had an application, serving lots of users, running 24/7. On top of this, the app tier was stateless so you could copy it, state go pushed down into the database, and in general things were optimized for throughput and utilization. You’d rent a box, and the box ran your code, and a lot of it existed to squeeze more human requests through fewer machines. So in essence the core cloud stack was:

• Virtual servers (EC2)

• File storage (S3)

• Managed Database (RDS)

• Load balancer

Now, however, agents have flipped and evolved the needs of classic cloud infrastructure. A new “agent cloud infrastructure” is emerging. The simple way I can try and describe this shift - the old cloud was built with the intention of allowing one app to serve millions of people. The agent cloud has to let millions of agents each run their own little app (and the type of workload is very diverse). Sometimes these little apps run for a few seconds, or a few days, or anything in-between.

On top of this, each app needs its own machine rather than a piece of a broader machine (because the code that is being run was written by a model a few seconds ago most likely). Every one needs to store it’s state, remember where it left off so it can start / stop, etc. And the workloads are unpredictable and arrive in bursts! Jensen framed it well at the $NVIDIA(NVDA)$ Vera launch earlier this year when he said the old CPUs were designed around lots of rentable cores, but an agent doesn’t want to rent a core, it wants the task finished quickly. Earlier I mentioned cloud infra workloads were more about optimizing utilization / throughput per app. Now for agent cloud infra what’s being optimized for is isolation, memory and clod start times, etc. Now the agent cloud stack is starting to look like:

• Sandboxes: This is the new virtual server. They are disposable, isolated computers per agent task

• Persistent state: This is the new file storage. A filesystem that lives longer than the individual task so agents can pause, snapshot, fork, etc. Shoutout to Tigris who was acquired this week by Dropbox!

• Inference: The obvious one! Inference clouds like Baseten / Fireworks / Modal are taking off.

• Router / Gateway: This is the new load balancer. Instead of spreading traffic across identical servers (ie the cloud infra stack pillar) you now want to spread workloads across different models.

• RL and training loops. Infra to turn open weight models into customized ones. You need environments and evals and many other services.

• CPUs, storage and network underneath. Boring stuff, but still needed! And need in a much bigger way than what was needed in more simple cloud infra

These are just the initial core pillars. There’s also another ring of agent cloud infra forming around this. Workflows / durable execution, browser infrastructure so agentc can use the web, CI/CD and test compute for all of the code that is written, new git backends that can store petabytes per day (shoutout Pierre Computer Company on their acquisition by OpenAI), identity and payent rails, all of the classic observability but now an order of magnitude greater given all the metric data agents spit off, memory stores, etc.

Net net - I think all of this agent cloud infra ends up dwarfing the size of classic cloud infra (and when I say classic cloud infra I’m referring more towards cloud application infra). The first cloud was compute / network / storage for applications humans were running. But the new cloud has the same core primitives , but rebuilt for agents, with a lot more underlying units.

I’m very excited to see how agent cloud infra evolves over the next 12 months - I think we’re due for an explosion!