The AI Agents Stack Aging

Published: November 14, 2024 | Original Post

---

A comprehensive analysis of the AI agents ecosystem organized into four primary layers.

What Makes Agents Unique

Agents require two critical capabilities absent from simple chatbots:

• State management -- Maintaining conversation history, long-term memories, and multiple LLM calls
• Tool execution -- Safely running actions the LLM outputs and returning results

The Four Layers

1. Model Serving

OpenAI, Anthropic, Together.AI, Fireworks, and Groq for API-based inference. vLLM dominates production-grade GPU serving.

2. Storage

Vector databases (Chroma, Weaviate, Pinecone, Qdrant) store agent external memory. Postgres with pgvector competes effectively.

3. Tools and Libraries

Tools use OpenAI's JSON schema standard for cross-framework compatibility. Composio, Browserbase, and Exa provide specialized tool ecosystems.

4. Agent Frameworks

Major frameworks include Letta, LangGraph, CrewAI, AutoGen, and phidata, differing in state management, context window structure, memory approaches, and open-model support.

Key Insight

Most agent frameworks today are designed for agents that do not exist outside of the Python script or Jupyter notebook they were written in. Future production systems will deploy agents as persistent services via REST APIs.