Skip to content

Stop using LLM frameworks

Build direct

The core pitch of LangChain was interchangeability. Plug-and-play components. Swap ChatGPT for Anthropic for Gemini for whatever. Replace your vector database, swap out your tools, mix and match to your heart's content. Build agents from standardized Lego bricks. It sounded great.

I think there's still a place for LangGraph for orchestration. But the rest of it? I don't think LangChain makes sense anymore. Here's why.

The APIs have diverged

In the early days, LLM APIs were basically identical. Every provider took a messages array, some params for the model name and temperature, and that was about it. A unified abstraction made sense because there wasn't much to abstract over.

That world is gone.

The Anthropic API and the OpenAI API are completely different now. OpenAI has two APIs that aren't even compatible with each other: the Chat Completions API and the newer Responses API. Gemini does its own thing entirely. And new features roll out constantly, adopted by different providers at different times and in different ways.

Consider what's happened in just the past year:

  • Reasoning: Extended thinking, summarized reasoning traces, tokenized reasoning, no reasoning. Every provider handles this differently.
  • Caching: OpenAI does it automatically with no code changes. Anthropic requires you to manually mark what should be cached. Completely different approaches.
  • Tool use: Basic function calling is standardized, but advanced patterns like interleaved thinking (reasoning between tool calls) are provider-specific. OpenAI handles reasoning with tools sequentially, Anthropic lets you interleave them.
  • Multimodal: Image inputs, image outputs, embeddings, audio. Some providers have it, some don't, and the APIs look nothing alike.

I don't blame the providers for this. Why would OpenAI design their API to be compatible with Anthropic's? They're competitors. It makes total sense that they'd innovate independently.

But it means you can't just swap one provider for another unless you're doing the most basic chat completion possible. The moment you want reasoning, caching, advanced tool use, or any of the features that actually make these models useful, you're locked into provider-specific code.

LangChain's abstractions are too narrow

LangChain tries to solve this by creating a common interface. The problem is that interface has to be the lowest common denominator. It can only expose what all providers support.

Their chat model abstraction is basically: messages in, response out. Model name, temperature, maybe max tokens. That's it. There's no room for reasoning configuration, prompt caching, extended context, or any of the features that distinguish one provider from another.

I've done the research. LangChain has almost no support for reasoning. Not for OpenAI's o1 reasoning, not for Claude's extended thinking, not for any of it. Reasoning doesn't fit into their spec, so it just doesn't exist in their world.

And reasoning is the paradigm right now. Almost every new model reasons by default. Some models require specific settings to disable reasoning. If your framework can't handle the dominant paradigm, what's the point?

Abstractions are for humans

Here's the thing I keep coming back to: abstractions are only useful for humans.

The whole point of a framework like LangChain is to reduce cognitive load. You don't have to learn the Anthropic SDK and the OpenAI SDK and the Gemini SDK. You learn one interface and it handles the rest. That's valuable if you're a developer who has to hold all this in your head.

But I don't write most of my code by hand anymore. I use coding agents. And coding agents don't need cognitive shortcuts.

When I want to integrate a new provider, I don't need to memorize their SDK. I pull up their docs, tell my coding agent "use these docs to write the integration," and it handles the details. The agent doesn't care if Anthropic's tool calling looks different from OpenAI's. It reads the documentation and writes the appropriate code.

The abstraction that made sense for human developers is actively harmful for agent-assisted development. A coding agent is an extremely capable engineer. Constraining it to work through someone else's opinionated, limited abstraction means you're leaving features on the table.

Build it yourself

If you really need provider interchangeability, build it yourself. Write your own wrapper functions. You'll have complete control over what features to expose and how to handle provider-specific functionality. When Anthropic ships a new feature, you can add support immediately instead of waiting for LangChain to implement it (if they ever do).

# Your own abstraction, your own rules
def call_llm(provider: str, messages: list, **kwargs):
    if provider == "anthropic":
        return call_anthropic(messages, **kwargs)
    elif provider == "openai":
        return call_openai(messages, **kwargs)
    # You decide what kwargs mean for each provider

It's more work upfront. But you're not dependent on a third party with VCs, a roadmap that has nothing to do with your use case, and a team that has to support every possible configuration.

The bottom line

LangChain made sense when LLM APIs were simple and uniform. Those days are over. The providers have diverged, the features have multiplied, and the unified abstraction can't keep up.

If you're building something serious, use the provider SDKs directly. If you need orchestration, LangGraph is fine. But the days of plug-and-play LLM interchangeability are behind us, and clinging to that abstraction means missing out on what these models can actually do.