# NodeLLM

> The Backend-First AI SDK for Node.js

NodeLLM is an open-source infrastructure layer for building provider-agnostic, production-grade LLM systems in Node.js. It standardizes integrations across OpenAI, Anthropic, Gemini, DeepSeek, Bedrock, OpenRouter, xAI, Ollama, and Mistral into a single, predictable API.

## Project Details

- **Type**: Node.js LLM SDK / Infrastructure Layer
- **Primary Use**: Building scalable AI workers, APIs, and agents
- **Languages**: JavaScript, TypeScript
- **License**: MIT
- **Package**: `@node-llm/core`
- **Testing Package**: `@node-llm/testing`
- **Repository**: `https://github.com/node-llm/node-llm`
- **Documentation**: `https://nodellm.dev`

## Core Features

- **Provider Agnostic**: Unified API for 540+ models (OpenAI, Anthropic, Gemini, Bedrock, DeepSeek, OpenRouter, xAI, Ollama, Mistral).
- **Backend-First**: Optimized for long-running processes, CRON jobs, and persistent agents (not just frontend streaming).
- **Static Model Registry**: Zero-latency, offline access to model metadata (context window, pricing, capabilities).
- **Middleware Architecture**: Intercept and modify LLM requests/responses globally or locally (PII masking, cost tracking, telemetry).
- **ORM Integration**: First-class support for persisting chat history and tools via `@node-llm/orm`.
- **Deterministic Testing**: Native support for recording/replaying LLM interactions (VCR) and fluent mocking via `@node-llm/testing`.

## Architecture

1.  **Core**: Light-weight, zero-dependency abstraction layer with middleware support.
2.  **Providers**: Pluggable adapters that normalize inputs/outputs.
3.  **Registry**: Static JSON database of model capabilities.
4.  **MiddlewareStack**: Multi-layer orchestration for Evals, Logging, and Redaction.

## Usage Example

```ts
import { NodeLLM } from "@node-llm/core";

// 1. Unified Interface with Middleware
const chat = NodeLLM.chat("gpt-4o", {
  middlewares: [piiMasker, costLogger]
});

// 2. Standardized Response
const response = await chat.ask("Explain infrastructure-as-code");
```

## Comparisons

- **vs Vercel AI SDK**: NodeLLM is optimized for backend/workers, whereas Vercel AI SDK is optimized for Frontend/Next.js streaming.
- **vs LangChain**: NodeLLM is a thin infrastructure layer with a static registry, whereas LangChain is a comprehensive framework with complex abstractions.
- **vs OpenAI SDK**: NodeLLM wraps the OpenAI SDK to provide multi-provider support with identical code.

## Quick Links

- **Get Started**: https://nodellm.dev/getting-started/overview
- **Architecture**: https://nodellm.dev/architecture
- **Testing**: https://nodellm.dev/core-features/testing
- **Contributing**: https://github.com/node-llm/node-llm/blob/main/CONTRIBUTING.md