Overview

High-level overview of NodeLLM components, design principles, and how the framework works.

Table of contents

1. Core Components
   1. Chat
   2. Providers
   3. Tools
   4. Configuration
2. Design Principles
   1. Unified Interface
   2. Streaming First
   3. Progressive Disclosure
3. Configuration Patterns
   1. 1. Fluent Builder API
   2. 2. Direct Configuration Object (Stateless)
4. How It Works

---

NodeLLM provides a seamless, unified interface for interacting with multiple Large Language Model (LLM) providers. Whether you are building a simple chat bot or a complex multi-modal agentic workflow, NodeLLM abstracts away the provider-specific complexities.

---

Core Components

Understanding these components will help you use the framework effectively.

Chat

The primary interface for conversational AI. NodeLLM.chat() creates a stateful object that manages conversation history.

const chat = llm.chat("gpt-4o");

Providers

Adapters that translate the unified NodeLLM format into provider-specific API calls (OpenAI, Anthropic, Gemini). You rarely interact with them directly; the library handles this based on the model ID you choose.

Tools

Functions that the AI can execute. You define the schema and the handler, and NodeLLM manages the execution loop automatically.

Configuration

Global settings for API keys and defaults.

const llm = createLLM({
  openaiApiKey: "sk-...",
  provider: "openai"
});

---

Design Principles

Unified Interface

Every provider works differently. NodeLLM normalizes inputs (messages, images) and outputs (content, usage stats) so your code doesn’t change when you switch models.

Streaming First

AI responses are slow. NodeLLM is built around AsyncIterator to make streaming text to the user as easy as a for await loop.

Progressive Disclosure

Start simple with NodeLLM.chat().ask("Hello"). As your needs grow, you can access advanced features like raw API responses, custom headers, and token usage tracking without breaking your initial code.

---

Configuration Patterns

NodeLLM supports two primary styles of configuration to match your preferred architectural pattern.

1. Fluent Builder API

Ideal for step-by-step configuration and readable “action” chains.

const chat = NodeLLM.chat("claude-3-7-sonnet")
  .withInstructions("You are a logic expert")
  .withTemperature(0.2)
  .withThinking({ budget: 16000 });

await chat.ask("Solve this puzzle");

2. Direct Configuration Object (Stateless)

Ideal for integrations that pass configuration dynamically or from a centralized settings object.

Enhanced in v1.7.0

// All options can be passed together at initialization
const chat = NodeLLM.chat("gpt-4o", {
  instructions: "You are a helpful assistant",
  temperature: 0.7,
  maxTokens: 500,
  thinking: { effort: "high" },
  headers: { "X-Tenant-ID": "123" }
});

// Or per-request for granular override
await chat.ask("Hello", {
  temperature: 0.1,
  maxToolCalls: 5
});

---

How It Works

1. Normalization: Your inputs (text, images, files) are converted into a standardized format.
2. Configuration: The library uses the provider and model you specify (e.g., GPT-4o with OpenAI).
3. Execution: The request is sent. If tools are called, the library executes them and feeds the result back to the model.
4. Response: The final response is normalized into a consistent object.