Configuration

Learn how to configure NodeLLM with API keys, custom base URLs, security limits, and per-request overrides.

Table of contents

  1. 1. Zero-Config (The “Direct” Pattern)
  2. 2. Explicit Factory (createLLM)
    1. Switching Providers
    2. Provider Configuration
      1. API Keys
      2. Custom Base URLs
      3. AWS Bedrock
      4. Loop Protection & Security Limits
  3. Supported Configuration Keys
  4. Inspecting Configuration
  5. Error Handling
    1. Snapshotting & Instance Initialization
  6. Best Practices
    1. Use Dotenv for Local Development
    2. Configure Once at Startup
    3. Scoped Configuration (Isolation)
      1. Isolated Provider State
      2. Scoped Credentials

NodeLLM provides three ways to configure providers: Zero-Config (via environment variables), Explicit Factory (via createLLM), and Isolated Branching (via .withProvider).

1. Zero-Config (The “Direct” Pattern)

The simplest way to use NodeLLM is by relying on environment variables. NodeLLM will automatically snapshot your environment at load time.

Environment variables (.env):

NODELLM_PROVIDER=openai
OPENAI_API_KEY=sk-....

Code:

import "dotenv/config";
import { NodeLLM } from "@node-llm/core";

// Zero setup required
const chat = NodeLLM.chat();

2. Explicit Factory (createLLM)

Recommended for production applications where you want to explicitly define provider behavior or manage multiple providers in one application.

Switching Providers

Since NodeLLM is immutable, you switch providers by creating a new instance using createLLM() or withProvider().

// Create an Anthropic instance
const llm = createLLM({
  provider: "anthropic",
  anthropicApiKey: process.env.ANTHROPIC_API_KEY
});

Provider Configuration

API Keys

Configure API keys in the configuration object.

const llm = createLLM({
  openaiApiKey: process.env.OPENAI_API_KEY,
  anthropicApiKey: process.env.ANTHROPIC_API_KEY,
  geminiApiKey: process.env.GEMINI_API_KEY,
  deepseekApiKey: process.env.DEEPSEEK_API_KEY,
  openrouterApiKey: process.env.OPENROUTER_API_KEY,
  xaiApiKey: process.env.XAI_API_KEY,
  mistralApiKey: process.env.MISTRAL_API_KEY
});

Custom Base URLs

Override the default API endpoints for custom deployments (e.g., Azure OpenAI):

const llm = createLLM({
  provider: "openai",
  openaiApiKey: process.env.AZURE_OPENAI_API_KEY,
  openaiApiBase: process.env.AZURE_OPENAI_API_BASE_ENDPOINT
});

AWS Bedrock

Bedrock uses AWS-native credentials rather than a single API key. NodeLLM supports two authentication styles:

// Option A: Long-term Bearer token (simplest)
const llm = createLLM({
  provider: "bedrock",
  bedrockApiKey: process.env.AWS_BEARER_TOKEN_BEDROCK,
  bedrockRegion: "us-east-1"
});

// Option B: SigV4 credentials (Access Key / Secret / optional Session Token)
const llm = createLLM({
  provider: "bedrock",
  bedrockAccessKeyId: process.env.AWS_ACCESS_KEY_ID,
  bedrockSecretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
  bedrockSessionToken: process.env.AWS_SESSION_TOKEN, // optional
  bedrockRegion: process.env.AWS_REGION,
  // Optional: attach an AWS Bedrock Guardrail
  bedrockGuardrailIdentifier: process.env.AWS_GUARDRAIL_ID,
  bedrockGuardrailVersion: process.env.AWS_GUARDRAIL_VERSION
});

Like the other providers, Bedrock also supports a custom endpoint override via bedrockApiBase (or the BEDROCK_API_BASE environment variable) — useful for routing requests through a proxy or private gateway instead of the standard AWS Bedrock endpoint for bedrockRegion.

Loop Protection & Security Limits

Prevent runaway costs, infinite loops, and hanging requests by setting execution and timeout limits:

const llm = createLLM({
  maxToolCalls: 5, // Stop after 5 sequential tool execution turns
  maxCorrections: 5, // Stop after 5 middleware-driven self-correction retries
  maxRetries: 2, // Retry network/server errors 2 times
  requestTimeout: 30000, // Timeout requests after 30 seconds (default)
  maxTokens: 4096 // Limit output to 4K tokens (default)
});

Security Benefits:

  • maxToolCalls: Prevents infinite tool execution loops
  • maxCorrections: Caps middleware-driven RETRY loops (e.g. schema self-correction) so a misbehaving middleware can’t retry — and bill — indefinitely
  • maxRetries: Prevents retry storms that could exhaust resources. Retries use exponential backoff with jitter and honor a provider’s Retry-After header
  • requestTimeout: Prevents hanging requests and DoS attacks
  • maxTokens: Prevents excessive output generation and cost overruns

Supported Configuration Keys

Key Description Default
openaiApiKey OpenAI API key process.env.OPENAI_API_KEY
openaiApiBase OpenAI API base URL process.env.OPENAI_API_BASE
anthropicApiKey Anthropic API key process.env.ANTHROPIC_API_KEY
anthropicApiBase Anthropic API base URL process.env.ANTHROPIC_API_BASE
geminiApiKey Google Gemini API key process.env.GEMINI_API_KEY
geminiApiBase Gemini API base URL process.env.GEMINI_API_BASE
deepseekApiKey DeepSeek API key process.env.DEEPSEEK_API_KEY
deepseekApiBase DeepSeek API base URL process.env.DEEPSEEK_API_BASE
openrouterApiKey OpenRouter API key process.env.OPENROUTER_API_KEY
openrouterApiBase OpenRouter API base URL process.env.OPENROUTER_API_BASE
xaiApiKey xAI API key process.env.XAI_API_KEY
xaiApiBase xAI API base URL process.env.XAI_API_BASE
mistralApiKey Mistral API key process.env.MISTRAL_API_KEY
mistralApiBase Mistral API base URL process.env.MISTRAL_API_BASE
ollamaApiBase Ollama API base URL process.env.OLLAMA_API_BASE
bedrockApiKey Bedrock long-term Bearer token process.env.AWS_BEARER_TOKEN_BEDROCK
bedrockAccessKeyId Bedrock SigV4 access key ID process.env.AWS_ACCESS_KEY_ID
bedrockSecretAccessKey Bedrock SigV4 secret access key process.env.AWS_SECRET_ACCESS_KEY
bedrockSessionToken Bedrock SigV4 session token process.env.AWS_SESSION_TOKEN
bedrockRegion AWS region for Bedrock process.env.AWS_REGION (us-east-1)
bedrockApiBase Bedrock custom endpoint override process.env.BEDROCK_API_BASE
bedrockGuardrailIdentifier Bedrock Guardrail ID process.env.AWS_GUARDRAIL_ID
bedrockGuardrailVersion Bedrock Guardrail version process.env.AWS_GUARDRAIL_VERSION
provider Active provider name (Zero-Config) process.env.NODELLM_PROVIDER
debug Verbose request/response logging process.env.NODELLM_DEBUG === "true"
defaultChatModel Default model for .chat() Provider default
defaultTranscriptionModel Default model for .transcribe() Provider default
defaultModerationModel Default model for .moderate() Provider default
defaultEmbeddingModel Default model for .embed() Provider default
maxToolCalls Max sequential tool execution turns 5
maxCorrections Max middleware-driven self-correction retries 5
maxRetries Max retries for provider errors (exponential backoff + jitter, honors Retry-After) 2
requestTimeout Request timeout in milliseconds 30000 (30s)
maxTokens Max output tokens per request 4096
toolExecution Tool execution mode (auto/confirm/dry-run) auto
toolConcurrency Run independent tool calls in the same turn concurrently (see Tools guide) false
retry Retry configuration (legacy) { attempts: 1, delayMs: 0 }

Inspecting Configuration

You can inspect the current internal configuration at any time.

console.log(NodeLLM.config.openaiApiKey);

Error Handling

Attempting to use an unconfigured provider will raise a clear error:

// If API key is not set
const llm = createLLM({ provider: "openai" });
// Error: openaiApiKey is not set in config...

Snapshotting & Instance Initialization

When you create an LLM instance (including the default NodeLLM export), it snapshots all relevant environment variables.

In the global NodeLLM instance, this initialization is lazy. It only snapshots process.env the first time you access a property or method (like .chat()). This makes it safe to use with dotenv/config or similar libraries in ESM, even if they are imported after the core library.

// ✅ Safe in NodeLLM v1.6.0+: Initialized on first call
import { NodeLLM } from "@node-llm/core";
import "dotenv/config";

const chat = NodeLLM.chat(); // Snapshots environment NOW

Best Practices

Use Dotenv for Local Development

import "dotenv/config";
import { createLLM } from "@node-llm/core";

const llm = createLLM({ provider: "openai" });

Configure Once at Startup

// app.ts
const llm = createLLM({
  openaiApiKey: process.env.OPENAI_API_KEY,
  anthropicApiKey: process.env.ANTHROPIC_API_KEY
});

Scoped Configuration (Isolation)

NodeLLM is a frozen, immutable instance. It cannot be mutated at runtime. This design ensures that configurations do not leak between parallel requests, making it safe for multi-tenant applications.

Use createLLM() or .withProvider() to create an isolated context.

Isolated Provider State

Run multiple providers in parallel safely without any side effects:

const [gpt, claude] = await Promise.all([
  NodeLLM.withProvider("openai").chat("gpt-5").ask(prompt),
  NodeLLM.withProvider("anthropic").chat("claude-sonnet-5").ask(prompt)
]);

Scoped Credentials

You can also pass a second argument to withProvider to override configuration keys (like API keys) for that specific instance only. This is useful for multi-tenant applications.

const userA = NodeLLM.withProvider("openai", {
  openaiApiKey: "USER_A_KEY"
});

const userB = NodeLLM.withProvider("openai", {
  openaiApiKey: "USER_B_KEY"
});

// These calls use different API keys simultaneously
const [resA, resB] = await Promise.all([
  userA.chat().ask("Hello from A"),
  userB.chat().ask("Hello from B")
]);

This ensures each parallel call uses the correct provider and credentials without interfering with others.