Models & Registry v1.0.0+

Programmatically discover available models, their capabilities, and real-time costs using our built-in registry powered by models.dev.

Table of contents

  1. Inspecting a Model
  2. Discovery by Capability v0.8.0+
    1. Finding Vision Models
    2. Finding Tool-Use Models
    3. Finding Audio Models
  3. Supported Providers
  4. Usage & Cost Tracking
    1. Manual Cost Calculation
  5. Custom Models & Endpoints
    1. Using assumeModelExists
    2. Custom Endpoints (e.g. Azure/Local)

`NodeLLM` includes a comprehensive, built-in registry of models using data from models.dev. This allows you to discover models and their capabilities programmatically.

Inspecting a Model

You can look up any supported model to check its context window, costs, and features.

import { createLLM } from "@node-llm/core";

const model = NodeLLM.models.find("gpt-5");

if (model) {
  console.log(`Provider: ${model.provider}`);
  console.log(`Context Window: ${model.context_window} tokens`);
  console.log(`Input Price: $${model.pricing.text_tokens.standard.input_per_million}/1M`);
  console.log(`Output Price: $${model.pricing.text_tokens.standard.output_per_million}/1M`);
}

Discovery by Capability v0.8.0+

You can filter the registry to find models that match your requirements.

Finding Vision Models

const visionModels = NodeLLM.models.all().filter((m) => m.capabilities.includes("vision"));

console.log(`Found ${visionModels.length} vision-capable models.`);
visionModels.forEach((m) => console.log(m.id));

Finding Tool-Use Models

const toolModels = NodeLLM.models.all().filter((m) => m.capabilities.includes("tools"));

Finding Audio Models

const audioModels = NodeLLM.models.all().filter((m) => m.capabilities.includes("audio_input"));

Supported Providers

The registry includes models from:

  • OpenAI (GPT-4o, GPT-3.5, DALL-E)
  • Anthropic (Claude 3.5 Sonnet, Haiku, Opus)
  • Google Gemini (Gemini 1.5 Pro, Flash)
  • DeepSeek (DeepSeek V3, R1)
  • AWS Bedrock (Nova, Titan, Claude) v1.8.0+
  • OpenRouter (400+ models)
  • xAI (Grok)
  • Ollama (Local models)
  • Mistral (Mistral Large, Codestral, Pixtral, Magistral) v1.14.0+

Usage & Cost Tracking

Every ChatResponseString returned from chat.ask() carries a .usage object plus convenience getters, so you can log spend without querying the registry yourself.

const response = await chat.ask("Summarize this document");

console.log(response.input_tokens); // Prompt tokens
console.log(response.output_tokens); // Completion tokens
console.log(response.cached_tokens); // Tokens served from a provider cache <span style="background-color: #0d9488; color: white; padding: 1px 6px; border-radius: 3px; font-size: 0.6em; font-weight: 600; vertical-align: middle;">v1.5.2+</span>
console.log(response.usage.cache_creation_tokens); // Tokens written to a new prompt cache (Anthropic)

console.log(response.cost); // Total cost in USD
console.log(response.input_cost); // Cost attributable to input tokens
console.log(response.output_cost); // Cost attributable to output tokens

Manual Cost Calculation

You can also calculate cost for a raw usage object (e.g., persisted usage loaded from your database) using ModelRegistry.calculateCost():

import { ModelRegistry } from "@node-llm/core";

const priced = ModelRegistry.calculateCost(
  { input_tokens: 1000, output_tokens: 500, total_tokens: 1500, cached_tokens: 200 },
  "gpt-5",
  "openai"
);

console.log(priced.cost); // Computed total cost in USD

Custom Models & Endpoints

Sometimes you need to use models not in the registry, such as Azure OpenAI deployments, Local Models (Ollama/LM Studio), or brand new releases.

Using assumeModelExists

This flag tells `NodeLLM` to bypass the registry check.

Important: You MUST specify the provider when using this flag, as the system cannot infer it from the ID.

const chat = NodeLLM.withProvider("openai").chat("my-custom-deployment", {
  assumeModelExists: true
});

// Note: Capability checks are bypassed (assumed true) for custom models.
await chat.ask("Hello");

Custom Endpoints (e.g. Azure/Local)

To point to a custom URL (like an Azure endpoint or local proxy), configure the base URL globally.

const llm = createLLM({
  openaiApiBase: "https://my-azure-resource.openai.azure.com",
  openaiApiKey: process.env.AZURE_API_KEY
});

// Now valid for all OpenAI requests
const chat = llm.chat("gpt-4", { provider: "openai" });