OpenAI v1.0.0+

Full support for the complete range of NodeLLM features including tool calling, vision, image generation, and the advanced Developer role.

Configuration
Specific Parameters
Features
Strict Mode
1. Using Strict Mode with Schema
2. Using Strict Mode with Tools
Predicted Outputs
Custom Endpoints
Getting an API Key

The OpenAI provider supports the full range of NodeLLM features, including robust tool calling, vision, and structured outputs.

Configuration

import { createLLM } from "@node-llm/core";

const llm = createLLM({ 
  provider: "openai", 
  openaiApiKey: process.env.OPENAI_API_KEY // Optional if set in env 
});

Specific Parameters

You can pass OpenAI-specific parameters using .withParams().

const chat = llm.chat("gpt-4o").withParams({
  seed: 42, // for deterministic output
  user: "user-123", // for user tracking
  presence_penalty: 0.5,
  frequency_penalty: 0.5
});

Features

Models: gpt-4o, gpt-4o-mini, gpt-4-turbo, etc.
Vision: Specific models like gpt-4o support image analysis.
Tools: Fully supported, including parallel tool execution.
Reasoning: Automatic tracking of reasoning tokens and costs for o1 and o3 models.
Smart Developer Role: Modern instructions are automatically mapped to the developer role for compatible models when using the official API.
Strict Structured Output: Supports OpenAI’s “Strict Mode” for 100% schema adherence.
Predicted Outputs: Optimize latency for code-editing tasks by providing expected output previews.

Strict Mode

OpenAI’s “Strict Mode” ensures that the model’s output exactly matches your JSON schema with 100% reliability. When enabled, OpenAI generates a context-free grammar from your schema to constrain the output.

Using Strict Mode with Schema

By default, NodeLLM enables strict mode when you use .withSchema().

const chat = llm.chat("gpt-4o-mini").withSchema(myZodSchema);
// Internally sets strict: true and enforces additionalProperties: false

Using Strict Mode with Tools

You can enable strict mode for individual tools by setting the strict property to true.

class MyTool extends Tool {
  name = "my_tool";
  strict = true; // Enables OpenAI Strict Mode for this tool
  schema = z.object({ ... });
}

Predicted Outputs

For tasks like code refactoring or document editing where much of the output is already known, you can use Predicted Outputs to significantly reduce latency.

const originalCode = "function foo() { return true; }";

const response = await chat
  .withPrediction(originalCode) // Provide the expected/existing content
  .ask("Refactor this to return false:\n\n" + originalCode);

When using .withPrediction(), OpenAI only generates the “diffs,” allowing it to stream results much faster. NodeLLM safely ignores this parameter for other providers, so your code remains portable.

Custom Endpoints

OpenAI’s client is also used for compatible services like Ollama, LocalAI, and Azure OpenAI. See Custom Endpoints for details.