Parallel Execution

Learn how to safely run multiple LLM providers concurrently using NodeLLM’s scoped context system to avoid global state race conditions.

Table of contents

1. The Problem
2. The Solution
3. How To Use It
   1. Simple Parallel Calls
4. Benefits
5. Example

---

The Problem

In previous versions, NodeLLM was a mutable singleton. Calling NodeLLM.configure() concurrently could lead to race conditions where one request would overwrite the configuration of another.

---

The Solution

As of v1.6.0, NodeLLM is a frozen, immutable instance. It cannot be mutated at runtime. For parallel execution with different providers or configurations, you use context branching via .withProvider() or create independent instances via createLLM().

---

How To Use It

Simple Parallel Calls

The most elegant way to run multiple providers is using .withProvider(). This creates a scoped, isolated instance for that specific call.

import { NodeLLM } from "@node-llm/core";

const [score1, score2, score3] = await Promise.all([
  NodeLLM.withProvider("openai").chat("gpt-4o").ask(prompt),
  NodeLLM.withProvider("anthropic").chat("claude-3-5-sonnet").ask(prompt),
  NodeLLM.withProvider("gemini").chat("gemini-2.0-flash").ask(prompt)
]);

---

Benefits

✅ Singleton Maintained: No need to use new NodeLLM() unless you want to.
✅ Race Condition Solved: Each .withProvider() call creates an isolated context.
✅ Clean Syntax: Chaining .withProvider().chat().ask() is intuitive and elegant.
✅ Automatic Key Sharing: Scoped instances inherit the global API keys by default.

---

Example

Check out the Parallel Scoring Example for a working demonstration.