Speculative Fan-out Architecture Pattern

What It Solves & Overview

Speculative Fan-out is the signature architecture pattern for System One models. Instead of chaining sequential API roundtrips, the application sends all potentially relevant questions at once. Host code then inspects the confidence scores to act only on high-certainty branches.

Suitability & Fit

When to Use

Multi-domain command interpretation (e.g. smart home, IoT control, multi-function tool dispatch).
Minimizing end-to-end user latency by replacing sequential LLM loops with a single fan-out call.
Complex policy evaluation where multiple independent rules must be assessed simultaneously.

When NOT to Use

Scenarios where subsequent questions strictly depend on the output value of a generative text step.

Installation & Setup

Built using @typesafe-ai/sdk or typesafe-sdk batch questions interface.

Offline Contract Verification Example

Verification Notice: The code snippet below demonstrates local structural validation of inputs and choice schemas without requiring live network credentials. Real-world API execution requires active connectivity and API credentials.

// Offline fan-out filtering demonstration
const speculativeResults = {
  is_lights: { score: 0.98, branch: 'lighting' },
  is_music: { score: 0.05, branch: 'audio' },
  is_security: { score: 0.02, branch: 'alarm' }
};
const activeBranches = Object.entries(speculativeResults).filter(([_, v]) => v.score > 0.8).map(([_, v]) => v.branch);
console.log('Active branches filtered offline:', activeBranches);

Prerequisites & Operational Boundaries

Prerequisites

Designing state representation that provides sufficient context for all speculative questions.

Known Limitations

Over-speculation beyond sensible bounds can increase API evaluation payload.

Official Source Provenance

This guide is compiled and fact-checked against official upstream assets:

TypeSafe Official Patterns: Speculative Fan-out ↗