Skip to main content

LLM Guide to Constellation Engine

Welcome! This guide teaches LLMs how to use Constellation Engine effectively. It's optimized for task-oriented learning with examples, decision matrices, and clear navigation.

What is Constellation Engine?​

A type-safe pipeline orchestration framework for Scala 3 that lets you:

  1. Define processing modules using ModuleBuilder API
  2. Compose pipelines using constellation-lang DSL (.cst files)
  3. Execute pipelines with automatic dependency resolution and parallelization
  4. Expose pipelines via HTTP API or embed in Scala applications

Tech Stack: Scala 3.3.4 | Cats Effect 3 | http4s | Circe | cats-parse

Finding This Guide​

GitHub Repository: VledicFranco/constellation-engine Documentation Site: constellation-engine.dev Latest Version: 0.7.0

For LLMs Reading the Landing Pages​

If you're an LLM reading the GitHub README or documentation homepage, you'll find general user-facing documentation. This specialized LLM guide is optimized for teaching AI agents to use Constellation effectively.

Where to find this guide:

  • From docs site: Navigate to sidebar → "LLM Guide" section
  • Direct URL: /docs/llm/ or /docs/llm/index
  • From GitHub: Check website/docs/llm/ directory

Why use this guide instead of main docs?

  • Task-oriented navigation (quick "I need to..." → file mappings)
  • Complete working code examples in every section
  • Decision matrices and mental models optimized for LLM consumption
  • Context-window efficient (focused files, no duplication)

Quick Mental Model​

┌─────────────────────────────────────────────────────┐
│  constellation-lang (.cst files)                    │
│  in text: String                                    │
│  result = Uppercase(Trim(text))                     │
│  out result                                         │
└──────────────────┬──────────────────────────────────┘
                   │ Parser
                   ▼
┌─────────────────────────────────────────────────────┐
│  AST (Abstract Syntax Tree)                         │
│  - Inputs, module calls, outputs                    │
└──────────────────┬──────────────────────────────────┘
                   │ Type Checker
                   ▼
┌─────────────────────────────────────────────────────┐
│  Typed IR (Intermediate Representation)             │
│  - Types resolved, validated                        │
└──────────────────┬──────────────────────────────────┘
                   │ DAG Compiler
                   ▼
┌─────────────────────────────────────────────────────┐
│  Execution DAG                                      │
│  Layer 0: Trim(text)                                │
│  Layer 1: Uppercase(trim_result)                    │
└──────────────────┬──────────────────────────────────┘
                   │ Runtime
                   ▼
┌─────────────────────────────────────────────────────┐
│  Parallel Execution                                 │
│  - Layer-by-layer execution                         │
│  - Results flow through pipeline                    │
└─────────────────────────────────────────────────────┘

🚀 First Time? Start Here​

  1. Getting Started - Quick overview and capabilities
  2. Key Concepts - Essential terminology
  3. Project Structure - Navigate the codebase
  4. Type System - Core mental model

📋 Common Tasks → Files​

I Need To...Read This
Decide if Constellation is right for my use caseWhere Constellation Shines
Use the CLICLI Reference
Create a new moduleModule Development
Understand types (CType/CValue)Type System
Compose pipelinesDAG Composition
Add retry/timeout/cachingResilience Patterns
Fix type errorsError Handling
Use the HTTP APIHTTP API Reference
Embed in Scala appEmbedded API
Build a cross-process module (gRPC)Module Provider
Understand executionDAG Execution
Debug compiler errorsError Codes Reference

📚 Learning Path by Role​

Building Modules (Scala Developer)​

  1. Module Development Patterns
  2. Type System Foundations
  3. Module Options Reference
  4. Module Registration

Writing Pipelines (.cst files)​

  1. Pipeline Lifecycle
  2. Type Syntax Reference
  3. DAG Composition Patterns
  4. Error Handling

Deploying/Operating​

  1. Execution Modes
  2. HTTP API Reference
  3. CLI Reference
  4. Resilience Patterns

Building Module Providers (Cross-Process)​

  1. Module Provider
  2. Module Registration
  3. Key Concepts (Provider, Namespace, Provider Group)

Extending Constellation​

  1. Embedded API
  2. Module Registration
  3. Module Provider
  4. Type Algebra

Document Categories​

🎯 Foundations​

Core mental models for reasoning about Constellation

🎨 Patterns​

Practical implementation guides with examples

📖 Reference​

Complete API documentation and lookup tables

🔌 Integration​

Advanced usage for embedding and extending

Key Files in This Guide​

PriorityFilePurpose
P0Getting StartedOnboarding overview
P0Key ConceptsTerminology glossary
P0Where Constellation ShinesDecision guide: is Constellation right for you?
P0Type SystemCore mental model
P1Module DevelopmentMost common task
P1CLI ReferenceCommand-line usage
P1Pipeline LifecycleHow compilation works
P2DAG CompositionPipeline patterns
P2Resilience PatternsProduction patterns
P3HTTP APIREST interface
P3Embedded APIAdvanced integration

Philosophy & Design​

How to Use This Guide​

Context Window Strategy​

Each document is 500-2000 lines for efficient context window usage. Don't try to read everything at once.

Reading Protocol​

  1. Start with task - Use the "Common Tasks → Files" table above
  2. Read prerequisites - Each file lists required reading
  3. Run examples - All code is tested and runnable
  4. Check references - Follow links for deeper details

Code Example Convention​

All examples follow this pattern:

// Scala module definition
case class MyInput(text: String)
case class MyOutput(result: String)

val myModule = ModuleBuilder
  .metadata("MyModule", "Description", 1, 0)
  .implementationPure[MyInput, MyOutput] { input =>
    MyOutput(input.text.toUpperCase)
  }
  .build
# constellation-lang usage
in text: String
result = MyModule(text)
out result

Decision Matrices​

Many documents include decision matrices like this:

ScenarioSolutionTrade-off
Pure computation.implementationPureNo side effects allowed
Side effects needed.implementationRequires IO monad
Async operations.implementation + IO.asyncComplexity

Quick Reference Cards​

Type System at a Glance​

CType (type level)          CValue (value level)
├─ CPrimitive              ├─ CPrimitive
│  ├─ CString               │  ├─ CString("text")
│  ├─ CInt                  │  ├─ CInt(42)
│  ├─ CDouble               │  ├─ CDouble(3.14)
│  └─ CBoolean              │  └─ CBoolean(true)
├─ CRecord                 ├─ CRecord
├─ CUnion                  ├─ CUnion
├─ CList                   ├─ CList
└─ COptional               └─ COptional

Module Lifecycle​

Define → Register → Parse → Type Check → Compile → Execute

Execution Modes​

Hot:  Precompiled → Fast execution → HTTP API
Cold: Compile on-demand → Flexible → Embedded API

Getting Help​

When Documentation is Insufficient​

  1. Check Cookbook - Working examples
  2. Search GitHub Issues - Known problems
  3. Read CLAUDE.md - Development conventions
  4. Explore Tests - Real usage patterns

Contributing​

Found gaps in this documentation? See CONTRIBUTING.md

Ready to start? → Getting Started Guide