Contributing to Constellation Engine

Thank you for your interest in contributing to Constellation Engine! This guide will help you get started with development.

First-Time Contributors

New to the codebase? Start with issues labeled good first issue on GitHub. These are self-contained tasks that introduce you to the project structure without requiring deep domain knowledge.

Prerequisites
Quick Start
Development Workflow
Project Structure
Running Tests
VSCode Setup
Common Tasks
Troubleshooting

Prerequisites

Before you begin, ensure you have:

Java 17+ (JDK 17 or later)
SBT 1.10+ (Scala Build Tool)
Node.js 18+ and npm
VSCode (recommended for extension development)

Verify Installation

java -version    # Should show Java 17+
sbt --version    # Should show SBT 1.10+
node --version   # Should show Node 18+
npm --version

Quick Start

1. Clone and Install Dependencies

git clone https://github.com/VledicFranco/constellation-engine.git
cd constellation-engine

# Install all dependencies
make install
# Or manually:
sbt update
cd vscode-extension && npm install && cd ..

2. Start Development Environment

Option A: Using Make (recommended)

make dev    # Starts server + TypeScript watch

Option B: Using Scripts

# Windows PowerShell
.\scripts\dev.ps1

# Unix/Mac
./scripts/dev.sh

Option C: Manual

# Terminal 1: Start server
sbt "exampleApp/runMain io.constellation.examples.app.server.ExampleServer"

# Terminal 2: Watch TypeScript
cd vscode-extension && npm run watch

3. Launch VSCode Extension

Open VSCode in the project root
Press F5 to launch the extension in a new window
Open a .cst file and start coding!

Development Workflow

Make Commands

Command	Description
`make dev`	Start full dev environment (server + watch)
`make server`	Start HTTP/LSP server only
`make watch`	Watch Scala and recompile on changes
`make ext-watch`	Watch TypeScript and recompile
`make test`	Run all tests
`make compile`	Compile all modules
`make clean`	Clean build artifacts
`make assembly`	Build fat JAR for deployment
`make docker-build`	Build Docker image
`make docker-run`	Run in Docker container

Hot Reload (Server Auto-Restart)

For automatic server restart on code changes:

make server-rerun
# Or: sbt "~exampleApp/reStart"

This requires the sbt-revolver plugin (already configured).

VSCode Tasks

Press Ctrl+Shift+P → "Tasks: Run Task" to see available tasks:

Compile All - Compile Scala modules
Run Tests - Run all tests
Start Server (ExampleLib) - Start the HTTP/LSP server
Watch Compile - Watch mode for Scala
Full Dev Setup - Start server + extension watch

Project Structure

constellation-engine/
├── modules/
│   ├── core/           # Type system (CType, CValue)
│   ├── runtime/        # Execution engine, ModuleBuilder
│   ├── lang-ast/       # AST definitions
│   ├── lang-parser/    # Parser (cats-parse)
│   ├── lang-compiler/  # Type checker, DAG compiler
│   ├── lang-stdlib/    # Standard library functions
│   ├── lang-lsp/       # Language Server Protocol
│   ├── http-api/       # HTTP server (http4s)
│   └── example-app/    # Example application
├── vscode-extension/   # VSCode extension (TypeScript)
├── docs/               # Documentation
├── scripts/            # Development scripts
├── Makefile            # Build automation
└── build.sbt           # SBT configuration

Module Dependencies

core → runtime → lang-compiler → http-api
       ↓                ↓
    lang-ast      lang-stdlib, lang-lsp, example-app
       ↓
   lang-parser

Rule: Modules can only depend on modules above them. No circular dependencies.

Running Tests

All Tests

Test Before Committing

Always run make test before pushing. CI runs the full test suite, and PRs with failing tests cannot be merged.

make test
# Or: sbt test

Module-Specific Tests

make test-core       # Test core module
make test-compiler   # Test compiler
make test-lsp        # Test LSP server
make test-parser     # Test parser

Watch Mode (Continuous Testing)

sbt "~test"          # Re-run tests on changes
sbt "~testQuick"     # Only re-run failed tests

VSCode Setup

Recommended Extensions

Scala (Metals) - Scala language support
Scala Syntax - Syntax highlighting

Settings

The project includes .vscode/tasks.json with pre-configured tasks.

Debugging the Extension

Open the vscode-extension/ folder in VSCode
Press F5 to launch
A new VSCode window opens with the extension loaded
Set breakpoints in TypeScript files

Debugging the Scala Backend

Start the server with debug port:
```
sbt -jvm-debug 5005 "exampleApp/run"
```
Attach a debugger to port 5005

Common Tasks

Adding a New Module Function

Define the module in modules/example-app/src/main/scala/.../modules/:

case class MyInput(value: String)
case class MyOutput(result: String)

val myModule: Module.Uninitialized = ModuleBuilder
  .metadata("MyModule", "Description", 1, 0)
  .implementationPure[MyInput, MyOutput] { input =>
    MyOutput(input.value.toUpperCase)
  }
  .build

Add signature to ExampleLib in ExampleLib.scala:

private val myModuleSig = FunctionSignature(
  name = "MyModule",
  params = List("value" -> SemanticType.SString),
  returns = SemanticType.SString,
  moduleName = "MyModule"
)

Register in allSignatures and add module to allModules
Restart server to pick up changes

Testing a Constellation-Lang Pipeline

Create a .cst file:

in text: String
result = MyModule(text)
out result

Run with Ctrl+Shift+R in VSCode or via HTTP:

curl -X POST http://localhost:8080/compile \
  -H "Content-Type: application/json" \
  -d '{"source": "in x: String\nresult = Uppercase(x)\nout result", "dagName": "test"}'

Troubleshooting

Server won't start

Check if port 8080 is in use:
```
netstat -an | grep 8080
```

Kill existing Java processes:

# Windows
taskkill /F /IM java.exe

# Unix
pkill -f "java.*constellation"

SBT compilation errors

Clean and recompile:
```
make clean
make compile
```
Update dependencies:
```
sbt update
```

Extension not connecting to LSP

Ensure server is running on port 8080
Check VSCode output panel for errors
Verify WebSocket URL in settings: ws://localhost:8080/lsp

TypeScript compilation errors

cd vscode-extension
rm -rf node_modules out
npm install
npm run compile

Code Style

Formatting Requirements

All Scala code must pass scalafmt checks before merging. Run sbt scalafmtCheck locally to verify formatting. PRs with formatting violations will fail CI.

Scala: Follow standard Scala 3 conventions
TypeScript: Follow existing patterns in the codebase
Commits: Use conventional commit messages
Tests: Add tests for new functionality

Error Handling Patterns

The codebase uses standardized error handling patterns for consistency and maintainability.

HTTP Routes (EitherT Pattern)

For HTTP endpoints that execute multiple operations, use EitherT to chain operations cleanly:

import cats.data.EitherT
import io.constellation.errors.{ApiError, ErrorHandling}

// Define a chain of operations using EitherT
private def executeByRef(req: ExecuteRequest): EitherT[IO, ApiError, DataSignature] =
  for {
    image   <- EitherT(constellation.pipelineStore.getByName(req.ref).map {
      case Some(img) => Right(img)
      case None      => Left(ApiError.NotFoundError("Pipeline", req.ref))
    })
    loaded  = LoadedPipeline(image, Map.empty)
    inputs  <- ErrorHandling.liftIO(convertInputs(req.inputs))(t => ApiError.InputError(t.getMessage))
    sig     <- ErrorHandling.liftIO(constellation.run(loaded, inputs))(t => ApiError.ExecutionError(t.getMessage))
  } yield sig

// In the route handler, pattern match on the result
case req @ POST -> Root / "execute" =>
  (for {
    execReq <- req.as[ExecuteRequest]
    result  <- executeStoredDag(execReq).value
    response <- result match {
      case Right(outputs) => Ok(ExecuteResponse(success = true, outputs = outputs))
      case Left(ApiError.NotFoundError(_, name)) => NotFound(...)
      case Left(ApiError.InputError(msg)) => BadRequest(...)
      case Left(error) => InternalServerError(...)
    }
  } yield response).handleErrorWith { error =>
    InternalServerError(...)  // Catch-all for unexpected errors
  }

Key principles:

Use ApiError sealed trait for typed errors
Use ErrorHandling.liftIO to wrap IO operations with error mapping
Keep outer handleErrorWith as catch-all for unexpected errors
Pattern match on specific error types to return appropriate HTTP status codes

LSP Notification Handlers (Logging Pattern)

For LSP notification handlers (didOpen, didChange, didClose), log errors instead of silently swallowing:

private def handleDidOpen(notification: Notification): IO[Unit] = {
  for {
    params <- IO.fromEither(...)
    _ <- documentManager.openDocument(...)
    _ <- validateDocument(...)
  } yield ()
}.handleErrorWith(e => logger.warn(s"Error in didOpen: ${e.getMessage}"))

Key principles:

Never silently swallow errors with handleErrorWith(_ => IO.unit)
Log errors with context (operation name, error message)
Notification handlers should complete successfully even on error (LSP protocol requirement)
Use logger.warn for recoverable errors

API Error Types

The io.constellation.errors.ApiError sealed trait provides typed errors:

sealed trait ApiError { def message: String }

object ApiError {
  case class InputError(message: String) extends ApiError       // JSON -> CValue conversion errors
  case class ExecutionError(message: String) extends ApiError   // DAG execution errors
  case class OutputError(message: String) extends ApiError      // CValue -> JSON conversion errors
  case class NotFoundError(resource: String, name: String) extends ApiError
  case class CompilationError(errors: List[String]) extends ApiError
}

ErrorHandling Utilities

The io.constellation.errors.ErrorHandling object provides helpers:

// Wrap an IO operation, mapping exceptions to typed errors
ErrorHandling.liftIO(riskyOperation)(t => ApiError.InputError(t.getMessage))

// Convert Either to EitherT
ErrorHandling.fromEither(either)

// Convert Option to EitherT with error for None
ErrorHandling.fromOption(opt, ApiError.NotFoundError("DAG", name))

Getting Help

Check existing issues on GitHub
Review documentation in /docs
See docs/architecture.md for comprehensive architecture guide

Happy coding! 🚀

Table of Contents​

Prerequisites​

Verify Installation​

Quick Start​

1. Clone and Install Dependencies​

2. Start Development Environment​

3. Launch VSCode Extension​

Development Workflow​

Make Commands​

Hot Reload (Server Auto-Restart)​

VSCode Tasks​

Project Structure​

Module Dependencies​

Running Tests​

All Tests​

Module-Specific Tests​

Watch Mode (Continuous Testing)​

VSCode Setup​

Recommended Extensions​

Settings​

Debugging the Extension​

Debugging the Scala Backend​

Common Tasks​

Adding a New Module Function​

Testing a Constellation-Lang Pipeline​

Troubleshooting​

Server won't start​

SBT compilation errors​

Extension not connecting to LSP​

TypeScript compilation errors​

Code Style​

Error Handling Patterns​

HTTP Routes (EitherT Pattern)​

LSP Notification Handlers (Logging Pattern)​

API Error Types​

ErrorHandling Utilities​

Getting Help​

Table of Contents