Skip to main content
Palyra employs a multi-layered verification strategy designed to ensure the reliability of its autonomous agent core while maintaining a high security posture for its distributed execution environment. This infrastructure spans local developer gates, extensive CI pipelines, and automated release packaging across Windows, macOS, and Linux.

​
Testing Strategy and Suites

The testing infrastructure is built around the concept of Deterministic Core verification. Because LLM-based systems are inherently non-deterministic, Palyra utilizes a combination of “fake” provider adapters and golden fixture snapshots to ensure that the internal state machine, tool dispatch logic, and journal serialization remain consistent across changes.

​
Core Verification Layers

  • Deterministic Core Suite: Validates the palyra-daemon and palyra-cli integration, including session bootstrapping, gRPC service persistence, and tool registry dispatch scripts/test/run-deterministic-core.sh#72-92.
  • Workflow Regression Matrix: A comprehensive test suite that runs agent scenarios against a contract-based harness to prevent regressions in complex tool-use flows scripts/test/run-workflow-regression.sh#1-10.
  • Fuzz Testing: Targeted fuzzing of high-risk parsers (A2UI JSON, Webhook payloads, Auth profiles) using cargo-fuzz justfile#172-185.
  • Performance and Soak Tests: Validates system stability under load and over long durations justfile#88-92.
For details, see Test Infrastructure and Test Suites.

​
Test Suite Execution Mapping

The following diagram maps high-level test suites to the specific scripts and code entities they exercise. Test Mapping: Logic to Code Sources: justfile#47-108, scripts/test/run-deterministic-core.sh#1-100, .github/workflows/ci.yml#103-140

​
CI/CD Pipelines and Security Gates

Palyra uses GitHub Actions to enforce quality and security standards on every pull request and push to the main branch. The pipeline is split into functional “Gates” that must be passed before code is considered merge-ready.

​
Pipeline Structure

  • CI Pipeline (ci.yml): Handles multi-OS builds (Ubuntu, macOS, Windows), workspace-wide testing, and UI build verification .github/workflows/ci.yml#16-34.
  • Security Gates (security.yml): Executes the security SDLC, including cargo-audit for vulnerabilities, cargo-deny for license/advisory checks, and gitleaks for secret detection .github/workflows/security.yml#95-124.
  • CodeQL (codeql.yml): Performs static analysis for Rust, JavaScript/TypeScript, and GitHub Actions .github/workflows/codeql.yml#19-48.
  • Pre-push Hooks: A local pre-push hook runs a “fast” version of the CI checks to provide immediate feedback to developers .githooks/pre-push#1-4.
For details, see CI/CD Pipelines and Security Gates.

​
Security Gate Workflow

The security pipeline generates a comprehensive set of artifacts, including a Software Bill of Materials (SBOM). Security Pipeline Diagram Sources: .github/workflows/security.yml#95-148, justfile#148-154

​
Release Process and Packaging

The release infrastructure automates the creation of portable bundles and platform-specific installers. It ensures that the daemon (palyrad), CLI (palyra), browser bridge (palyra-browserd), and the web dashboard are correctly colocated and functional.

​
Release Lifecycle

  1. Metadata Preparation: Derives versioning and generates release/migration notes based on repository state .github/workflows/release.yml#30-119.
  2. Portable Bundling: Compiles binaries and packages them with the UI assets into .zip and .tar.gz archives for all supported OSs .github/workflows/release.yml#120-173.
  3. Release Smoke Test: Executes a full lifecycle test (install -> setup -> gateway start -> onboarding) on the final packaged artifacts to ensure no missing dependencies .github/workflows/cli-full-regression.yml#54-107.
  4. Attestation: Generates build attestations and provenance sidecars for the released assets .github/workflows/release.yml#22-23.
For details, see Release Process and Packaging.

​
Release Task Summary

The project uses just (or make) as a task runner to provide a consistent interface for release-related operations.
TaskCommandDescription
Release Smokejust release-smokeRuns full packaging and lifecycle verification justfile#76-77.
Version Auditassert-version-coherence.ps1Ensures versions match across all manifests .github/workflows/release.yml#45.
SBOM Generationjust sbomGenerates CycloneDX SBOM for the workspace justfile#164-165.
Doctor Checkjust doctorValidates local environment readiness for builds justfile#7-11.
Sources: justfile#1-189, Makefile#1-179, .github/workflows/release.yml#1-173

​
Child Pages