Skip to main content
Palyra employs a multi-layered testing strategy designed to ensure reliability across diverse operating systems and execution environments. This infrastructure ranges from deterministic unit and integration tests to long-running soak tests, CLI smoke tests, and automated fuzzing of critical parsers.

​
Deterministic Core and Soak Testing

The core testing infrastructure focuses on reproducibility by using deterministic providers and fixed-seed state initialization. The run-deterministic-core scripts orchestrate these checks across the Rust workspace and the React web console.

​
Execution Flow

The runner ensures that the environment is prepared (JS workspaces and desktop UI built) before executing a suite of targeted tests that verify:
  • State Initialization: Seeds and onboarding defaults in the Tauri desktop app [scripts/test/run-deterministic-core.ps1#17-17](http://scripts/test/run-deterministic-core.ps1#17-17).
  • Authentication Flows: OpenAI API key and OAuth bootstrap cycles [scripts/test/run-deterministic-core.ps1#18-20](http://scripts/test/run-deterministic-core.ps1#18-20).
  • Gateway Logic: gRPC route messaging, attachment metadata preservation, and tool approval persistence [scripts/test/run-deterministic-core.ps1#26-28](http://scripts/test/run-deterministic-core.ps1#26-28).
  • Web Console: Critical React component tests including consoleApi.test.ts [scripts/test/run-deterministic-core.ps1#30-35](http://scripts/test/run-deterministic-core.ps1#30-35).

​
Deterministic Fixture Management

To prevent regression in complex message routing, Palyra uses JSON fixtures. The check-deterministic-fixtures script validates that the current execution matches the stored channel_simulator_expected.json [scripts/test/check-deterministic-fixtures.ps1#6-6](http://scripts/test/check-deterministic-fixtures.ps1#6-6). If the logic changes, developers must explicitly run update-deterministic-fixtures.sh to acknowledge the change in output [scripts/test/check-deterministic-fixtures.ps1#14-14](http://scripts/test/check-deterministic-fixtures.ps1#14-14). Sources: [scripts/test/run-deterministic-core.ps1](), [scripts/test/run-deterministic-core.sh](), [scripts/test/check-deterministic-fixtures.ps1]().

​
Workflow Regression Matrix

The workflow regression suite ensures that the interaction between palyrad, palyra (CLI), and palyra-browserd remains stable across versions.

​
Regression Components

The run-workflow-regression script triggers a matrix of tests:
  1. Compatibility Tests: Checks the compat::tests module in the daemon [scripts/test/run-workflow-regression.ps1#12-12](http://scripts/test/run-workflow-regression.ps1#12-12).
  2. CLI Wizard: Validates the non-interactive and interactive setup flows [scripts/test/run-workflow-regression.ps1#13-13](http://scripts/test/run-workflow-regression.ps1#13-13).
  3. ACP Shim: Tests the Agent Control Protocol shim used for stdio-to-agent bridging [scripts/test/run-workflow-regression.ps1#14-14](http://scripts/test/run-workflow-regression.ps1#14-14).
  4. Workflow Matrix: A comprehensive test in palyra-cli that exercises the workflow_regression_matrix [scripts/test/run-workflow-regression.ps1#15-15](http://scripts/test/run-workflow-regression.ps1#15-15).
Sources: [scripts/test/run-workflow-regression.ps1](), [scripts/test/run-workflow-regression.sh]().

​
CLI Install Smoke Tests

The CLI smoke harness (run-cli-install-smoke.ps1) simulates a real-world installation and first-run experience. It operates on the actual compiled binary rather than via cargo test wrappers.

​
Smoke Test Architecture

The harness creates a New-ScenarioContext which isolates the test from the host system by redirecting all configuration and state paths:
  • Environment Isolation: Sets PALYRA_CONFIG, PALYRA_STATE_ROOT, and PALYRA_VAULT_DIR to temporary directories [scripts/test/run-cli-install-smoke.ps1#91-100](http://scripts/test/run-cli-install-smoke.ps1#91-100).
  • Transcript Logging: Every command execution is logged via Invoke-TranscriptCommand, capturing stdout, stderr, and exit codes into a structured report [scripts/test/run-cli-install-smoke.ps1#188-200](http://scripts/test/run-cli-install-smoke.ps1#188-200).

​
Verified Commands

The installed_smoke.rs test file verifies the binary’s health:
  • palyra version and palyra --help [crates/palyra-cli/tests/installed_smoke.rs#69-70](http://crates/palyra-cli/tests/installed_smoke.rs#69-70).
  • palyra doctor --json to ensure diagnostic integrity [crates/palyra-cli/tests/installed_smoke.rs#72-75](http://crates/palyra-cli/tests/installed_smoke.rs#72-75).
  • palyra setup --wizard in non-interactive mode to verify quickstart flows [crates/palyra-cli/tests/installed_smoke.rs#168-190](http://crates/palyra-cli/tests/installed_smoke.rs#168-190).
  • palyra update --dry-run and palyra uninstall --dry-run to test lifecycle management [crates/palyra-cli/tests/installed_smoke.rs#102-145](http://crates/palyra-cli/tests/installed_smoke.rs#102-145).

​
CLI Parity Matrix

An inventory of all CLI commands is generated using emit_cli_install_smoke_inventory.rs. This tool parses the CliParityMatrix to ensure that every command has a corresponding help snapshot and is covered by the smoke test [crates/palyra-cli/examples/emit_cli_install_smoke_inventory.rs#35-55](http://crates/palyra-cli/examples/emit_cli_install_smoke_inventory.rs#35-55). Sources: [scripts/test/run-cli-install-smoke.ps1](), [crates/palyra-cli/tests/installed_smoke.rs](), [crates/palyra-cli/examples/emit_cli_install_smoke_inventory.rs]().

​
Fuzzing Infrastructure

Palyra utilizes cargo-fuzz (based on libFuzzer) to identify edge cases in parsers and validators. The fuzzing harness is located in the /fuzz directory and targets components where untrusted input is processed.

​
Fuzz Targets

The system includes 9 fuzz targets, focusing on:
  • Workspace Patch Parser: Validates the robust handling of diff/patch formats used in tool execution [fuzz/fuzz_targets/workspace_patch_parser.rs]().
  • Process Runner Input: Fuzzes the process_runner_input logic in palyra-common to prevent injection or crashes during command serialization [fuzz/fuzz_targets/process_runner_input_parser.rs]().
  • Protocol Buffers: Validates the deserialization of gRPC messages.
  • Cedar Policies: Tests the policy evaluator against malformed context data.

​
Fuzzing Data Flow

The arbitrary crate is used to transform raw fuzzer bytes into structured Rust types for targeted testing. Fuzzer Input Mapping Sources: [fuzz/Cargo.toml](), [fuzz/fuzz_targets/workspace_patch_parser.rs](), [fuzz/fuzz_targets/process_runner_input_parser.rs]().

​
CI/CD Integration

Testing is integrated into GitHub Actions with specific gates for regression and smoke testing.
WorkflowPurposeTrigger
cli-install-smoke.ymlValidates binary on Ubuntu, macOS, and Windows [.github/workflows/cli-install-smoke.yml#23-27](http://.github/workflows/cli-install-smoke.yml#23-27)Push to main
cli-full-regression.ymlRuns the full workflow matrix and ACP shim testsScheduled/Manual
ci.ymlUnit tests, clippy, and rustfmtPR/Push
Sources: [.github/workflows/cli-install-smoke.yml](), [.github/workflows/cli-full-regression.yml]().

​
Performance Smoke Tests

The run-performance-smoke.ps1 script executes high-load scenarios to detect latency regressions in the palyrad gateway and journal store. It measures:
  • Message Throughput: Latency of the run_stream pipeline.
  • Journal Append Latency: Time to commit events to the SQLite-backed JournalStore.
  • Memory Search Recall: Latency of vector/FTS lookups during prompt augmentation.
Sources: [scripts/test/run-performance-smoke.ps1]().