> ## Documentation Index
> Fetch the complete documentation index at: https://docs-code.palyra.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Test Infrastructure and Test Suites

<details>
  <summary>Relevant source files</summary>

  The following files were used as context for generating this wiki page:

  * crates/palyra-cli/examples/emit\_cli\_install\_smoke\_inventory.rs
  * crates/palyra-cli/examples/run\_release\_eval\_gate.rs
  * crates/palyra-cli/examples/run\_workflow\_regression.rs
  * crates/palyra-cli/src/workflow\_regression.rs
  * crates/palyra-cli/tests/cli\_v1\_acp\_shim.rs
  * crates/palyra-cli/tests/installed\_smoke.rs
  * crates/palyra-cli/tests/support/bin\_under\_test.rs
  * crates/palyra-cli/tests/support/cli\_harness.rs
  * crates/palyra-cli/tests/support/mod.rs
  * crates/palyra-cli/tests/workflow\_regression\_contract.rs
  * crates/palyra-cli/tests/workflow\_regression\_matrix.rs
  * crates/palyra-common/src/release\_evals/catalog.rs
  * crates/palyra-common/src/release\_evals/evaluator.rs
  * crates/palyra-common/src/release\_evals/mod.rs
  * crates/palyra-common/src/release\_evals/projections.rs
  * crates/palyra-common/src/release\_evals/schema.rs
  * crates/palyra-common/src/runtime\_preview\.rs
  * crates/palyra-common/tests/release\_eval\_contract.rs
  * crates/palyra-connectors/tests/fixtures/channel\_simulator\_expected.json
  * crates/palyra-egress-proxy/src/lib.rs
  * crates/palyra-egress-proxy/tests/critical\_attack\_scenarios.rs
  * crates/palyra-safety/tests/critical\_attack\_scenarios.rs
  * crates/palyra-workerd/src/lib.rs
  * crates/palyra-workerd/tests/critical\_attack\_scenarios.rs
  * fixtures/golden/release\_eval\_inventory.json
  * infra/release/compat-hardening-readiness.json
  * infra/release/workflow-regression-matrix.json
  * scripts/test/check-deterministic-fixtures.ps1
  * scripts/test/check-deterministic-fixtures.sh
  * scripts/test/run-cli-install-smoke.ps1
  * scripts/test/run-cli-install-smoke.sh
  * scripts/test/run-deterministic-soak.ps1
  * scripts/test/run-deterministic-soak.sh
  * scripts/test/run-workflow-regression.ps1
  * scripts/test/run-workflow-regression.sh
  * scripts/test/update-deterministic-fixtures.ps1
  * scripts/test/update-deterministic-fixtures.sh
</details>

Palyra employs a multi-layered testing strategy designed to ensure deterministic behavior across its core agent logic, CLI parity, and complex integration scenarios involving external connectors and worker fleets. The infrastructure ranges from low-level unit tests to high-level "golden" fixture comparisons and a comprehensive workflow regression matrix that gates releases.

## Workflow Regression Matrix

The workflow regression matrix is the primary integration gate for the Palyra ecosystem. It validates end-to-end scenarios by spawning real `palyrad` and `browserd` processes on dynamic loopback ports, interacting with them via the CLI and gRPC clients.

### Implementation and Data Flow

The matrix is driven by a JSON manifest `infra/release/workflow-regression-matrix.json` [infra/release/workflow-regression-matrix.json#1-121](http://infra/release/workflow-regression-matrix.json#1-121), which defines:

* **Profiles**: `fast` (per-push) and `full` (nightly/pre-release) [infra/release/workflow-regression-matrix.json#5-51](http://infra/release/workflow-regression-matrix.json#5-51).
* **Subsystems**: Logical groupings of functionality like `session_queue_policy`, `retrieval_dual_path`, and `networked_workers` [infra/release/workflow-regression-matrix.json#52-121](http://infra/release/workflow-regression-matrix.json#52-121).
* **Scenarios**: Specific test cases mapped to CLI commands or unit tests [infra/release/workflow-regression-matrix.json#152-190](http://infra/release/workflow-regression-matrix.json#152-190).

The `run_workflow_regression` example [crates/palyra-cli/examples/run\_workflow\_regression.rs#24-180](http://crates/palyra-cli/examples/run_workflow_regression.rs#24-180) orchestrates the execution. It loads the manifest, filters scenarios based on the active profile, and executes them, capturing logs and generating a `WorkflowRegressionRunReport`.

### Regression Components

| Component                         | Responsibility                                       | Source                                                                                                                                    |
| :-------------------------------- | :--------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------- |
| `workflow_regression_matrix.rs`   | Main test runner for local/remote/session lifecycle. | [crates/palyra-cli/tests/workflow\_regression\_matrix.rs#38-158](http://crates/palyra-cli/tests/workflow_regression_matrix.rs#38-158)     |
| `workflow_regression_contract.rs` | Validates consistency between manifest and code.     | [crates/palyra-cli/tests/workflow\_regression\_contract.rs#45-107](http://crates/palyra-cli/tests/workflow_regression_contract.rs#45-107) |
| `run-workflow-regression.sh`      | Entry point for CI environments.                     | [scripts/test/run-workflow-regression.sh#40-49](http://scripts/test/run-workflow-regression.sh#40-49)                                     |

**Sources:** [infra/release/workflow-regression-matrix.json#1-190](http://infra/release/workflow-regression-matrix.json#1-190), [crates/palyra-cli/examples/run\_workflow\_regression.rs#24-180](http://crates/palyra-cli/examples/run_workflow_regression.rs#24-180), [crates/palyra-cli/tests/workflow\_regression\_matrix.rs#1-158](http://crates/palyra-cli/tests/workflow_regression_matrix.rs#1-158), [crates/palyra-cli/tests/workflow\_regression\_contract.rs#1-107](http://crates/palyra-cli/tests/workflow_regression_contract.rs#1-107).

***

## CLI Parity and ACP Shim Harness

The CLI infrastructure ensures that the `palyra` binary maintains a stable contract for both human operators and automated shims (like the Agent Control Protocol - ACP).

### Help Snapshot and Status Tests

The suite validates that CLI output remains consistent across versions. The `status_reports_http_grpc_and_admin_health` test [crates/palyra-cli/tests/cli\_v1\_acp\_shim.rs#26-61](http://crates/palyra-cli/tests/cli_v1_acp_shim.rs#26-61) verifies that the `status` command correctly reports on all gateway transports.

### ACP NDJSON Shim

The ACP shim allows external tools to consume agent events as NDJSON. The `agent_acp_shim_emits_ndjson_events` test [crates/palyra-cli/tests/cli\_v1\_acp\_shim.rs#204-207](http://crates/palyra-cli/tests/cli_v1_acp_shim.rs#204-207) ensures that when `--output-format json` is used, the CLI emits valid, machine-readable events instead of formatted text.

**Sources:** [crates/palyra-cli/tests/cli\_v1\_acp\_shim.rs#1-207](http://crates/palyra-cli/tests/cli_v1_acp_shim.rs#1-207).

***

## Golden Fixture and Installed Smoke Tests

To prevent regression in the distribution artifacts, Palyra runs "smoke tests" against fully packaged binaries.

### Installed Smoke Context

The `installed_smoke.rs` suite uses environment variables like `PALYRA_BIN_UNDER_TEST` to locate a production-build binary [crates/palyra-cli/tests/installed\_smoke.rs#28-40](http://crates/palyra-cli/tests/installed_smoke.rs#28-40). It performs:

* **Baseline Commands**: `version`, `--help`, and `doctor` [crates/palyra-cli/tests/installed\_smoke.rs#86-96](http://crates/palyra-cli/tests/installed_smoke.rs#86-96).
* **Configuration Validation**: `config validate` and `setup --wizard` [crates/palyra-cli/tests/installed\_smoke.rs#102-109](http://crates/palyra-cli/tests/installed_smoke.rs#102-109).
* **Update/Uninstall Dry-runs**: Validates the logic for managing the installation root [crates/palyra-cli/tests/installed\_smoke.rs#119-163](http://crates/palyra-cli/tests/installed_smoke.rs#119-163).

### Scenario Orchestration

The PowerShell script `run-cli-install-smoke.ps1` [scripts/test/run-cli-install-smoke.ps1#41-83](http://scripts/test/run-cli-install-smoke.ps1#41-83) creates isolated scenario contexts (temporary home directories, config paths, and state roots) to ensure that the binary behaves correctly in clean-room environments.

**Sources:** [crates/palyra-cli/tests/installed\_smoke.rs#1-163](http://crates/palyra-cli/tests/installed_smoke.rs#1-163), [scripts/test/run-cli-install-smoke.ps1#1-101](http://scripts/test/run-cli-install-smoke.ps1#1-101).

***

## Worker Fleet and Egress Testing

Testing the worker fleet requires simulating attestation and network isolation.

### Worker Attestation Harness

The `palyra-workerd` crate defines `WorkerAttestation` [crates/palyra-workerd/src/lib.rs#35-62](http://crates/palyra-workerd/src/lib.rs#35-62). Tests validate the `validate` function [crates/palyra-workerd/src/lib.rs#113-153](http://crates/palyra-workerd/src/lib.rs#113-153) against `WorkerAttestationExpectation` to ensure that only workers with valid image digests and attested egress proxies can join the fleet.

### Egress Proxy Verification

The `palyra-egress-proxy` suite ensures that the `EgressProxyPolicyService` [crates/palyra-egress-proxy/src/lib.rs#125-127](http://crates/palyra-egress-proxy/src/lib.rs#125-127) correctly blocks forbidden schemes and non-allowlisted hosts. The `evaluate_request` function [crates/palyra-egress-proxy/src/lib.rs#140-150](http://crates/palyra-egress-proxy/src/lib.rs#140-150) is tested against malicious inputs to confirm fail-closed behavior.

**Sources:** [crates/palyra-workerd/src/lib.rs#35-153](http://crates/palyra-workerd/src/lib.rs#35-153), [crates/palyra-egress-proxy/src/lib.rs#1-150](http://crates/palyra-egress-proxy/src/lib.rs#1-150).

***

## Release Evaluation Gate

The Release Evaluation Gate is the final check before a build is promoted. It uses the `CompatReleaseReadinessChecklist` [crates/palyra-cli/tests/workflow\_regression\_contract.rs#12-13](http://crates/palyra-cli/tests/workflow_regression_contract.rs#12-13) to ensure that all "must-pass" scenarios for a given release scope are met.

### Readiness Logic

The gate logic is defined in `infra/release/compat-hardening-readiness.json` [infra/release/compat-hardening-readiness.json#1-37](http://infra/release/compat-hardening-readiness.json#1-37), which tracks:

* **Rollout Controls**: Feature flags that must be in a specific state (e.g., `networked_worker_preview_rollout`) [infra/release/compat-hardening-readiness.json#28-31](http://infra/release/compat-hardening-readiness.json#28-31).
* **Evidence Requirements**: Maps specific evidence IDs (like `workflow_regression_fast`) to the subsystems and scenarios that must pass [infra/release/compat-hardening-readiness.json#76-123](http://infra/release/compat-hardening-readiness.json#76-123).

### System Data Flow: Regression to Release

This diagram illustrates how test results flow from individual scenarios into the final release gate.

```mermaid theme={null}
graph TD
    subgraph "Execution Space"
        S1["Scenario: queued_input_lifecycle"]
        S2["Scenario: dual_path_retrieval"]
        T1["Test: installed_smoke.rs"]
    end

    subgraph "Orchestration Space"
        WRM["WorkflowRegressionManifest"]
        RWR["run_workflow_regression.rs"]
    end

    subgraph "Gate Space"
        CRR["CompatReleaseReadinessChecklist"]
        REG["run_release_eval_gate.rs"]
    end

    S1 --> RWR
    S2 --> RWR
    T1 --> RWR
    WRM --> RWR
    RWR -->|"WorkflowRegressionRunReport"| REG
    CRR --> REG
    REG -->|"Verdict: PASS/FAIL"| Release["Release Artifact"]

    style REG stroke-width:2px
    style RWR stroke-width:2px
```

**Sources:** [infra/release/compat-hardening-readiness.json#1-123](http://infra/release/compat-hardening-readiness.json#1-123), [crates/palyra-cli/tests/workflow\_regression\_contract.rs#1-155](http://crates/palyra-cli/tests/workflow_regression_contract.rs#1-155), [crates/palyra-cli/examples/run\_workflow\_regression.rs#1-180](http://crates/palyra-cli/examples/run_workflow_regression.rs#1-180).

***

## Test Infrastructure Code Entities

The following diagram bridges the natural language concepts of "Test Profiles" and "Validation" to the specific Rust structs and JSON schemas used in the codebase.

```mermaid theme={null}
classDiagram
    class WorkflowRegressionManifest {
        +BTreeMap profiles
        +Vec setup_steps
        +Vec scenarios
    }
    class WorkflowRegressionScenario {
        +String id
        +Vec subsystems
        +Vec profiles
        +Vec command
    }
    class CompatReleaseReadinessChecklist {
        +String release_scope
        +Vec rollout_controls
        +Vec evidence
    }
    class EgressProxyRequest {
        +String url
        +Vec allowed_hosts
        +bool allow_private_targets
    }
    class WorkerAttestation {
        +String worker_id
        +String image_digest_sha256
        +bool egress_proxy_attested
    }

    WorkflowRegressionManifest "1" *-- "many" WorkflowRegressionScenario : contains
    CompatReleaseReadinessChecklist ..> WorkflowRegressionManifest : references matrix_manifest
    EgressProxyRequest ..> EgressProxyPolicyService : evaluated by
    WorkerAttestation ..> WorkerFleetManager : validated by
```

**Sources:** [infra/release/workflow-regression-matrix.json#1-190](http://infra/release/workflow-regression-matrix.json#1-190), [infra/release/compat-hardening-readiness.json#1-50](http://infra/release/compat-hardening-readiness.json#1-50), [crates/palyra-egress-proxy/src/lib.rs#33-48](http://crates/palyra-egress-proxy/src/lib.rs#33-48), [crates/palyra-workerd/src/lib.rs#35-62](http://crates/palyra-workerd/src/lib.rs#35-62).