CLI Parity Matrix and Reporting
TheCliParityMatrix serves as the source of truth for the intended surface area of the CLI. It defines the expected commands, subcommands, aliases, and flags that must exist in the binary.
Data Structures
The parity system is implemented incrates/palyra-cli/src/cli_parity.rs.
CliParityMatrix: A collection of entries representing the entire CLI tree crates/palyra-cli/src/cli_parity.rs#95-99.CliParityMatrixEntry: Defines a specific command path (e.g.,palyra gateway status), its category, and its implementation status (Done,Partial, etc.) crates/palyra-cli/src/cli_parity.rs#79-93.CliParityReport: The result of comparing a liveclap::Commandinstance against theCliParityMatrix. It identifies missing commands, missing aliases, or missing flags crates/palyra-cli/src/cli_parity.rs#155-159.
Verification Logic
The functionbuild_cli_parity_report takes the matrix and the root command generated by Cli::command() and performs a deep inspection crates/palyra-cli/src/cli_parity.rs#165-171. It uses find_command to traverse the clap command tree and verify that all required elements defined in the matrix are present crates/palyra-cli/src/cli_parity.rs#179-200.
Parity Data Flow
The following diagram illustrates how the parity matrix is used to generate reports and enforce consistency. CLI Parity Verification Flow Sources: crates/palyra-cli/src/cli_parity.rs#165-200, crates/palyra-cli/src/lib.rs#76-80Help Snapshot Testing
To prevent regressions in documentation and command discoverability, Palyra maintains a suite of help-text snapshots. These are verified bycrates/palyra-cli/tests/help_snapshots.rs.
Snapshot Specification
TheCliParitySnapshotSpec allows for platform-specific help files, accounting for differences like file extensions (.exe on Windows) or platform-specific subcommands crates/palyra-cli/src/cli_parity.rs#29-38.
- Unix Snapshots: Usually stored in files like
root-help-unix.txtcrates/palyra-cli/tests/help_snapshots/root-help-unix.txt#1-10. - Windows Snapshots: Stored in
root-help-windows.txtcrates/palyra-cli/tests/help_snapshots/root-help-windows.txt#1-10.
Normalization and Comparison
During testing, the CLI output is normalized to ensure that minor environment differences (like terminal width or line endings) do not cause false failures.- Terminal Width: Forced to 200 columns via the
COLUMNSenvironment variable crates/palyra-cli/tests/help_snapshots.rs#26-31. - Line Endings: CRLF is converted to LF crates/palyra-cli/tests/help_snapshots.rs#22-24.
- Binary Name:
palyra.exeis normalized topalyracrates/palyra-cli/tests/help_snapshots.rs#23-23. - Whitespace: Trailing whitespace is trimmed and redundant blank lines are collapsed crates/palyra-cli/tests/help_snapshots.rs#41-52.
Snapshot Update Workflow
If the CLI structure changes intentionally, developers can update snapshots by setting thePALYRA_UPDATE_HELP_SNAPSHOTS environment variable crates/palyra-cli/tests/help_snapshots.rs#8-12.
Sources: crates/palyra-cli/src/cli_parity.rs#29-51, crates/palyra-cli/tests/help_snapshots.rs#22-53
CLI Install Smoke Tests
The smoke test suite ensures that the packaged CLI binary functions correctly in a real installation environment across Linux, macOS, and Windows.Implementation
The smoke tests are driven by two main components:- Rust Test Harness:
crates/palyra-cli/tests/installed_smoke.rsexecutes the actual binary under test using environment variables to locate the installation root and configuration crates/palyra-cli/tests/installed_smoke.rs#17-36. - PowerShell Orchestrator:
scripts/test/run-cli-install-smoke.ps1manages the lifecycle of the test environment, including creating temporary state directories and executing the CLI across different scenarios scripts/test/run-cli-install-smoke.ps1#41-83.
Smoke Test Scenarios
The suite validates several critical non-interactive flows:- Baseline Commands:
version,--help, andprotocol versioncrates/palyra-cli/tests/installed_smoke.rs#69-84. - Doctor Diagnostics: Running
palyra doctor --jsonto verify the diagnostic pipeline crates/palyra-cli/tests/installed_smoke.rs#72-80. - Setup Wizard: Executing a non-interactive
setupwith various flags like--mode local,--flow quickstart, and--accept-riskcrates/palyra-cli/tests/installed_smoke.rs#167-195. - Update/Uninstall Dry-runs: Verifying that the update and uninstall logic correctly identifies the installation root and potential actions without performing destructive changes crates/palyra-cli/tests/installed_smoke.rs#102-146.
Workflow Regression Matrix
The regression matrix is a component of the CI pipeline that ensures complex multi-step workflows (like onboarding or secret rotation) do not break.emit_cli_install_smoke_inventory: This example tool generates a JSON inventory of all help commands defined in the parity matrix, which is then used by the smoke harness to ensure every documented command is at least executable crates/palyra-cli/examples/emit_cli_install_smoke_inventory.rs#17-20.- CI Integration: The
cli-install-smoke.ymlworkflow runs on every push tomain, executing the smoke scripts on all three major operating systems to catch platform-specific regressions early .github/workflows/cli-install-smoke.yml#24-27.