Skip to content

verify: baseline of checks, CI matrix, Ruff exceptions and known failures #49

Open

Description

@NaNIsaac

Baseline for the verification pass. Read-only: nothing in the repository was changed. Checked on origin/main 83b960e, 2026-10-27. I did not run the test suite locally; the results below are the CI runs on that commit (Rust, Tests and lint succeeded, 2026-10-26).

What is verified, and how

162 test functions in 23 files plus fake_github.py and fixtures/; 5 skips, all platform conditions (a shell that is not installed, names reserved on Windows, CreateProcess and PowerShell lookups that exist only on Windows); no xfail. The Python target is 3.15 (requires-python >=3.15.0b1).

  • Extraction and output: test_pack.py, test_pack_context.py, test_output.py, test_formats.py, test_structure.py, test_change_output.py, test_semdiff_output.py, test_shadow.py, test_status.py, test_presets.py, test_cap_options.py, test_codigest_dir.py.
  • Remote scan against a fake GitHub (fake_github.py): test_remote_scan.py, including entries that are not plain files, deleted files and Windows-reserved names.
  • Secrets: test_secrets.py (credential files such as .env, id_rsa, *.pem are recognised and templates such as .env.example are not; token masking in code, diffs and terminal output; scan and pack never print secrets; vendor-prefix tokens; commands that print credentials are denied) and test_term_secrets.py, tests/fixtures/capture_secrets.json.
  • Terminal capture and hooks: test_term_install.py, test_term_uninstall.py, test_termstore.py, test_capture_limits.py, test_hook_history.py, test_shell_help.py, test_agent_use.py.

CI matrix

  • ci.yml (Tests): ubuntu-latest with Python 3.15 (uv python install 3.15, uv run --with pytest pytest -q); powershell-help: windows-latest with 3.15 checks Get-Help codigest, native CLI forwarding, and Windows PowerShell 5.1 compatibility.
  • rust.yml (paths rust/**, src/**, lockfiles): ubuntu, windows, macos for fmt, clippy and tests; a second job on ubuntu and windows compares the Rust build with the Python build.
  • installers.yml (install.sh and install.ps1: latest, pinned, corrupted download refused, PowerShell 7 and 5.1), termux.yml (ubuntu-24.04-arm, cargo test and install.sh in Termux), rust-release.yml, publish.yml, docs.yml, lint.yml (uvx ruff@0.17.1 check and format --check), readme.yml.

Ruff exceptions

extend-per-file-ignores: 32 per-file lines (the figure in the brief is confirmed), 72 file/rule pairs (I001 26, BLE001 16, B008 10, S112 4, PLW1510 3, S110 3, others fewer). No # noqa comments. With the per-file ignores removed from a scratch copy, ruff 0.17.1 reports 97 violations (BLE001 30, I001 23, B008 11, S110 8, S112 7, PLW1510 6, SIM102 5, others fewer). Rules selected: B, I, UP plus ruff's defaults.

Version note: CI uses ruff 0.17.1 and passes. Ruff 0.15.15 reports 2 errors on the same commit (E402 1, E731 1) and warns that Python 3.15 support is under development.

Known failures

No test fails on main. Three known limits, from the README "Limitations" list for terminal capture:

  1. In bash (4+) and zsh, multi-line commands (loops typed over several lines, heredocs), background jobs (cmd &) and lines using ! history expansion are not captured; they run normally. In bash, lines that start with a subshell ( ... ) are not captured either.
  2. In bash the hook uses the DEBUG trap and keeps one set before the first prompt; a DEBUG trap set later replaces it.
  3. Captured programs write to a pipe, not a TTY, so some lose colors or progress bars.

Also open: #24 (direction check: capture machinery and analyzers have outgrown the extraction core).

Next in the plan: robustness on 5 or more real open-source repositories (large, broken files, circular imports, deleted files, no git) and a default-exclusion check for secrets (VERIFY item 4).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    verifyVerification baseline and evidence

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions