Skip to content

test(otel): clarify the case-25 errorless callback baseline test #172

Open

Description

@zhongkechen

test_case25_requires_error_for_callback_without_error_details only asserts that the baseline trace validates successfully. Its name suggests a negative control proving that ERROR status is required.

Follow-up to PR #131's test-name review comment.

Verified scope

The test remains unchanged in packages/aws-durable-execution-conformance-tests-otel/tests/test_callback_no_error_requirements.py on main 9a5ead5e9ab8051bedfd300726472cbd443923cb, checked on 2026-10-27:

@pytest.mark.parametrize("view", ["invocation", "execution"])
def test_case25_requires_error_for_callback_without_error_details(view: str) -> None:
    assert validate_trace(_trace(view), _assertions(view), _query()) == []

This is a naming/readability cleanup. The separate test_case25_rejects_wrong_terminal_callback_status_on_lossless_backend already rejects both OK and UNSET in both views. Validation at PR head 04f194f44a5136074dfb17c8add1d75b24055cad confirmed those status-specific failures and a baseline callback status of ERROR.

Acceptance criteria

  • Rename the test to describe acceptance of the valid errorless-failure baseline, or make its name and explicit assertions agree.
  • Retain a passing baseline control and the existing negative status controls for both views.
  • If asserting the fixture's zero exception-event count for clarity, keep it a fixture assertion: case 25 deliberately does not require every SDK to omit exception events. A real exception may escape the callback span while the enclosing workflow catches it.
  • Run the focused callback requirement tests.

Related: #167 and #168 improve attribution in negative tests in the same module. Their described scope and current diff do not rename this positive baseline test; coordinate the small change to avoid conflicts.

Activity

  1. added
    documentationImprovements or additions to documentation
    pkg:otelPackage: aws-durable-execution-conformance-tests-otel
    and removed
    needs-triageAwaiting operator triage
    on Oct 28, 2026
  2. zhongkechen commented on Nov 2, 2026

    @zhongkechen
    ContributorAuthor

    Severity - low

    What is wrong. test_case25_requires_error_for_callback_without_error_details is a positive baseline check: it only asserts that the valid trace passes validation. Its name reads like a negative control proving that an incorrect callback status is rejected.

    What is correct. Separate negative controls already supply OK and UNSET callback statuses and check for status validation errors in both Invocation and Execution views. The misleading baseline name does not itself weaken the requirement or change which SDK traces pass.

    Why this severity. This is test readability and maintenance cleanup, with no demonstrated customer runtime impact or missing status coverage. Renaming the baseline, or making its assertions match its name, is sufficient while retaining the existing positive and negative controls. Any assertion about zero exception events should describe this fixture, not impose a new SDK-wide omission requirement.

    Source review: baseline and negative status controls at 0eaf7341. No new test run was performed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationpkg:otelPackage: aws-durable-execution-conformance-tests-otel

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions