Skip to content

docs(ai): bundled skill examples match main (#1591) - #1592

Merged
dgunning merged 1 commit into
dgunning:mainfrom
synfonia-llc:fix/skill-no-xbrl-docs
Oct 18, 2026
Merged

dgunning merged 1 commit into
dgunning:mainfrom
synfonia-llc:fix/skill-no-xbrl-docs

Conversation

@synfonia-llc

Copy link
Copy Markdown
Contributor

What this changes

The AI skill files that agents read before writing edgartools code described behaviour that main does not have. Two statements are wrong on main: nothing raises XBRLFilingWithNoXbrlData, so the NO_XBRL_DATA error the skill promised for a filing with no XBRL cannot occur (get_financials() returns None; on v5.62.0 the error appeared only with the opt-in EDGARTOOLS_STRICT_ERRORS=1, and by default the call returned an empty Financials), and a missing item such as tenk["Item 99"] raises SectionNotFoundError where v5.62.0 returned None. Three were already wrong on v5.62.0: filing.document is a property returning an Attachment, not a method; section text is a str, so .text and .title do not exist on it; and find("AAPL") returns an Entity, which has no get_financials(), not a Company. This PR changes only the skill YAML under edgar/ai/skills/ (core and reports, skill.yaml and sharp-edges.yaml), adds the regression test, and adds a changelog fragment. There are no code changes.

Fixes #1591

Verification

  • Ground truth: at least one assertion checks a specific value against a real SEC filing I verified by hand. Filing and accession: Apple Inc. 10-K for fiscal 2024, accession 0000320193-24-000123, filed 2024-11-01 (the SGML copy in data/sgml/). Its cover page states 15,115,823,000 shares of common stock outstanding. The regression test asserts that value appears in filing.document.text(), that tenk["Item 99"] raises SectionNotFoundError, and that tenk.risk_factors is a string longer than 50,000 characters (68,887 here).
  • Regression test for a bug is in tests/issues/regression/test_issue_1591.py, and its docstring links the issue.
  • Silence check: bad or missing input produces a useful error, not None. The corrected text says a missing item raises SectionNotFoundError; the test checks that it raises, that the message names the item, and that get(item, default) returns the default.
  • Docs: any documented example this touches still runs. Each corrected form was run against the Apple filing. find("AAPL") was run live and returns an Entity with no get_financials(). The 40-F example is checked with a stand-in object, not a live 40-F: the repo has no committed 40-F filing, and the existing network test tests/issues/regression/test_sx7y_report_getitem_signals_miss.py pins FortyF against a real one (Avino Silver & Gold, 0001477932-24-001577).
  • Changelog: changelog.d/1591.fixed.md

6.0

  • This PR adds a deprecation, a 6.0 warning, or a "removed in 6.0" note, and docs/upgrade/6.0.md says what users see now and what to do instead

CI fails a PR that stages a 6.0 change without touching the upgrade guide. If it flags something that isn't a 6.0 change, add the no-upgrade-guide label, say why here, and re-run the job.

AI assistance

  • None
  • Assisted. Tool and model: Claude Code. Claude Sonnet 5.5 (extra-high effort) diagnosed the defect; Claude Opus 5.5 (extra-high effort) scoped the change; Claude Haiku 5.5 (extra-high effort) wrote the fix and the regression test; Claude Sonnet 5.5 (extra-high effort) reviewed it.

Working context (optional)

Plan:

  1. Checked each stale statement in the issue against the code on the unchanged commit (8cd5799), against docs/upgrade/6.0.md, and against the v5.62.0 results the issue records.
  2. Wrote the regression test first and ran it on the unchanged skill files: 6 of 12 fail (the NO_XBRL_DATA check and five stale-form checks). The six behaviour checks pass on the unchanged code, because they test the code, which is already correct.
  3. Corrected the four skill files. In review, the static check was widened to scan every example string, including the avoid entry's pattern and the detection patterns, so a stale call cannot hide in a regex key. Re-ran the test: 12 of 12 pass.

Evidence checked:

Not changed:

Test runs:

  • New regression test on unchanged 8cd5799: 6 failed, 6 passed.
  • New regression test after the fix: 12 passed.
  • Provenance check (scripts/check_regression_provenance.py) on a copy of the test with 1591 replaced by a number: OK.
  • Offline suite (-m fast -n auto) on this branch: 53 failed, 42,345 passed, 3 skipped, 1 xfailed, 147 errors (200 failing or erroring tests). Each of the 200 was re-run on the unchanged 8cd5799 in the same environment. 198 fail or error there too, so they are pre-existing. The other two, test_parsing_speed_threshold and test_large_document_performance, are wall-clock thresholds. Run alone today they fail on the unchanged base as well (base 4.9 to 5.4 s against a 2 s threshold and 9.8 to 11.0 s against 8.5 s; branch 4.6 to 5.6 s and 9.4 to 12.8 s), so the machine's load, not this change, decides them. No failure is new.
  • ruff check on the new test file: clean. ruff check edgar: 1,075 errors on both 8cd5799 and this branch (pre-existing). No Python file under edgar/ changed, so none were added.

The AI skill files under edgar/ai/skills described behaviour that main does
not have, so the calls an agent wrote failed or read the wrong value. Five
statements were checked against main (8cd5799) and against v5.62.0:

- A filing with no XBRL was said to raise the MCP error NO_XBRL_DATA, and
  filing.document() was named as the way to its text. Nothing raises
  XBRLFilingWithNoXbrlData on main: Company.get_financials() returns None
  (docs/upgrade/6.0.md). Stale in 6.0: on v5.62.0 the same call returned an
  empty Financials with a FutureWarning.
- A missing item was said to read as None (tenk['Item 15'], forty_f['Item 1']).
  It raises SectionNotFoundError on main (tenk['Item 99'] on Apple's FY2024
  10-K, accession 0000320193-24-000123); tenk.get(item, default) never raises.
  Stale in 6.0: v5.62.0 returned None with a FutureWarning.
- filing.document was written as a call. It is a property returning an
  Attachment: filing.document() raises TypeError on both builds, and
  filing.document.text() returns the filing text (262,262 characters for
  Apple's FY2024 10-K). Wrong on v5.62.0 as well.
- Section text was read with .text. tenk.risk_factors and tenk['Item 1A']
  are str (68,887 characters for Apple's FY2024 10-K), so .text does not
  exist on them. Wrong on v5.62.0 as well.
- find('AAPL') was said to return a Company. It returns an Entity, which has
  no get_financials(); Company("AAPL") is the call for financials. Wrong on
  v5.62.0 as well.

Root cause: two statements went stale in 6.0 (d07209b, dgunning#1557, made
get_financials() return None instead of an empty Financials object (the 5.x
warning had said it would raise), and the missing-item lookup began to raise).
Three were already wrong before 6.0. The skill text was never checked against
the code, and no test read the examples.

Fix: change only the four skill files under edgar/ai/skills (core and reports,
skill.yaml and sharp-edges.yaml). No code changes. Each corrected form was run
against Apple's FY2024 10-K SGML copy in data/sgml, and find("AAPL") was run
live: it returns an Entity with no get_financials(). The static check scans
every example string, including the avoid entry's pattern and the detection
patterns, so a stale call cannot hide in a regex key.

Not changed, because they are outside edgar/ai: the same stale lines in
docs/examples/user_journeys.md, docs/guides/proxystatement-data-object-guide.md
and docs/common-pitfalls.md, and the find() docstring in edgar/__init__.py,
which says Company where the code returns Entity. Also not changed: the reports
skill says FortyF properties return None for an absent section (reports
skill.yaml:135, sharp-edges forty-f-sections-may-be-absent). On main they raise
SectionNotFoundError (forty_f.py capital_structure goes through __getitem__ and
report_lookup_miss). Whether they should raise or return None is the public
behaviour question in dgunning#1581, so that text waits for the answer.

tests/issues/regression/test_issue_1591.py: on the unchanged files 6 of 12 tests
fail: the NO_XBRL_DATA text check and five stale-form checks (the filing.document()
call, risk_factors.text, item.text, "Ticker → Company", and "is None" for a
missing item). After the fix all 12 pass. The six behaviour tests check the
corrected forms against the Apple filing and pass on both builds.

Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dgunning

Copy link
Copy Markdown
Owner

Thanks for this. It's a tidy, well-scoped fix. I checked it against main and it holds up:

  • Nothing on main raises XBRLFilingWithNoXbrlData, find("AAPL") goes to Entity(...), and tenk.risk_factors is self.get('Item 1A'), so the corrected wording matches the code.
  • I confirmed the cover-page value against SEC's copy of Apple's FY2024 10-K (0000320193-24-000123): EntityCommonStockSharesOutstanding is 15,115,823,000.
  • With the four skill files reverted and the test kept, 6 of 12 tests fail. With your changes all 12 pass, and the existing tests/ai skill tests still pass on top of current main.
  • A grep across edgar/ai/skills finds no other copies of the stale forms.

A few small things, none of them blocking:

  1. The 40-F test builds the object with object.__new__ and sets _filing and aif_text by hand. That's fine for now since there's no committed 40-F, but it depends on internals, so it may need touching if FortyF is refactored.
  2. Until FortyF and TwentyF section properties raise SectionNotFoundError where TenK's four return None #1581 is settled, the reports skill disagrees with itself: forty_f['Item 1'] now raises, but the 40-F note and the if cap: sharp-edge still say the properties return None. Let's get that decided soon so agents don't get mixed messages.
  3. The same stale lines in docs/ and the find() docstring are good candidates for a follow-up issue. Could you open one? The docs-name gate won't catch .document(), because document exists as an attribute.

Thanks again.

@dgunning
dgunning merged commit 9260b8f into dgunning:main Oct 18, 2026
11 checks passed
@synfonia-llc

Copy link
Copy Markdown
Contributor Author

On it.

@synfonia-llc

Copy link
Copy Markdown
Contributor Author

Thanks for the review and the merge. As you asked, the follow-up for docs/ and the find() docstring is #1614.

It covers the find() docstring, the three docs/ pages this PR listed as out of scope, and two more found while checking: docs/guides/finding-companies.md, which says find("AAPL") searches names rather than tickers, and docs/examples.md, whose 10-K example indexes a filing by item and calls .text on section text. Each example was run as written on main at 4f11882 and on v5.62.0, with the same failures on both. The issue also records your point that the documented-API check does not catch filing.document().

How the issue was made, by model and role:

  • Claude Opus 5.5 (extra-high effort), in Claude Code, found the stale lines on main, ran every example on both builds and drafted the issue.
  • Claude Sonnet 5.5 (extra-high effort) checked every claim, citation and quote against main, and found docs/examples.md.
  • GPT-6.1 Sol in Codex (ultra reasoning effort) reviewed the draft adversarially and caught two errors in the suggested corrections: the Microsoft journey still fails after its first fix, because Financials.extract() takes a filing, and the docs/common-pitfalls.md example is marked # WRONG on purpose.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Installed skill docs: NO_XBRL_DATA is unreachable on main, and find(), document() and risk_factors examples fail

2 participants