Repository navigation
docs(ai): bundled skill examples match main (#1591) - #1592
Conversation
The AI skill files under edgar/ai/skills described behaviour that main does not have, so the calls an agent wrote failed or read the wrong value. Five statements were checked against main (8cd5799) and against v5.62.0: - A filing with no XBRL was said to raise the MCP error NO_XBRL_DATA, and filing.document() was named as the way to its text. Nothing raises XBRLFilingWithNoXbrlData on main: Company.get_financials() returns None (docs/upgrade/6.0.md). Stale in 6.0: on v5.62.0 the same call returned an empty Financials with a FutureWarning. - A missing item was said to read as None (tenk['Item 15'], forty_f['Item 1']). It raises SectionNotFoundError on main (tenk['Item 99'] on Apple's FY2024 10-K, accession 0000320193-24-000123); tenk.get(item, default) never raises. Stale in 6.0: v5.62.0 returned None with a FutureWarning. - filing.document was written as a call. It is a property returning an Attachment: filing.document() raises TypeError on both builds, and filing.document.text() returns the filing text (262,262 characters for Apple's FY2024 10-K). Wrong on v5.62.0 as well. - Section text was read with .text. tenk.risk_factors and tenk['Item 1A'] are str (68,887 characters for Apple's FY2024 10-K), so .text does not exist on them. Wrong on v5.62.0 as well. - find('AAPL') was said to return a Company. It returns an Entity, which has no get_financials(); Company("AAPL") is the call for financials. Wrong on v5.62.0 as well. Root cause: two statements went stale in 6.0 (d07209b, dgunning#1557, made get_financials() return None instead of an empty Financials object (the 5.x warning had said it would raise), and the missing-item lookup began to raise). Three were already wrong before 6.0. The skill text was never checked against the code, and no test read the examples. Fix: change only the four skill files under edgar/ai/skills (core and reports, skill.yaml and sharp-edges.yaml). No code changes. Each corrected form was run against Apple's FY2024 10-K SGML copy in data/sgml, and find("AAPL") was run live: it returns an Entity with no get_financials(). The static check scans every example string, including the avoid entry's pattern and the detection patterns, so a stale call cannot hide in a regex key. Not changed, because they are outside edgar/ai: the same stale lines in docs/examples/user_journeys.md, docs/guides/proxystatement-data-object-guide.md and docs/common-pitfalls.md, and the find() docstring in edgar/__init__.py, which says Company where the code returns Entity. Also not changed: the reports skill says FortyF properties return None for an absent section (reports skill.yaml:135, sharp-edges forty-f-sections-may-be-absent). On main they raise SectionNotFoundError (forty_f.py capital_structure goes through __getitem__ and report_lookup_miss). Whether they should raise or return None is the public behaviour question in dgunning#1581, so that text waits for the answer. tests/issues/regression/test_issue_1591.py: on the unchanged files 6 of 12 tests fail: the NO_XBRL_DATA text check and five stale-form checks (the filing.document() call, risk_factors.text, item.text, "Ticker → Company", and "is None" for a missing item). After the fix all 12 pass. The six behaviour tests check the corrected forms against the Apple filing and pass on both builds. Co-Authored-By: Claude Haiku 5.5 <noreply@anthropic.com> Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks for this. It's a tidy, well-scoped fix. I checked it against main and it holds up:
A few small things, none of them blocking:
Thanks again. |
|
On it. |
|
Thanks for the review and the merge. As you asked, the follow-up for It covers the How the issue was made, by model and role:
|
What this changes
The AI skill files that agents read before writing edgartools code described behaviour that main does not have. Two statements are wrong on main: nothing raises
XBRLFilingWithNoXbrlData, so theNO_XBRL_DATAerror the skill promised for a filing with no XBRL cannot occur (get_financials()returnsNone; on v5.62.0 the error appeared only with the opt-inEDGARTOOLS_STRICT_ERRORS=1, and by default the call returned an emptyFinancials), and a missing item such astenk["Item 99"]raisesSectionNotFoundErrorwhere v5.62.0 returnedNone. Three were already wrong on v5.62.0:filing.documentis a property returning anAttachment, not a method; section text is astr, so.textand.titledo not exist on it; andfind("AAPL")returns anEntity, which has noget_financials(), not aCompany. This PR changes only the skill YAML underedgar/ai/skills/(core and reports,skill.yamlandsharp-edges.yaml), adds the regression test, and adds a changelog fragment. There are no code changes.Fixes #1591
Verification
data/sgml/). Its cover page states 15,115,823,000 shares of common stock outstanding. The regression test asserts that value appears infiling.document.text(), thattenk["Item 99"]raisesSectionNotFoundError, and thattenk.risk_factorsis a string longer than 50,000 characters (68,887 here).tests/issues/regression/test_issue_1591.py, and its docstring links the issue.None. The corrected text says a missing item raisesSectionNotFoundError; the test checks that it raises, that the message names the item, and thatget(item, default)returns the default.find("AAPL")was run live and returns anEntitywith noget_financials(). The 40-F example is checked with a stand-in object, not a live 40-F: the repo has no committed 40-F filing, and the existing network testtests/issues/regression/test_sx7y_report_getitem_signals_miss.pypins FortyF against a real one (Avino Silver & Gold, 0001477932-24-001577).changelog.d/1591.fixed.md6.0
docs/upgrade/6.0.mdsays what users see now and what to do insteadCI fails a PR that stages a 6.0 change without touching the upgrade guide. If it flags something that isn't a 6.0 change, add the
no-upgrade-guidelabel, say why here, and re-run the job.AI assistance
Working context (optional)
Plan:
docs/upgrade/6.0.md, and against the v5.62.0 results the issue records.avoidentry's pattern and the detection patterns, so a stale call cannot hide in a regex key. Re-ran the test: 12 of 12 pass.Evidence checked:
filing.documentis anAttachment;filing.document.text()returns 262,262 characters containing the cover-page share count;filing.document()raisesTypeError;tenk["Item 99"]raisesSectionNotFoundError;tenk.get("Item 99", "")returns"";tenk.risk_factorsandtenk["Item 1A"]are equalstrvalues of 68,887 characters.find("AAPL")returnsEntity, and that object has noget_financials.XBRL.from_filingreturningNone,Financials.extractreturnsNone, which is the stepget_financials()takes.forty_f.capital_structuregoes throughFortyF.__getitem__, which callsreport_lookup_miss. On main a missing section therefore raisesSectionNotFoundError. The reports skill's statement that these properties returnNone(reportsskill.yaml, the 40-F note) and itsif cap:guard (reportssharp-edges.yaml,forty-f-sections-may-be-absent) are wrong today in the same way. Whether they should raise or returnNoneis the public-behaviour question in FortyF and TwentyF section properties raise SectionNotFoundError where TenK's four return None #1581, so that text waits for the answer.find()docstring is the only code-side mismatch, and it is outside this subsystem.edgar/ai/skills/or this test file. feat!: remove the seven deprecated exception aliases (07lk.10) #1573 touchesedgar/ai/mcp/tools/base.py, which this PR does not change.Not changed:
edgar/ai/: the same stale lines indocs/examples/user_journeys.md,docs/guides/proxystatement-data-object-guide.mdanddocs/common-pitfalls.md, and thefind()docstring inedgar/__init__.py, which saysCompanywhere the code returnsEntity.edgar/ai/skills/constitution.yaml:35, an anti-pattern regex (document\(\)\.text) used as an evaluation input. It names the call form that is still wrong, so it stays.if cap:guard, which wait on FortyF and TwentyF section properties raise SectionNotFoundError where TenK's four return None #1581 as described above.Test runs:
scripts/check_regression_provenance.py) on a copy of the test with1591replaced by a number: OK.-m fast -n auto) on this branch: 53 failed, 42,345 passed, 3 skipped, 1 xfailed, 147 errors (200 failing or erroring tests). Each of the 200 was re-run on the unchanged 8cd5799 in the same environment. 198 fail or error there too, so they are pre-existing. The other two,test_parsing_speed_thresholdandtest_large_document_performance, are wall-clock thresholds. Run alone today they fail on the unchanged base as well (base 4.9 to 5.4 s against a 2 s threshold and 9.8 to 11.0 s against 8.5 s; branch 4.6 to 5.6 s and 9.4 to 12.8 s), so the machine's load, not this change, decides them. No failure is new.ruff checkon the new test file: clean.ruff check edgar: 1,075 errors on both 8cd5799 and this branch (pre-existing). No Python file underedgar/changed, so none were added.