Make the default silent-chunk-drop on parse errors discoverable - #521
Conversation
extract()'s default (suppress_parse_errors=True) drops a chunk with no exception on a parse or schema error, which is easy to miss: caller-side retry logic keyed on catching that exception never fires, because it never reaches the except block. Traced and reported by @reichaves on google#358 after independently hitting it in github.com/reichaves/langextract-fundos; not a bug, an existing knob that wasn't visible enough. No behavior change here, only visibility of the existing default: - resolver.py: the two warning-level log lines now say the data is gone and cannot be retried, and name the exact opt-out (resolver_params={'suppress_parse_errors': False}), instead of just "Skipping chunk: ...". - extraction.py: extract()'s docstring gets a Note callout right after the summary, ahead of the full Args list, instead of the behavior being reachable only via one line buried under resolver_params. Adds a regression test asserting the parse-error log message names both "no retry possible" and the opt-out, and strengthens the existing schema-error log test the same way.
|
Your branch is 1 commits behind git fetch origin main
git merge origin/main
git pushNote: Enable "Allow edits by maintainers" to allow automatic updates. |
|
Reporter of #358 here — this wording change addresses exactly what bit me, and I want to add the field numbers behind that report since they may be useful for review. The document was a Brazilian FIDC fund regulation (a CVM regulatory filing, ~160K chars of PDF text reduced to ~50K before extraction). I split ~22 entity types across three That chain never once fired. Every parse failure was suppressed by the default, so the What made this hard to spot is that the failure is silent in both directions: no exception, and the One small note in favor of the docstring No changes requested from me. |
|
Your branch is 4 commits behind git fetch origin main
git merge origin/main
git pushNote: Enable "Allow edits by maintainers" to allow automatic updates. |
Description
extract()'s default (suppress_parse_errors=True) drops a chunk on a parse or schema error with no exception raised — the chunk's data is simply missing from the result. This is easy to miss: any caller-side retry logic written around catching a parse exception fromextract()never fires, because the exception never reaches it.Traced and reported by @reichaves on #358, after independently hitting it in
reichaves/langextract-fundos— not a bug, an existing knob (resolver_params={'suppress_parse_errors': False}) that just wasn't visible enough. No behavior change — only visibility of the existing default, per that comment: "Might be worth a doc callout or a debug-level log distinguishing 'chunk silently dropped, no retry possible' from other warnings."resolver.py: both warnings explicitly say the chunk was omitted, identify the parsing or schema error, and show the existingresolver_params={'suppress_parse_errors': False}option for raising an exception.extraction.py:extract()'s docstring gets aNote:callout right after the summary — ahead of the fullArgslist — instead of the behavior being reachable only via one line buried underresolver_params.Related to #358
Type: Documentation (log-message wording + docstring only; no behavior change).
How Has This Been Tested?
python -m pytest tests/resolver_test.py tests/annotation_test.py tests/init_test.py— 146 passed.ea2094518f32ad9779e199ecc3cfc91fce2353d4onto currentmain: 12 live API tests passed; 3 credential-dependent or opt-in tests skipped.Checklist:
Note:callout is the fix).pylintover the affected code.