Skip to content

Preserve content list aliases in multimodal chunks - #273

Merged
LarFii merged 2 commits into
HKUDS:mainfrom
FU-max-boop:fix/content-list-alias-handling
May 11, 2026
Merged

Preserve content list aliases in multimodal chunks#273
LarFii merged 2 commits into
HKUDS:mainfrom
FU-max-boop:fix/content-list-alias-handling

Conversation

@FU-max-boop

Copy link
Copy Markdown
Contributor

Summary

This PR makes direct content_list multimodal processing preserve common alias fields instead of silently dropping or misformatting them.

Changes:

  • preserve each multimodal item's original content_list index after text items are separated, so context lookup still points at the original surrounding items
  • normalize string/list caption and footnote fields before joining them into image/table chunks
  • read table content from table_body, table_data, or text
  • read equation content from text, latex, or equation, and infer LaTeX format when only latex is provided
  • add an offline regression test for these alias-handling cases

Why

The direct content_list path can receive pre-parsed multimodal items whose field names differ slightly by source. In that path, a table that only has table_data can produce an empty table body, an equation that only has latex can lose its formula text, and a string caption can be joined character-by-character.

The same path also separates text and multimodal items before later context extraction. Without preserving the original item index, context lookup can point to the compacted multimodal list position instead of the original content_list position.

Validation

From the repo root:

python3 -m py_compile raganything/utils.py raganything/processor.py raganything/modalprocessors.py tests/test_content_list_alias_handling.py
python3 -m unittest tests/test_content_list_alias_handling.py
git diff --check

The targeted unittest covers:

  • original content_list index preservation
  • string caption normalization for table chunks
  • table_data alias preservation
  • latex equation alias preservation

Scope / limits

This PR does not change parsers, model calls, retrieval behavior, or prompt templates. It only normalizes already-parsed multimodal item fields before building prompts/chunks.

FU-max-boop and others added 2 commits May 10, 2026 10:07
- format_table_body now renders list-of-lists as a Markdown table so the LLM
  sees structured rows instead of a Python repr (the previous str(...) path
  produced "[['Method', 'Score']]" in prompts).
- get_equation_text_and_format no longer concatenates the description into
  equation_text; the equation_chunk template already has a separate
  enhanced_caption slot. Field priority: text > latex > equation, with
  text_format defaulting to "latex" only when reading from the latex alias.
- Updated unit tests to cover Markdown rendering and the simplified
  equation field selection.
@LarFii
LarFii force-pushed the fix/content-list-alias-handling branch from df89f30 to 48a56f9 Compare May 11, 2026 09:16
@LarFii
LarFii merged commit c47bdf3 into HKUDS:main May 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants