Skip to content

fix(follow-up): keep non-Latin follow-up questions intact - #2387

Merged
felladrin merged 1 commit into
felladrin:mainfrom
uuzzrm:fix/followup-non-latin
Aug 16, 2026
Merged

fix(follow-up): keep non-Latin follow-up questions intact#2387
felladrin merged 1 commit into
felladrin:mainfrom
uuzzrm:fix/followup-non-latin

Conversation

@uuzzrm

@uuzzrm uuzzrm commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

What & why

Follow-up question generation is documented as language-matching ("The follow-up question must be written in the exact same language as both the original question and response") — but two places in generateFollowUpQuestion only understand Latin scripts, so questions in Chinese, Japanese, Korean, Arabic, and any other non-Latin script come back empty:

  1. The line search only matches an ASCII ? at the end of the line, so a question ending in the fullwidth (or Arabic ؟) is never found and the function returns "" early.
  2. The leading-punctuation strip uses /^[^a-zA-Z]+/, which matches every character of a non-Latin question — the whole question gets replaced with an empty string.

Verified against the current code:

"你觉得这个方案怎么样?".endsWith("?")            // false → never found
"你觉得这个方案怎么样?".replace(/^[^a-zA-Z]+/, "")  // "" → wiped
"1. What about the alternatives?".replace(/^[^a-zA-Z]+/, "") // "What about the alternatives?" (unchanged behavior)

Fix:

  • Find the question line by any of ?, , ؟.
  • Strip leading bullets/numbering/quotes with /^[^\p{L}]+/u — letters in every script survive, so the strip now only removes what it was meant to remove.

English behavior is unchanged (the new regex still strips leading digits and punctuation).

Test plan

New client/modules/followUpQuestions.test.ts (6 tests): English numbering, fullwidth , a Japanese bulleted line, Arabic ؟, a line with no question mark, and the suppressed path. npx vitest run client/modules/followUpQuestions.test.ts → 6 pass. npx tsc and npx biome check pass on the changed files.

Signed-off-by: Ruiming Zhao <uuzzrm@gmail.com>
@felladrin
felladrin force-pushed the fix/followup-non-latin branch from 8ee1173 to daf8303 Compare August 16, 2026 00:15

@felladrin felladrin left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's another great catch! Your contributions are much appreciated!

@felladrin
felladrin merged commit 27b56fd into felladrin:main Aug 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants