Skip to content

bugfix: support Jinja undefined checks in chat templates - #90

Merged
yt-zeng merged 1 commit into
xLLM-AI:mainfrom
yt-zeng:bugfix/normalize-undefined-template-checks
Sep 9, 2026
Merged

yt-zeng merged 1 commit into
xLLM-AI:mainfrom
yt-zeng:bugfix/normalize-undefined-template-checks

Conversation

@yt-zeng

@yt-zeng yt-zeng commented Sep 8, 2026

Copy link
Copy Markdown
Collaborator

Fix chat template rendering for templates that use Jinja's is undefined / is not undefined tests, such as Qwen3.8 templates.

Minja represents missing optional template arguments as none and does not support the undefined test directly. Normalize these tests before creating the Minja chat template:

  • is undefined → is none
  • is not undefined → is not none

The rewrite only applies inside Jinja expression and statement blocks, and ignores quoted string literals.

@yt-zeng
yt-zeng requested a review from yingxudeng September 8, 2026 12:10

@yingxudeng yingxudeng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yt-zeng
yt-zeng requested a review from shifengmin September 8, 2026 12:12

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 30a0797f28

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 30a0797 to ea80e25 Compare September 8, 2026 12:20

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ea80e254b8

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from ea80e25 to 1719173 Compare September 8, 2026 12:31

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1719173741

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 1719173 to 725f1d9 Compare September 8, 2026 12:53

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 725f1d9bf9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 725f1d9 to 2b9d8f8 Compare September 8, 2026 13:04

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2b9d8f8b54

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 2b9d8f8 to 0eb7c77 Compare September 8, 2026 13:18

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0eb7c775d0

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 0eb7c77 to 9b3d5f4 Compare September 8, 2026 13:37

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9b3d5f4dcc

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 9b3d5f4 to 115dfee Compare September 8, 2026 13:56

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 115dfee06c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 115dfee to 7d6eb29 Compare September 8, 2026 14:08

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7d6eb2983c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 7d6eb29 to 3be6a74 Compare September 8, 2026 14:27

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3be6a742b5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +281 to +283
if (block.compare(token_end, kUndefined.size(), kUndefined) != 0 ||
(token_end + kUndefined.size() < block.size() &&
is_identifier_character(block[token_end + kUndefined.size()]))) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Consume parentheses on undefined test calls

When a model uses Jinja's valid no-argument test-call syntax, such as enable_thinking is undefined(), the replacement stops after undefined and leaves () behind. The normalized expression therefore becomes (enable_thinking is none and ... )(), which attempts to call the resulting boolean and causes template rendering to fail; consume an optional empty argument list as part of the test rewrite.

Useful? React with 👍 / 👎.

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 3be6a74 to 1b3b97c Compare September 8, 2026 14:41

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1b3b97c998

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 1b3b97c to 1de3dda Compare September 8, 2026 14:55

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1de3ddaa2c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from 1de3dda to b45060e Compare September 8, 2026 15:14

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b45060eda9

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from b45060e to a0e5c56 Compare September 9, 2026 01:32

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a0e5c560b5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +231 to +234
if (preceding_pos > 0 && text[preceding_pos - 1] == '|') {
return std::string_view();
}
return std::string_view(text.data() + pos, path_end - pos);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep binary operands intact when rewriting tests

When is undefined follows a compound operand whose final term is a path—for example, {% if 1 + value is undefined %} with value = 2—Jinja applies the test to the complete sum and evaluates it as defined. This helper instead extracts only value, producing 1 + (value is none and ...), which becomes truthy or raises a type error rather than selecting the defined branch. The fresh evidence after the earlier filter report is that the current predecessor guard excludes only |; it must reject other non-simple operands or rewrite the complete tested expression.

Useful? React with 👍 / 👎.

Rewrite Jinja undefined tests to Minja-supported none tests before rendering.

Cover missing and false enable_thinking values.
@yt-zeng
yt-zeng force-pushed the bugfix/normalize-undefined-template-checks branch from a0e5c56 to 58014b6 Compare September 9, 2026 02:11

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 58014b61c2

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +344 to +346
if (!content.empty() && content.front() == '-') {
content.remove_prefix(1);
content = trim_whitespace(content);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Recognize leading + whitespace-control markers

When a template uses Jinja's valid leading + whitespace-control form, such as {%+ set value = none %}{% if value is undefined %}, this parser strips only -, so the assignment is not instrumented. The subsequent rewrite therefore sees a null value with no /value presence entry and incorrectly takes the undefined branch; handle + alongside - when parsing block contents.

Useful? React with 👍 / 👎.

@yt-zeng
yt-zeng merged commit b232b70 into xLLM-AI:main Sep 9, 2026
1 check passed
@yt-zeng
yt-zeng deleted the bugfix/normalize-undefined-template-checks branch September 9, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants