Skip to content

createMixedStreamParser: keep line breaks and blank lines, and export a whole-text split that matches the stream #423

Open

Description

@peter-cynomi

Version: @json-render/core 0.21.0

createMixedStreamParser calls onText(line) without the line break and skips blank lines entirely. The prose it rebuilds therefore differs from what createJsonRenderTransform streamed for the same text.

That matters for a server that saves the model's raw transcript, with its ```spec fences, and splits it into text and data-spec parts again when a conversation reloads. There is no exported function that gives exactly the parts the stream gave, so a reload can differ from what the user saw live.

Repro

Feed Here is your report.\n\n```spec\n{"op":"add","path":"/root","value":"report"}\n```\n\nLet me know. to both:

  • createJsonRenderTransform streams Here is your report.\n\n\nLet me know. as prose.
  • createMixedStreamParser calls onText with Here is your report. and Let me know.. Both line breaks and blank lines are gone.

Ask

  • Keep the line break and blank lines in onText.
  • Export a whole-text split, for example splitSpecText(text) → { segments, invalid }, that gives exactly what the transform streams for the same text, with prose and patch segments in order.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions