fix(insight): Harden insight facet normalization and empty qualitative handling

[BingqingLyu]

TLDR

This PR hardens the insight generation system by adding robust normalization for session facet data and improving handling of empty qualitative sections in the insight report UI.

Key Changes:

• DataProcessor.ts: Added comprehensive normalization functions (, , , ) to handle malformed or incomplete LLM responses gracefully
• DataProcessor.ts: Added helper to filter out empty/invalid qualitative data
• App.tsx & Qualitative.tsx: Updated UI components to conditionally render sections only when they contain meaningful data, preventing empty sections from appearing in reports
• Tests: Added comprehensive test coverage for malformed facet handling and empty qualitative scenarios

Screenshots / Video Demo

N/A — internal data handling improvements with no visible UI changes when data is present. When qualitative data is empty, sections are now hidden instead of showing blank areas.

Dive Deeper

The insight generation pipeline processes session data through LLM analysis to produce both quantitative metrics and qualitative insights. Previously, the system had several vulnerabilities:

1. Malformed LLM responses: When the LLM returned null values, unexpected types, or invalid enum values for facet fields, the aggregation logic would fail or produce incorrect results
2. Empty qualitative data: When all qualitative sections were empty, the UI would still render section headers and navigation links to nowhere
3. Cached facet reuse: Previously cached facets weren't validated when reused, potentially propagating malformed data

This PR addresses these issues by:

1. Normalization layer: All facet data is now normalized through type-safe functions that coerce invalid values to sensible defaults (e.g., invalid enum values fall back to defaults, null objects become empty records)

2. Meaningfulness checks: The UI now checks if qualitative sections have actual content before rendering them, using a recursive function that handles strings, numbers, arrays, and objects

3. Defensive aggregation: Count record aggregation now safely handles malformed inputs using

4. Cached facet validation: When reusing cached facets from previous runs, they are now normalized before being returned

Reviewer Test Plan

1. Run the insight generation on a dataset with mixed session types
2. Verify that malformed LLM responses (simulated in tests) are handled gracefully
3. Check that the insight report UI renders correctly with partial or empty qualitative data
4. Run the new test cases:

✓ src/services/insight/generators/DataProcessor.test.ts (42 tests) 68ms

Test Files 1 passed (1)
Tests 42 passed (42)
Start at 18:28:01
Duration 6.19s (transform 1.17s, setup 22ms, collect 2.32s, tests 68ms, environment 384ms, prepare 98ms)

Testing Matrix

	🍏	🪟	🐧
npm run	✅	❓	❓
npx	✅	❓	❓
Docker	❓	❓	❓
Podman	-	-	-
Seatbelt	❓	-	-

Tested on macOS with npm run build + npm run test

Linked issues / bugs

No linked issues

---

🤖 Generated with Qwen Code

[BingqingLyu]

Conflict Group 2

This PR shares modified functions with 1 other PR(s): #45.

These PRs should be reviewed as a batch — merging one may affect the others.

Function	File	Also modified by
AtAGlance	Qualitative.tsx	#45
NavToc	Qualitative.tsx	#45
analyzeSession	DataProcessor.ts	#45
generateQualitativeInsights	DataProcessor.ts	#45

graph LR
    PR108["PR #108"]
    FAtAGlance_7020["AtAGlance<br>Qualitative.tsx"]
    PR108 -->|modifies| FAtAGlance_7020
    PR45["PR #45"]
    PR45 -->|modifies| FAtAGlance_7020
    FNavToc_7020["NavToc<br>Qualitative.tsx"]
    PR108 -->|modifies| FNavToc_7020
    PR45 -->|modifies| FNavToc_7020
    FanalyzeSession_8719["analyzeSession<br>DataProcessor.ts"]
    PR108 -->|modifies| FanalyzeSession_8719
    PR45 -->|modifies| FanalyzeSession_8719
    FgenerateQualitativeInsights_8719["generateQualitativeInsights<br>DataProcessor.ts"]
    PR108 -->|modifies| FgenerateQualitativeInsights_8719
    PR45 -->|modifies| FgenerateQualitativeInsights_8719

Loading

---

Posted by codegraph-ai conflict detection.