Add Mintlify analytics fetch scripts for assistant insights - #1222
Merged
Conversation
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
|
Warning Review limit reached
Next review available in: 37 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (11)
📝 WalkthroughWalkthroughThis PR adds a Mintlify analytics cache pipeline: API fetching with pagination and retries, checkpointed local persistence, per-day Markdown/JSON outputs, a CLI to run assistant/search/feedback exports, and updated environment/docs/script wiring. ChangesMintlify Analytics Cache Fetch Pipeline
Sequence Diagram(s)sequenceDiagram
participant User
participant fetch-assistant-insights
participant analytics-checkpoint
participant mintlify-analytics
participant analytics-writers
User->>fetch-assistant-insights: pnpm analytics:fetch
fetch-assistant-insights->>analytics-checkpoint: loadCheckpoint / createCheckpoint
fetch-assistant-insights->>mintlify-analytics: fetch assistant conversations chunked
mintlify-analytics-->>fetch-assistant-insights: conversation batch
fetch-assistant-insights->>analytics-checkpoint: markChunkComplete, saveConversationStore
fetch-assistant-insights->>mintlify-analytics: fetch search queries chunked
fetch-assistant-insights->>mintlify-analytics: fetch feedback chunked
fetch-assistant-insights->>analytics-writers: writeSplitSummaries(range, data)
analytics-writers-->>fetch-assistant-insights: dayFiles, dayStats
fetch-assistant-insights->>analytics-checkpoint: clearCheckpoint
fetch-assistant-insights-->>User: manifest.json + summary report
Compact metadata
Related issues: None referenced. Related PRs: None referenced. Suggested labels: enhancement, tooling, documentation Suggested reviewers: Reviewer familiar with Bun/TypeScript CLI scripts and Mintlify API integration A cache for chats, a cache for clues, 🚥 Pre-merge checks | ✅ 2✅ Passed checks (2 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
✨ Simplify code
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 13
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/scripts/analytics/analytics-checkpoint.ts:
- Around line 76-79: The writeJsonFile helper is doing a direct overwrite of the
target checkpoint/store file, which can leave a truncated or invalid JSON file
if the run is interrupted. Update writeJsonFile in analytics-checkpoint.ts to
write the JSON to a temporary file first and then atomically replace the
destination with a same-filesystem rename, so readJsonFile, saveCheckpoint, and
markChunkComplete never observe partial writes.
- Around line 91-97: clearCheckpoint currently depends on readJsonFile failing
JSON.parse of an empty file instead of actually clearing state. Update
clearCheckpoint to remove the checkpoint file directly (or otherwise write an
explicit sentinel value) using checkpointPath and the filesystem API, and keep
the existing error swallowing behavior so callers still treat missing/cleared
checkpoints as non-fatal.
- Around line 150-167: The mergeSearches function is double-counting hits during
incremental refreshes because overlapping ranges re-fetch the same query window.
Update mergeSearches to avoid adding row.hits to an existing SearchRow; instead
preserve the latest row’s hits (or otherwise dedupe overlapping range data)
while still keeping the newer lastSearchedAt, topClickedPage, and ctr in
mergeSearches. Use the existing mergeSearches and incrementalRangeFromManifest
behavior as the key places to adjust the aggregation logic.
In @.github/scripts/analytics/analytics-writers.ts:
- Around line 26-38: The day-report writer is persisting raw user queries
without any truncation or redaction, so sensitive input can leak into shareable
analytics artifacts. Update toSlim and the downstream buildDayMarkdown flow to
scrub or redact the query before it is written to Markdown/JSON, similar to how
responsePreview is handled. Keep the fix localized to the analytics-writers.ts
helpers so all report generation paths use the same sanitized value.
In @.github/scripts/analytics/fetch-assistant-insights.ts:
- Around line 430-433: The search store is being loaded twice in the same block,
causing redundant disk and JSON parsing work. Update the logic around
loadSearchStore(CACHE_DIR) to read it once, store the result in a single local
variable, and destructure both searches and totalSearches from that object
alongside the existing loadConversationStore and loadFeedbackStore reads.
- Around line 122-129: The env-derived paging values in
fetch-assistant-insights.ts are not using the same bounds as the CLI flags, so
ANALYTICS_PAGE_LIMIT can exceed the allowed maximum and get passed through to
fetchOpts.limit. Update the options construction in the fetch-assistant-insights
logic to apply the same 1..1000 clamp used by the CLI page-limit parsing, while
keeping ANALYTICS_PAGE_DELAY_MS validated with the existing non-negative check,
so both env and CLI paths share consistent paging limits.
- Around line 461-464: The chunkDays guard in fetch-assistant-insights.ts is
using fetchRange.dateFrom !== fetchRange.dateTo, which is always true and makes
the DEFAULT_DAYS cap unreachable. Update the condition in the chunkDays
assignment to check the actual date range span instead of comparing endpoints,
so the Math.min(options.chunkDays, DEFAULT_DAYS) branch only applies for small
windows.
In @.github/scripts/analytics/mintlify-analytics.ts:
- Around line 101-103: The chunkKey implementation is duplicated in
analytics-checkpoint, so keep a single source of truth by importing the exported
chunkKey from mintlify-analytics.ts instead of redefining it. Update the
analytics-checkpoint module to reuse chunkKey directly and remove the verbatim
copy so both callers stay in sync if the DateRange key format changes.
- Around line 479-531: The chunked fetch helpers are overriding caller progress
callbacks instead of forwarding them, so `fetchSearchQueriesChunked` (and the
matching `fetchFeedbackChunked`) drop `options.onProgress` entirely while
printing progress. Update the inline `onProgress` wrapper in
`fetchSearchQueriesChunked` to invoke `options.onProgress?.(progress)` before or
alongside the console write, and apply the same change in `fetchFeedbackChunked`
so it matches `fetchAssistantConversationsChunked` behavior and preserves
`PaginatedFetchOptions` hooks.
- Around line 236-279: The pagination loop in fetchPaginated can repeat forever
if the API keeps returning the same cursor or a cursor that normalizes to an
already-seen value. Update fetchPaginated to track the previous cursor before
assigning cursor and break with a warning when the next cursor does not advance,
or add a sane max-pages safety cap. Use the existing fetchPaginated, hasMore,
normalizeCursor, and cursor handling logic to keep the fix localized.
- Around line 326-349: fetchAllSearchQueries is overwriting totalSearches for
each fetched page, so bisected fetches only keep the last branch’s count. Update
the logic inside fetchAllSearchQueries (and the fetchPaginatedWithBisect
callback path it uses) so totalSearches is accumulated per page/recursive branch
instead of reassigned, while still returning the merged searches array and
correct overall total.
- Line 7: The default page delay is still too low and can exceed Mintlify’s
request limit when parseArgs() falls back to DEFAULT_PAGE_DELAY_MS and the fetch
flow uses it unchanged. Update the DEFAULT_PAGE_DELAY_MS constant in
mintlify-analytics.ts to match the 100 req/hour cap (about 36 seconds) or
compute it from that limit, so the default run throttles correctly without
requiring callers to override pageDelayMs. Ensure the new default is used
consistently by the parseArgs() fallback and the downstream fetch path.
In @.github/scripts/analytics/README.md:
- Around line 131-140: The README text in the analytics output section
hard-codes a 14-day chunk, but the batching is configurable and chunk-based.
Update the description under the gitignored output for the fetcher/docs so it
refers generically to chunk/window-based updates instead of a fixed 14-day
period, keeping the wording aligned with the analytics fetcher behavior and the
output paths like assistant-summary.md and by-day/YYYY-MM-DD.md.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: b4b1efc3-3317-471b-adfd-08c4d10f43b3
📒 Files selected for processing (9)
.env.local.example.github/scripts/analytics/README.md.github/scripts/analytics/analytics-checkpoint.ts.github/scripts/analytics/analytics-writers.ts.github/scripts/analytics/fetch-assistant-insights.ts.github/scripts/analytics/mintlify-analytics.ts.gitignoreAGENTS.mdpackage.json
Comment on lines
+101
to
+103
| export function chunkKey(range: DateRange): string { | ||
| return `${range.dateFrom}_${range.dateTo}`; | ||
| } |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
chunkKey duplicated verbatim in analytics-checkpoint.ts.
Same implementation exists at analytics-checkpoint.ts Lines 56-58. Since this file already exports chunkKey, the checkpoint module should import it instead of redefining, avoiding future drift between the two copies.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/scripts/analytics/mintlify-analytics.ts around lines 101 - 103, The
chunkKey implementation is duplicated in analytics-checkpoint, so keep a single
source of truth by importing the exported chunkKey from mintlify-analytics.ts
instead of redefining it. Update the analytics-checkpoint module to reuse
chunkKey directly and remove the verbatim copy so both callers stay in sync if
the DateRange key format changes.
comfyui-wiki
added a commit
that referenced
this pull request
Jul 4, 2026
Atomic JSON writes, fix search merge double-counting, pagination guards, rate-limit defaults, and other robustness improvements from PR #1222 review.
Introduce a local analytics cache CLI (assistant, searches, feedback) with checkpoint resume and rate limiting. Centralize env var documentation in .github/ENV.local.md and keep AGENTS.md to brief references only.
comfyui-wiki
force-pushed
the
feat/mintlify-analytics-cache
branch
from
July 4, 2026 17:16
db9e782 to
7ba1235
Compare
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
.github/scripts/analytics/fetch-assistant-insights.ts (1)
629-651: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winPersist
totalSearchesbefore writing outputs
fetchSearchQueriesChunkedreturns an aggregatedtotalSearches, but this call site drops it and never writes back tosearchStore.totalSearches. Fresh runs will pass0intowriteOutputs(...), so the total-searches summary can come out empty even when rows exist.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/scripts/analytics/fetch-assistant-insights.ts around lines 629 - 651, The search-query fetch flow is ignoring the aggregated totalSearches returned by fetchSearchQueriesChunked, so searchStore.totalSearches never gets updated before later output writing. Update the fetchSearchQueriesChunked call site in fetch-assistant-insights.ts to capture the returned totalSearches and assign it back onto searchStore.totalSearches before moving on to checkpointing or writeOutputs, using the existing searchStore and fetchSearchQueriesChunked symbols to locate the change..github/scripts/analytics/analytics-checkpoint.ts (1)
155-173: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winDon’t reuse
mergeSearchesfor per-chunk search aggregation
splitDateRangecreates non-overlapping windows, andfetchSearchQueriesChunkedalready sums hits across them. TheonChunkCompletepath still merges each raw chunk withmergeSearches, so the “keep the fresher row” branch can overwritehitsand drop earlier chunks for the same query. Keep that freshness-based merge for overlapping incremental re-fetches only, and use the chunked function’s merged return value for the intra-run store.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/scripts/analytics/analytics-checkpoint.ts around lines 155 - 173, The current use of mergeSearches in onChunkComplete is mixing overlapping incremental re-fetch logic with non-overlapping chunk aggregation, which can overwrite summed hits for the same query. Keep mergeSearches reserved for freshness-based incremental updates only, and switch the chunked path to use the merged return value from fetchSearchQueriesChunked for the intra-run store. Ensure splitDateRange-based chunks are accumulated by summing their results rather than applying the “keep the fresher row” branch in mergeSearches.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In @.github/scripts/analytics/analytics-checkpoint.ts:
- Around line 155-173: The current use of mergeSearches in onChunkComplete is
mixing overlapping incremental re-fetch logic with non-overlapping chunk
aggregation, which can overwrite summed hits for the same query. Keep
mergeSearches reserved for freshness-based incremental updates only, and switch
the chunked path to use the merged return value from fetchSearchQueriesChunked
for the intra-run store. Ensure splitDateRange-based chunks are accumulated by
summing their results rather than applying the “keep the fresher row” branch in
mergeSearches.
In @.github/scripts/analytics/fetch-assistant-insights.ts:
- Around line 629-651: The search-query fetch flow is ignoring the aggregated
totalSearches returned by fetchSearchQueriesChunked, so
searchStore.totalSearches never gets updated before later output writing. Update
the fetchSearchQueriesChunked call site in fetch-assistant-insights.ts to
capture the returned totalSearches and assign it back onto
searchStore.totalSearches before moving on to checkpointing or writeOutputs,
using the existing searchStore and fetchSearchQueriesChunked symbols to locate
the change.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 79486107-c4d0-4657-807c-fef8c719e323
📒 Files selected for processing (5)
.github/scripts/analytics/README.md.github/scripts/analytics/analytics-checkpoint.ts.github/scripts/analytics/analytics-writers.ts.github/scripts/analytics/fetch-assistant-insights.ts.github/scripts/analytics/mintlify-analytics.ts
Drop the redundant env doc and restore inline comments in the example file; update cross-references in AGENTS.md and script READMEs.
Keep AGENTS.md to skills plus doc links; document changelog fork, commands, and agent rules in i18n and cms READMEs.
Keep English MDX editing rules in the agent entrypoint; i18n README links back instead of duplicating them.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.github/scripts/analytics/to pull Mintlify AI assistant conversations, search terms, and user feedback into a local gitignored cache (tmp/analytics-cache/).analytics:fetch:all), checkpoint resume after 504/429/interrupt, rate limiting, custom date ranges, and--assistant-onlyfor AI Q&A-first workflows.MINTLIFY_PROJECT_ID+ optional throttle env vars to.env.local.example.Test plan
.env.local.example→.env.localand setMINTLIFY_API_KEY+MINTLIFY_PROJECT_IDpnpm analytics:fetch:dry-runand confirm date range / mode loggingpnpm analytics:fetch:assistant -- --days 7and verifytmp/analytics-cache/assistant-summary.md+by-day/outputpnpm analytics:fetch -- --resumecontinues from checkpointtmp/analytics-cache/stays gitignored