Repository navigation
fix(feed): bound the markdown the RSS route parses per request - #103
Conversation
The feed rendered every document in a 50-record page from markdown to HTML, synchronously, on an unauthenticated path with a 10 ms CPU budget. Measured with this plugin list on realistic prose: 4.5 ms for 2 000 characters, 32 ms for 10 000, ~3 000 ms for a full feed page. A single average post already exceeded the invocation's whole budget, so this was a 1102 on any real publication, not a slow response. The existing MAX_CONTENT_CHARS cap didn't help: it truncates at escape time, after the parse it was meant to bound. And the read cache didn't close it — readCacheKey varied on `cursor`, which the feed ignores, so any shape-valid value minted a fresh full-cost MISS for identical bytes. Now a whole-request character budget, spent newest-first, so the items a reader actually sees carry full text and the tail falls back to its excerpt. Per-item caps were the wrong shape: fifty items that each fit still terminate the Worker. The cursor now varies the key only on the path that paginates. The budget only covers short posts, so /leaving-substack no longer claims full text — the guard for that claim moves into the copy test rather than being deleted quietly. Earning it back means rendering HTML once at publish and storing it, not parsing per request.
📝 WalkthroughWalkthroughThe RSS route now enforces a shared 2,000-character markdown budget and uses descriptions after exhaustion. Cache keys scope cursor variation to paginated profile paths. Marketing copy and claim tests remove unsupported full-text RSS statements. ChangesRSS delivery behavior
Full-text claim corrections
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant RSSRoute
participant markdownBudget
participant markdownToHtml
RSSRoute->>markdownBudget: create one feed budget
RSSRoute->>markdownBudget: render newest item content
markdownBudget->>markdownToHtml: parse content within remaining capacity
markdownBudget-->>RSSRoute: HTML or empty result
RSSRoute-->>RSSRoute: use description after budget exhaustion
Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
goldroad-staging | 6ae4ae4 | Commit Preview URL Branch Preview URL |
Jul 31 2026, 10:11 PM |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/__tests__/marketing-claims.test.ts`:
- Around line 50-57: Update the correction-date annotations in the marketing
claims test entries for “Full-text RSS” and “carrying the whole post, not an
excerpt” so they use the actual correction date, which must not be later than
July 31, 2026; remove the date if the actual date is unavailable.
In `@src/lib/markdown-html.ts`:
- Around line 89-93: Update both render() and markdownToHtml() to check
markdown.length > remaining before calling trim(), returning an empty string
immediately for oversized input; preserve the existing blank-string handling and
conversion behavior for inputs within the remaining budget.
In `@src/lib/read-cache.ts`:
- Around line 64-66: Update readCacheKey() to construct the cache URL from
canonicalOrigin(url.origin) instead of url.origin, preserving the existing
pathname and URL-string return behavior so loopback development uses the request
origin while production and previews use the configured canonical origin.
- Around line 60-68: Update readCacheKey and PAGINATED_PATH_RE so cursor-bearing
cache keys use the same pathname shape accepted by the /@{$handle}/$rkey route,
including consistent handling of encoded delimiters such as /@h%2Fpost. Either
reject those paths in the cache-key regex to match routing or explicitly support
them consistently, without allowing cache behavior to diverge from route
matching.
In `@src/routes/leaving-substack.tsx`:
- Line 97: Update the RSS copy in src/routes/leaving-substack.tsx at lines 97
and 285-287 to describe a single author-scoped feed covering the author’s
publications, not a distinct feed for each publication. Replace both the “RSS
for every publication” and “A feed for every publication” claims and adjust the
accompanying per-publication description accordingly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 75e63c1f-177e-47de-ada9-362881480307
📒 Files selected for processing (7)
src/__tests__/feed-full-text.test.tssrc/__tests__/marketing-claims.test.tssrc/__tests__/read-cache.test.tssrc/lib/markdown-html.tssrc/lib/read-cache.tssrc/routes/@{$handle}.rss[.]xml.tssrc/routes/leaving-substack.tsx
| { | ||
| phrase: "Full-text RSS", | ||
| why: "the feed renders markdown at request time, and a 10 ms CPU budget only covers short posts — full text needs HTML rendered once at publish, not per request (corrected 2026-08-01)", | ||
| }, | ||
| { | ||
| phrase: "carrying the whole post, not an excerpt", | ||
| why: "same claim, and it explicitly contrasted with Substack on the one axis we could not hold (corrected 2026-08-01)", | ||
| }, |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Use a non-future correction date.
August 1, 2026 is after the review date, July 31, 2026. Replace the date with the actual correction date or remove it.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/__tests__/marketing-claims.test.ts` around lines 50 - 57, Update the
correction-date annotations in the marketing claims test entries for “Full-text
RSS” and “carrying the whole post, not an excerpt” so they use the actual
correction date, which must not be later than July 31, 2026; remove the date if
the actual date is unavailable.
| render(markdown: unknown): string { | ||
| if (typeof markdown !== "string" || markdown.trim() === "") return ""; | ||
| if (markdown.length > remaining) return ""; | ||
| remaining -= markdown.length; | ||
| return markdownToHtml(markdown); |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
# Run only in an isolated, non-privileged checkout.
pnpm gateRepository: peterkibuchi/goldroad
Length of output: 38764
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
printf '--- src/lib/markdown-html.ts ---\n'
nl -ba src/lib/markdown-html.ts | sed -n '1,140p'
printf '\n--- usages of MarkdownTokenizer/MarkdownRender/render ---\n'
rg -n "MarkdownTokenizer|MarkdownRender|render\\(" src/lib src -g '!src/lib/markdown-html.ts' || true
printf '\n--- existing tests mentioning markdown-html or tokenizer budget ---\n'
rg -n "markdown-html|MarkdownTokenizer|markdown.*trim|remaining|budget|exceeded|render" src/__tests__ src/lib src/routes src/services src/api src/shared -g '*.ts' -g '*.tsx' || trueRepository: peterkibuchi/goldroad
Length of output: 250
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- src/lib/markdown-html.ts ---'
nl -ba src/lib/markdown-html.ts | sed -n '1,140p'
printf '%s\n' ''
printf '%s\n' '--- usages of MarkdownTokenizer/MarkdownRender/render ---'
rg -n "MarkdownTokenizer|MarkdownRender|render\\(" src/lib src -g '!src/lib/markdown-html.ts' || true
printf '%s\n' ''
printf '%s\n' '--- existing tests mentioning markdown-html or tokenizer budget ---'
rg -n "markdown-html|MarkdownTokenizer|markdown.*trim|remaining|budget|exceeded|render" src/__tests__ src/lib src/routes src/services src/api src/shared -g '*.ts' -g '*.tsx' || trueRepository: peterkibuchi/goldroad
Length of output: 232
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
echo "--- src/lib/markdown-html.ts ---"
awk '{printf "%6d\t%s\n", NR, $0}' src/lib/markdown-html.ts | sed -n '1,140p'
echo
echo "--- usages of MarkdownTokenizer/MarkdownRender/render ---"
rg -n "MarkdownTokenizer|MarkdownRender|render\\(" src/lib src --glob '!src/lib/markdown-html.ts' || true
echo
echo "--- existing tests mentioning markdown-html or tokenizer budget ---"
rg -n "markdown-html|MarkdownTokenizer|markdown.*trim|remaining|budget|exceeded|render" src/__tests__ src/lib src/routes src/services src/api src/shared --glob '*.ts' --glob '*.tsx' || true
echo
echo "--- node behavior for long blank string with trim ---"
node - <<'JS'
const s = ' '.repeat(200000000);
console.log('length:', s.length);
const start = process.hrtime.bigint();
const t = s.trim();
const delta = process.hrtime.bigint() - start;
console.log('trimResultLength:', t.length, 'durationMs:', Number(delta / 1000000n));
JSRepository: peterkibuchi/goldroad
Length of output: 50377
Reject oversized markdown before calling trim().
render() receives PDS record text before applying the character budget. A very long whitespace post is already unbounded at this line, so markdown.trim() scans input that markdown.length > remaining would reject afterward. Move the length check before the blankness check in both render() and markdownToHtml().
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/lib/markdown-html.ts` around lines 89 - 93, Update both render() and
markdownToHtml() to check markdown.length > remaining before calling trim(),
returning an empty string immediately for oversized input; preserve the existing
blank-string handling and conversion behavior for inputs within the remaining
budget.
Source: Coding guidelines
| const PAGINATED_PATH_RE = /^\/@[^/]+\/?$/; | ||
|
|
||
| /** Normalized cache key: origin + pathname, plus a valid `cursor` param on the | ||
| * one path that reads it. Stripping the rest prevents cache-key pollution. */ | ||
| export function readCacheKey(request: Request): string { | ||
| const url = new URL(request.url); | ||
| const key = new URL(url.origin + url.pathname); | ||
| const cursor = url.searchParams.get("cursor"); | ||
| if (cursor && isValidCursor(cursor)) key.searchParams.set("cursor", cursor); | ||
| if (cursor && isValidCursor(cursor) && PAGINATED_PATH_RE.test(url.pathname)) { |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
node <<'NODE'
const predicate = /^\/@[^/]+\/?$/;
for (const rawPath of ["/%40h", "/@h%2Fpost"]) {
const url = new URL(`https://trygoldroad.com${rawPath}?cursor=3lyk73wxnok2f`);
console.log({
rawPath,
pathname: url.pathname,
predicateMatch: predicate.test(url.pathname),
decodedPathname: decodeURIComponent(url.pathname),
});
}
NODE
rg -n -C 4 'decodeURIComponent|decodeURI|readCacheKey|Older posts' srcRepository: peterkibuchi/goldroad
Length of output: 13457
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '--- candidate files ---\n'
fd -a '(\.html|wrangler|work\.|vite|tanstack|middleware|components\.server)' . | sed 's#^\./##' | head -200
printf '\n--- route package/runtime clues ---\n'
for f in package.json bun.lockb bun.lock package-lock.json pnpm-lock.yaml yarn.lock wrangler.toml .dev.vars .env; do
[ -e "$f" ] && printf '\n# %s\n' "$f" && sed -n '1,220p' "$f"
done
printf '\n--- all route file names ---\n'
git ls-files 'src/routes/**' | sed -n '1,220p'
printf '\n--- imports of read-cache ---\n'
rg -n "read-cache|readCacheKey|serveWithReadCache|isCacheableReadRequest" src --glob '!**/__tests__/**'
printf '\n--- read-cache implementation ---\n'
cat -n src/lib/read-cache.tsRepository: peterkibuchi/goldroad
Length of output: 250
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- candidate files ---'
fd '(\.html|wrangler|work\.|vite|tanstack|middleware|components\.server)' . | sed 's#^\./##' | head -200
printf '%s\n' ''
printf '%s\n' '--- route package/runtime clues ---'
for f in package.json bun.lockb bun.lock package-lock.json pnpm-lock.yaml yarn.lock wrangler.toml .dev.vars .env; do
if [ -e "$f" ]; then
printf '%s\n' "# $f"
sed -n '1,220p' "$f"
fi
done
printf '%s\n' ''
printf '%s\n' '--- all route file names ---'
git ls-files 'src/routes/**' | sed -n '1,220p'
printf '%s\n' ''
printf '%s\n' '--- imports of read-cache ---'
rg -n "read-cache|readCacheKey|serveWithReadCache|isCacheableReadRequest" src --glob '!**/__tests__/**'
printf '%s\n' ''
printf '%s\n' '--- read-cache implementation ---'
cat -n src/lib/read-cache.tsRepository: peterkibuchi/goldroad
Length of output: 21634
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- src/server.ts ---'
cat -n src/server.ts
printf '%s\n' ''
printf '%s\n' '--- route implementation snippets ---'
for f in 'src/routes/@{$handle}.index.tsx' 'src/routes/@{$handle}.$rkey.tsx' 'src/routes/@{$handle}.rss[.]xml.ts'; do
printf '%s\n' "# $f"
sed -n '40,140p' "$f" | cat -n -v
done
printf '%s\n' ''
printf '%s\n' '--- read-cache test normalization section ---'
sed -n '83,124p' src/__tests__/read-cache.test.ts | cat -n -vRepository: peterkibuchi/goldroad
Length of output: 15949
🌐 Web query:
TanStack React Start file route params handle query decoding percent encoded path match
💡 Result:
In TanStack Start and TanStack Router, path parameters are automatically URL-encoded and decoded during navigation and route matching [1][2]. However, the framework's approach to handling percent-encoded characters has undergone significant refinements to prevent common issues like redirect loops, "URI malformed" errors, and unintended double-decoding [3][4][5][6][7]. Key aspects of how TanStack handles these parameters include: Automatic Encoding and Decoding By default, path parameters provided via the params object in navigation functions like Link or navigate are encoded using encodeURIComponent [1][2]. When the router parses the URL, it decodes these parameters so they are available in your application as the original string [1]. Configuration of Allowed Characters If you need specific characters (such as @, +, or ;) to remain unencoded in the URL path, you can configure an allowlist in your router options [1][2]: const router = createRouter({ routeTree, pathParamsAllowedCharacters: [';', ':', '@', '&', '=', '+', '$', ','], }) Recent Improvements and Fixes The framework has addressed several edge cases related to path and query parameter encoding: 1. Preservation of Unsafe Characters: Recent updates (e.g., in decodeSegment) ensure that specific percent-encoded characters—which might be meaningful to the router or considered "unsafe" by standards—are preserved rather than aggressively decoded, which previously caused infinite redirect loops in Server-Side Rendering (SSR) [3][8][5][6]. 2. Splat Route Encoding: For wildcard (splat) routes, the router now correctly uses encodeURIComponent on individual segments to avoid issues with spaces or special characters that encodeURI might leave unencoded [5][9]. 3. Query Parameter Integrity: Previous issues with "double-decoding" of query parameters were resolved by ensuring that parameters are decoded exactly once, preventing scenarios where encoded characters (like %2540) were prematurely stripped of their encoding [7][10][11]. 4. Raw Search Option: For advanced use cases where you need to opt-out of automatic JSON parsing and decoding for search params, you can use the search: { raw: true } option [12]. Best Practices - Do not manually double-encode parameters before passing them to navigate() or Link. The router manages encoding automatically [4][1][2]. - If you encounter "URI malformed" errors with characters like % in your path, verify that you are on the latest version of TanStack Router/Start, as recent patches have significantly improved the robustness of path normalization [3][4][6]. - Use params.parse and params.stringify within your route definitions if you need custom logic to transform or validate parameters beyond basic string handling [1][2].
Citations:
- 1: https://tanstack-router-54.mintlify.app/router/concepts/path-params
- 2: https://tanstack.com/router/v1/docs/guide/path-params
- 3: fix(router-core): preserve percent-encoded URL-unsafe chars in decodeSegment TanStack/router#7695
- 4: Path params do not support
%TanStack/router#3402 - 5: SSR redirect loop for splat routes with URL-encoded characters (%20, parentheses) TanStack/router#6519
- 6: TanStack/router@9196ae4
- 7: Fix double decoding of query parameters in qss.ts TanStack/router#4300
- 8: [Start] Encoded URL params cause redirect loop on page refresh TanStack/router#4514
- 9: https://github.com/TanStack/router/blob/edf55759/packages/router-core/src/path.ts
- 10: URLs cannot safely be stored in search parameters TanStack/router#4282
- 11: Router navigation decodes URL params twice TanStack/router#4294
- 12: feat: Add flag for disabling JSON-first search param parsing TanStack/router#6000
🌐 Web query:
TanStack Router 1.170 path decodeSegment decodeURIComponent percent path params match
💡 Result:
In TanStack Router (including version 1.170+), the handling of path parameter decoding has evolved to prioritize URL safety, security, and the prevention of infinite redirect loops [1][2][3][4]. Key technical details regarding path parameter decoding, decodeSegment, and decodeURIComponent include: 1. Evolution from decodeURIComponent: While earlier versions of the router relied heavily on direct calls to native decodeURIComponent for parameter extraction [5][6], this approach caused issues with URL-unsafe characters. Unsafe characters that were accidentally decoded could lead to mismatches between the raw request URL and the router's internal path representation, triggering infinite 307 redirect loops [1][2]. 2. Improvements in decodeSegment: To resolve these issues, TanStack Router introduced a more robust decodeSegment utility [1][7]. Instead of simple, global decoding, this function is designed to: - Preserve percent-encoded characters that are considered "URL-unsafe" or meaningful to routing (e.g., control characters, spaces, and certain reserved characters) [1]. - Process adjacent percent-encoded bytes as a run to correctly handle multi-byte UTF-8 sequences (like emojis) rather than decoding each %XX group independently [1]. - Prevent the accidental decoding of characters like backslashes (%5C) or slashes (%2F) in specific contexts (like splat parameters), which ensures that path separators remain distinguishable from segment content [3][4]. 3. Path Parameter Matching: When matching routes, the router extracts raw parameter values from the URL [5][6]. By default, path parameters are automatically escaped using encodeURIComponent when navigating [8]. If you need to allow characters that are typically restricted (such as @ or +), these must be explicitly configured in your RouterOptions using the pathParamsAllowedCharacters property [8]. 4. Security Hardening: Recent versions have introduced sanitization logic to strip ASCII control characters and prevent protocol-relative URL interpretation, further hardening the router against open redirects and XSS vulnerabilities [9][7]. In summary, while decodeURIComponent remains a fundamental underlying mechanism, TanStack Router now employs specialized wrapping logic (such as decodeSegment) to selectively decode URL components, ensuring that sensitive characters are preserved and that the router's internal state remains synchronized with the browser's URL [1][3][4].
Citations:
- 1: fix(router-core): preserve percent-encoded URL-unsafe chars in decodeSegment TanStack/router#7695
- 2: Infinite redirect loop ("ERR_TOO_MANY_REDIRECTS") caused by encoded unsafe characters in URL pathname TanStack/router#7587
- 3: refactor(router-core): more stable url decoding TanStack/router#5663
- 4: fix(router-core): preserve encoded slashes in splat params TanStack/router#7893
- 5: https://github.com/TanStack/router/blob/87a51603/packages/router-core/src/new-process-route-tree.ts
- 6: TanStack/router@8578a52
- 7: fix: open redirect TanStack/router#6309
- 8: https://tanstack.com/router/latest/docs/guide/path-params
- 9: TanStack/router@e5fd28b
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- TanStack Router path decode implementation in lockfile sources ---'
tmpdir="$(mktemp -d)"
tar_file="$(sed -n '/@tanstack\/router-core@1.171.15/,+20p' pnpm-lock.yaml | grep '/node_modules/@tanstack/router-core' | sed -n '1p' | awk '{print $2}')"
printf '%s\n' "${tar_file:-NOT_FOUND}"
if [ "${tar_file#NOT_FOUND}" != "NOT_FOUND" ]; then
python3 - <<'PY' "$tar_file" "$tmpdir"
import pathlib, tarfile, sys
tar = sys.argv[1]
tmpdir = sys.argv[2]
pathlib.Path(tmpdir).mkdir()
tf = tarfile.open(tar, 'r')
for name in ('packages/router-core/src/path.ts','packages/router-core/src/utils.ts','packages/router-core/src/types.ts'):
member = next((m for m in tf.getmembers() if m.name.endswith(name)), None)
if member:
print(f'\n# {name}')
tf.extract(member, tmpdir)
out = pathlib.Path(tmpdir)/member.name
data = out.read_text(errors='replace')
for i, line in enumerate(data.splitlines(), 1):
if 'decode' in line.lower() or 'escape' in line.lower() or 'URI' in line.upper():
lo=max(1,i-8); hi=min(len(data.splitlines()), i+10)
lines=data.splitlines()
for j in range(lo,hi+1):
print(f'{j}: {lines[j-1]}')
print('---')
PY
fi
printf '%s\n' '--- package tarballs in sandbox if node_modules already exists ---'
find node_modules -path '*`@tanstack/router-core`*' -type f | sed -n '1,80p' || trueRepository: peterkibuchi/goldroad
Length of output: 230
🌐 Web query:
github TanStack router 1.171.15 router-core path.ts decodeSegment
💡 Result:
In TanStack Router version 1.171.15, the decodeSegment function is located in packages/router-core/src/utils.ts, not path.ts [1]. decodeSegment is a critical utility responsible for decoding URI path segments while maintaining security and preventing unintended behavior [1][2]. Recent versions have evolved this function to address several issues: 1. Preservation of URL-unsafe characters: To fix issues where decoded paths differed from raw request URLs—leading to infinite 307 redirect loops in SSR—decodeSegment was updated to avoid blanket use of decodeURI(). It now uses a custom implementation that preserves specific characters (such as ASCII control characters and WHATWG URL "path percent-encode set" characters like space, <, >, {, and }) [2]. 2. Security Hardening: The function is designed to handle malformed encodings gracefully and works in conjunction with sanitizePathSegment to strip ASCII control characters (such as CR, LF, and DEL), which helps prevent open redirect vulnerabilities [1][3]. 3. Integration: It is called by decodePath, which also manages complex cases like protocol-relative URL sanitization and backslash handling [1]. While packages/router-core/src/path.ts handles path interpolation and parsing (e.g., using InterpolatePathOptions and encodeParam), the actual decoding logic for segments is centralized in utils.ts [4][5][6][1]. Version 1.171.15 specifically introduced improvements to scroll restoration logic but maintained these existing security and decoding patterns in utils.ts [7][8].
Citations:
- 1: https://github.com/TanStack/router/blob/24cc08f8/packages/router-core/src/utils.ts
- 2: fix(router-core): preserve percent-encoded URL-unsafe chars in decodeSegment TanStack/router#7695
- 3: fix: open redirect TanStack/router#6309
- 4: https://github.com/TanStack/router/blob/a4154714/packages/router-core/src/path.ts
- 5: https://github.com/TanStack/router/blob/9f9f371e/packages/router-core/src/path.ts
- 6: https://github.com/TanStack/router/blob/ecbbd9a8/packages/router-core/src/path.ts
- 7: https://github.com/TanStack/router/releases/tag/@tanstack%2Frouter-core@1.171.15
- 8: fix(router-core): handle window and element scroll restoration independently TanStack/router#7807
🌐 Web query:
raw.githubusercontent.com TanStack router packages/router-core/src/utils.ts decodeSegment encodeURIComponent segment.split('%2F')
💡 Result:
In TanStack Router, the functions and logic referenced (specifically within packages/router-core/src/utils.ts) are used to handle URL path segment decoding and encoding safely [1][2]. The decodeSegment function is designed to decode percent-encoded URL segments [1][3]. It first attempts to use the built-in decodeURI function [2][4]. If decoding fails (e.g., due to malformed sequences), it employs a fallback strategy that uses a regular expression (/%[0-9A-F]{2}/gi) to identify and individually attempt to decode each percent-encoded triplet, preserving any that remain invalid [1][2]. Regarding the specific segment handling mentioned, TanStack Router processes paths by splitting them into segments [5]. When dealing with encoded paths or constructing URLs, the library ensures that forward slashes ('/') are treated as separators rather than being encoded, while other characters (such as whitespace and non-ASCII characters) are encoded appropriately using encodeURIComponent or similar logic [1][6]. Splitting segments, often represented as segment.split('%2F') in various URL processing contexts, is a technique used to handle encoded path segments correctly without incorrectly splitting on escaped forward slashes [1][2]. These utilities are part of TanStack Router's effort to provide robust SSR and client-side routing by ensuring that href values and path parameters are accurately parsed and reconstructed without the overhead or limitations of full URL object parsing [2][6]. Sources: [1] https://github.com/TanStack/router/blob/ecbbd9a8/packages/router-core/src/utils.ts [2] https://github.com/tanstack/router/blob/1afa90e8/packages/router-core/src/utils.ts [6] https://github.com/TanStack/router/blob/a4154714/packages/router-core/src/utils.ts [3] https://github.com/TanStack/router/blob/edf55759/packages/router-core/src/utils.ts [4] https://github.com/TanStack/router/blob/24cc08f8/packages/router-core/src/utils.ts [5] https://github.com/TanStack/router/blob/ecbbd9a8/packages/router-core/src/path.ts
Citations:
- 1: https://github.com/TanStack/router/blob/ecbbd9a8/packages/router-core/src/utils.ts
- 2: https://github.com/tanstack/router/blob/1afa90e8/packages/router-core/src/utils.ts
- 3: https://github.com/TanStack/router/blob/edf55759/packages/router-core/src/utils.ts
- 4: https://github.com/TanStack/router/blob/24cc08f8/packages/router-core/src/utils.ts
- 5: https://github.com/TanStack/router/blob/ecbbd9a8/packages/router-core/src/path.ts
- 6: https://github.com/TanStack/router/blob/a4154714/packages/router-core/src/utils.ts
🏁 Script executed:
#!/bin/bash
set -euo pipefail
curl -sSL 'https://raw.githubusercontent.com/TanStack/router/v1.171.15/packages/router-core/src/utils.ts' | sed -n '1,220p'
printf '%s\n' ''
curl -sSL 'https://raw.githubusercontent.com/TanStack/router/v1.171.15/packages/router-core/src/path.ts' | sed -n '1,260p'Repository: peterkibuchi/goldroad
Length of output: 188
🌐 Web query:
"decodeSegment" "TanStack Router" "encodeURI" "2F"
💡 Result:
In TanStack Router, the interaction between path segments and URL encoding—specifically regarding %2F (the encoded forward slash)—is a critical aspect of how the router manages URL parsing, SSR redirects, and route parameter matching [1][2][3]. Historically, TanStack Router used decodeURI() to decode path segments [4][5]. However, this approach was problematic because decodeURI() would decode characters that are technically unsafe in URL paths according to the WHATWG URL specification [1]. Specifically, decoding %2F into a literal / within a single route segment caused the router to misinterpret segments, leading to issues such as path mismatches and infinite 307 redirect loops during server-side rendering (SSR) [1][2]. To address these issues, TanStack Router has refined its decodeSegment utility and related path processing logic [1][2][3]: 1. Preservation of Encoded Slashes: The router now ensures that percent-encoded slashes (%2F) within route parameters (especially splat or wildcard params) remain encoded [3]. This distinguishes them from actual path separators, allowing them to be treated as part of a single path segment rather than as delimiters that would break route matching [3]. 2. Improved Decoding Strategy: Instead of relying solely on decodeURI(), recent updates have introduced a more robust decodeSegment approach that selectively preserves certain percent-encoded characters [1][2]. This includes characters defined in the WHATWG "path percent-encode set" and ASCII control characters, ensuring that these characters are not incorrectly decoded into the router's internal state [1][2]. 3. Re-encoding Mechanism: The router utilizes a sanitizePathSegment or similar re-encoding step that effectively re-encodes "unsafe" characters after an initial decode, maintaining consistency between the raw request URL and the internal path representation used by the router [1][2]. In summary, TanStack Router intentionally keeps %2F encoded within path segments to maintain structural integrity and prevent routing vulnerabilities, such as open redirects or incorrect segment splitting [1][2][5][3]. Developers working with custom path parameters or splats should expect %2F to be preserved as an encoded sequence rather than being decoded into a literal slash [3].
Citations:
- 1: fix(router-core): preserve percent-encoded URL-unsafe chars in decodeSegment TanStack/router#7695
- 2: TanStack/router@9809a06
- 3: fix(router-core): preserve encoded slashes in splat params TanStack/router#7893
- 4: https://github.com/TanStack/router/blob/24cc08f8/packages/router-core/src/utils.ts
- 5: https://github.com/TanStack/router/blob/a4154714/packages/router-core/src/utils.ts
Keep readCacheKey aligned with the requested pathname shape.
PAGINATED_PATH_RE applies to URL.pathname, so /@h%2Fpost keeps cursor-based cache keys while the /@{$handle}/$rkey route rejects that path. Decide whether encoded delimiters are a routing/security issue or whether the cache key should support them explicitly.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/lib/read-cache.ts` around lines 60 - 68, Update readCacheKey and
PAGINATED_PATH_RE so cursor-bearing cache keys use the same pathname shape
accepted by the /@{$handle}/$rkey route, including consistent handling of
encoded delimiters such as /@h%2Fpost. Either reject those paths in the
cache-key regex to match routing or explicitly support them consistently,
without allowing cache behavior to diverge from route matching.
| export function readCacheKey(request: Request): string { | ||
| const url = new URL(request.url); | ||
| const key = new URL(url.origin + url.pathname); |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
rg -n -C 4 'lib/origin|VITE_PUBLIC_ORIGIN|readCacheKey|cache\.(match|put)' srcRepository: peterkibuchi/goldroad
Length of output: 36864
🏁 Script executed:
#!/usr/bin/env bash
set -euo pipefail
sed -n '1,135p' src/lib/read-cache.ts
printf '\n--- origin helper(s) ---\n'
sed -n '1,140p' src/lib/origin.ts
printf '\n--- targeted helper names ---\n'
rg -n 'export (const|function|async function) (canonicalOrigin|isLoopbackOrigin|requestOriginToCanonical|ownOrigins|canonicalRedirect|CANONICAL_ORIGIN)' src/lib/origin.ts src/lib/read-cache.tsRepository: peterkibuchi/goldroad
Length of output: 11725
Build the stored cache key from the canonical origin.
readCacheKey() returns a URL string passed to cache.match() and cache.put(), so non-canonical request origins can populate separate cache namespaces. Build it from canonicalOrigin(url.origin), where dev loopback keeps the request origin and production/previews use CANONICAL_ORIGIN (VITE_PUBLIC_ORIGIN).
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/lib/read-cache.ts` around lines 64 - 66, Update readCacheKey() to
construct the cache URL from canonicalOrigin(url.origin) instead of url.origin,
preserving the existing pathname and URL-string return behavior so loopback
development uses the request origin while production and previews use the
configured canonical origin.
Source: Coding guidelines
| { | ||
| label: "Feeds", | ||
| goldroad: "Full-text RSS for every publication", | ||
| goldroad: "RSS for every publication, on by default", |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Do not claim a distinct feed for every publication.
The RSS route has one author-scoped URL, /@{$handle}/rss.xml. It loads the author’s document records across publications. Update this copy to describe one RSS feed for the author’s publications.
src/routes/leaving-substack.tsx#L97-L97: Replace “RSS for every publication” with an author-scoped claim.src/routes/leaving-substack.tsx#L285-L287: Replace “A feed for every publication” and the per-publication description with an author-scoped claim.
📍 Affects 1 file
src/routes/leaving-substack.tsx#L97-L97(this comment)src/routes/leaving-substack.tsx#L285-L287
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/routes/leaving-substack.tsx` at line 97, Update the RSS copy in
src/routes/leaving-substack.tsx at lines 97 and 285-287 to describe a single
author-scoped feed covering the author’s publications, not a distinct feed for
each publication. Replace both the “RSS for every publication” and “A feed for
every publication” claims and adjust the accompanying per-publication
description accordingly.
1 participant
The bug
/@handle/rss.xmlrendered every document in a 50-record page from markdown to HTML synchronously, on an unauthenticated path, inside a 10 ms CPU budget.Measured with this repo's plugin list, warm, on realistic prose (links, emphasis, headings, inline code):
A single average-length post already exceeded the whole budget. This was a hard 1102 on any real publication, not a slow feed. Cloudflare's free plan allows flexibility for infrequent overruns; a feed reader polling on a schedule is the consistent case.
Two things that looked like they bounded it, and didn't:
MAX_CONTENT_CHARSinfeed.tstruncates inxmlValue()— at escape time, after the parse it was meant to bound. It caps output size, not parse cost.cursor, which this route ignores entirely.isValidCursoris shape-only, so?cursor=<any 512 chars>minted a fresh full-cost MISS for byte-identical content. The cheap pages document that amplifier; behind the most expensive handler in the app it was the whole exploit.Changes
markdownBudget()— a whole-request character budget, spent newest-first (the route already sorts before mapping). Per-item caps are the wrong shape: fifty items that each fit still terminate the Worker. A document larger than the remaining budget is refused whole rather than truncated, because cutting markdown mid-document emits a dangling emphasis run or half a code fence, and the parse is already paid by the time you could measure output.description, which is the existing bounded excerpt — so a feed reader still shows something for every post.readCacheKeykeepscursoronly on a bare/@handle, the one path with an "Older posts" link.feed-full-text.test.tshad a test named for bounding hostile input that asserted only that the output serializer truncates —item()takes already-rendered HTML, so the unbounded step was never exercised and the suite read as covered. Renamed to what it actually proves, and the parse-side bound now has its own tests.The claim this costs us
/leaving-substacksaid "Full-text RSS for every publication" and "carrying the whole post, not an excerpt", in a row explicitly contrasting Substack's excerpts. Under this budget that is true only for short posts, so the copy now says what the feed does. Both phrasings are added to the marketing-claims guard rather than just deleted, so they cannot drift back before they are earned.Earning them back means not parsing at request time at all: render the HTML once when a post is published (the editor already has the blocks, so this can cost the server nothing) and store it. That is a schema change and a publish-path change, so it is deliberately not in this PR.
Verification
pnpm gateexits 0 — 1,609 tests.The CPU figures above are my own measurements against the repo's installed dependency versions, with the processor frozen and the JIT warmed; an earlier 20-iteration run overstated every number by roughly 5×, which is why the table is from 200-iteration runs. The shipped budget is pinned by a test in both directions, since it is a number derived from a measurement and "let's serve a bit more" would silently reintroduce the 1102.
Summary by CodeRabbit