grammars prefetch: retry transient GitHub download failures (CI failed on 504) #131

mcp-index grammars prefetch (mcp_server/utils/grammar_cache.py) makes a single attempt at tree-sitter-language-pack's download: the manifest parsers.json and the parsers-<platform>.tar.zst bundle, both served from GitHub release assets. On 2026-11-09, around 06:04–06:08 UTC, GitHub returned HTTP 504 for those assets. Because prefetch has no retry, CI failed twice on #130:

• the Docker build (Alpha Gate - Docker Build And Smoke);
• the Python 3.14 agent gate, at its "Prefetch tree-sitter grammars" step.

The same thing will happen in Release Automation, in image builds and in rollouts on any transient GitHub error. Since #116, prefetch is mandatory, so every build that needs grammars depends on it.

Fix

• Retry transient download failures (HTTP 5xx and 429, connection reset, timeout) with bounded exponential backoff and jitter, for example 5 attempts over about 2 minutes. Never retry 4xx such as 404, or a checksum mismatch.
• Log each retry with the URL and status, never headers or tokens, and fail with a clear message after the last attempt.
• Optionally, let CI and image builds restore a cached grammar bundle keyed by the language-pack version, so a GitHub outage doesn't block them.

Acceptance

• Unit tests with a fake downloader: a 504, then success, passes. A persistent 504 fails after N attempts within the time bound. A 404 fails immediately.
• CI, Release Automation and the Dockerfiles use the retrying prefetch.