Skip to content

Install lsql offline in CI to avoid GitHub API rate-limit 403s#4865

Merged
FastLee merged 1 commit into
mainfrom
fix/lsql-install-offline-cache
Jun 8, 2026
Merged

Install lsql offline in CI to avoid GitHub API rate-limit 403s#4865
FastLee merged 1 commit into
mainfrom
fix/lsql-install-offline-cache

Conversation

@FastLee

@FastLee FastLee commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Summary

The Install LSQL step in push.yml intermittently fails with:

Error: version: versions: refresh: github request failed: 403 Forbidden
  GET https://api.github.com/repos/databrickslabs/lsql/releases

databricks labs install fetches both the release list and the source zipball from api.github.com anonymously — the CLI's cmd/labs/github client uses http.DefaultClient with no Authorization header and reads no GITHUB_TOKEN/GH_TOKEN on any version (verified against v0.292.0 and main). So it hits GitHub's 60 req/hr unauthenticated rate limit, which is shared across GitHub Actions runner IPs.

PR #4857 added GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} to the step, but that variable is silently ignored by the CLI — it was a no-op, and the run only went green because it happened to land in a window with rate-limit budget.

Fix

Pre-fetch the release list and source archive with authenticated gh (5,000 req/hr), seed the CLI's local cache (~/.databricks/labs/lsql/cache) and lib dir (~/.databricks/labs/lsql/lib), then run databricks labs install lsql --offline so the CLI makes no GitHub API calls at all.

Test plan

  • Verified locally end-to-end: seed cache + unpack lib + install --offline produces a working lsql plugin (databricks labs installed lists lsql v0.17.0) and databricks labs lsql fmt runs with no diff.
  • CI build / fmt job passes the Install LSQL step.

This pull request and its description were written by Isaac.

The databricks CLI's 'labs install' fetches the release list and source
zipball from api.github.com anonymously (it never reads GITHUB_TOKEN), so
it routinely hits GitHub's 60 req/hr unauthenticated rate limit on shared
runner IPs and fails with '403 Forbidden'. PR #4857's GITHUB_TOKEN env was
a no-op because the CLI ignores that variable.

Pre-fetch the release list and source archive with authenticated 'gh'
(5000 req/hr), seed the CLI's local cache and lib dir, then install with
--offline so no GitHub API calls are made.

Co-authored-by: Isaac
@FastLee FastLee requested a review from a team as a code owner June 5, 2026 16:57
@FastLee FastLee temporarily deployed to account-admin June 5, 2026 16:57 — with GitHub Actions Inactive
@github-actions

github-actions Bot commented Jun 5, 2026

Copy link
Copy Markdown

✅ 2/2 passed, 20s total

Running from acceptance #9036

@codecov

codecov Bot commented Jun 5, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.79%. Comparing base (22ba51e) to head (ed30c40).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4865   +/-   ##
=======================================
  Coverage   87.79%   87.79%           
=======================================
  Files         123      123           
  Lines       17595    17595           
  Branches     3717     3717           
=======================================
  Hits        15448    15448           
  Misses       1458     1458           
  Partials      689      689           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@FastLee FastLee added this pull request to the merge queue Jun 8, 2026
Merged via the queue into main with commit 4451cbe Jun 8, 2026
9 checks passed
@FastLee FastLee deleted the fix/lsql-install-offline-cache branch June 8, 2026 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants