Migration Guide: Converting Lecture Repos to Use QuantEcon Actions

This guide provides step-by-step instructions for migrating a QuantEcon lecture repository to use the centralized composite actions.

Overview

Goal: Replace repetitive workflow setup code with reusable, cached composite actions.

Benefits:

⚡ 5-6 minute setup time reduction (via Conda caching)
🔧 Centralized maintenance (update once, benefit everywhere)
📊 Consistent build environments across all repos
🐛 Easier troubleshooting and updates
🎯 Unified environment setup (single action for Conda + LaTeX)

Time Required: ~1-2 hours per repository (including testing)

Prerequisites

Before starting migration:

Ensure quantecon/actions repository is published
Have a test branch ready in target repository
Know which features your repo uses (ML libs, LaTeX, etc.)

Quick Migration: Using setup-environment

Recommended approach:

Before (Manual Setup)

- name: Setup Anaconda
  uses: conda-incubator/setup-miniconda@v3
  with:
    python-version: "3.13"
    environment-file: environment.yml  # This is setup-miniconda's own input name
    activate-environment: quantecon

- name: Install LaTeX
  run: |
    sudo apt-get update
    sudo apt-get install -y texlive-latex-extra ...

After (Unified Action)

- uses: quantecon/actions/setup-environment@v1
  with:
    python-version: '3.13'
    environment: 'environment.yml'
    install-latex: 'true'
    latex-requirements-file: 'latex-requirements.txt'
    environment-name: 'quantecon'

Benefits: Simpler configuration, Conda caching (~5-6 min saved), unified setup.

Detailed Step-by-Step Migration

Step 1: Identify Your Repository Type

Determine which features your repository needs:

Repository	ML Libs (JAX/PyTorch)	LaTeX	Netlify	Notes
lecture-python.myst	✅ Yes	✅ Yes	✅ Yes	Most complex
lecture-python-programming.myst	❌ No	✅ Yes	✅ Yes	Standard
lecture-python-intro	❌ No	✅ Yes	✅ Yes	Standard
lecture-python-advanced.myst	❌ No	✅ Yes	✅ Yes	Standard

Step 2: Create Test Branch

cd /path/to/your/lecture-repo
git checkout -b migrate/composite-actions

Step 3: Backup Current Workflows

cp -r .github/workflows .github/workflows.backup

Step 4: Update `ci.yml`

Before (example from lecture-python.myst):

name: Build Project [using jupyter-book]
on: [pull_request]

jobs:
  preview:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Setup Anaconda
        uses: conda-incubator/setup-miniconda@v3
        with:
          auto-update-conda: true
          auto-activate-base: true
          miniconda-version: 'latest'
          python-version: "3.13"
          environment-file: environment.yml  # This is setup-miniconda's own input name
          activate-environment: quantecon
      
      - name: Install latex dependencies
        run: |
          sudo apt-get -qq update
          sudo apt-get install -y texlive-latex-recommended texlive-latex-extra ...
      
      # ... 40+ more lines of setup
      
      - name: Build HTML
        shell: bash -l {0}
        run: |
          jb build lectures --path-output ./ -n -W --keep-going

After:

name: Build Project [using jupyter-book]
on: [pull_request]

jobs:
  preview:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      
      # Setup environment with caching
      - uses: quantecon/actions/setup-environment@v1
        with:
          python-version: '3.13'
          environment: 'environment.yml'
          environment-name: 'quantecon'
          install-latex: 'true'
      
      # Build lectures (with cache restore for fast incremental builds)
      - uses: quantecon/actions/restore-jupyter-cache@v1
        with:
          cache-type: 'build'
      
      - uses: quantecon/actions/build-lectures@v1
        id: build
        with:
          source-dir: 'lectures'
          builder: 'html'
          upload-failure-reports: true  # Upload reports if build fails
      
      # Deploy preview
      - uses: quantecon/actions/preview-netlify@v1
        with:
          netlify-auth-token: ${{ secrets.NETLIFY_AUTH_TOKEN }}
          netlify-site-id: ${{ secrets.NETLIFY_SITE_ID }}
          build-dir: ${{ steps.build.outputs.build-path }}

Key Changes:

Replaced ~60 lines with ~10 lines
Added restore-jupyter-cache for fast PR builds
Unified error handling
Clearer intent with named actions

Step 5: Update `cache.yml`

Before:

name: Build Cache [using jupyter-book]
on:
  schedule:
    - cron: '0 3 * * 1'
  workflow_dispatch:

jobs:
  cache:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      
      - name: Setup Anaconda
        uses: conda-incubator/setup-miniconda@v3
        with:
          auto-update-conda: true
          # ... many lines
      
      - name: Build HTML
        shell: bash -l {0}
        run: |
          jb build lectures --path-output ./ -W --keep-going
      
      - name: Upload "_build" folder (cache)
        uses: actions/upload-artifact@v5
        with:
          name: build-cache
          path: _build

After (Recommended: Using build-jupyter-cache Action):

name: Build Cache
on:
  schedule:
    - cron: '0 0 * * 0'  # Weekly Sunday midnight UTC
  workflow_dispatch:      # Manual trigger
  push:
    branches:
      - main
    paths:
      - 'environment.yml'  # Auto-rebuild when env changes

jobs:
  cache:
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/quantecon/quantecon:latest
    permissions:
      contents: read
      issues: write
      packages: read
    steps:
      - uses: actions/checkout@v4
      
      - uses: quantecon/actions/build-jupyter-cache@v1
        with:
          builders: 'html'
          create-issue-on-failure: true

Key Changes:

Uses dedicated build-jupyter-cache action
Automatically handles cache key generation (build-{env-hash}-{run-id})
Creates issues on failure (with duplicate prevention)
Verifies build before saving cache
Push trigger rebuilds cache when environment.yml changes on main

Step 6: Update `publish.yml`

Before:

name: Build & Publish to GH Pages
on:
  push:
    tags:
      - 'publish*'

jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v5
      
      - name: Setup Anaconda
        uses: conda-incubator/setup-miniconda@v3
        # ... many lines
      
      - name: Build HTML
        shell: bash -l {0}
        run: |
          jb build lectures --path-output ./ -n -W --keep-going
      
      - name: Deploy to gh-pages
        uses: peaceiris/actions-gh-pages@v4
        # ...

After:

name: Build & Publish to GH Pages
on:
  push:
    tags:
      - 'publish*'

permissions:
  contents: read
  pages: write
  id-token: write

concurrency:
  group: "pages"
  cancel-in-progress: false

jobs:
  publish:
    if: github.event_name == 'push' && startsWith(github.event.ref, 'refs/tags')
    runs-on: ubuntu-latest
    environment:
      name: github-pages
      url: ${{ steps.deploy.outputs.page-url }}
    steps:
      - uses: actions/checkout@v4
      
      - uses: quantecon/actions/setup-environment@v1
        with:
          install-latex: 'true'
      
      # Build PDF
      - uses: quantecon/actions/build-lectures@v1
        with:
          builder: 'pdflatex'
          upload-failure-reports: true
      
      # Build notebooks
      - uses: quantecon/actions/build-lectures@v1
        with:
          builder: 'jupyter'
          upload-failure-reports: true
      
      # Build HTML and assemble all assets
      - uses: quantecon/actions/build-lectures@v1
        id: build
        with:
          builder: 'html'
          html-copy-pdf: true
          html-copy-notebooks: true
          upload-failure-reports: true
      
      # Deploy to GitHub Pages
      - uses: quantecon/actions/publish-gh-pages@v1
        id: deploy
        with:
          build-dir: ${{ steps.build.outputs.build-path }}
          cname: 'python.quantecon.org'  # Adjust per repo

Key Changes:

Uses native GitHub Pages deployment (requires pages: write and id-token: write permissions)
Builds PDF, notebooks, then HTML with asset assembly
html-copy-pdf and html-copy-notebooks assemble all formats into HTML folder
upload-failure-reports helps debug build failures

Step 7: Repository-Specific Adjustments

For `lecture-python.myst` (with GPU/ML libs):

# In ci.yml, cache.yml, publish.yml:
# ML packages (JAX, PyTorch, numpyro) are specified in the repo's
# environment.yml or environment-update.yml, not in the action.
- uses: quantecon/actions/setup-environment@v1
  with:
    environment-update: 'environment-update.yml'

# Also update runner if needed:
jobs:
  preview:
    runs-on: "runs-on=${{ github.run_id }}/family=g4dn.2xlarge/..."

For `collab.yml` (Google Colab container):

This workflow uses a different container and may need custom handling:

# collab.yml may need to stay as-is or use partial actions
# The Google Colab container environment is different from standard Ubuntu

# Option 1: Keep as-is for now
# Option 2: Use only build-lectures action
- uses: quantecon/actions/build-lectures@v1
  with:
    build-html: 'true'
    cache-workflow: 'cache.yml'

Step 8: Testing

Test your migrated workflows:

Create test workflow (don't modify existing yet):

cp .github/workflows/ci.yml .github/workflows/test-ci.yml

Modify test-ci.yml to use actions and manual trigger:

name: Test CI (New Actions)
on: workflow_dispatch  # Manual trigger only

Push test branch and run workflow:

git add .github/workflows/test-ci.yml
git commit -m "Add test workflow with composite actions"
git push origin migrate/composite-actions

Manually trigger the workflow from GitHub Actions UI
Compare results:
- Build time (should improve on second run)
- Build artifacts (should be identical)
- Execution reports

Step 9: Validation Checklist

Before merging, verify:

Test workflow completes successfully
Build time reduced on cached runs
HTML output identical to production build
PDF output identical (if applicable)
Notebooks identical (if applicable)
No new errors or warnings
Cache created and restored properly
All secrets still work (Netlify, GitHub tokens)

Step 10: Merge and Monitor

Update actual workflows (not test ones):

# Copy working test-ci.yml to ci.yml
cp .github/workflows/test-ci.yml .github/workflows/ci.yml
# Remove test workflow
rm .github/workflows/test-ci.yml

Create PR:

git add .github/workflows/
git commit -m "Migrate to quantecon/actions composite actions

- Adds caching for conda, pip, and LaTeX
- Reduces setup time from 8-12 min to ~1 min (cached)
- Centralizes workflow logic for easier maintenance
- Tested with manual workflow runs"

git push origin migrate/composite-actions

Review and merge
Monitor first 5 workflow runs:
- Check execution times
- Verify caching behavior
- Watch for any errors

Clean up after 1 week:

# Remove backup if everything is working
rm -rf .github/workflows.backup

Repository-Specific Notes

lecture-python.myst

Special requirements:

GPU runners: runs-on: "runs-on=${{ github.run_id }}/family=g4dn.2xlarge/..."
ML libraries: Specified in repo's environment.yml / environment-update.yml (JAX, PyTorch, numpyro)

Example ci.yml excerpt:

jobs:
  preview:
    runs-on: "runs-on=${{ github.run_id }}/family=g4dn.2xlarge/image=quantecon_ubuntu2404/disk=large"
    steps:
      - uses: actions/checkout@v4
      
      - uses: quantecon/actions/setup-environment@v1
        with:
          environment-update: 'environment-update.yml'
      
      - uses: quantecon/actions/restore-jupyter-cache@v1
        with:
          cache-type: 'build'
      
      - uses: quantecon/actions/build-lectures@v1
        id: build

lecture-python-programming.myst

Standard setup - no special requirements

Example ci.yml excerpt:

jobs:
  preview:
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/quantecon/quantecon-build:latest
    permissions:
      contents: read
      packages: read
    steps:
      - uses: actions/checkout@v4
      
      - uses: quantecon/actions/setup-environment@v1
        with:
          environment-update: 'environment-update.yml'  # Delta packages for container
      
      - uses: quantecon/actions/restore-jupyter-cache@v1
        with:
          cache-type: 'build'
      
      - uses: quantecon/actions/build-lectures@v1
        id: build

lecture-python-intro

Standard setup - no special requirements
Same as lecture-python-programming.myst

lecture-python-advanced.myst

Standard setup - no special requirements
Same as lecture-python-programming.myst

Troubleshooting

Issue: Cache not working

Symptom: Every run installs packages from scratch

Check:

Verify cache key format matches
Check runner OS (cache keys include OS)
Verify paths are correct

Solution:

# Add debug step
- name: Debug cache info
  run: |
    echo "OS: ${{ runner.os }}"
    echo "Hash: ${{ hashFiles('environment.yml') }}"

Issue: Build output differs

Symptom: HTML/PDF differs from production

Check:

Package versions (conda list, pip list)
Jupyter Book version
Build commands

Solution: Pin versions in environment.yml

Issue: Secrets not working

Symptom: Netlify/GitHub Pages deployment fails

Check:

Secret names match (case-sensitive)
Secrets are set in repository settings
Permissions are correct

Solution: Verify in repository Settings → Secrets and variables → Actions

Rollback Procedure

If issues occur after migration:

Emergency Rollback

# Restore from backup
rm -rf .github/workflows
cp -r .github/workflows.backup .github/workflows
git add .github/workflows
git commit -m "Rollback to pre-migration workflows"
git push origin main

Partial Rollback

# In affected workflow file, comment out action and restore original:

# - uses: quantecon/actions/setup-environment@v1

- name: Setup Anaconda
  uses: conda-incubator/setup-miniconda@v3
  with:
    # ... original configuration

Getting Help

If you encounter issues during migration:

Review action README files for configuration options
Open an issue in quantecon/actions with:
- Repository name
- Workflow file
- Error logs
- What you've tried

Post-Migration

After successful migration:

Document the change:
- Update repository README if needed
- Note in changelog
Share lessons learned:
- Update this guide with any issues found
- Add troubleshooting tips
Monitor metrics:
- Track build times (should improve significantly)
- Watch for any degradation
Update other repositories:
- Use this experience to migrate next repo
- Refine process each time

Migration Checklist Template

Copy this for each repository:

## Migration: [Repository Name]

### Preparation
- [ ] Review action README files
- [ ] Identify repo type (ML libs? LaTeX? etc.)
- [ ] Create test branch
- [ ] Backup workflows

### Migration
- [ ] Update ci.yml
- [ ] Update cache.yml
- [ ] Update publish.yml
- [ ] Update other workflows (linkcheck, collab, etc.)
- [ ] Adjust for repository-specific needs

### Testing
- [ ] Create test workflow
- [ ] Run test workflow successfully
- [ ] Compare build artifacts
- [ ] Verify caching works
- [ ] Run 3+ times to confirm stability

### Deployment
- [ ] Update actual workflows
- [ ] Create PR with detailed description
- [ ] Code review
- [ ] Merge to main
- [ ] Monitor first 5 runs

### Cleanup
- [ ] Remove test workflows
- [ ] Remove backups (after 1 week)
- [ ] Update documentation
- [ ] Share lessons learned

### Metrics
- Build time before: _____ min
- Build time after (first run): _____ min
- Build time after (cached): _____ min
- Issues encountered: _____
- Resolution time: _____

Uh oh!

FilesExpand file tree

MIGRATION-GUIDE.md

Latest commit

History

MIGRATION-GUIDE.md

File metadata and controls

Migration Guide: Converting Lecture Repos to Use QuantEcon Actions

Overview

Prerequisites

Quick Migration: Using setup-environment

Before (Manual Setup)

After (Unified Action)

Detailed Step-by-Step Migration

Step 1: Identify Your Repository Type

Step 2: Create Test Branch

Step 3: Backup Current Workflows

Step 4: Update ci.yml

Before (example from lecture-python.myst):

After:

Step 5: Update cache.yml

Before:

After (Recommended: Using build-jupyter-cache Action):

Step 6: Update publish.yml

Before:

After:

Step 7: Repository-Specific Adjustments

For lecture-python.myst (with GPU/ML libs):

For collab.yml (Google Colab container):

Step 8: Testing

Step 9: Validation Checklist

Step 10: Merge and Monitor

Repository-Specific Notes

lecture-python.myst

lecture-python-programming.myst

lecture-python-intro

lecture-python-advanced.myst

Troubleshooting

Issue: Cache not working

Issue: Build output differs

Issue: Secrets not working

Rollback Procedure

Emergency Rollback

Partial Rollback

Getting Help

Post-Migration

Migration Checklist Template

Step 4: Update `ci.yml`

Step 5: Update `cache.yml`

Step 6: Update `publish.yml`

For `lecture-python.myst` (with GPU/ML libs):

For `collab.yml` (Google Colab container):