Skip to content

Skill discovery follows symlinks into large directories, causing 120s+ startup on slow filesystems #27027

Description

@taozy1020

Description

External skill discovery uses Bun.Glob("skills/**/SKILL.md") with followSymlinks: true. If any skill directory contains a subdirectory symlink that points to a large tree, opencode walks the entire target on every cold start.

On my setup (home directory on NFS, a couple of skills with references/ symlinks pointing to a shared docs tree) the /agent endpoint took ~122 s every cold start. After moving those skills out of the way, it dropped to ~0.6 s — same opencode binary, same machine, same network.

This should hit anyone with a slow or network filesystem (NFS, WSL /mnt, SMB, sshfs) once a skill directory ends up containing a symlink to something non-trivial.

Numbers from my machine

Scenario /agent time
Skills containing symlinks to large external trees 122.43 s
Same setup, ~/.agents/skills renamed away 0.58 s
Warm cache 0.014 s

A few things I checked while bisecting:

  • opencode --help / --version: ~1.2 s — binary load is fine
  • opencode debug startup: ~1.1 s
  • Provider discovery on a cold models.json cache: ~10–19 s (already cached after first run, unrelated to this issue)
  • HEAD to models.dev / opencode.ai / registry.npmjs.org: ~0.1 s each — network is fine
  • /agent has been slow on every cold session I have logs for, roughly 60–110 s historically

find -L <skill> counts on the skills I had installed:

skill-a   ~9890 files (find timed out at 30 s)
skill-b     750 files
skill-c     753 files
others      1–21 files

The three large ones each contain a references/<name> subdirectory symlink pointing to a shared docs/source tree. The SKILL.md files themselves are 10–20 KB — the cost is entirely in the recursive walk through the symlink.

Root Cause

packages/opencode/src/skill/skill.ts:

const EXTERNAL_SKILL_GLOB = new Bun.Glob("skills/**/SKILL.md")

EXTERNAL_SKILL_GLOB.scan({
  cwd: root,
  absolute: true,
  onlyFiles: true,
  followSymlinks: true,
  dot: true,
})

Three things combine here:

  • ** makes Bun.Glob recurse to any depth looking for SKILL.md
  • followSymlinks: true makes the walker follow every directory symlink
  • dot: true means dotfile-named symlinks aren't excluded either

So every file under every directory or symlink target inside each skill gets stated. On a local FS a stat is ~0.001 ms and nobody notices. On NFS each one is 1–50 ms, so a 10k-file tree turns into anywhere from 10 s to several minutes.

The docs at https://opencode.ai/docs/skills describe the layout as ~/.agents/skills/*/SKILL.md (a single level), so the current ** glob is also broader than what's documented.

Expected behavior

Skill discovery should be roughly O(number of skills), not O(transitive file count under each skill). A skill containing a symlink to a big tree shouldn't multiply cold-start time by 200x.

Possible fixes

A few options, roughly in order of how invasive they are:

  • Drop followSymlinks: true. One-line change. SKILL.md files are still discovered, only directory symlinks aren't recursed into. Skills that keep reference material under a symlink can still read it at runtime — discovery just doesn't need to.
  • Use skills/*/SKILL.md (single level). One-line change, matches the docs, but breaks anyone relying on nested layouts.
  • realpath the target and only follow if it stays under the parent skill directory. A handful of lines. Keeps nested layouts and intra-skill symlinks working, only blocks "skill references a giant external tree".

Option 1 or 3 by itself would fully resolve what I'm seeing, with minimal risk. Happy to send a PR once someone confirms which direction is preferred.

A few things I checked along the way that turned out not to matter: bun installing on NFS (the binary itself loads fine), models.dev provider discovery (different code path, cached after the first run), and plain network latency (the relevant HTTPS endpoints all respond in ~100 ms).

Plugins

No

OpenCode version

1.14.48

Steps to reproduce

# 1. Pick or create a directory with many files. Any cloned repo works.
LARGE_DIR=$(mktemp -d)
git clone --depth 1 https://github.com/nodejs/node "$LARGE_DIR/big" >/dev/null 2>&1
# ~50k files; substitute any tree you have handy.

# 2. Create a skill that symlinks into it.
mkdir -p ~/.agents/skills/demo-slow-skill
cat > ~/.agents/skills/demo-slow-skill/SKILL.md <<'EOF'
---
name: demo-slow-skill
description: Reproducer for slow skill discovery via symlinks
---
EOF
mkdir ~/.agents/skills/demo-slow-skill/references
ln -s "$LARGE_DIR/big" ~/.agents/skills/demo-slow-skill/references/big

# 3. Time the /agent endpoint.
opencode serve --hostname 127.0.0.1 --port 8765 &
PID=$!
sleep 3
time curl -s -o /dev/null http://127.0.0.1:8765/agent
kill $PID

# 4. Disable the skill and try again.
mv ~/.agents/skills ~/.agents/skills.disabled
opencode serve --hostname 127.0.0.1 --port 8765 &
PID=$!
sleep 3
time curl -s -o /dev/null http://127.0.0.1:8765/agent
kill $PID
mv ~/.agents/skills.disabled ~/.agents/skills

Slowdown scales with (files in symlink target) × (per-stat latency). On a fast local FS the difference is small; on NFS / WSL / SMB it is dramatic.

Screenshot and/or share link

No response

Operating System

Linux x86_64, kernel 4.18

Terminal

Mate Terminal

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions