Skip to content

fix(plugin): resolve skills against the installed package - #1971

Closed
dayongkr wants to merge 6 commits into
toss:mainfrom
dayongkr:fix/plugin-skills-installed-package
Closed

dayongkr wants to merge 6 commits into
toss:mainfrom
dayongkr:fix/plugin-skills-installed-package

Conversation

@dayongkr

@dayongkr dayongkr commented Jul 29, 2026 •

Copy link
Copy Markdown
Collaborator

Why

The skills read src/ and docs/ paths that only exist in this repository, not where the plugin runs. Each skill carried a docs -> ../../../docs symlink to compensate, which installers that copy a single skill directory leave dangling (#1932) — and which dragged 12MB / 2,507 files behind a 16KB skill.

Measured against a real consumer project, a small model (Haiku 4.5) produced imports from the wrong entry point for 7 of 15 lodash functions, reproducing 21/21 across repeated runs. Each one type-checks and then resolves to undefined at runtime. After the rewrite: 0 of 15.

What changed

Resolution instead of recall. Entry points are resolved by importing them at runtime; signatures come from the installed package's .d.ts. Nothing is hardcoded, so answers track the version the user has. This also removes the reason the symlinks existed.

compat is the default for migrations. The skill previously preferred the strict API, inverting the path documented in docs/compat/intro.md — swap the import to es-toolkit/compat and leave call sites alone, then move to es-toolkit gradually. Moving to strict changes behavior, so it is now a separate stage rather than something bundled into the import swap.

Size estimation without installing anything. Adding esbuild to someone's service project to report a number is too invasive. The estimator scans the source for lodash usage — including single-function packages like lodash.orderby — and walks the already-installed packages using only Node built-ins. It prints AS-IS / TO-BE plus what moving to strict would additionally save.

Dropped the guide skill. Install commands and import style are things models already answer correctly, and its doc links 404'd.

Fixed doc URLs. The site 404s without .html, and compat pages live under /compat/reference/.

Validation

Ran against eight installed environments (pnpm workspaces, Yarn PnP, npm) rather than one synthetic fixture. Everything non-trivial broke, and each failure is fixed and re-verified:

before after
es-toolkit < 1.49.0 ERR_PACKAGE_PATH_NOT_EXPORTED, whole check dead works; absent entry points reported
Yarn PnP raw ENOENT stack trace resolves into .yarn/cache/*.zip
pnpm monorepo root nothing resolves tells you to run inside the workspace package
stray ~/node_modules/lodash confident 145KB baseline for projects with no lodash warns that resolution left the repo
strict/compat picker matched dist/fp/... as strict 148 files → 6 files, -(-9)% → -94%
dependency walker followed imports inside JSDoc @example comments (112 dist files have one) comments stripped

The usage scanner was checked against a manual grep across six pnpm services — 31 import sites, no false positives or negatives.

For reviewers

Byte counts are labelled raw source, not shipped size. Against real esbuild bundles the percentages hold within ~5pp, but absolute numbers run 2–5x high, so the skill says to quote the percentage and never the bytes.

A bare import { x } from 'lodash' resolving to lodash's 546KB bundle is correct, not a bug — CJS lodash does not tree-shake, and esbuild pulls 226KB raw for three functions versus 43KB for deep imports. The entry points are printed so this is visible rather than surprising.

Worth a look: dropping guide is a judgement call, and the failure-rate measurement is one small model over 15 functions.

The skills pointed at repository paths (`src/`, `docs/`) that do not exist
where the plugin actually runs. Each skill carried a `docs -> ../../../docs`
symlink to compensate, which installers that copy a single skill directory
leave dangling — reported in toss#1932. That symlink also pulled 12MB / 2,507
files behind a 16KB skill.

Measured against the installed package instead, a small model produced
imports from the wrong entry point in 7 of 15 lodash functions, reproducing
21/21 across repeated runs. Each one type-checks and resolves to `undefined`
at runtime. `get`, `castArray`, `defaultsDeep` and `toArray` are compat-only;
`pipe` is fp-only; `chain`, `tap` and `sortedUniq` exist nowhere.

So both skills now resolve entry points by importing them at runtime and
read signatures from `node_modules/es-toolkit/dist/**/*.d.ts`. The check
lists every entry point a function appears in, since `map` is in both `fp`
and `compat` while `chunk` is in all three. Re-running the same 15 probes
against the rewritten skills leaves 0 wrong imports.

Also:
- Drop the `guide` skill. Its topics (install commands, import style) are
  ones models already answer correctly, and its doc links 404'd.
- Stop constructing doc URLs without the `.html` suffix, which the site
  404s on, and correct the compat path to `/compat/reference/`.
- Treat `fp`, `server` and `types` as first-class; use `compat` only with
  a stated reason, since it is feature-complete and takes no new functions.
- Offer bundle-size measurement after a migration, on request only, and
  require stating the baseline: the same migration measures -97% against
  CJS `lodash` but -75% against `lodash-es`.
@vercel

vercel Bot commented Jul 29, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
es-toolkit Ready Ready Preview Jul 30, 2026 2:31am

Request Review

If the project marks dependencies as external — library builds, SSR bundles,
an explicit `rollupOptions.external` — neither lodash nor es-toolkit lands in
the output, so a before/after comparison shows no change or an apparent
regression. The skill now tells users to drop that setting for the test build,
and frames the built-in script as an estimate: the number that matters comes
from re-running the project's own production build.

Also records that summing file sizes on disk understates the gain (77% vs the
97% a bundled measurement showed), names the model the entry-point failure
rate was measured on, and lists `server` and `types` alongside the other
entry points.
dayongkr added 3 commits July 30, 2026 10:40
…ler to measure

The skill told migrations to prefer the strict API, which inverts the
documented path in docs/compat/intro.md: swap the import to
`es-toolkit/compat` first and leave call sites alone, then move to
`es-toolkit` gradually. Moving straight to strict changes behavior in the
same pass as the import swap, so the two are now separate stages.

Measurement no longer installs esbuild. Adding a dependency and a lockfile
entry to someone's service project to report a number is too invasive, so the
estimate now walks the already-installed lodash and es-toolkit files with Node
built-ins, following each function's requires/imports. It reports AS-IS/TO-BE
plus what moving on to strict would additionally save, and names the functions
that must stay on compat.

The trade-off is stated in the skill: this undercounts, because it measures
unminified bytes and ignores tree shaking (68% where a bundled measurement of
the same code showed 96%), and it cannot distinguish `lodash` from `lodash-es`.
For the number that matters the skill points at the project's own production
build, with a warning that external-ised dependencies make that comparison
meaningless until the setting is dropped.

Also drops hardcoded function counts, which go stale every release, and adds a
closing summary written for someone who has never used es-toolkit: what
changed, how much smaller, what moving to strict would improve, where to read.
Validated the scripts against eight installed environments (pnpm workspaces,
Yarn PnP, npm) instead of only the synthetic one they were written against.
They broke in every non-trivial case:

- A bare `Promise.all` over the entry points threw
  ERR_PACKAGE_PATH_NOT_EXPORTED on any install older than 1.49.0, since `fp`
  landed in 1.49.0 and `server` in 1.47.0. Each import is now tolerated
  individually and absent entry points are reported.
- "Run from the project root" is wrong for monorepos — nothing resolves at a
  pnpm workspace root. It now says to run inside the package that depends on
  es-toolkit, and to use `yarn node` where a `.pnp.cjs` exists.
- The hardcoded `node_modules/es-toolkit/dist` path crashed with a raw ENOENT
  stack trace under PnP and when es-toolkit was absent. The dist directory is
  now derived from `require.resolve`, which resolves into PnP's zips (verified
  reading `.yarn/cache/es-toolkit-npm-1.39.8-*.zip/node_modules/es-toolkit/dist`).
- Worst case: `require.resolve('lodash/orderBy.js')` walked up to a stray
  `~/node_modules/lodash` and reported a confident 145KB baseline for projects
  with no lodash dependency at all. Resolution outside the project is now
  detected and warned about, and a zero baseline exits instead of printing NaN.
- The strict/compat picker matched `dist/fp/...` as "strict", and the
  dependency walker followed `es-toolkit/*` imports that appear inside JSDoc
  `@example` comments — 112 dist files carry one — dragging in the whole fp
  barrel. Comments are stripped and `fp` is excluded from the strict pick.
  One case went from 148 files / -(-9)% to 6 files / -94%.

Also notes that single-function packages (`lodash.orderby`, `lodash.omit`)
migrate the same way; in the codebases checked they were the only form of
lodash usage present.
… list

The estimator required hand-editing a function list, which meant it measured
whatever the caller happened to type rather than what the project uses. It now
scans the source directories for lodash imports — including single-function
packages like `lodash.orderby`, which in the codebases checked were the only
form of lodash usage present — and reports which forms it could not follow
(default, namespace, re-export, dynamic import) instead of dropping them
silently.

Validated against six pnpm workspace services: the scanner matched a manual
grep exactly, with no false positives or negatives across 31 import sites.

Two correctness fixes that invalidated every number before this:

- Resolution was anchored to the script's own location, so running it from a
  directory with its own node_modules measured that copy instead of the target
  project. Six different services reported byte-identical results. It now
  anchors to the working directory, and `es-toolkit/compat` is imported through
  a resolved file URL so it follows the same anchor.
- "Outside this project" was judged against the cwd, which false-positives on
  every pnpm store path. It now resolves the repository root first, which still
  catches a stray `~/node_modules/lodash` while accepting `.pnpm` paths.

Byte counts are labelled as raw source, not shipped size. Checked against real
esbuild bundles the percentages hold within ~5pp, but the absolute numbers run
2-5x high, so the skill now says to quote the percentage and never the bytes.
A bare `import { x } from 'lodash'` legitimately resolves to lodash's 546KB
bundle — CJS lodash does not tree-shake — so the entry points are printed to
make that visible rather than surprising.
Summing source bytes was chosen to avoid installing a bundler, but it only
gets the percentage right — absolute numbers ran several times high because
they count unminified source and ignore tree shaking.

It turns out installing was never necessary: app projects generally already
carry esbuild as a transitive dependency (Vite, Vitest and friends). Checked
across the services on hand, every one resolved esbuild 0.25.12. So the
estimator now bundles through whatever esbuild is present and reports real
minified and gzipped sizes, in ~700ms for three functions. Nothing is
installed and no lockfile is touched.

Where esbuild is genuinely absent it falls back to the source-byte walk and
labels the output accordingly, so the two paths never get confused. On one
service the fallback read -73% against the bundled -69%, which is the ~5pp
band the skill claims.

Also stops suggesting the user re-run their own production build for a better
figure — it takes far longer than this and the ratio barely moves. The
external-dependency warning is kept for anyone who compares builds anyway,
since that comparison is meaningless while lodash and es-toolkit are both
externalised out of the output.

This branch was successfully deployed

1 active deployment
Preview — 171d8e46 Deployed Jul 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant