fix: detect every binding in export var/let/const declarations - #199
Conversation
58694b5 to
94d3cf4
Compare
|
Really nice to finally get every binding name out of these declarations, and the destructuring walk in My one substantive concern is export const f = () => { return /,/g }, b = 2; // → throws "Parse error"
The deeper point is that
Two ways to share the real machinery, in order of preference: 1. Ride the main loop. The only thing the binding scan genuinely needs that the tokenizer doesn't already give us is "is this 2. Minimal: factor just the decision. Pull Either way the The genuinely new logic that has to stay in the export path regardless: ASI termination (line break after a value), since the main loop doesn't care about statement boundaries, and the binding-target/pattern reading. Both are fine as-is. |
94d3cf4 to
4c9faa4
Compare
4c9faa4 to
e21a484
Compare
|
I updated the code to use as much of the main loop as possible. I tried to make sure the performance stays the same while being able to parse more and that was the reason I pulled out |
A multi-declarator export reported only the first binding:
`export const a = 1, b = 2` yielded `a`, never `b` - the initializer scanner
stopped at the first declarator instead of skipping each initializer to its
separating comma.
skipExpression now rides consumeToken, the one tokenizer the main loop uses, so
its regex, keyword and import resolution cannot diverge from the main scan. That
also fixes four cases a separate, simplified scanner got wrong:
1. A '/' after a value keyword (return/typeof/yield/void) opens a regex, so the
comma in `export const f = () => { return /,/g }, b = 2` no longer truncates
the list.
2. A '}' closing a block leaves the next '/' in statement position (a regex).
3. import() in an initializer or a destructuring default is detected.
4. A numeric or BigInt property key such as `{ 0.5: b }` is consumed whole.
a5e5a16 to
9c25cc0
Compare
…2.3.0 in /studio [skip ci] Bumps [es-module-lexer](https://github.com/guybedford/es-module-lexer) from 2.1.0 to 2.3.0. Release notes *Sourced from [es-module-lexer's releases](https://github.com/guybedford/es-module-lexer/releases).* > 2.2.0 > ----- > > What's Changed > -------------- > > * build(deps-dev): update dev dependencies to latest by [`@BridgeAR`](https://github.com/BridgeAR) in [guybedford/es-module-lexer#200](https://redirect.github.com/guybedford/es-module-lexer/pull/200) > * fix: report n for no-substitution template dynamic imports by [`@BridgeAR`](https://github.com/BridgeAR) in [guybedford/es-module-lexer#201](https://redirect.github.com/guybedford/es-module-lexer/pull/201) > * Automatic asm.js dictionary extraction and clang toolchain update by [`@guybedford`](https://github.com/guybedford) in [guybedford/es-module-lexer#203](https://redirect.github.com/guybedford/es-module-lexer/pull/203) > * fix: don't treat a method named `import` with 2+ args as a dynamic import by [`@soberm`](https://github.com/soberm) in [guybedford/es-module-lexer#207](https://redirect.github.com/guybedford/es-module-lexer/pull/207) > * fix: detect every binding in export var/let/const declarations by [`@BridgeAR`](https://github.com/BridgeAR) in [guybedford/es-module-lexer#199](https://redirect.github.com/guybedford/es-module-lexer/pull/199) > * feat: expose export statement start position as ss by [`@BridgeAR`](https://github.com/BridgeAR) in [guybedford/es-module-lexer#206](https://redirect.github.com/guybedford/es-module-lexer/pull/206) > > New Contributors > ---------------- > > * [`@BridgeAR`](https://github.com/BridgeAR) made their first contribution in [guybedford/es-module-lexer#200](https://redirect.github.com/guybedford/es-module-lexer/pull/200) > * [`@soberm`](https://github.com/soberm) made their first contribution in [guybedford/es-module-lexer#207](https://redirect.github.com/guybedford/es-module-lexer/pull/207) > > **Full Changelog**: <guybedford/es-module-lexer@2.1.0...2.2.0> Commits * [`dbac1c3`](guybedford/es-module-lexer@dbac1c3) 2.3.0 * [`f3c972d`](guybedford/es-module-lexer@f3c972d) fix: allow wasm memory growth in the minimal build, document it in the readme... * [`e9b611e`](guybedford/es-module-lexer@e9b611e) fix: allow wasm memory growth for sources over ~4MB ([#217](https://redirect.github.com/guybedford/es-module-lexer/issues/217)) * [`6c90425`](guybedford/es-module-lexer@6c90425) feat: add minimal build for es-module-shims (es-module-lexer/minimal) ([#211](https://redirect.github.com/guybedford/es-module-lexer/issues/211)) * [`70010fb`](guybedford/es-module-lexer@70010fb) docs: correct stale limitations and gzip size in README ([#212](https://redirect.github.com/guybedford/es-module-lexer/issues/212)) * [`1f19494`](guybedford/es-module-lexer@1f19494) 2.2.0 * [`d61e97a`](guybedford/es-module-lexer@d61e97a) feat: expose export statement start position as ss ([#206](https://redirect.github.com/guybedford/es-module-lexer/issues/206)) * [`064d0db`](guybedford/es-module-lexer@064d0db) fix: detect every binding in export var/let/const declarations ([#199](https://redirect.github.com/guybedford/es-module-lexer/issues/199)) * [`99b5226`](guybedford/es-module-lexer@99b5226) fix: don't treat a method named `import` with 2+ args as a dynamic import ([#207](https://redirect.github.com/guybedford/es-module-lexer/issues/207)) * [`fa1ade6`](guybedford/es-module-lexer@fa1ade6) build: update to upstream emsdk toolchain and derive asm.js dictionary from t... * Additional commits viewable in [compare view](guybedford/es-module-lexer@2.1.0...2.3.0) [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
export var/let/constdeclarations reported only their first binding. The declaration scanner stopped at the first= initializer, so every later declarator was dropped (export const a = 1, b = 2reported only a), and the destructuring path reported the property key rather than the bound name (export const { a: b } = xreported a), missing defaults, nested patterns and rest elements.The scanner now walks the whole binding list: it reads each bound name and skips every initializer and default value expression-aware, so a comma inside one no longer ends the list, and the list terminates at
;, EOF or an ASI line break (never reading the next statement as a binding).That expression skip shares the main tokenizer (a single
consumeToken), so its regex / keyword / import() resolution cannot diverge from the main scan.Fixes #176
Fixes #175