Repository navigation
fix(compat): count string size by code points for multi-byte characters - #1853
Merged
Merged
Conversation
`size` returned `String#length`, i.e. the number of UTF-16 code units,
so a string containing astral characters was over-counted, e.g.
`size('😀😁😂')` returned `6` instead of lodash's `3`.
Count strings by Unicode code points, matching lodash and the existing
`truncate` handling. Non multi-byte strings keep using `length` to avoid
the allocation.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
gwennlbh
pushed a commit
to cigaleapp/cigale
that referenced
this pull request
Aug 3, 2026
##### [v1.50.0](https://github.com/toss/es-toolkit/blob/HEAD/CHANGELOG.md#Version-v1500) Released on July 24th, 2026. - Added the `es-toolkit/types` entrypoint: a declaration-only module of compile-time type utilities TypeScript lacks natively — `ValueOf`, `Simplify`, `Writable`, `NonEmptyArray`, `DeepPartial`, and `DeepReadonly`. (\[[#1818](toss/es-toolkit#1818)]) - Added `flow` to `es-toolkit/fp`, a reusable data-last composition built on `pipe`. (\[[#1812](toss/es-toolkit#1812)]) The following bring `es-toolkit/compat` closer to Lodash. Behavior for the affected edge cases now matches Lodash, so results may differ if you relied on the previous output. - Fixed `compat/intersectionBy` to dedupe a single array (including by the iteratee) and to keep results consistent with Lodash. (\[[#1935](toss/es-toolkit#1935)]) - Fixed `compat/findIndex` and `compat/findLastIndex` to coerce a `NaN` `fromIndex` to `0` and to convert `fromIndex` to an integer. (\[[#1828](toss/es-toolkit#1828)], \[[#1834](toss/es-toolkit#1834)], \[[#1938](toss/es-toolkit#1938)]) - Fixed `compat/lastIndexOf` to coerce `fromIndex` like Lodash. (\[[#1832](toss/es-toolkit#1832)]) - Fixed `compat/every` to treat a falsy, non-nullish `doesMatch` as a `_.property` shorthand instead of identity. (\[[#1940](toss/es-toolkit#1940)]) - Fixed `compat/nth` to support string inputs. (\[[#1833](toss/es-toolkit#1833)]) - Fixed `compat/inRange` to not throw on a lone negative bound. (\[[#1835](toss/es-toolkit#1835)]) - Fixed `compat/toArray` to convert `Set`s to arrays like Lodash. (\[[#1840](toss/es-toolkit#1840)]) - Fixed `compat/values` to treat sparse array holes as `undefined`. (\[[#1894](toss/es-toolkit#1894)]) - Fixed `compat/maxBy` and `compat/minBy` to restore Lodash-compatible comparison. (\[[#1893](toss/es-toolkit#1893)]) - Fixed `compat/includes` to exclude the `length` property when matching array-like values. (\[[#1886](toss/es-toolkit#1886)]) - Fixed the case functions to split ordinal numbers like Lodash, and to count string size and padding by code points for multi-byte characters. (\[[#1836](toss/es-toolkit#1836)], \[[#1852](toss/es-toolkit#1852)], \[[#1853](toss/es-toolkit#1853)]) - Fixed `compat/unset` to not treat an own literal dotted key as a deep path. (\[[#1808](toss/es-toolkit#1808)]) - Moved `compat/flattenDepth`'s flattening logic out of `flatten`. (\[[#1847](toss/es-toolkit#1847)]) - Restored the standalone `eq` and `templateSettings` entry points. (\[[#1895](toss/es-toolkit#1895)]) - Fixed `deburr` to remove all combining diacritical marks, matching Lodash. (\[[#1807](toss/es-toolkit#1807)]) - Fixed `partition` to accept any predicate return value, like `Array.prototype.filter`. (\[[#1937](toss/es-toolkit#1937)]) - Fixed `has` and `hasIn` to prioritize nullish literal keys over paths. (\[[#1810](toss/es-toolkit#1810)]) - Fixed `invokeMap` to normalize a string path via `toPath` for `this` binding. (\[[#1814](toss/es-toolkit#1814)]) - Fixed `isDeepKey` to detect deep keys with stricter patterns. (\[[#1621](toss/es-toolkit#1621)]) - Fixed `unzipWith` to return an empty array instead of throwing on empty input. (\[[#1816](toss/es-toolkit#1816)]) - Improved documentation, JSDoc accuracy, and internal refactoring across `compat` and `fp`. (\[[#1817](toss/es-toolkit#1817)], \[[#1820](toss/es-toolkit#1820)], \[[#1822](toss/es-toolkit#1822)], \[[#1823](toss/es-toolkit#1823)], \[[#1825](toss/es-toolkit#1825)], \[[#1829](toss/es-toolkit#1829)], \[[#1830](toss/es-toolkit#1830)], \[[#1842](toss/es-toolkit#1842)], \[[#1845](toss/es-toolkit#1845)], \[[#1851](toss/es-toolkit#1851)], \[[#1858](toss/es-toolkit#1858)], \[[#1867](toss/es-toolkit#1867)], \[[#1879](toss/es-toolkit#1879)]) We sincerely thank [@Antoliny0919](https://github.com/Antoliny0919), [@raon0211](https://github.com/raon0211), [@dayongkr](https://github.com/dayongkr), [@spokodev](https://github.com/spokodev), [@sarathfrancis90](https://github.com/sarathfrancis90), [@mayur-shenoy](https://github.com/mayur-shenoy), [@mahirhir](https://github.com/mahirhir), [@kojesung](https://github.com/kojesung), [@Hprogram](https://github.com/Hprogram), [@BangDori](https://github.com/BangDori), and everyone else who contributed. We appreciate your great efforts! Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
raon0211
pushed a commit
that referenced
this pull request
Aug 15, 2026
… to match lodash (#1900) compat/size used Array.from(str).length on the multibyte branch, which counts code points. lodash's size counts grapheme clusters, so combined sequences (regional-indicator flags, ZWJ emoji, Fitzpatrick skin-tone modifiers, variation selectors, combining marks) were over-counted: size('🇺🇸') returned 2 instead of 1. Port lodash's _unicodeSize helper (its reUnicode grapheme regex) and use it on the multibyte branch. Independent emoji still count as before (size('😀😁😂') === 3). This completes #1853, which moved size from code units to code points 'matching lodash' but stopped short of graphemes. Signed-off-by: Andrew Chen <48723787+chuenchen309@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
sizereturnsString#lengthfor strings — the number of UTF-16 code units — so a string with astral characters is over-counted:Count strings by Unicode code points, matching lodash and the existing
truncatehandling. Strings without multi-byte characters keep usinglengthto avoid the extra allocation.