Bug Description
std.base64() fails when the input string contains characters with codepoints greater than 255, such as emoji, CJK characters, and other non-Latin scripts.
Reproduction
Expected Behavior
(This is the correct base64 encoding of the UTF-8 bytes f0 9f 8e 89 for 🎉)
Actual Behavior
RUNTIME ERROR: base64 encountered invalid codepoint value in the array (must be 0 <= X <= 255), got 127881
More Examples
std.base64("你好") // fails - Chinese characters
std.base64("مرحبا") // fails - Arabic characters
std.base64("héllo") // works - all codepoints <= 255
std.base64("hello") // works - ASCII
Version
$ jsonnet --version
Jsonnet commandline interpreter (Go implementation) v0.22.0
Analysis
The implementation appears to convert the string to an array of Unicode codepoints and then try to treat each codepoint as a byte. This fails for any character with a codepoint > 255.
The correct behavior should be:
- Encode the string as UTF-8 bytes
- Base64-encode the resulting byte array
sjsonnet handles this correctly:
std.base64("🎉") // returns "8J+OiQ==" (correct)
std.base64("你好") // returns "5L2g5aW9" (correct)
Verification that the expected output is correct:
echo -n "🎉" | base64
# Output: 8J+OiQ==
Bug Description
std.base64()fails when the input string contains characters with codepoints greater than 255, such as emoji, CJK characters, and other non-Latin scripts.Reproduction
Expected Behavior
"8J+OiQ=="(This is the correct base64 encoding of the UTF-8 bytes
f0 9f 8e 89for 🎉)Actual Behavior
More Examples
Version
Analysis
The implementation appears to convert the string to an array of Unicode codepoints and then try to treat each codepoint as a byte. This fails for any character with a codepoint > 255.
The correct behavior should be:
sjsonnet handles this correctly:
Verification that the expected output is correct: