Repository navigation
Some Unicode characters(U+2714) displayed incorrectly #12537
Description
Windows Terminal version
1.12.10393.0
Windows build number
10.0.19044.0
Other Software
PowerShell 7.2.1
wsl.exe
Steps to reproduce
Using PowerShell 7.2.1 to print some Unicode characters like "✔" and the size is too small
Expected Behavior
In Fluent Terminal
Actual Behavior
In WT
Activity
elsaco commentedon Feb 21, 2022
The experimental text rendering engine does a better job:
and w/out:
j4james commentedon Feb 21, 2022
Note that
U+2714is defined in Unicode as a neutral character, which is meant to occupy a single column, whileU+274Cis defined as a wide character, and is meant to occupy two columns.As a point of reference, here's what those characters look like in a bunch of different terminals. My test case is:
printf "+-+--+\n|\u2714|\u274c|\n+-+--+\n"From left to right, top to bottom: XTerm, Gnome Terminal, MLTerm, WezTerm, Contour, St, Alacritty, and Kitty.
Gnome Terminal might be letting
U+2714overlap into the following column to a certain extent, but it's clearly still a single column character. Everyone else seems to be quiet strictly constrained to the one column. So in my opinion, the rendering in Fluent Terminal is incorrect, but it's possible that's deliberate.That said, the Atlas renderer clearly does a better job of dealing with fonts like this than the DX renderer. However, I don't think we should ever expect it to take up two columns.
lhecker commentedon Feb 23, 2022
If you look closely you can see how the glyph is cut off on both sides in the experimental rendering engine ("AtlasEngine").
I'll likely submit a PR in the future which will make the AtlasEngine look similar to our current standard renderer by downsizing glyphs larger than number of cells allocated for them (1 cell in case of ✔️).
Not because I want to make the size of Emojis/symbols inconsistent like in the screenshot, but rather because some scripts only have proportional fonts, which require to be downsized to still be readable (most noticeable with Hebrew on a standard Windows setup). Maybe I can come up with something clever to find a good middle ground...
Jaykul commentedon Aug 5, 2022
I believe there's a bug here in how terminal is handling emoji. Many emoji also exist as characters:
Anyway ... there's a lot.
The point is: Unicode has variation selectors, and they were made part of the emoji spec in 2018. Bottom line:
fe0eselects text mode, andfe0fselects emoji mode and when the selector is missing, it's up to the software to choose which way to render it.Here's the current list of all the variation sequences for 14.0.0.
I believe that Windows Terminal is doing the wrong thing here.
It appears to me that terminal is trying to render the emoji in a single character space whenever there's no variation selector. I think this should not be done -- but if you were going to do it, surely you should only do it when it's the "text" variation, and not when the variation is unspecified?!
I, for one, can live with always rendering emoji --as far as I know, no fixed-width font has characters for these anyway-- but making them small when the selector is missing is absolutely a bug.
For what it's worth, a concrete example. Here's what happens in the terminal:
When I copy/paste that from Terminal into github, Github renders it properly. The first is rendered as a character. The second is an emoji. The third one is rendered as a character in code blocks, and an emoji elsewhere (except for the face, hmmmm).
❯ "`u{23f1}`u{fe0e} `u{23f1}`u{fe0f} `u{23f1}"⏱︎ ⏱️ ⏱
❯ "`u{2714}`u{fe0e} `u{2714}`u{fe0f} `u{2714}"✔︎ ✔️ ✔
❯ "`u{2764}`u{fe0e} `u{2764}`u{fe0f} `u{2764}"❤︎ ❤️ ❤
❯ "`u{1F610}`u{fe0e} `u{1F610}`u{fe0f} `u{1F610}"😐︎ 😐️ 😐
lhecker commentedon Aug 5, 2022
BTW I did this just recently in #13549, which will be part of v1.16.
Hmm we're feeding the U+FE0E straight to Direct2D to draw it and I was hoping it would just do the correct thing automatically. I'll check with them whether there's anything we can do differently to fix that.
I believe this is because of what James Holderness (@j4james) said:
Additionally, if you notice any weird whitespace gaps around Emojis: That's because of #8000 and I'm currently working on fixing that. 🙂
Jaykul commentedon Aug 5, 2022
I suppose they are. As I noted earlier, the fonts we're all using don't have these characters in them, usually, because the characters don't fit in a single cell -- that is, they're totally indecipherable when scaled down.
I'm concerned about this:
I'm not sure about that one ... downsizing emoji to fit in a single character cell is not the right move, in my opinion. isn't that going to result in the super-tiny examples above (the third character in the each row of the screenshot)? I'm not clear, guess I'll have to wait and see -- because if the only options are "shrink" it to one character or "stretch" it to two wide, I think emoji are going to end up uniformly hideous 😉
lhecker commentedon Aug 5, 2022
The downsizing I implemented is at least somewhat better at this than our current approach.
DxEngine (current):
AtlasEngine (future):
We don't want to let glyphs overlap rows for performance and correctness reasons (at the moment, because overlapping glyphs are anything but trivial) and so shrinking large glyphs is an okay-ish and simple solution. For instance this string is about 1.5 rows tall: ⟨င်္က္ကျြွှေို့်ာှီ့ၤဲံ့းႍ⟩
Without scaling:
With scaling:
(As explained before the extra whitespace is because we assign at least 1 cell to each code point we process. Since that string has 27 code points it's also 27 cells wide. As part of #8000 this situation will be improved though.)
With scaling in general being ultimately necessary, scaling Emojis is basically just a side effect and not actually intentional. Unrelated to the U+FE0E issue, Emojis like U+2714 (❤) without variant selector are fit into a single cell however, simply because that's exactly what other terminals do too (see #2066). So at least that part won't change any time soon. There's also things like #12512 (comment) which I'll probably try to fix as part of #9999.
Jaykul commentedon Aug 8, 2022
Well, I'll have to see what I can do about making my emojis explicitly use the emoji variant. I have to say though -- it seems that if you're going to force the non-variant to a single cell, you really should force the text variant to a single cell too. That's the one where the author meant for it to be a single text character, right?
DHowett commentedon Aug 8, 2022
That the text variant takes up two cells hearkens back to an ages-old issue in the console:
FE0Fis allocated its own column, because we've never supported zero-width glyphs, and so the grapheme clusterHEART + VARIATION SELECTOR 15is allocated two columns. :)Combine that with DirectWrite's propensity for emoji-fying it even when the variation selector is present, and you get a double-width full-color heart. Bah.
j4james commentedon Aug 9, 2022
FYI, the issue of variant selectors was discussed at length in #8970. It's a more complicated problem than just supporting zero-width glyphs, because you're changing the width of the character after it has already been output, and original width can affect the layout of the page in ways that are irreversible. Last I checked, most terminals just ignored them (at least regarding the width change), which seems the most sensible thing to do.
Rewrite AtlasEngine to allow arbitrary overhangs (#14959)