Skip to content

Add a means by which UIA clients can determine whether a build of Windows Terminal does or doesn't support notifications #13601

Open

Description

Description of the new feature/enhancement

#12358 adds support for UIA notifications, allowing screen readers and other UIA clients to easily determine exactly what new text to report without doing any of the heavy lifting themselves (diffing, etc..). However, as not all terminals support notifications, UIA clients need a way to determine when this support is available and when they should fall back to performing diffing themseles. (if clients diff even when notifications are available, we risk doing extra work and/or double-reporting incoming text).

Proposed technical implementation details (optional)

  • In Use UIA notifications for text output #12358 and over email at various times, we discussed using a different class name (TermCtrl2) for situations when notifications are available. NVDA supports this new ID today, but there were concerns from JAWS.
  • We also discussed sending an "API version" of sorts through a custom property. This is necessary since wt and conhost versions aren't necessarily tied to particular Windows releases/aren't reliable for feature detection.

Activity

  1. added
    Issue-FeatureComplex enough to require an in depth planning process and actual budgeted, scheduled work.
    on Jul 26, 2022
  2. ghost added
    Needs-Tag-FixDoesn't match tag requirements
    on Jul 26, 2022
  3. ghost added
    Needs-TriageIt's a new issue that the core contributor team needs to triage at the next triage meeting
    on Jul 26, 2022
  4. codeofdusk commented on Jul 26, 2022

    @codeofdusk
    ContributorAuthor
  5. LeonarddeR commented on Jul 26, 2022

    @LeonarddeR

    It could also be considered to add an automationID (it is currently empty)

  6. codeofdusk commented on Jul 26, 2022

    @codeofdusk
    ContributorAuthor

    It could also be considered to add an automationID (it is currently empty)

    Leonard de Ruijter (@LeonarddeR) That idea was rejected during initial implementation.

  7. carlos-zamora commented on Jul 27, 2022

    @carlos-zamora

    Chatted with Bill Dengler (@codeofdusk). This is a feature/task. We'll chat more about it at sync but the short version is this:

    • In a world where Terminal doesn't send notifications for text output...
      • NVDA reads the text buffer, performs a diff on what was stored, and reads out what changed.
      • Diffing is (unsurprisingly) pretty expensive! This is a common practice among screen readers.
    • Today, Terminal sends notifications for text output to tell the screen reader exactly what is relevant...
      • NVDA would normally read these notifications, resulting in a "local echo" where it would read the notification and the result of the diff. Not good! So NVDA straight up ignores those notifications.
      • That said, these notifications uncovered a massive opportunity for screen readers! What if they just trusted the notification instead of diffing themselves? That would be a huge performance enhancement!
    • The Problem: NVDA needs some kind of check to know if Terminal will send out these notifications. If that check succeeds, then they know they can trust our notifications, and they can avoid diffing altogether.
      • NOTE: NVDA already does these "checks" on UIA. There's 3 "API Levels":
        • 0 --> "GetVisibleRange" returns one range per line of text --> initial UIA implementation (pre-UIA provider refactor)
        • 1 --> "GetVisibleRange" returns one range total --> after my UIA provider refactor
        • 2 --> text formatting (i.e. color/font) is exposed --> most recent UIA implementation
  8. added
    Issue-TaskIt's a feature request, but it doesn't really need a major design.
    and removed
    Issue-FeatureComplex enough to require an in depth planning process and actual budgeted, scheduled work.
    Needs-TriageIt's a new issue that the core contributor team needs to triage at the next triage meeting
    Needs-DiscussionSomething that requires a team discussion before we can proceed
    on Aug 15, 2022
  9. ghost removed
    Needs-Tag-FixDoesn't match tag requirements
    on Aug 15, 2022
  10. zadjii-msft commented on Aug 15, 2022

    @zadjii-msft
    Member

    discussion: sure let's do it

  11. carlos-zamora commented on Aug 16, 2022

    @carlos-zamora

    Bill Dengler (@codeofdusk) here's an idea actually. So, all versions of Windows Terminal in active support going back to v1.12 send notifications. At this point, why not just assume that if NVDA is connected to Windows Terminal, it has support for notifications?

    The key part here is "connected to Windows Terminal" because Conhost won't support notifications. So if the class name is TermControl, you know you're connected to Windows Terminal, and by extension, you know we support notifications.

    CC Dustin L. Howett (@DHowett)

  12. DHowett commented on Aug 16, 2022

    @DHowett
    Member

    Conhost won't support notifications

    Until SV2

  13. carlos-zamora commented on Aug 16, 2022

    @carlos-zamora

    Conhost won't support notifications

    Until SV2

    Nah. Conhost doesn't use the UiaRenderer. So the whole notifications thing doesn't affect Conhost. It operates the same way it always has (I believe it's attached to the Write API but don't quote me on that one, I'd have to look into it).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Area-AccessibilityIssues related to accessibilityArea-TerminalControlIssues pertaining to the terminal control (input, selection, keybindings, mouse interaction, etc.)Issue-TaskIt's a feature request, but it doesn't really need a major design.Product-TerminalThe new Windows Terminal.

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions