Interactivity bugs post-#9820

This is a tracking thread for things I probably broke in #9820. I'd rather not file 100 bugs for these

(A) Mousedown, select, SCROLL does a weird thing with endpoints that doesn't happen in stable
- fixed in 4f4df01
- fixed in First three interactivity fixes #9980
(B) Trackpad scrolling down with small increments seems buggy. But scrolling with a big increment goes fast? And scrolling with the mouse is fine. And trackpad scrolling in mouse mode is fine.
- fixed in First three interactivity fixes #9980
(C) Looks like there's a selection issue when you click and drag too quickly. Seems like the start endpoint doesn't always get set.
- fixed in 9d33e89
- fixed in First three interactivity fixes #9980
(D) Mysterious DX crash - this never reprod, so we're assuming it's unrelated

Mysterious DX crash

I experienced a crash when I killed DWM, and it took particularly long to recover

crash was on:

            RETURN_IF_FAILED(_dxgiFactory2->CreateSwapChainForComposition(_d3dDevice.Get(),
                                                                          &_swapChainDesc,
                                                                          nullptr,
                                                                          &_dxgiSwapChain));

in

0:000> k
 # Child-SP          RetAddr               Call Site
00 00000043`cef8ec80 00007ff9`3a4ac5a1     Microsoft_Terminal_Control!Microsoft::Console::Render::DxEngine::_CreateDeviceResources+0x47b [E:\BA\242\s\src\renderer\dx\DxRenderer.cpp @ 630] 
01 (Inline Function) --------`--------     Microsoft_Terminal_Control!Microsoft::Console::Render::DxEngine::GetSwapChain+0x14 [E:\BA\242\s\src\renderer\dx\DxRenderer.cpp @ 1010] 
02 00000043`cef8ed50 00007ff9`3a55667d     Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::ControlCore::GetSwapChain+0x21 [E:\BA\242\s\src\cascadia\TerminalControl\ControlCore.cpp @ 1183] 
03 00000043`cef8ed80 00007ff9`3a4e1f71     Microsoft_Terminal_Control!winrt::Microsoft::Terminal::Control::implementation::TermControl::RenderEngineSwapChainChanged$_ResumeCoro$1+0x2bd [E:\BA\242\s\src\cascadia\TerminalControl\TermControl.cpp @ 582] 
04 (Inline Function) --------`--------     Microsoft_Terminal_Control!std::experimental::coroutine_handle<void>::resume+0x9 [C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\experimental\coroutine @ 107] 
05 (Inline Function) --------`--------     Microsoft_Terminal_Control!std::experimental::coroutine_handle<void>::operator()+0x9 [C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\VC\Tools\MSVC\14.28.29910\include\experimental\coroutine @ 99] 
06 00000043`cef8ee50 00007ff9`5e2623a6     Microsoft_Terminal_Control!winrt::impl::resume_apartment_callback+0x11 [E:\BA\242\s\src\cascadia\TerminalControl\Generated Files\winrt\windows.ui.core.h @ 1088] 
07 00000043`cef8ee80 00007ff9`5e2af93f     Windows_UI!<lambda_59517943c03487243f9bea31c6c1a784>::operator()+0x84 [onecoreuap\windows\advcore\winrt\onecoreiwindow\corewindow\common\dispatcher.cpp @ 908] 
08 00000043`cef8eeb0 00007ff9`7de6a440     Windows_UI!Microsoft::WRL::Details::DelegateArgTraits<long (__cdecl Windows::System::IDispatcherQueueHandler::*)(void)>::DelegateInvokeHelper<Microsoft::WRL::Implements<Microsoft::WRL::RuntimeClassFlags<2>,Windows::System::IDispatcherQueueHandler,Microsoft::WRL::FtmBase>,<lambda_59517943c03487243f9bea31c6c1a784>,-1>::Invoke+0xf [onecore\external\sdk\inc\wrl\event.h @ 354] 
09 00000043`cef8eee0 00007ff9`7de4229c     CoreMessaging!Windows::System::DispatcherQueue::DeferInvokeCallback+0x20 [mincore\coreui\dev\dispatcherqueue\wrtdispatcherqueue.cpp @ 903] 
0a (Inline Function) --------`--------     CoreMessaging!Microsoft::CoreUI::ActionCallback::ImportAdapter$::__l2::<lambda_a81ff790741c2a62f2197c2561f5fe49>::operator()+0x1f [mincore\CoreUI\Dev\System\Api\IExportMessageSession.cs @ 22]

the _dxgiFactory2 was null. That's unexpected. It shouldn't be null.

(E) It doesn't seem like we're calling this when we create the automation peer -- we are not guaranteed to always call UpdateSettings after creating a peer, so the first peer may have no padding. See Only access ControlInteractivity through the projection #10051 (comment). (fixed by Initialize the padding for the Control UIA provider #10874)

Looks like there's a selection issue when you click and drag too quickly.

(Sorry for the terse body; I have to post this using powershell because of a new MSIT rule about personal devices :|)

Looks like there's a selection issue when you click and drag too quickly.

There sure is, just hit like 4 times this morning. Added to the list ☺️

haha, found it

if (_singleClickTouchdownPos)
{
    // Figure out if the user's moved a quarter of a cell's smaller axis away from the clickdown point
    auto& touchdownPoint{ *_singleClickTouchdownPos };
    auto distance{ std::sqrtf(std::powf(cursorPosition.X - touchdownPoint.X, 2) + std::powf(cursorPosition.Y - touchdownPoint.Y, 2)) };
    const til::size fontSize{ _actualFont.GetSize() };

    const auto fontSizeInDips = fontSize.scale(til::math::rounding, 1.0f / _renderEngine->GetScaling());
    if (distance >= (std::min(fontSizeInDips.width(), fontSizeInDips.height()) / 4.f))
    {
        _terminal->SetSelectionAnchor(_GetTerminalPosition(touchdownPoint));
        // stop tracking the touchdown point
        _singleClickTouchdownPos = std::nullopt;
    }
}

if (_singleClickTouchdownPos)
{
    // Figure out if the user's moved a quarter of a cell's smaller axis away from the clickdown point
    auto& touchdownPoint{ *_singleClickTouchdownPos };
    float dx = ::base::saturated_cast<float>(pixelPosition.x() - touchdownPoint.x());
    float dy = ::base::saturated_cast<float>(pixelPosition.y() - touchdownPoint.y());
    auto distance{ std::sqrtf(std::powf(dx, 2) +
                              std::powf(dy, 2)) };

    const auto fontSizeInDips{ _core->FontSizeInDips() };
    if (distance >= (std::min(fontSizeInDips.width(), fontSizeInDips.height()) / 4.f))
    {
        _core->SetSelectionAnchor(terminalPosition);
        // stop tracking the touchdown point
        _singleClickTouchdownPos = std::nullopt;
    }
}

        _terminal->SetSelectionAnchor(_GetTerminalPosition(touchdownPoint));

vs

        _core->SetSelectionAnchor(terminalPosition);

We're now using the location of the drag event as the selection anchor, instead of the location that the user initially clicked. Oops. Now to write a test.

This is a test for #9955.c

Fixes #9955.C

Add a test for #9955.a

Fixes #9955.a

Oof. B is hard, I think. The touchpad sends very many small scroll deltas, less than one row at a time. The control scrollbar can store a double, so small deltas can accumulate. Originally, these would accumulate in the scrollbar, and we'd only read that out as an int in the scrollbar updater, which is throttled.

In the interactivity split, there's no place for us to store that double. We immediately narrow to an int for ControlInteractivity::_updateScrollbar.

Theoretically, we could store a double in Interactivity that's a fake scrollbar position, to accumulate to. Then, ControlInteractivity would need to listen to scroll events from the core, unsure how easy that is right now...

This fixes #9955.b

add tests for #9955.b as well

Point [E] may cause an accessibility regression in the 1.11 timeframe.

This is a fix for #9955.e, just make sure to initialize the padding

Initialize the padding for the Control UIA provider (#10874)

Navigation Menu

Repository navigation

Interactivity bugs post-#9820 #9955

2 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Interactivity bugs post-#9820 #9955

Description

Activity

DHowett commented on Apr 28, 2021

zadjii-msft commented on Apr 28, 2021

zadjii-msft commented on Apr 28, 2021

zadjii-msft commented on Apr 28, 2021

2 remaining items

DHowett commented on Jul 19, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions