Repository navigation
WinUI DevTools: inspect, comment on and live-edit a running WinUI app - #943
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ilters - -a falls back to title matching (preferring ApplicationFrameHost frames) when the matched process owns no top-level window, and fails instead of reporting an empty target - inspect --interactive includes Document and writable-value elements - --type/--root/--class-name on every single-selector ui command; invoke accepts them without --action Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…d examples - `winapp ui --help` renders plain text: golden path, -a/<selector> definitions, and commands grouped by task (2,741 bytes redirected, down from 6,538) - every ui command gets plain-text help with 1-3 placeholder examples and a one-line global options pointer; examples are data (IHelpExamples) and are parse-tested - `winapp ui <unknown>` fails with exit 1 (even with --help) before sandbox routing, suggesting up to two commands; --json emits the error envelope with `suggestions` - `find` and `tree` are aliases for `search` and `inspect` - root help points agents at `winapp ui --help` - a missing option value ends with a pointer to --help instead of the full help - inspect's footer example uses set-value for editable-only elements Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ghten unknown-command detection - `-a` falls back only to ApplicationFrameHost frames; a process with no window yet keeps the process-scoped target so `wait-for` and `status` behave as before - bare `winapp ui` still prints the group help after "Required command was not provided" - only the first unmatched token counts as a command, so `ui -a Notepad` and `ui -- inspect` report the normal parse error instead of an unknown command Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Build Metrics ReportValidation passed. All required build and validation jobs succeeded. Binary Sizes
.NET Test Results (TRX reports)Other suites are reflected in the overall validation status above. ✅ 10419 passed, 43 skipped out of 10462 tests in 1652.5s (+1622 tests, +107.4s vs. baseline) Test Coverage✅ 87.2% line coverage, 81.3% branch coverage · CLI Startup Time46ms median (x64, Updated 2026-10-16 06:02:40 UTC · commit |
d25242b to
d69d6cb
Compare
Triage fixes (groups 1 and 2)Seven commits on top of the original squash. Each fix has a regression test that fails without it. Live checks ran against AI Dev Gallery and FixedPicking text with no source picks the authored control. Hover and click share the path; Just my XAML off keeps the raw hit. A comment on an element that still has no source says
Comment anchors. A unique declaration or
Rerun while the app is running Identical Smaller fixes Also fixed: the comments pane says "not linked to source" for unanchored comments and pluralizes the footnote correctly; Layout adorners is a UIA toggle ( Group 2: Not reproducedQuick-edit panel off-screen. On a 150% single monitor, the panel stays inside the work area for a full-width TextBox and a Gallery expander, with the Popup unconstrained. The existing placement tests already cover the flip, shift and height cap. I added the element and viewport to the Also
|
|
Quick-edit panel off-screen: not a bug. I reproduced the reported setup: a window as wide as the monitor, then picks on a full-width element and on one near the bottom, at 150% (monitor 3840×2160 px, work area 3840×2088 px). The panel stays inside the work area: The original report compared physical coordinates with a logical screen size. |
Group 3 decisions implementedFour commits. Each change has a test that fails without it. "Before" is the pre-fix build; "after" is from this branch, run on Pin (kept) is a toolbar toggle. It uses the same accent on-state as Pick and Layout adorners, and the corner dot is gone. Theme toggle (kept on the toolbar). Tooltip "Toggle the inspected app between Light and Dark themes" is now "Switch app theme". Inspector Properties pane. Order is now: type and name, ancestors, source line + Open, then the filter and the grid. The box model and the offered/took explanation move into a Layout section after the grid. It is collapsed by default, and its state is remembered across runs (
No
Guide split. Unchanged: |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…hored UI comments `winapp run --devtools` injects an inspection agent into a WinUI 3 app: an in-app overlay (pick, quick edit, comments), an inspector window, and a `winapp devtools` command group for agents (inspect, search, get/set property, get-source, diagnose-binding, comments). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Keep walking past a nearer generated scope that does not reference the target
(a UserControl whose content the page supplies) instead of reporting unavailable.
- Enrol windows held in static App fields (App.MainWindow { get; private set; }).
- Resolve non-public members for {x:Bind}, which compiles against its owner.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…omments A hit with no source (a Button's string content, a TextBox placeholder, a nav item label) now promotes to its nearest ancestor with authored source, the same as a template part. Hover and click share this path; Just my XAML off keeps the raw hit. A comment saved on an element that still has no source reports 'Saved. Not linked to source.' in the panel. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- A declaration that matches exactly one source declaration, or a unique x:Name, is a confirmed anchor; runtime instance counts no longer weaken it. - Drop 'via type' candidates when a declaration or x:Name match exists. - Show the historical creation location only when there is no current match, and never print an empty line number. - A comment without a source file is weak, requires confirmation, and says it is not linked to source. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Comments pane: an unanchored comment says 'not linked to source' instead of 'not on this screen' and is not counted as off screen; the footnote agrees in number. - Layout adorners is a ToggleButton, so UI Automation exposes its state; the tooltip no longer says 'Show' while it is on. - The precedence chain shows a null default as empty instead of the raw handle '0'. - The inspector's extra-width note appears only when HorizontalAlignment is not Stretch. - Log the quick-edit panel's element and viewport for placement triage. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…arnings - When staging fails because the app is still running, name its PID(s) by matching the process image to the staged path, instead of a generic access-denied error. - Collapse identical source-location warnings into one entry with a count, and say to rebuild when the build output does not match the XAML sources. - run --devtools --detach ends with 'Next: winapp devtools inspect -a <pid>' and no longer prints 'running bytes are not verified'. - devTools fields in run --json are camelCase like the rest of the new fields. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- set-property warns when it replaces a {Binding} or overrides an x:Bind (replacedBinding in JSON).
- get-property/set-property take the property positionally; -p still works.
- search falls back to the whole tree when nothing in your XAML matches (items created
from data), and no longer claims displayed text is not searched.
- inspect --filter says when matches may be below --depth.
- comments: -a alias; list --status footer counts what was hidden; --on is hidden
where it is always rejected.
- Relaxed JSON escaping so captured XAML is readable; 'properties' instead of 'propert(ies)'.
- Hide the healthy-binding scope disclaimer from human diagnose-binding output.
- Binding-owner e2e: static-held windows diagnose; unique x:Name comments confirm.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Pinned uses the same accent on-state as Pick and Layout adorners and is a ToggleButton, so UI Automation reports its state. The corner dot, which read as a notification badge, is removed. - The theme button's tooltip is 'Switch app theme' so it is not clipped at the edge. - Move the settings helpers to DevToolsSettings.h so the inspector can use them. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The Properties pane shows the header (type and name, ancestors, source line and Open), then the filter and the grid. The box model and the offered/took explanation move into a Layout section after the grid, collapsed by default and remembered across runs. Source-verification notes are one line; the full reason is on hover and exposed to screen readers. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…inding - winapp devtools JSON (attach, list, live commands, errors) uses processId like winapp ui and winapp target. run --json keeps ProcessId; --pid and -a are unchanged. - The set-property binding warning ends with 'Restart the app to restore the binding.' Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
devtools.md starts with numbered steps and two screenshots, then covers reviewing comments with an agent, finding elements, bindings, live changes, refreshing, the terminal workflow and failures. devtools-advanced.md covers Sandbox, attaching to a running app, window and subtree targeting, comment storage and the protocol. Links in usage, Sandbox docs and skills point at the right page. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
DevTools.negotiate (and the legacy hello reply) report processId instead of pid, matching the CLI payloads. The protocol schema and the CLI's negotiate parser change with it; there is no alias because the protocol has not shipped. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Generalize #942's ui help renderer and unknown-command handling to any compact help group; winapp ui output is unchanged apart from one pointer line. - winapp devtools --help: purpose, the main workflow, what -a and <selector> mean, commands grouped by task, and when to use winapp ui instead. Every devtools command has examples, stored as data and parsed in tests. Unknown devtools commands suggest the closest one. - Root help gets a short 'Start here' block for winapp ui and winapp devtools. - A winapp ui command whose target has the DevTools agent prints one tip line on stderr (not with --json or --quiet). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
#942 added the options to ui record; the hand-written argument builder did not forward them, which failed the generated-options sync test. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The winui-devtools skill now matches requests to change, translate or restyle a running UI without editing source; winapp-ui-automation says it reads, clicks and types in any app and defers live WinUI edits to winui-devtools. The UI automation and DevTools docs mention the new help and the ui tip. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Based on time before you go out we don't have to address all these in this PR but filing here for us to track going forward. Agent found a few functions which seem to duplicate logic:
Code Cleanup For Styles/Controls:
Screenshots can include the toolbarNow that devtools are on by default, an agent using winapp ui screenshot will by default take a screenshot of the app which includes the devtools overlay. Should we add a note to the documentation or winapp cli skills that tells agents they should run with the devtools off or no overlay before taking an app screenshot? |
# Conflicts: # src/winapp-CLI/WinApp.Cli.Tests/FakeDotNetService.cs # src/winapp-CLI/WinApp.UIAutomation/Services/UiAutomationService.Query.cs # src/winapp-CLI/WinApp.UIAutomation/Services/UiAutomationService.cs
In a packaged app that references a XAML library, the library's XAML is copied loose next to the exe while the app's own pages are compiled into resources.pri. The layout scan then trusted the loose file and hid the app's pages: devtools inspect showed only the library's TextBlock. With a source root (any winapp run launch), the default tree now uses the same rule as picking: a file in the project on disk. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Merged main (acacbd4) into this PR: 13e5431, plus one fix found while checking (e51afe3)
Checked with main's changes
Found while checking #1032 (e51afe3): before this fix,
Validation: Release build of the solution with Main moved again (ad0a62c), merged in 6bbbb83:
|
# Conflicts: # src/winapp-CLI/WinApp.Cli/Models/ProjectRunModels.cs
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
With DevTools on by default, a screenshot showed its toolbar, markers, highlight and comment flyout. The screenshot now asks the engine to make that chrome transparent for the capture and puts it back afterwards, so the toolbar, selection, pick mode and focus are unchanged. DevTools' own windows (the inspector and the flyout's popup) are left out of the composite. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…f them BuildOneTreeRow re-declared AppendUiTo, NewTextBlock and SetAutomationTextOn as identical lambdas. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Route1Layout and CmdElementLayout each carried a copy of the same reads. Both now call ReadElementLayout_ui; Layout.get still adds the parent's wire handle. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
… window The engine now names where each Property.get row's value came from (origin), using the same rule as the window's pill, and get-property reports it as source instead of deriving its own copy. Output is unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
DeriveEditKind asks CardMapType instead of listing Foreground and Background again. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Previews and query results each cut values at a literal 120. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Each native test file declared its own Check/CheckEq helpers and failure counter. They now include DevToolsTestCheck.h, which gives every file its own counter. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ProjectContextDetector re-parsed packages.config itself. It now uses PackagesConfigReader, so a malformed packages.config no longer hides UseWinUI and turns the project Unknown. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The five toolbar ToggleButtons each repeated the same six transparent checked-state brushes. They now come from the toolbar Canvas resources. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The C++-built pane markup repeated the secondary text color, the note text and note box, and the section caption. They are now DT_* constants defined once in DevToolsWindow.cpp. They are not keyed Styles in window-shell.xaml: XamlReader.Load parses each pane fragment on its own, so a StaticResource or ThemeResource that points at the shell fails to load (tried both; the pane showed 'Unable to render properties'). Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The two overlay palettes move from DevToolsOverlay.State.inc to DevToolsTheme.h. The toolbar, comment flyout and selection builders each mapped palette fields to $TOKEN$s; they now share one DevToolsXamlSubst overload that fills every palette token before the builder's own values. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Conflict in docs/usage.md (C++ run notes): kept the DevTools line and main's -c / -p:Configuration line from #1085. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
winapp ui screenshot now leaves DevTools chrome out, so the comment-marker check saw no markers. The check now captures the window with PrintWindow and measures the label offset from the window's physical rect. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
In a blank app the collapsed toolbar said "Comments": the label treated visible comment markers, which are on by default, as a comment state, and hid the count when it was 0. The pill now reads DevTools at rest, and Picking or Commenting only while clicks go to DevTools. The pill no longer shows a comment count; the Comments button's badge still does. Its UI Automation name stays "DevTools (open toolbar)" and adds the active mode. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The pick-mode check ran after a menu hover had left the toolbar expanded, so the pill was not on screen. It now runs right after the at-rest check and also checks leaving pick mode. The later check with saved comments first moves the pointer off the toolbar. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Writing a string to TitleBar.Content made the setter fail with E_NOINTERFACE, but WinUI kept the string. The TitleBar then failed on it at the next window activation and WinUI fail-fasted the app (0xC000027B), taking DevTools with it. Every DevTools write (DevTools window, set-property, the protocol) now remembers the property's local value, puts it back when the setter fails, and clears the error WinUI left on the thread. The DevTools window shows the reason on the row, for example: Couldn't set Content: TitleBar rejected 'x' (String). The previous value was kept. Also fixes the binding-confirm prompt text, which was written through the wrong interface. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
After pick mode the DevTools window is open and the toolbar stays expanded, so the pill isn't on screen. The label rule takes no comment count, which the native test covers. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Chiara Mooney (@chiaramooney) thanks for the review. Each point below is its own commit. README command lists: Screenshots include the toolbar: Duplicated logic
Styles and controls
No visible change: before/after screenshots of the toolbar, both toolbar menus, the comment flyout and the inspector (tree, Set view, Comments), in light and dark, match pixel for pixel. The only differences are the PID in the title, caret blink and Acrylic noise. Also fixed: the collapsed toolbar said "Comments" in a blank app (8cc3836). Visible comment markers (on by default) made the label say "Comments", and it hid the count when there were 0 comments. Now the collapsed toolbar reads DevTools at rest, whatever the comments or markers state. It reads Picking or Commenting only while that mode is on, because then clicks go to DevTools instead of the app. The comment count stays on the Comments button's badge. The UI Automation name stays "DevTools (open toolbar)" and adds the mode while one is on.
Performance: baseline 6bbbb83 (right after the main merge) vs aa85f33 (all of the above). Later commits merge main, fix test scripts, change the collapsed-toolbar label (runs only when a mode changes), and add a read of the previous value before each DevTools property write (not on any hot path). Method: the same machine, alternating A/B three times per build. No other builds or benchmarks ran during the window. Engine numbers come from the engine's own UI-thread counters; wall times include the CLI round trip.
Nothing is worse than its run-to-run spread except the screenshot, which is the new hide/restore cost and stays under the 100 ms target. |
Description
Adds WinUI DevTools to winapp. Before this change,
Select element opens the inspector on the clicked element. The tree scrolls to it and the panel shows its source line and properties.
Saved comments show as markers when the app starts, including comments an agent added from the CLI.
## Additional NotesValidation- CI: unit and integration tests, plus native engine and protocol tests (the engine is loaded into a real WinUI test app). A UI test drives the toolbar, picking, comments, the inspector and keyboard focus in a real window. The sample tests (Electron, C++, Rust, WPF, .NET, MAUI, Flutter, Tauri, WinUI) are unchanged and pass.- Real apps: checked hands-on against AI Dev Gallery, WinUI Gallery, Daylight,
winapp runstarted a WinUI 3 app and agents could only see it through UI Automation (winapp ui). UI Automation shows controls, but not the XAML that produced them, their bindings or their live properties. There was also no way for a developer to point at an element and tell an agent what to change.Now, for a WinUI 3 project,winapp runstarts the app with DevTools by default:- For the developer: an in-app toolbar. - Select element opens an inspector window on the clicked element: visual tree, live properties, its XAML file and line. - Comments mode lets you click an element and leave a comment. The comment is saved to.winapp/ui-comments.jsonand anchored to the element's XAML declaration.- For agents: thewinapp devtoolscommand group reads and changes the same live tree: -inspectandsearchshow the tree with each element's source line. -get-property,set-propertyanddiagnose-bindingread properties, change them live and explain bindings. -comments list/get/updateread the developer's comments and resolve them.It works for C# (.csprojwithUseWinUI), C++ (.vcxprojwith the Windows App SDK and XAML pages),dotnet runthrough theMicrosoft.Windows.SDK.BuildTools.WinApppackage, and Windows Sandbox (--on sandbox --devtools on).When it's on:- Off automatically: in CI (theCIvariable is set), with--no-launch,--without-alias,--profileor--on sandbox, and for build-output folders,.csfiles and projects that aren't WinUI.- Choose per run:--devtools on|headless|off, or-p:WinAppRunDevTools=...withdotnet run.headlessdraws nothing in the app; Ctrl+Shift+F12 brings up the toolbar.- Change the default:winapp config set run.devtools off|headless|on(winapp config listshows your per-user settings). The toolbar's ⋯ menu changes the same setting.- Never breaks a run: when DevTools comes from the default and can't start, the app still runs and winapp prints one line saying why. If the app is already running,runcloses it so DevTools can start it, and says so.Non-WinUI runs are unchanged: same output, same JSON ({AUMID, ProcessId}) and the same exit codes.## Usage Examplepowershellwinapp run . # WinUI project: DevTools on by defaultwinapp devtools comments list # what the developer asked forwinapp devtools inspect --depth 4 # live tree with source lineswinapp devtools search Submitwinapp devtools get-property SubmitButton Contentwinapp devtools set-property SubmitButton Content "Send"winapp devtools diagnose-binding <selector> Textwinapp run . --devtools headless # DevTools without the toolbarwinapp config set run.devtools off # turn the default off for your accountdotnet run -p:WinAppRunDevTools=off # same choice through the NuGet packageObserved onsamples/winui-app(winapp run . --detach, thenwinapp devtools inspect -a <pid> --depth 6, trimmed):text📝 DevTools on (default) · turn off: --devtools off or winapp config set run.devtools off...[grid-38abee175a] Grid MainWindow.xaml:14 [AppTitleBar] TitleBar MainWindow.xaml:20 [scrollviewer-ed8e9dae6e] ScrollViewer MainWindow.xaml:26 aid=MainScroller [stackpanel-c5c6c99888] StackPanel MainWindow.xaml:27 [textblock-13e9b6d8d7] TextBlock MainWindow.xaml:29 aid=PageTitle Text="WinUI Sample App" [CounterButton] Button MainWindow.xaml:41 Content="Click Me" AutomationProperties.Name="Counter Button" [SubmitButton] Button MainWindow.xaml:83 Content="Submit" AutomationProperties.Name="Submit Button"## Related IssueNone.## Type of Change- ✨ New feature- 📝 Documentation- 🔧 Config/build- 🧪 Test update## Checklist- [x] New tests added for new functionality- [x] Tested locally on Windows- [x] Main README.md updated- [x] docs/usage.md updated- [x] Language-specific guides updated: newdocs/guides/devtools.mdanddevtools-advanced.md;dotnet.md,dotnet-run-support.mdand the NuGet README updated- [x] Shipped skills updated inplugins/winapp/skills/: newwinui-devtools;winapp-ui-automation,winapp-sandboxandwinapp-troubleshootpoint to it## Screenshots / DemoComments mode: click an element, type, press Enter. The panel shows the element's XAML file and line.samples/winui-app,samples/cpp-winui-appanddotnet run, plus Windows Sandbox.-winapp uisees only the app: with DevTools on or headless,winapp ui inspectandsearchshow the same elements as--devtools off;winapp ui inspect DevToolsOverlayshows the toolbar. With--devtools headless, the app area is also pixel-identical.Performance. DevTools is on by default, so it costs launch time and memory. Measured onsamples/winui-app(Xeon 8370C, 64 GB, app already built,winapp run . --detach --no-build, 3 rounds each):- Launch: 2.8–3.9 s with--devtools off, 8.6–11.5 s with it on.- Memory: working set right after launch went from about 160–177 MB to about 187 MB.- Build: the source-capture build is not slower. On Daylight, from a clean build, it took 21.5–21.9 s against 22.7–22.9 s with DevTools off.The app also uses more memory over a long session, as with Visual Studio's F5 with XAML Hot Reload.--devtools off,winapp config set run.devtools offand-p:WinAppRunDevTools=offturn it off.Limits- Binding diagnosis needs the .NET runtime, so it reportsunavailablefor C++ and Native AOT apps.get-propertystill shows the value and where it comes from.- A comment added withcomments add --file/--name, without a live element, counts in the badge but has no marker in the app.-dotnet runwith--on sandboxdoesn't support DevTools. With the default it is off; an explicitWinAppRunDevTools=onwith Sandbox is rejected with a message saying so.- When default DevTools has to step aside before launch (for example, the running app can't be closed), a--with-aliasrun falls back to launching by AUMID, so the app's console output isn't shown in the terminal for that run.Size. Against main, the CLI grows by 5.0 MB (x64 and ARM64), the MSIX by 2.1 MB (x64) / 2.0 MB (ARM64), and the npm and NuGet packages by about 4.2 MB. About 2.6 MB of that is the native engine and 2.4 MB is new CLI code. The engine is built with/Gy /Gwand linked with/OPT:REF /OPT:ICF; before that change the CLI grew by 8.0 MB (x64) / 9.3 MB (ARM64), because the engine was 5.5 MB (x64) / 6.8 MB (ARM64). Inspection speed was unchanged in a side-by-side check.