Add customization workflow to generate-sdk-locally skill#14821
Merged
Conversation
samvaity
commented
Mar 30, 2026
samvaity
commented
Member
- Add azsdk_typespec_customized_code_update and azsdk_run_typespec_validation to MCP tools table
- Add explicit Customize step (step 4) for build failure recovery
- Add trigger phrases for customization/breaking changes in description
- Add customization-workflow.md reference with detailed steps and common scenarios
- Update sdk-repos.md build failure section with customization tool references
- Add customization-workflow.yaml eval task for build failure + customization flow
- Update edge-case.yaml to expect azsdk_typespec_customized_code_update tool call
- Bump skill version to 1.1.0
samvaity
requested review from
Copilot,
praveenkuttappan,
raych1 and
ronniegeraghty
and removed request for
Copilot
samvaity
force-pushed
the
savaity/add-customization-to-generate-sdk-skill
branch
from
f8f03ce to
082f3db
Compare
samvaity
force-pushed
the
savaity/add-customization-to-generate-sdk-skill
branch
2 times, most recently
from
1b50956 to
ecaff19
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a customization-focused workflow and accompanying evaluation tasks to the generate-sdk-locally skill, so the skill can recover from build failures and handle common TypeSpec-driven SDK adjustments (renames, hiding operations, analyzer issues, breaking changes).
Changes:
- Bumps the skill version and expands the skill’s documented workflow to include an explicit Customize step using
azsdk_typespec_customized_code_update. - Adds multiple new evaluation tasks covering customization-trigger scenarios (rename, hide operation, analyzer errors, breaking changes, duplicate-field conflicts).
- Adds a new reference doc (
customization-workflow.md) and updates existing references to point to the new workflow and validation tool.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| .github/skills/generate-sdk-locally/SKILL.md | Updates skill version, tool list, steps, and examples to include customization workflow. |
| .github/skills/generate-sdk-locally/references/sdk-repos.md | Updates build-failure guidance to reference customization + TypeSpec validation and links to the workflow doc. |
| .github/skills/generate-sdk-locally/references/customization-workflow.md | New detailed reference describing entry points, steps, scenarios, and retry logic for customization. |
| .github/skills/generate-sdk-locally/tasks/edge-case.yaml | Updates build failure recovery expectations to include customization tool invocation. |
| .github/skills/generate-sdk-locally/tasks/rename-client.yaml | New eval task for rename requests intended to trigger customization. |
| .github/skills/generate-sdk-locally/tasks/hide-operation.yaml | New eval task for hiding an internal operation via customization. |
| .github/skills/generate-sdk-locally/tasks/analyzer-errors.yaml | New eval task for .NET analyzer error remediation via customization. |
| .github/skills/generate-sdk-locally/tasks/breaking-changes.yaml | New eval task for breaking changes/customization drift scenarios. |
| .github/skills/generate-sdk-locally/tasks/customization-workflow.yaml | New eval task for duplicate-field conflicts after TypeSpec updates. |
samvaity
added
azsdk-cli
samvaity
force-pushed
the
savaity/add-customization-to-generate-sdk-skill
branch
from
ecaff19 to
8e12148
Compare
Expand the skill to cover all customization entry points from the spec: - Build failures (compilation, analyzers, linting) - Breaking changes from spec updates - User prompts (rename, hide, restructure) - API review feedback - .NET analyzer errors (AZC0030, AZC0012) - Customization drift after TypeSpec renames - Duplicate field conflicts Changes: - SKILL.md: Add azsdk_typespec_customized_code_update and azsdk_run_typespec_validation to MCP tools, add Customize step (step 4), expand description with all trigger phrases, add diverse examples covering all entry points, bump to v1.1.0 - references/customization-workflow.md: Document entry points table, two-phase workflow (Phase A: TypeSpec decorators, Phase B: code repairs), common scenarios with phase mapping, retry logic with max iterations - references/sdk-repos.md: Update build failure section with tool references - tasks/customization-workflow.yaml: Duplicate field conflict scenario (Java) - tasks/breaking-changes.yaml: Customization drift after TypeSpec rename (Java) - tasks/rename-client.yaml: API review rename request (.NET) - tasks/analyzer-errors.yaml: .NET analyzer errors (AZC0030/AZC0012) - tasks/hide-operation.yaml: Hide internal operation (Python) - tasks/edge-case.yaml: Expect azsdk_typespec_customized_code_update tool call Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
samvaity
force-pushed
the
savaity/add-customization-to-generate-sdk-skill
branch
from
8e12148 to
961fce1
Compare
praveenkuttappan
approved these changes
Mar 31, 2026
ronniegeraghty
left a comment
ronniegeraghty
left a comment
Member
There was a problem hiding this comment.
How does the tool handle different starting states? And should the skill be what accounts for that?
Examples of different starting states:
| # | Working Repo | TypeSpec State | Description |
|---|---|---|---|
| 1 | Azure REST API Specs | Local changes | Editing TypeSpec locally within the specs repo before publishing or syncing |
| 2 | Azure SDK Repo | Local changes | Working in SDK repo with locally modified TypeSpec definitions |
| 3 | Azure SDK Repo | Remote changes (not pulled) | SDK repo is local, but TypeSpec updates exist only in the remote repo |
Assuming SDK repos are cloned locally for all these.
…orkflow.md Co-authored-by: Ronnie Geraghty <ronniegerag@gmail.com>
raych1
reviewed
Mar 31, 2026
Member
Author
The CustoomizedCodeUpdateTool, it takes does take two inputs packagePath (SDK repo) and tspProjectPath (TypeSpec project) so it works cross-repo. But it requires both repos to be cloned locally (scenario 2 prereq). I believe the #entry points should be enough for the tool invocation in that case and the tool just operates on local paths regardless of the git state. |
Co-authored-by: Ray Chen <raychen@microsoft.com>
raych1
reviewed
Mar 31, 2026
raych1
reviewed
Mar 31, 2026
raych1
reviewed
Mar 31, 2026
raych1
approved these changes
Apr 1, 2026
ronniegeraghty
approved these changes
Apr 1, 2026
Member
Author
|
/check-enforcer override |
This was referenced Apr 1, 2026
This was referenced Apr 9, 2026
ronniegeraghty
added
the
AzSDK Tools Agent
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.