OCPMCP-437: openshift-mcp-server: establish CI pass gates - #82846
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughThe MCP evaluation configuration sets task and assertion pass-rate thresholds to ChangesMCP evaluation
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 15✅ Passed checks (15 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
8b2c656 to
98a0581
Compare
|
@grokspawn: |
|
/lgtm |
|
/pj-rehearse pull-ci-openshift-openshift-mcp-server-main-mcpchecker-eval-anthropic |
grokspawn
changed the title
|
@grokspawn: your |
|
/pj-rehearse pull-ci-openshift-openshift-mcp-server-main-mcpchecker-eval-anthropic |
|
@grokspawn: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@grokspawn: requesting more than one rehearsal in one comment is not supported. If you would like to rehearse multiple specific jobs, please separate the job names by a space in a single command. |
|
/pj-rehearse pull-ci-openshift-openshift-mcp-server-main-mcpchecker-eval-google |
|
@grokspawn: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
rehearsals failed as desired. Now to fix the service account mapping for higher privilege. |
openshift-ci
Bot
added
do-not-merge/hold
|
/pj-rehearse pull-ci-openshift-openshift-mcp-server-main-mcpchecker-eval-google |
|
@grokspawn: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
grokspawn
changed the title
openshift-ci-robot
added
the
jira/valid-reference
|
@grokspawn: This pull request references OCPMCP-437 which is a valid jira issue. Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
de25aed to
648d7c9
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
openshift-ci
Bot
added
the
approved
648d7c9 to
f3048c1
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@ci-operator/step-registry/openshift/mcp-server/mcpchecker-eval/openshift-mcp-server-mcpchecker-eval-commands.sh`:
- Line 29: Update the server startup command near make run-server in the
evaluation script to use an existing openshift-mcp-server startup target, or
define run-server with readiness handling before invoking make run-evals. Ensure
server startup succeeds and is ready before evaluations run.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Central YAML (inherited)
Review profile: CHILL
Plan: Pro Plus
Run ID: a27cc22c-0283-4f87-beb1-1bc3ed830dca
📒 Files selected for processing (2)
ci-operator/config/openshift/openshift-mcp-server/openshift-openshift-mcp-server-main.yamlci-operator/step-registry/openshift/mcp-server/mcpchecker-eval/openshift-mcp-server-mcpchecker-eval-commands.sh
🚧 Files skipped from review as they are similar to previous changes (1)
- ci-operator/config/openshift/openshift-mcp-server/openshift-openshift-mcp-server-main.yaml
|
/pj-rehearse pull-ci-openshift-openshift-mcp-server-main-mcpchecker-eval-anthropic |
|
@grokspawn: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
openshift-ci
Bot
added
the
needs-rebase
bb81630 to
f3048c1
Compare
|
@grokspawn, Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
Signed-off-by: grokspawn <jordan@nimblewidget.com>
Signed-off-by: grokspawn <jordan@nimblewidget.com>
f3048c1 to
d10c44e
Compare
openshift-ci
Bot
removed
the
needs-rebase
Signed-off-by: grokspawn <jordan@nimblewidget.com>
|
/pj-rehearse pull-ci-openshift-openshift-mcp-server-main-mcpchecker-eval-google |
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
@grokspawn: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-openshift-openshift-mcp-server-main-mcpchecker-eval-google |
|
@grokspawn: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-openshift-openshift-mcp-server-main-mcpchecker-eval-anthropic |
|
@grokspawn: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/hold cancel |
openshift-ci
Bot
removed
the
do-not-merge/hold
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: dlom, grokspawn The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@grokspawn: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
openshift-merge-bot
Bot
added
the
rehearsals-ack
|
@grokspawn: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
04b4808
into
openshift:main
3 participants
build off #81118 and establish 80% pass gates for the jobs
Summary by CodeRabbit
The OpenShift MCP server CI now enforces 80% task and assertion pass-rate gates for Google and Anthropic MCP checker evaluations. The evaluation step exports
MCP_EVAL_KUBECONFIGand uses the repository Makefile to start and validate the MCP server.