feat(evals): update mcpchecker to use labels #696
Conversation
manusa
changed the title
|
Waiting for the new release of mcpchecker with labels support |
|
@josunect here is the release: https://github.com/mcpchecker/mcpchecker/releases/tag/v0.0.4 |
josunect
force-pushed
the
issue642
branch
2 times, most recently
from
968e100 to
8eec95b
Compare
manusa
left a comment
manusa
left a comment
There was a problem hiding this comment.
LGTM, thx.
At some point we should improve the pipeline to support kubevirt too.
.github/workflows/mcpchecker.yaml
Outdated
| # Backward compatible behavior: | ||
| # - If suite is explicitly provided, it wins. | ||
| # - Otherwise, if task-filter mentions "kiali", treat this as a kiali run. | ||
| # - Otherwise default to kubernetes-only. | ||
| SUITE="kubernetes" | ||
| if [[ -n "$SUITE_INPUT" ]]; then | ||
| SUITE="$SUITE_INPUT" | ||
| elif [[ "$TASK_FILTER" =~ kiali ]]; then | ||
| SUITE="kiali" | ||
| fi | ||
|
|
||
| # Select label-selector and infrastructure based on suite | ||
| # All suites use the same eval.yaml file; suite controls label-selector + infra. | ||
| case "$SUITE" in | ||
| kiali) | ||
| echo "label-selector=suite=kiali" >> $GITHUB_OUTPUT | ||
| echo "kiali-run=true" >> $GITHUB_OUTPUT | ||
| ;; | ||
| all) | ||
| echo "label-selector=" >> $GITHUB_OUTPUT # No filter: run all taskSets | ||
| echo "kiali-run=true" >> $GITHUB_OUTPUT | ||
| ;; | ||
| *) | ||
| echo "label-selector=suite=kubernetes" >> $GITHUB_OUTPUT | ||
| echo "kiali-run=false" >> $GITHUB_OUTPUT | ||
| ;; |
There was a problem hiding this comment.
IMO we don't need to keep the backwards compatible behaviour, especially since it is complicating this already complex github action. Since this is only triggerable by maintainers, I think it is reasonable to expect that it is called correctly with the new command
There was a problem hiding this comment.
Ok, so I will remove it as well
| metadata: | ||
| labels: | ||
| suite: kiali | ||
| requires: istio |
There was a problem hiding this comment.
what are we using these requires labels for?
There was a problem hiding this comment.
At the moment they are not used, it was an example of classification. But, I will remove it.
Signed-off-by: josunect <jcordoba@redhat.com>
Signed-off-by: josunect <jcordoba@redhat.com>
Signed-off-by: josunect <jcordoba@redhat.com>
Signed-off-by: josunect <jcordoba@redhat.com>
Signed-off-by: josunect <jcordoba@redhat.com>
Signed-off-by: josunect <jcordoba@redhat.com>
Signed-off-by: josunect <jcordoba@redhat.com>
Signed-off-by: josunect <jcordoba@redhat.com>
|
@Cali0707 conflicts solved! |
|
@Cali0707 can this be merged? |
4 participants
Update mcpchecker workflow to use labels to filter different tasks.
Requires: mcpchecker/mcpchecker#111
Fixes #642