TEST: Add coverage enforcement (fail-under 78%, diff-cover 90%)#1605
Open
romanlutz wants to merge 5 commits intomicrosoft:mainfrom
Open
TEST: Add coverage enforcement (fail-under 78%, diff-cover 90%)#1605romanlutz wants to merge 5 commits intomicrosoft:mainfrom
romanlutz wants to merge 5 commits intomicrosoft:mainfrom
Conversation
- Add --cov-fail-under=78 to all unit-test Makefile targets to prevent overall coverage regression (current baseline: ~79%) - Add diff-cover>=9.0.0 as dev dependency - Add diff-cover step to CI workflow (PR-only) requiring >=90% coverage on new/changed lines - Add 'make diff-cover' target for local use Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Runs in parallel with build_and_test on a single Ubuntu/Python 3.12 runner instead of blocking the full OS x Python matrix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Use 'python -m diff_cover.diff_cover_tool' instead of '-m diff_cover' (diff_cover is a package, not directly executable) - Fix test_simple_initializer assertion: check for 'converter_target' instead of 'PromptConverter.converter_target' since apply_defaults registers concrete class names, not the abstract base Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove --cov from build_and_test.yml matrix jobs (24 jobs now run faster without coverage overhead) - Move CodeCoverageSummary report to coverage workflow - coverage workflow runs once on Ubuntu/Python 3.12: 1. Overall threshold: --cov-fail-under=78 2. Coverage summary report (badge + markdown) 3. Diff coverage on PRs: >=90% on changed lines - Rename diff_cover.yml workflow to 'coverage' - Add unit-test-cov Makefile target (local coverage without xml/junit) - Keep unit-test target clean (no coverage, for quick local runs) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add unit-test-junit target (tests + junit xml, no coverage) - Add diff-cover-only target (just diff-cover, assumes coverage.xml exists) - Both workflows now use make instead of inline commands - Removed --doctest-modules from unit-test-cov-xml (not needed for coverage) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds automated coverage enforcement to prevent regressions and ensure new code is well-tested.
Changes
1. Overall coverage floor: 78%
--cov-fail-under=78to all unit-test Makefile targets (unit-test,unit-test-cov-html,unit-test-cov-xml)2. Diff coverage: 90% on new/changed lines
diff-cover(>=9.0.0) as a dev dependencypyrit/are covered by testsmake diff-covertarget for local useHow it works in CI
--cov-fail-under=78(fails build if overall coverage drops below 78%)origin/mainand fails if <90% of changed lines are coveredLocal usage