Skip to content

Conversation

@JennyPng
Copy link
Member

@JennyPng JennyPng commented Feb 4, 2026

#13784

thoughts

  • reliably loads skill and references when prompted "verify my setup" or "verify my setup for [language]" or "what should i install"
  • correctness of install instructions is generally correct, but sometimes less reliable, copilot might come up with different instructions that weren't specified in the .md file
  • copilot may also vary how it checks that a requirement exists
  • sometimes copilot chains many check commands, other times it runs each check individually meaning manual approval is needed for each, slowing down the verification
  • correctness of version checking is also less reliable - claude-sonnet-4 claimed 2x that my pytest version 8.3.4 exceeds the minimum required 8.3.5

example output

image image

example of incorrectness - claims pytest exceeds minimum required version when it clearly doesn't

image

@JennyPng JennyPng requested a review from samvaity February 4, 2026 00:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant