test: add Pest v1 security test infrastructure#251
test: add Pest v1 security test infrastructure#251somethingwithproof wants to merge 6 commits intoCacti:developfrom
Conversation
Add source-scan tests verifying security patterns (prepared statements, output escaping, auth guards, PHP 7.4 compatibility) remain in place across refactors. Tests run with Pest v1 (PHP 7.3+) and stub the Cacti framework so plugins can be tested in isolation. Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
There was a problem hiding this comment.
Pull request overview
Adds a Pest v1-based test harness for the Flowview plugin, focused on static security/regression checks and basic PHP 7.4 compatibility guards.
Changes:
- Introduces Pest bootstrap/config and security-focused Pest test suites.
- Adds source-scanning tests for prepared-statement usage and setup.php structure.
- Adds a minimal
composer.jsonfor dev dependencies and a Dependabot configuration file.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
tests/Pest.php |
Pest entrypoint that loads the test bootstrap. |
tests/bootstrap.php |
Defines Cacti/framework function stubs for isolated testing. |
tests/Security/SetupStructureTest.php |
Static checks that setup.php contains required plugin hook functions and version keys. |
tests/Security/PreparedStatementConsistencyTest.php |
Static scan intended to prevent reintroduction of raw db_* calls. |
tests/Security/Php74CompatibilityTest.php |
Static scan for a few PHP 8.0-only constructs. |
composer.json |
Adds Pest v1 as a dev dependency and dev autoloading. |
.github/dependabot.yml |
Adds Dependabot configuration (currently npm + GitHub Actions). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| it('uses prepared DB helpers in all plugin files', function () { | ||
| $targetFiles = array( | ||
| 'flowview.php', | ||
| 'flowview_devices.php', | ||
| 'flowview_filters.php', | ||
| 'flowview_schedules.php', | ||
| 'setup.php', |
There was a problem hiding this comment.
setup.php is included in $targetFiles, but the current setup.php in this repo contains raw db_fetch_cell( / db_execute( calls (e.g. setup.php:110, setup.php:324, setup.php:1024). This test will fail immediately under vendor/bin/pest even without any regressions. Either (a) migrate those calls to prepared variants / wrappers, or (b) narrow the check (e.g. exclude setup.php and/or ignore DDL-only db_execute() usage) so the new test suite is green on the current codebase.
| it('uses prepared DB helpers in all plugin files', function () { | |
| $targetFiles = array( | |
| 'flowview.php', | |
| 'flowview_devices.php', | |
| 'flowview_filters.php', | |
| 'flowview_schedules.php', | |
| 'setup.php', | |
| it('uses prepared DB helpers in all migrated plugin files', function () { | |
| $targetFiles = array( | |
| 'flowview.php', | |
| 'flowview_devices.php', | |
| 'flowview_filters.php', | |
| 'flowview_schedules.php', |
| $lines = explode("\n", $contents); | ||
| $rawCallsOutsideComments = 0; | ||
|
|
||
| foreach ($lines as $line) { | ||
| $trimmed = ltrim($line); | ||
|
|
||
| if (strpos($trimmed, '//') === 0 || strpos($trimmed, '*') === 0 || strpos($trimmed, '#') === 0) { | ||
| continue; | ||
| } | ||
|
|
||
| if (preg_match($rawPattern, $line) && !preg_match($preparedPattern, $line)) { | ||
| $rawCallsOutsideComments++; | ||
| } | ||
| } | ||
|
|
There was a problem hiding this comment.
The line-based comment skipping here is not robust: block comments starting with /* (or inline /* ... */) and string literals containing db_execute( will still be scanned and can cause false positives. Consider tokenizing with token_get_all() and ignoring T_COMMENT, T_DOC_COMMENT, and T_CONSTANT_ENCAPSED_STRING tokens before applying the regex, to make this test stable.
| $lines = explode("\n", $contents); | |
| $rawCallsOutsideComments = 0; | |
| foreach ($lines as $line) { | |
| $trimmed = ltrim($line); | |
| if (strpos($trimmed, '//') === 0 || strpos($trimmed, '*') === 0 || strpos($trimmed, '#') === 0) { | |
| continue; | |
| } | |
| if (preg_match($rawPattern, $line) && !preg_match($preparedPattern, $line)) { | |
| $rawCallsOutsideComments++; | |
| } | |
| } | |
| $tokens = token_get_all($contents); | |
| $sanitizedContents = ''; | |
| foreach ($tokens as $token) { | |
| if (is_array($token)) { | |
| $tokenId = $token[0]; | |
| $tokenText = $token[1]; | |
| if ($tokenId === T_COMMENT || $tokenId === T_DOC_COMMENT || $tokenId === T_CONSTANT_ENCAPSED_STRING) { | |
| $sanitizedContents .= preg_replace('/[^\r\n]/', ' ', $tokenText); | |
| } else { | |
| $sanitizedContents .= $tokenText; | |
| } | |
| } else { | |
| $sanitizedContents .= $token; | |
| } | |
| } | |
| preg_match_all($rawPattern, $sanitizedContents, $matches); | |
| $rawCallsOutsideComments = count($matches[0]); |
| $source = file_get_contents(realpath(__DIR__ . '/../../setup.php')); | ||
|
|
There was a problem hiding this comment.
realpath() can return false and file_get_contents() can return false, which will emit warnings and then make the subsequent toContain() assertions fail with an unhelpful type/value. Add explicit assertions that the path exists and the file contents were read successfully (fail fast with a clear message) before running the string checks.
| $source = file_get_contents(realpath(__DIR__ . '/../../setup.php')); | |
| $setupPath = realpath(__DIR__ . '/../../setup.php'); | |
| \PHPUnit\Framework\Assert::assertNotFalse($setupPath, 'Failed to resolve setup.php path.'); | |
| $source = file_get_contents($setupPath); | |
| \PHPUnit\Framework\Assert::assertNotFalse($source, 'Failed to read setup.php contents.'); |
| function db_execute($sql) { | ||
| $GLOBALS['__test_db_calls'][] = array('fn' => 'db_execute', 'sql' => $sql, 'params' => array()); | ||
| return true; | ||
| } | ||
| } | ||
|
|
||
| if (!function_exists('db_execute_prepared')) { | ||
| function db_execute_prepared($sql, $params = array()) { | ||
| $GLOBALS['__test_db_calls'][] = array('fn' => 'db_execute_prepared', 'sql' => $sql, 'params' => $params); | ||
| return true; | ||
| } | ||
| } | ||
|
|
||
| if (!function_exists('db_fetch_assoc')) { | ||
| function db_fetch_assoc($sql) { | ||
| return array(); | ||
| } | ||
| } | ||
|
|
||
| if (!function_exists('db_fetch_assoc_prepared')) { | ||
| function db_fetch_assoc_prepared($sql, $params = array()) { | ||
| return array(); | ||
| } | ||
| } | ||
|
|
||
| if (!function_exists('db_fetch_row')) { | ||
| function db_fetch_row($sql) { | ||
| return array(); | ||
| } | ||
| } | ||
|
|
||
| if (!function_exists('db_fetch_row_prepared')) { | ||
| function db_fetch_row_prepared($sql, $params = array()) { | ||
| return array(); | ||
| } | ||
| } | ||
|
|
||
| if (!function_exists('db_fetch_cell')) { | ||
| function db_fetch_cell($sql) { | ||
| return ''; | ||
| } | ||
| } | ||
|
|
||
| if (!function_exists('db_fetch_cell_prepared')) { | ||
| function db_fetch_cell_prepared($sql, $params = array()) { |
There was a problem hiding this comment.
The db_execute() stub only accepts one parameter, but plugin code calls db_execute($sql, $log, $conn) via flowview_db_execute() (see database.php:70). If any test starts loading plugin code, this will raise an argument count error. Update the stub signatures to accept the optional parameters used in production (and similarly for fetch helpers) so the bootstrap is compatible with the plugin.
| function db_execute($sql) { | |
| $GLOBALS['__test_db_calls'][] = array('fn' => 'db_execute', 'sql' => $sql, 'params' => array()); | |
| return true; | |
| } | |
| } | |
| if (!function_exists('db_execute_prepared')) { | |
| function db_execute_prepared($sql, $params = array()) { | |
| $GLOBALS['__test_db_calls'][] = array('fn' => 'db_execute_prepared', 'sql' => $sql, 'params' => $params); | |
| return true; | |
| } | |
| } | |
| if (!function_exists('db_fetch_assoc')) { | |
| function db_fetch_assoc($sql) { | |
| return array(); | |
| } | |
| } | |
| if (!function_exists('db_fetch_assoc_prepared')) { | |
| function db_fetch_assoc_prepared($sql, $params = array()) { | |
| return array(); | |
| } | |
| } | |
| if (!function_exists('db_fetch_row')) { | |
| function db_fetch_row($sql) { | |
| return array(); | |
| } | |
| } | |
| if (!function_exists('db_fetch_row_prepared')) { | |
| function db_fetch_row_prepared($sql, $params = array()) { | |
| return array(); | |
| } | |
| } | |
| if (!function_exists('db_fetch_cell')) { | |
| function db_fetch_cell($sql) { | |
| return ''; | |
| } | |
| } | |
| if (!function_exists('db_fetch_cell_prepared')) { | |
| function db_fetch_cell_prepared($sql, $params = array()) { | |
| function db_execute($sql, $log = true, $conn = false) { | |
| $GLOBALS['__test_db_calls'][] = array('fn' => 'db_execute', 'sql' => $sql, 'params' => array()); | |
| return true; | |
| } | |
| } | |
| if (!function_exists('db_execute_prepared')) { | |
| function db_execute_prepared($sql, $params = array(), $conn = false) { | |
| $GLOBALS['__test_db_calls'][] = array('fn' => 'db_execute_prepared', 'sql' => $sql, 'params' => $params); | |
| return true; | |
| } | |
| } | |
| if (!function_exists('db_fetch_assoc')) { | |
| function db_fetch_assoc($sql, $log = true, $conn = false) { | |
| return array(); | |
| } | |
| } | |
| if (!function_exists('db_fetch_assoc_prepared')) { | |
| function db_fetch_assoc_prepared($sql, $params = array(), $conn = false) { | |
| return array(); | |
| } | |
| } | |
| if (!function_exists('db_fetch_row')) { | |
| function db_fetch_row($sql, $log = true, $conn = false) { | |
| return array(); | |
| } | |
| } | |
| if (!function_exists('db_fetch_row_prepared')) { | |
| function db_fetch_row_prepared($sql, $params = array(), $conn = false) { | |
| return array(); | |
| } | |
| } | |
| if (!function_exists('db_fetch_cell')) { | |
| function db_fetch_cell($sql, $log = true, $conn = false) { | |
| return ''; | |
| } | |
| } | |
| if (!function_exists('db_fetch_cell_prepared')) { | |
| function db_fetch_cell_prepared($sql, $params = array(), $conn = false) { |
| } | ||
|
|
||
| if (!function_exists('__esc')) { | ||
| function __esc($text, $domain = '') { |
There was a problem hiding this comment.
__esc() is used throughout the plugin with sprintf-style arguments (e.g. __esc('Listener [edit: %s]', $device['name'], 'flowview')), but the stub only accepts ($text, $domain = ''). This will break as soon as any test evaluates code paths that call __esc. Consider implementing the stub with a variadic signature and applying vsprintf when additional args are provided.
| function __esc($text, $domain = '') { | |
| function __esc($text, ...$args) { | |
| $domain = ''; | |
| if (!empty($args) && is_string($args[count($args) - 1])) { | |
| $domain = array_pop($args); | |
| } | |
| if (!empty($args)) { | |
| $text = vsprintf($text, $args); | |
| } |
| version: 2 | ||
| updates: | ||
| - package-ecosystem: "npm" | ||
| directory: "/" | ||
| schedule: | ||
| interval: "weekly" | ||
| open-pull-requests-limit: 10 | ||
| - package-ecosystem: "github-actions" |
There was a problem hiding this comment.
Dependabot is configured for the npm ecosystem, but this repository has no package.json / npm dependencies. With the new composer.json, it would be more accurate to switch this entry to the composer ecosystem (and keep github-actions if desired) to avoid dependabot configuration errors/no-op updates.
| if ($path === false) { | ||
| continue; | ||
| } | ||
|
|
||
| $contents = file_get_contents($path); | ||
|
|
||
| if ($contents === false) { | ||
| continue; | ||
| } |
There was a problem hiding this comment.
These continue branches mean the test can pass while silently skipping missing/unreadable files (e.g. if the repo layout changes or a file is deleted), which undermines the purpose of a regression/security check. Prefer failing fast with an assertion when realpath() or file_get_contents() fails so the test suite can’t go green without actually validating the files.
| if ($path === false) { | |
| continue; | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| continue; | |
| } | |
| expect($path)->not->toBeFalse( | |
| "Failed to resolve expected plugin file: {$relativeFile}" | |
| ); | |
| $contents = file_get_contents($path); | |
| expect($contents)->not->toBeFalse( | |
| "Failed to read expected plugin file: {$relativeFile}" | |
| ); |
| it('does not use str_contains (PHP 8.0)', function () use ($files) { | ||
| foreach ($files as $relativeFile) { | ||
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | ||
|
|
||
| if ($path === false) { | ||
| continue; | ||
| } | ||
|
|
||
| $contents = file_get_contents($path); | ||
|
|
||
| if ($contents === false) { | ||
| continue; | ||
| } | ||
|
|
||
| expect(preg_match('/\bstr_contains\s*\(/', $contents))->toBe(0, | ||
| "{$relativeFile} uses str_contains() which requires PHP 8.0" | ||
| ); | ||
| } | ||
| }); | ||
|
|
||
| it('does not use str_starts_with (PHP 8.0)', function () use ($files) { | ||
| foreach ($files as $relativeFile) { | ||
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | ||
|
|
||
| if ($path === false) { | ||
| continue; | ||
| } | ||
|
|
||
| $contents = file_get_contents($path); | ||
|
|
||
| if ($contents === false) { | ||
| continue; | ||
| } | ||
|
|
||
| expect(preg_match('/\bstr_starts_with\s*\(/', $contents))->toBe(0, | ||
| "{$relativeFile} uses str_starts_with() which requires PHP 8.0" | ||
| ); | ||
| } | ||
| }); | ||
|
|
||
| it('does not use str_ends_with (PHP 8.0)', function () use ($files) { | ||
| foreach ($files as $relativeFile) { | ||
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | ||
|
|
||
| if ($path === false) { | ||
| continue; | ||
| } | ||
|
|
||
| $contents = file_get_contents($path); | ||
|
|
||
| if ($contents === false) { | ||
| continue; | ||
| } | ||
|
|
||
| expect(preg_match('/\bstr_ends_with\s*\(/', $contents))->toBe(0, | ||
| "{$relativeFile} uses str_ends_with() which requires PHP 8.0" | ||
| ); | ||
| } | ||
| }); | ||
|
|
||
| it('does not use nullsafe operator (PHP 8.0)', function () use ($files) { | ||
| foreach ($files as $relativeFile) { | ||
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | ||
|
|
||
| if ($path === false) { | ||
| continue; | ||
| } | ||
|
|
||
| $contents = file_get_contents($path); | ||
|
|
||
| if ($contents === false) { | ||
| continue; | ||
| } | ||
|
|
There was a problem hiding this comment.
Each loop continues when a target file is missing/unreadable, which can produce false-green results (compatibility constraints aren’t actually enforced for that file). Consider asserting that realpath() and file_get_contents() succeed for every expected plugin file so the test fails if it can’t validate the source.
| it('does not use str_contains (PHP 8.0)', function () use ($files) { | |
| foreach ($files as $relativeFile) { | |
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | |
| if ($path === false) { | |
| continue; | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| continue; | |
| } | |
| expect(preg_match('/\bstr_contains\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_contains() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use str_starts_with (PHP 8.0)', function () use ($files) { | |
| foreach ($files as $relativeFile) { | |
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | |
| if ($path === false) { | |
| continue; | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| continue; | |
| } | |
| expect(preg_match('/\bstr_starts_with\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_starts_with() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use str_ends_with (PHP 8.0)', function () use ($files) { | |
| foreach ($files as $relativeFile) { | |
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | |
| if ($path === false) { | |
| continue; | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| continue; | |
| } | |
| expect(preg_match('/\bstr_ends_with\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_ends_with() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use nullsafe operator (PHP 8.0)', function () use ($files) { | |
| foreach ($files as $relativeFile) { | |
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | |
| if ($path === false) { | |
| continue; | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| continue; | |
| } | |
| $readPluginFile = function (string $relativeFile): string { | |
| $path = realpath(__DIR__ . '/../../' . $relativeFile); | |
| if ($path === false) { | |
| throw new \RuntimeException("Failed to resolve expected plugin file: {$relativeFile}"); | |
| } | |
| $contents = file_get_contents($path); | |
| if ($contents === false) { | |
| throw new \RuntimeException("Failed to read expected plugin file: {$relativeFile}"); | |
| } | |
| return $contents; | |
| }; | |
| it('does not use str_contains (PHP 8.0)', function () use ($files, $readPluginFile) { | |
| foreach ($files as $relativeFile) { | |
| $contents = $readPluginFile($relativeFile); | |
| expect(preg_match('/\bstr_contains\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_contains() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use str_starts_with (PHP 8.0)', function () use ($files, $readPluginFile) { | |
| foreach ($files as $relativeFile) { | |
| $contents = $readPluginFile($relativeFile); | |
| expect(preg_match('/\bstr_starts_with\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_starts_with() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use str_ends_with (PHP 8.0)', function () use ($files, $readPluginFile) { | |
| foreach ($files as $relativeFile) { | |
| $contents = $readPluginFile($relativeFile); | |
| expect(preg_match('/\bstr_ends_with\s*\(/', $contents))->toBe(0, | |
| "{$relativeFile} uses str_ends_with() which requires PHP 8.0" | |
| ); | |
| } | |
| }); | |
| it('does not use nullsafe operator (PHP 8.0)', function () use ($files, $readPluginFile) { | |
| foreach ($files as $relativeFile) { | |
| $contents = $readPluginFile($relativeFile); |
…dabot - Throw RuntimeException when realpath/file_get_contents fails (previously silent continue hid unscanned files) - Fix Dependabot ecosystem from npm to composer - Remove committed .omc session artifacts, add .omc/ to .gitignore Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
Signed-off-by: Thomas Vincent <thomasvincent@gmail.com>
|
Converted to draft to serialize the stack in this repo. Blocked by #250; will un-draft after that merges to avoid cross-PR merge conflicts. |
Summary
Test plan
composer install && vendor/bin/pestpasses