Skip to content

Notify in CI when a slow full analysis ran without a pre-existing result cache#5954

Open
phpstan-bot wants to merge 4 commits into
phpstan:2.2.xfrom
phpstan-bot:create-pull-request/patch-a9i3o77
Open

Notify in CI when a slow full analysis ran without a pre-existing result cache#5954
phpstan-bot wants to merge 4 commits into
phpstan:2.2.xfrom
phpstan-bot:create-pull-request/patch-a9i3o77

Conversation

@phpstan-bot

Copy link
Copy Markdown
Collaborator

Summary

In long-running CI pipelines that do not persist PHPStan's result cache, every run is a full analysis and can be slow. This adds a DX hint that tells the user to persist the result cache directory when a full run was slow and no result cache existed at start.

The message is shown only when all of these hold:

  • PHPStan is running in CI (detected via OndraM\CiDetector)
  • The run is a full analysis over real paths (not --only-files)
  • The result cache file did not exist when PHPStan started
  • The analysis took longer than 60 seconds

It is deliberately not shown when the result cache existed but was invalid (stale metadata, expired, corrupted, changed extension file, …) and therefore triggered a full re-analysis.

Changes

  • src/Analyser/ResultCache/ResultCacheManager.php: add resultCacheExists() which checks is_file($this->cacheFilePath). It is queried before restore() so it is unaffected by the @unlink() calls restore() performs on a corrupted cache.
  • src/Command/AnalyseApplication.php: capture resultCacheManager->resultCacheExists() before restoring and pass it to AnalysisResult.
  • src/Command/AnalysisResult.php: new constructor parameter resultCacheExisted (defaults to true) plus resultCacheExisted() getter; propagated in withFileSpecificErrors().
  • src/Command/AnalyseCommand.php: new RESULT_CACHE_CI_NOTIFICATION_ELAPSED_LIMIT constant and reportMissingResultCacheInCi(), called at the end of a full analysis; preserve the new flag when the AnalysisResult is rebuilt for the internal-errors path.

Root cause

This is a feature request, not a bug. The key distinction the issue asks for — "cache did not exist" vs "cache existed but was invalid" — was not expressible: AnalysisResult only knew whether the cache was used (isResultCacheUsed()), which is false in both cases. The fix adds the missing piece of information (did the cache file exist at start) and surfaces it where the decision is made. Capturing it before restore() matters because restore() deletes a corrupted cache file, which would otherwise make a present-but-invalid cache look like it never existed.

Test

tests/PHPStan/Command/ResultCacheCiNotificationTest.php drives AnalyseCommand via CommandTester in an isolated temp working directory (own tmpDir), forcing the elapsed-time condition by passing an analysisStartTime 120 seconds in the past and toggling the GITHUB_ACTIONS env var:

  • notifies on a slow full analysis in CI with no result cache (fails without the fix);
  • does not notify on the second run once the cache exists;
  • does not notify when the analysis is fast;
  • does not notify outside CI.

The reported scenario is the only construct on this axis: the baseline-generation and PHPStan Pro paths return before reaching this code (and Pro is blocked in CI anyway), and the internal-errors path intentionally skips the hint. No analogous sibling paths needed the same change.

Fixes phpstan/phpstan#14881

…ult cache

- Add `ResultCacheManager::resultCacheExists()` to report whether the result
  cache file was on disk when PHPStan started, before `restore()` can unlink it.
- Thread that fact through `AnalyseApplication` into a new
  `AnalysisResult::resultCacheExisted()` getter (defaults to true so a stale but
  present cache that forced a full re-analysis does not trigger the message).
- In `AnalyseCommand`, after a full analysis, print a hint to persist the result
  cache directory when: running in CI (detected via `CiDetector`), analysing
  full paths (not `--only-files`), the cache file did not exist at start, and the
  run took longer than `RESULT_CACHE_CI_NOTIFICATION_ELAPSED_LIMIT` (60s).
- The message is intentionally suppressed when the cache existed but was invalid.
Comment thread src/Command/AnalyseCommand.php Outdated
Add the notifyAboutMissingResultCacheInCi feature toggle (off by default,
on under bleedingEdge.neon) and only emit the missing-result-cache CI hint
when it is enabled.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@ondrejmirtes

Copy link
Copy Markdown
Member

@staabm I'd do it for everyone, not just bleeding edge (it doesn't break anything, doesn't make sense).

I wouldn't do this kind of test in tests/, I hate these.

Instead, I'd make the time configurable and do some E2E tests in e2e-tests.yml.

Also, the message should change to have more "call-to-action". Everyone knows it "took more than one minute" but it should say that the CI runs could be much faster.

Also would be nice to pair this with phpstan/phpstan#8599.

@staabm staabm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@phpstan-bot adjust the PR according to feedback in #5954 (comment)

Also would be nice to pair this with phpstan/phpstan#8599.

do not work on this item. we will work on it in a separate PR

phpstan-bot and others added 2 commits June 30, 2026 07:56
…ld configurable

Drops the bleeding-edge-only feature toggle so the slow-full-analysis hint
runs for all users, and replaces the hardcoded 60s constant with a
resultCacheCiNotificationSeconds parameter. The message is reworded to be a
clearer call to action about speeding up CI.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an e2e/result-cache-ci-notification case driven from e2e-tests.yml:
a fresh run emits the hint, the cached re-run does not. Replaces the
removed CommandTester-based unit test.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@phpstan-bot

Copy link
Copy Markdown
Collaborator Author

I've addressed the review feedback from ondrejmirtes' comment. Here's what changed:

1. Run for everyone, not just bleeding edge

Removed the notifyAboutMissingResultCacheInCi feature toggle entirely (conf/bleedingEdge.neon, conf/config.neon, conf/parametersSchema.neon) and dropped the early-return gate in reportMissingResultCacheInCi(). The hint now runs for all users.

2. Made the time configurable

Replaced the hardcoded RESULT_CACHE_CI_NOTIFICATION_ELAPSED_LIMIT = 60.0 constant with a new resultCacheCiNotificationSeconds parameter (default 60), read from the container at the call site. This lets the threshold be lowered in tests/config.

3. Replaced the tests/ test with an E2E test

Deleted ResultCacheCiNotificationTest.php (the CommandTester-driven test ondrejmirtes disliked) and added e2e/result-cache-ci-notification/ plus a job in .github/workflows/e2e-tests.yml. The e2e config sets resultCacheCiNotificationSeconds: 0 so the slow-analysis condition is always met; the first run (no cache) emits the hint, the cached re-run does not.

4. Stronger call-to-action message

Reworded from "This analysis took more than a minute…" to:

Tip: This CI run analysed your whole project from scratch, which is why it was slow.
Persist PHPStan's result cache directory between CI runs to make your pipeline dramatically faster.
Only changed files and their dependencies are re-analysed, so most runs finish in a fraction of the time.
Learn how to set it up: https://phpstan.org/user-guide/result-cache

I did not touch the #8599 pairing, as instructed.

Verification

  • make phpstanOK, no errors (and it fired the notification live, since make phpstan clears the cache first and runs in CI — end-to-end proof).
  • Ran the E2E scenario locally: run 1 (no cache) shows the hint, run 2 (cache present) does not.
  • tests/PHPStan/Command + tests/PHPStan/DependencyInjection — 205 tests green (confirms the schema/container still load after the toggle removal and new parameter).

Two commits pushed to create-pull-request/patch-a9i3o77. (Note: the unit-test deletion landed in the first commit since it was pre-staged from git rm, rather than alongside the E2E addition — functionally equivalent, both are in the PR.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DX: PHPStan should message about missing result cache in long running CI

3 participants