Your CI is not flaky. Your cache expires every seven days.

Zero issues, and red anyway

The log said two things that should never appear together. First: 0 issues. Then: Timeout exceeded.

The linter had finished its work. It had read every file, found nothing wrong, and then died on the clock. It ran for 311 seconds against a budget of 300.

Eleven seconds over. On a pull request that touched one page of prose and no Go code at all.

Why re-running it made things worse

The obvious move is to press re-run. It passed. That felt like proof that the first failure was noise.

It was proof of the opposite. Two lines above the timeout, the log said: Cache not found for input keys.

The first run had a cold cache and had to do the full analysis from scratch. It also populated the cache. The second run read that cache and finished in a fraction of the time. Re-running did not clear a random glitch. It removed the only condition under which the job fails.

So the green run taught us nothing, and it hid the fact that the budget was too small.

Three questions that separate a flake from a costume

Ask them before you press re-run. They take a minute and they work for any job, in any language.

One: did the work finish? A job that produced its result and then failed is not flaky. Something outside the work killed it. Look for a completed output line above the error.

Two: what was different about this run? Cache hit or miss, cold container, a dependency fetched from the network, a first run after a weekend. Real flakes have no pattern. This one had a name printed in its own log.

Three: how close was it? A job that finishes in 40 percent of its budget is healthy. One that finishes at 90 percent is a scheduled outage. Print the duration and compare it to the limit, every run.

Find out how often the condition returns

This is the step that turns a shrug into a decision. Our cache configuration said the invalidation interval is seven days.

That means the first run after every weekly expiry is cold, and every one of those runs was over budget. Not a rare glitch. A weekly failure, hidden by the six warm days around it.

The rest of the week papers over it. That is exactly why it looked random: the failures are regular, the observations are not.

The cost of calling it a flake

A red check that says nothing about the code is not free. It teaches people that red sometimes means nothing.

After the third time, nobody reads the log. They re-run it. And the day a red check means something real, it gets the same treatment.

That is the actual damage, and it is done to your team, not to your pipeline.

What it took to fix

One line. The timeout went from 5 minutes to 15. The job still runs the same checks and still finds the same problems.

The comment above that line is longer than the change. It names the numbers: 311 seconds, 300 budget, cold cache, seven day expiry. The next person will not need to rediscover it.

That is the whole trade. One minute of reading the log instead of pressing re-run, and a class of failure disappears.