In one sentence: require two consecutive runs in the same direction before alerting, trigger on the edge rather than the level, and make every alert name the evidence — otherwise people mute the channel within a month and the system is worse than nothing.
Bug one: firing on a single run
Generative answers vary between identical runs. A metric built on them will move several points with no cause, every cycle. An alert wired to a single run therefore fires on noise, and the second or third false alarm teaches everyone to skim past it. After that the real alert arrives into a channel nobody reads.
Two consecutive runs in the same direction, or it is not an alert. Variance is rarely directional twice.
Bug two: level instead of edge
We shipped this one. An alert written as "fire when mention rate is below X" re-fires on every run for as long as the condition holds. A brand sitting below the threshold for three months generates three months of identical alerts, which is indistinguishable from spam.
The correct trigger is the transition: fire when the condition becomes true, and again only when it becomes false and then true again. State has to be stored per site per condition, which is more work than a threshold check and is the entire difference between a useful alert and a muted channel.
What a useful alert contains
| Element | Why |
|---|---|
| What changed, with both values | "Mention rate 34% → 21%" is actionable; "visibility dropped" is not |
| Which runs, with dates | Lets the reader check whether a collection failure explains it |
| Which questions moved | A change concentrated in one intent group has a cause; an even spread is noise |
| A link into the evidence | The answer text is what distinguishes a detection bug from a real change |
| What did not run | Prevents reading a collection gap as a decline |
What is worth alerting on at all
Less than people expect. Most metrics in this category move too much to be worth a notification. The ones that earn it:
- A named competitor appearing or disappearing consistently. An event with a findable cause.
- Loss of crawl access. Binary, catastrophic, and usually caused by a deploy that nobody connected to this.
- Citation source mix shifting. Moves slowly, so movement means something.
- A collection failure streak. The alert that protects every other alert.
The rule that keeps the channel alive
Every alert should be one someone would act on today. If an alert is informational, it belongs in the weekly summary, not in a notification. A channel where nine of ten messages need no action is a channel that has already been muted; you just have not been told yet.