The incident record says the problem began at 14:32.
That is the timestamp on the first alert, the moment a latency graph crossed a line drawn months earlier. It is precise, defensible, and where the postmortem will start counting.
It is not when the problem began.
The chain had already been forming. A pull request changed a caching path the week before. A deployment carried it into production that morning. Telemetry had been drifting for an hour, below the threshold. Two engineers traded a half-joke in chat about the service "feeling slow," and a support agent tagged a ticket "probably nothing." Each was inconclusive alone. Together, they formed the chain that would become the incident.
What the start time actually measures
The moment an alert fires is a fact about the alert, not about the failure.
Alerting thresholds are detection lines. They tell you when a signal grew large or persistent enough to demand attention. They say nothing about when the underlying condition began to form. A system that pages at 14:32 may have been degrading since 13:40, and the change behind it merged days earlier.
Treating the alert as the origin has a cost. It frames the incident as a moment, something that happened, rather than a chain, something that developed. A moment invites a single cause. A chain invites an investigation.
The alert is not where the incident starts. It is the first link in the chain that was loud enough to hear.
The evidence was already there
Almost everything a responder needs in order to understand an incident usually exists before the incident is recognised.
The change that introduced the risk is in version control, with an author and a diff. The deployment that carried it is in a pipeline log. The drift is in a metrics store. The unease is in a review comment or a chat message. The first customer signal is in a support queue, ahead of monitoring. None of it is hidden; it is simply scattered.
The scattering is structural, not a failure of any particular product. Monitoring sees production behaviour, but not always the engineering activity that produced it. Source control records what changed, not what it did downstream. Chat holds the hunches and hesitations, in a stream that is hard to search by the time it matters. Ticketing records work only after it has a name. Each system holds one true fragment, and none was built to reason across the whole chain. The spaces between them are where an incident quietly assembles itself.
The reconstruction tax
So when the alert finally fires, the work begins from the wrong end. The responder starts at the visible failure and rebuilds backwards: comparing timestamps, scanning deployment history, scrolling chat for a half-remembered comment, paging whoever touched that service last. Piece by piece, under pressure, they reconstruct a chain that already existed in the systems in front of them.
This is the reconstruction tax, and it falls due at the worst possible time. The obvious cost is minutes added to every incident. The quieter cost is worse. It consumes senior attention when that attention is scarcest, and rewards speed over rigour. Twenty minutes into assembling fragments, a responder is under pressure to commit to the first story that fits, before the evidence is complete.
From observation to grounded recommendation
There is a better shape for this work. It helps to name the stages between raw activity and a decision worth acting on.
Observations
→ Evidence
→ Context
→ Assessments
→ Recommendations
An observation is something that happened: a deploy, a metric sample, a message, a ticket. Evidence is an observation weighed against a claim, something that can support or challenge it. Context ties that evidence to systems, changes and time, connecting a deployment to the service that degraded and the migration it carried. An assessment interprets the available evidence and says how confident that reading is. A recommendation proposes the next action while preserving the uncertainty that remains.
One property runs through all of it: provenance. Every assessment should trace back to its evidence, and every piece of evidence to the observation it came from. A conclusion that cannot be traced is only a guess in a confident voice.
Continuous engineering intelligence
The stronger model does not wait to be asked. Most tooling is interrogative: it sits quietly until an incident is declared, then answers as fast as it can. Continuous engineering intelligence inverts that posture. It watches engineering activity as it happens, preserves provenance, and connects related evidence while the connections are cheap, so the context a responder needs is already assembled and traceable.
This is not about producing more alerts. Alert volume was never the problem. It is not a chatbot summarising a channel after the damage is done, nor a model licensed to assert causes it cannot support. It is something quieter and more durable: a standing, evidence-grounded understanding of engineering activity that carries forward through time.
That is the direction Ember is being built toward: not a louder alarm, but a memory that was paying attention before the alarm was needed.
Better decisions, not more certainty
The goal is not to manufacture certainty faster. A system worth trusting holds four things apart: what was observed, what the evidence supports, what remains uncertain, and what to do next. Collapsing those into one confident answer is how incidents get misdiagnosed.
That discipline matters most around cause. A deployment that landed minutes before an alert, in the affected service, is worth investigating first. It is not, on that basis alone, the cause. A good assessment can rank it as the place to start and still decline to call it guilty until the evidence closes the gap. Correlation earns attention. Evidence earns a conclusion.
The system that was paying attention
Incident response has spent years getting faster at the moment of failure: quicker alerts, tighter runbooks and better dashboards once the pager goes off. The larger opportunity sits earlier, while the chain is still forming.
The incident did not start at 14:32.
The understanding of it should not have to either.