Where Automation Should Stop and Judgment Should Start

Somewhere in most heavily automated workflows there’s a rule that was written to handle a decision nobody could actually articulate cleanly, and the rule handles it by picking a threshold — a dollar amount, a score, a percentage — that sounds precise but was really just someone’s best guess about where to draw a line through a genuinely ambiguous judgment call. The automation runs perfectly, confidently, on both sides of that threshold. It just isn’t clear the threshold was ever the right place to draw the line, and because it’s automated, nobody’s actively reconsidering it the way a human making the same call manually would naturally reconsider a borderline case.

This is the real risk in pushing automation further into a workflow than the underlying decision logic actually supports: automation doesn’t eliminate judgment calls, it just encodes whatever judgment call existed at the moment the rule was written, and then applies it forever, uniformly, without the ongoing reconsideration a human decision-maker would naturally bring to a case that felt different from the ones before it.

Automation Is Excellent at Applying Rules, Not at Noticing When Rules Don’t Fit

The genuine strength of automation is consistency — it applies the same logic to every case, every time, without fatigue, mood, or favoritism affecting the outcome. That consistency is exactly what makes it a poor fit for decisions that inherently require weighing several soft, context-dependent factors against each other in a way that resists being reduced to a formula. A human reviewing a borderline case notices when the situation in front of them doesn’t quite match the pattern the rule was written for, and adjusts. An automated rule has no such capacity — it applies the same threshold regardless of whether the specific case in front of it is a good fit for that threshold’s original reasoning or not.

Signs a Decision Point Was Automated Too Far

There’s a recognizable pattern when automation has been pushed past the point the underlying decision logic can support: an increasing number of edge cases requiring manual override after the fact, a threshold that gets adjusted reactively every time a bad outcome makes news internally, or a rule so complex — dozens of nested conditions trying to capture every scenario — that it’s actually trying to simulate judgment through sheer branching complexity rather than admitting the decision needs an actual person weighing context.

Signal	What it suggests
Frequent manual overrides of an automated decision	The rule doesn’t match real-world case variety well
Threshold adjusted reactively after each bad outcome	The line was never grounded in a stable underlying principle
Rule has grown extremely complex with many nested conditions	Automation is trying to simulate judgment rather than apply a clear rule
Cases cluster suspiciously right at the threshold	People are gaming a known cutoff rather than the underlying intent being served

A Useful Test: Can the Rule Be Explained Without Reference to the Threshold Itself

One diagnostic worth applying to any automated decision point: can someone explain why the threshold sits where it does, in terms of the actual underlying goal, without just restating the number itself? “Claims under this amount get approved automatically because the historical fraud rate below this line is low enough to accept the residual risk” is a real, defensible rationale grounded in something outside the number. “Claims under this amount get approved automatically because that’s the threshold we set” is not a rationale — it’s just restating the rule, and it’s a signal that the threshold may have been chosen more out of the need to draw some line than out of a defensible connection to the actual goal the rule is supposed to serve.

Building in a Human Checkpoint Without Losing the Efficiency Gains of Automation

The answer isn’t necessarily to pull the whole decision back into manual handling — that discards the genuine efficiency gains automation was providing for the cases where the rule really does fit well. A better middle ground routes the clear-cut majority through full automation and routes cases near the boundary, or cases with unusual combinations of factors the rule wasn’t really designed to weigh, to a human reviewer specifically. This keeps most of the efficiency gain intact while reserving judgment for exactly the cases where judgment is actually needed, rather than either full automation with no exceptions or full manual review that discards the benefit automation was providing for the easy cases.

Revisiting Thresholds on Purpose, Not Just After a Failure

Thresholds set once and left alone tend to drift out of alignment with reality as conditions change, and the only trigger that usually prompts a review is a bad outcome dramatic enough to generate internal attention — which means the threshold gets adjusted reactively, under pressure, rather than proactively as part of routine maintenance. Building a scheduled, periodic review of any automated decision threshold, independent of whether anything has recently gone wrong, catches drift before it produces a headline-worthy failure and allows for calmer, more reasoned adjustment than a review conducted in the aftermath of a visible mistake.

People Will Learn Where the Line Is, and Some Will Aim For It

Any fixed, known threshold in an automated workflow creates an incentive for anyone whose outcome depends on it to structure their case to land on the favorable side of that line, whether or not doing so reflects the actual substance the rule was meant to evaluate. This is worth anticipating rather than treating as a surprising discovery each time it happens — clustering right at a threshold is a predictable consequence of publishing a fixed, gameable line, and it’s a strong argument, on its own, for keeping the most consequential thresholds under periodic human review rather than treating them as permanently settled.

Automation Should Handle the Clear Cases and Surface the Unclear Ones

The workflows that get this right treat automation as a tool for handling volume efficiently on the cases where the underlying logic genuinely holds, while explicitly building a path for judgment to re-enter wherever a case sits close to a boundary, involves an unusual combination of factors, or simply doesn’t resemble the pattern the automated rule was built around. That’s a more deliberate design than either extreme — full manual review or full automation with no exceptions — and it’s the version that actually respects what automation is good at, and what it was never going to be able to replace.