Navigating the AI WorldPart A · For everyone who uses AI at workChapter 4 of 28

Judgement, overreliance and the fluency trap

The analytics and contract examples later in this chapter show what a good check looks like. This section is about what happens when people stop checking.

There is a recognizable pattern in how teams adopt these tools. Week one, everyone verifies. Month three, the tool has been right often enough that verification feels like ceremony. Month six, a wrong answer reaches a client, a regulator or a board, and the person who forwarded it cannot explain how it was produced.

That is not carelessness. It is how trust works. The tool earns it the same way a colleague would, but it does not reliably signal when confidence is misplaced. A fluent answer can sound settled even when the evidence is weak.

Three habits hold against it.

Keep the authoritative system authoritative. Numbers come from the query. Clauses come from the signed document. Balances come from the system of record. The model interprets, drafts and explains. It does not become the source. If your workflow lets a generated figure become the figure of record, fix the workflow.

Name the error that would change the decision. "Review the output" is not an instruction. "Confirm that the termination trigger in the summary matches clause 8.2" is. Before you read the answer, write down the one thing that would make it wrong. Then look for that.

Notice what you can no longer do. If you have not written a first draft, reconciled a figure or read a full contract in six months, your ability to judge the tool's output on those tasks is decaying. That is a career risk, not a productivity gain. Deliberately do some of the work by hand.

Bias and decisions about people. Any use that touches hiring, ranking, performance, health or discipline changes the harm profile completely. Accurate extraction from a record does not make a decision about a person defensible. If a tool is being used to decide about people, someone with authority should have decided that it may, on what criteria, and how those people can contest the result. Chapter 20 returns to this from the governance side.

Different professions need different checks. A generic instruction to "review the AI output" is too vague to be useful. Decide which part of the work remains authoritative and what error would change the decision.

Analytics with numbers that reconcile

An analyst asks for an explanation of monthly revenue. In a fictional example, May revenue is 1.20 million and June revenue is 1.08 million in the same currency. The decrease is 0.12 million, or 10 percent relative to May. A query or spreadsheet should calculate those values. The model can describe the result and propose questions to investigate.

Suppose the model blames customer churn. The supplied totals establish a decline, but provide no customer retention evidence. The analyst marks churn as an untested hypothesis and checks customer counts, refunds, recognition timing and currency treatment. A polished narrative that assigns a cause prematurely could send management after the wrong problem.

A reusable report records the data snapshot, query version, reporting period, units and reconciliation to the source. The application rejects output with inconsistent units or an unexplained total. Customer-level data stays within the approved analytics boundary. When row-level access differs, the query enforces it before results enter context.

Contracts with the clause in view

A manager compares a supplier proposal with a fictional agreement. Clause 8.2 says, “Either party may terminate for material breach after 30 days to cure.” The proposal summary calls this “termination for convenience with 30 days' notice.” That changes both the trigger and the meaning of the period.

A useful assistant identifies the discrepancy, links the relevant clause and leaves the commercial decision to the reviewer. It records the agreement version, separates quoted terms from interpretation and flags missing schedules or amendments. It should not conclude that a provision is enforceable from the wording alone.

The review output can be a short list of deviations and questions for the contract owner. It should preserve unresolved issues rather than hide them behind a yes-or-no recommendation. The manager verifies the complete agreement and applicable review process before committing the company.

These examples establish a durable division of labour. Use deterministic systems for authoritative numbers and records. Use AI for interpretation, extraction and drafting where it helps. Keep professional judgement attached to the evidence that could change it.

This part turns the question of what to learn into a set of practical choices. Start with the work you want to improve, decide how much depth the role actually needs, and measure whether the learning changes what you can deliver.

  • 5 Where to start when everything is moving
  • 6 Choose what to know, use and master
  • 7 Build a roadmap you can revise
  • 8 Put the technology in its place
  • 9 What it costs and why
Report a correction

Corrections go to the editor and are never published automatically. No account needed.