Correcting Intelligence Failures: Structure, Process, Review

Correcting Intelligence Failures: Structure, Process, Review

CommandEleven Intelligence Desk // Tradecraft Desk, Tradecraft Primer. Part of the series How CommandEleven Intelligence Thinks.

Bottom Line Up Front

The official reviews that followed the 9/11 attacks and the Iraq WMD assessments did not stop at describing what went wrong. They recommended corrections, and those corrections fall into three kinds. Structural corrections change who leads and who shares. Process corrections change how analysts reason and how they write. Review corrections check the work and feed the lessons back.

Both reviews were clear about limits. The 9/11 Commission wrote that structure alone is not enough, and Heuer reports that awareness of bias does not reliably remove it. This primer sets out what was recommended, where the recommendations meet the failure points from the earlier primers, and what they can and cannot be expected to fix. It is the last primer in the series.

Corrections Follow the Failures

The first primer in this group located failures at steps of the intelligence cycle, and the second explained the causes in the mind, the evidence and the organization. The recommendations in the official reviews line up with both. The table below pairs each failure point with the correction recommended for it.

Table pairing six failure points with the correction recommended for each, from the 9/11 Commission, the WMD Commission and the analytic standards that followed

Structural Corrections

The 9/11 Commission argued that the government’s problem was a lack of unity of effort, and its recommendations reorganized who is responsible. It proposed a National Counterterrorism Center that would pool foreign and domestic information, produce net assessments and provide warning, with an operations arm that would plan case management and assign tasks to agencies without directing how they carried them out. It proposed replacing the Director of Central Intelligence with a National Intelligence Director who would oversee national intelligence centers and manage the national intelligence program. On information sharing, it argued that the Cold War habit of need to know should give way to a culture of need to share, and that information should be separated from sources and methods when reports are written so that more users can reach it.

The 2004 reform law established a Director of National Intelligence, and the WMD Commission, reporting in March 2005, built on it. It recommended that the Director use the budget and personnel powers the law granted to lead the community, and that Mission Managers on the Director’s staff direct collection, oversee analysis and act as a clearinghouse for priority targets. It also recommended integrated management of collection across agencies and a new Human Intelligence Directorate at the CIA to coordinate espionage operations. Its statement on information sharing was that information belongs to the government as a whole, not to individual agencies.

Process Corrections

Structural change does not change how an individual analyst thinks, so both the reviews and the standards that followed reach into the analytic process.

The WMD Commission recommended competing and independent analysis, including a designated devil’s advocate for consensus views, an outside research institute, more transparent reasoning in finished products, more analyst training and a National Intelligence University. It also recommended that Mission Managers encourage alternative views and make sure dissenting opinions reach customers. For the daily brief, it recommended dropping misleading headline summaries and having the Director oversee the brief so that it reflects alternative views.

Intelligence Community Directive 203 turns much of this into everyday standards. It asks analysts to identify and explain uncertainty, to separate intelligence from assumptions and judgments, to consider alternatives, to flag changes from earlier judgments with an explanation of what caused them, and to make judgments precise enough to be assessed for accuracy later. It also says analysis must not be shaped by advocacy for an audience, agenda or policy preference, and that analysts should weigh alternative views and contrary evidence. Heuer’s Analysis of Competing Hypotheses, which we cover in Structured Analysis, is a technique for the same purpose: it tells analysts to proceed by trying to disprove hypotheses, not prove them.

Review Corrections

The third kind of correction checks the work after it is done. The reform law assigns an individual or entity to ensure that finished intelligence is timely, objective, independent of political considerations and based on proper analytic tradecraft. That person or office may oversee analysis but may not be directly involved in producing specific products. It reviews products regularly, and the statute lists what a review should ask: whether the product used all available sources, caveated its uncertainties, separated intelligence from the analysts’ assumptions and, where appropriate, incorporated alternative analyses. The lessons are meant to feed into training.

ICD 203 adds that each community element must run a program that evaluates its own analytic products against the standards. It also gives analysts a route to raise concerns about bias or politicization, first inside their element and then to an Analytic Ombuds. The WMD Commission likewise recommended lessons-learned reviews as part of sustained leadership and oversight.

A Worked Example: The Review Questions on the Aviation Warning

This is an illustration, continuing the aviation example from the earlier primers. It applies the four review questions from the statute to a hypothetical assessment of the warning. It is not a review of any real product.

Four review questions from the statute on analytic integrity applied to a teaching example, each with the answer a reviewer would look for

Each question is a plain check that can be done before a judgment is published. A product that cannot answer one of them has a gap that a reader, or a reviewer later, will find. The value of the list is that the questions are fixed in advance, so they do not depend on whether the analyst happens to feel doubtful.

What Corrections Cannot Fix

The reviews are candid about limits. The 9/11 Commission wrote that structure alone is not enough, and that good people can overcome bad structures though they should not have to. It acknowledged that a new center would not remove interagency disputes, that concentrating authority carries its own risk, and that agencies would have to give up some turf. Heuer reports that awareness of a bias does not reliably remove it, which is why the corrections above change the process around the analyst instead of asking the analyst to be better. And no correction can make evidence complete, so judgment under uncertainty will remain.

Our test of a correction is whether it changes what happens at a break point. A reform that does not change who asks the question, who sees the dissent, or who checks the work leaves the failure where it was.

How We Apply This

We apply the review corrections on a small scale. Each product states what is confirmed and what is judged, and our confidence tiers make the weight of each claim visible. When a judgment changes, our Revision Annex records what changed and why. Our Foresight Record lets readers check our past judgments against what happened. We do not have a separate review office, and we do not claim the protections of one. What we can do is publish our reasoning in a form that others can check.

Common Errors

  • Treating reorganization as the cure. Both reviews said structure alone is not enough.
  • Relying on awareness. Knowing about a bias does not reliably remove it.
  • Adding dissent without a route to the reader. A devil’s advocate only helps if the dissent reaches the person deciding.
  • Reviewing only after a failure. The review questions work as routine checks, not only as a postmortem.
  • Declaring the problem fixed. Corrections reduce risk. Neither review claims they remove it.

Frequently Asked Questions

What did the 9/11 Commission recommend for intelligence?

A National Counterterrorism Center to pool foreign and domestic information, a National Intelligence Director to lead the community, and a shift from need to know to need to share.

What did the WMD Commission recommend for analysis?

Competing and independent analysis, a designated devil’s advocate, an outside research institute, more training and transparent reasoning, and making sure dissenting views reach customers.

What is ICD 203 for?

It sets the analytic standards for the US Intelligence Community, and requires each element to evaluate its analytic products against them.

Do these reforms prevent intelligence failures?

No. The reviews themselves say structure alone is not enough, and Heuer reports that awareness of bias does not reliably correct it. The reforms aim to change what happens at the break points.

How does CommandEleven Intelligence apply these ideas?

By labeling confidence, separating confirmed fact from judgment, recording changes in judgments and publishing a record of our past forecasts.

Where do I start with the series?

How CommandEleven Intelligence Thinks lists every primer in order.

Sources

How to Read Our Labels

Every claim in a CommandEleven Intelligence dossier carries one of these.

CONFIRMED

Multiple independent sources, direct corroboration, or reporting from a major established outlet. Stated as fact, with its source.

ASSESSED

Our reasoned judgment from the available evidence. Uncertainty remains.

REPORTED

One outside report we have not corroborated, from a source without established standing. Attributed, not vouched for.

ALLEGED

A party's claim about someone else's conduct. Attributed, not treated as established.

SPECULATIVE

Plausible but thinly supported. Context only, and never weighs on a conclusion.

DATA DEFICIT

Notation: the information does not exist or is out of reach. Stated, not guessed.

Read the full primer: Confidence Tiers, How We Label What We Know

Related dossiers

More from CommandEleven Intelligence

Opens in a new tab
Vol. I No. 1 is out
The monthly magazine

Convergence

Our dossiers, drawn together each month into one edition.

Get the first edition →
Publishing mid-December
The annual report

2027 Business Threat Exposure

Ten threats to American business, ranked by exposure.

Get notified →
The scorecard

Foresight Record

Every call we have made, scored in public against what happened.

Open →
The audit

GCTA Audit

The April 2026 forecasts, checked section by section.

Open →
The panel

Survival Dispatch News

Our regular panel appearances, with the full conversation.

Open →
The weekly show

Soft Targets America

Weekly on Tuesdays, with co-host Jason Piechura.

Open →
For organizations
Our core service

Intelligence as a Service

Private intelligence, delivered to your team.

Engage us →