Skip to content

← Articles

Quiet Rooms and Pencils: How the Delphi Method Gathers Expert Judgment

Delphi puts experts through anonymous rounds of questions and feedback. It can blunt the loudest voice in the room, but the result depends on choices made before the first answer.

A stack of blank questionnaires with a pencil resting on top, on a desk in quiet window light, suggesting an anonymous expert panel at work.
Blank questionnaires and a pencil on a desk: the plain working tools of a Delphi panel, where judgments are gathered in private rounds rather than open debate.

A committee table has a weather of its own. Someone speaks first and firmly, someone senior nods, and by the third speaker the room has a climate. The Delphi method was built to get experts out of that weather and into something closer to a quiet room with a pencil.

A stack of blank questionnaires with a pencil resting on top, on a desk in quiet window light, suggesting an anonymous expert panel at work.
A stack of blank questionnaires on a desk, the plain working material of an anonymous expert panel.

Delphi, also called the Delphi technique or Estimate-Talk-Estimate, is a structured way to forecast and reach consensus. A panel answers several rounds of questionnaires. Between rounds, a facilitator feeds back a summary of what the group said. The aim is simple: let judgment improve without letting the loudest voice win.

Where the method came from

Delphi was developed at the RAND Corporation in the 1950s. Olaf Helmer, Norman Dalkey and Nicholas Rescher are the names usually attached to it, according to the Ian Khan overview and Wikipedia. One account ties it to Cold War military forecasting.

The motive is easy to see. Face-to-face committees reward confidence and rank. A hesitant expert with better evidence can lose to a fluent one with a better title. Delphi was a structured answer to vocal individuals, halo effects and social pressure.

It has since moved well beyond defense. Business forecasters use it. So do groups that develop professional guidelines, including in many health-related fields, per Wikipedia. That same source notes that Delphi has advantages over prediction markets in certain respects, though it does not say how the two compare on accuracy.

How the rounds run

The cycle is short to describe. First, assemble a panel with relevant and varied knowledge. Typical panels run from 10 to 50 experts. Second, send an open-ended first questionnaire, so people give forecasts and reasoning anonymously. Third, the facilitator analyzes the replies and returns a summary, such as the median, the range and the key arguments. Experts reconsider and answer again.

Then the loop repeats. Most studies run 2 to 4 rounds, and 3 is the most common, according to the monday.com guide. Stopping can happen when answers stabilize, when disagreement narrows to a few clear issues, or when a preset number of rounds is done.

The facilitator matters more than the tidy description suggests. That person writes the questions, summarizes the replies and decides when to stop. Each of those is a judgment, and each can tilt the result.

Anonymity and what the output looks like

Anonymity is the feature that separates Delphi from a panel discussion. If nobody knows who answered what, an expert can change a number without backing down in public. Reputation stops doing the arguing.

The price is that nobody can defend a position in person. That is why the better designs ask for reasons, not just numbers. A typical output is a consensus forecast, such as a 70% probability of an event by 2030 (an illustrative figure from the Khan overview). It can also include a prioritized list of drivers and a disagreement analysis that keeps minority reasoning on the record.

Variants exist. Modified Delphi, Real-Time Delphi and Wide-Band Delphi all adjust the basic design, per Wikipedia and monday.com. The sources I could verify do not detail how each differs, so check a methods text before choosing one.

How agreement is measured

Delphi facilitators do not usually count votes. They track the median and the interquartile range (IQR) across rounds. When the IQR falls within a threshold set in advance, the group is said to have reached statistical consensus.

The words "set in advance" carry the weight. A threshold chosen after seeing the data can be bent to fit a desired answer. The sources do not explain how to pick a sensible IQR threshold or how to handle experts who drop out between rounds. Those gaps are real, and a reader should expect a study to state its own choices.

  • Median: the middle answer, not a majority position.
  • IQR: the spread of the middle half of answers. A smaller number means a tighter panel.
  • Threshold: the pre-defined IQR level that counts as consensus.

Questions to ask of any Delphi result

A converged answer is only as good as the process behind it. Four questions help.

  1. Does the panel hold varied perspectives, or could it be an echo chamber?
  2. Was the consensus threshold set before the rounds began?
  3. How many rounds were planned, and what happened if answers did not settle?
  4. Were minority views and their reasoning recorded, not just the median?

Two red flags follow. One is reading the median as if it were a majority vote. The other is trusting a consensus from a panel without relevant domain knowledge. Judgmental forecasting improves when forecasters know the field and have timely information, as the Hyndman and Athanasopoulos text on forecasting notes. It also stresses documented, systematic process.

A narrow spread tells you the panel agrees. It does not tell you why.

The sources give little hard evidence on how accurate Delphi forecasts are against other methods, and their treatment of the method's limits is brief. Delphi fits best where hard data is thin and judgment is all there is. In that setting it is a reasonable tool. Ask to see the threshold, the rounds and the dissent.

Sources

Marcus Adeyemi-Holt

Marcus Adeyemi-Holt is a house writer for Deeperpectives, where he examines how ordinary routines reveal larger patterns in work, memory, and place. He draws on years of reporting on municipal records and oral histories to trace the quiet forces that shape everyday life.

Keep reading.

Puzzles · Nov 1, 2026

Einstein's Riddle: The Legend and the 1962 Magazine Page

The famous five-house logic puzzle has no documented link to Einstein. Its earliest known appearance is a 1962 issue of Life International, and its author is still unknown.

By Siobhan O'Reilly

Paradoxes · Nov 1, 2026

The Jam Table Problem and the Limits of a Famous Experiment

A 2000 study of tasting booths became a case for choosing less, but the wider evidence points to something more conditional.

By Marcus Adeyemi-Holt

Paradoxes · Nov 1, 2026

A Shortcut That Slows Everyone: Braess's Paradox and the Limits of the Map

A small traffic model shows how a new connector can leave every driver worse off, and why the result is a reason to ask harder questions about road proposals rather than a verdict on any particular road.

By Lila Serrano

Quiet Rooms and Pencils: How the Delphi Method Gathers Expert Judgment | Deeperpectives