Hindsight Score: How Old Predictions Are Scored

Pages in Past Futures score old forecasts against what happened. The rules below are fixed before any scoring starts and are the same for every scored page on Hoax and Hindsight, so a reader can redo a score or challenge a single judgment.

Rule 1: the full text, then the items

The scored page reproduces the prediction in full, or cites exactly where the full text can be read: the publication, the date and the page number. A forecast often makes several claims at once, so it is split into items, each a claim that can be checked on its own. Passages that the text does not present as a forecast, such as plot, invented names and jokes, are not split into items; the scored page lists what was set aside and why. The list of items is fixed and published before any item is scored, so the split cannot be adjusted to improve the result.

Rule 2: a deadline and a test for each item

Each item gets two things in writing. The first is a deadline. If the original text gives one, that date is used. If it does not, the deadline comes from the title or context of the publication; a forecast for “the next hundred years” printed in 1900 is read as running to 2000. Where a deadline comes from context rather than from the text, the page says it is the site's reading. The second is the observable test: the fact that will decide whether the item came true.

Rule 3: four grades

Each item receives one of four grades.

• Hit: the claim came true.
• Partial hit: part of the claim came true; the page says which part.
• Miss: the claim did not come true.
• Not scorable: the wording is too vague to name a test, the deadline has not yet passed, or no source read for the page can settle the test. Finding no record of something is not, by itself, enough to grade it a miss.

Items graded not scorable are left out of the denominator, so vague claims cannot pad or sink a score. Their number is still published with the score.

Rule 4: two hit rates

Each scored page gives two numbers. The strict rate counts hits only. The lenient rate counts hits and partial hits together. Both are written as a count over the number of scorable items, not as a bare percentage. Every item comes with a one-sentence reason for its grade and the source the grade rests on, and the full list can be downloaded as a CSV file beside the article, so both rates can be recomputed from the table.

Rule 5: changes are kept, not erased

If a rule or a single judgment is later changed, the old judgment stays in the data file, in a column for earlier grades, and the change is logged on the Corrections page and at the foot of the article. Scores are never silently rewritten.

Reading the Docket card on a scored page

On a scored page, the field headed The verdict begins with the words Hindsight score:. It then gives the strict rate and the lenient rate with their denominators, the number of items graded not scorable, and the date of judgment in year-month-day form. The Status field says whether every deadline has passed, or how many items are not yet due. When a deadline falls due, the page is revisited and rescored under these same rules, and the date of judgment moves with it.

What is not scored

Not every page in Past Futures is a scored forecast. A page that traces a famous quotation back to the earliest printing that can be found checks a source, not a prediction, so it carries no Hindsight score; its verdict field reports the finding of the source check instead.

A score measures one thing: how the items fared against what happened. It does not grade the people who made the forecast. The site's interest in an old prediction lies as much in what it shows about its own time as in whether it came true; see the time-stamped wish.

The general rules for sources, quotations and data files are on the Retrial method page.