Failure-First Ledger

What failed

• Namib sand sea was a negative control for the Moon run. It scored at percentile 90.2, where the pre-registered maximum was 25.0.

  What this means for the analogy
  A flat sand sea is not like a polar lunar ridge, yet it ranked high. The score picks up broad terrain traits and cannot tell a poor analog from a good one.

  Failed | Provisional

  Source: v1: results/moon/validation.json (68c9ae4d4477)

• Rub' al Khali was a negative control for the Moon run. It scored at percentile 75.6, where the pre-registered maximum was 25.0.

  What this means for the analogy
  A flat sand sea is not like a polar lunar ridge, yet it ranked high. The score picks up broad terrain traits and cannot tell a poor analog from a good one.

  Failed | Provisional

  Source: v1: results/moon/validation.json (68c9ae4d4477)

• The Mars primary site median was percentile 61.3, below the pre-registered minimum of 90.0.

  What this means for the analogy
  Known Mars analog sites did not rank consistently high. Mars stays a low-confidence secondary result.

  Failed | Provisional

  Source: v1: results/mars/validation.json (7df4fe948dda)

Pre-registered criteria, by run

Source: v1: results/moon/validation.json (68c9ae4d4477)

Negative controls

Controls are places that should score low. Scores are percentiles among scored tiles. The pre-registered maximum was percentile 25.0.

Mars primary site median: percentile 61.3, against a pre-registered minimum of percentile 90.0.

Source: v1: results/moon/validation.json (68c9ae4d4477)

The Moon score is terrain only

• For the Moon, removing the illumination weight leaves the top 1% of tiles unchanged (overlap 1.0000).

  What this means for the analogy
  The Moon score is effectively terrain only. Polar lighting, a defining feature of the target, plays no part in the ranking.

  confirmed by v1 files

  Source: v1: results/moon/shackleton_rim__base/alt_weight_checks.json (5c40526cf72f)

Where the analogy breaks down

These physical differences come from the v1 limits ledger. Most entries are provisional and have not been verified by us.

• Air pressure. Target: vacuum Moon (night surface pressure 3 x 10^-15 bar); Mars 6.36 mb at mean radius, under 1% of Earth. Earth: about half sea level on the highest roads (Umling La 5,798 m about 48%, NASA Glenn formula).

  What this means for the analogy
  Earth always has far more air than the Moon or Mars. Heat loss, dust and descent differ.

  Does not match | Provisional

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Cold traps. Target: below 110 K (cold trapping water over a billion years); Diviner measured surface bolometric brightness temperatures as low as 29 K at mid-day in southern summer (Paige et al. 2010, Science 330:479, doi 10.1126/science.1187726, checked against a text copy, to confirm in the Science PDF). The coldest Diviner reading anywhere is 25 K in Hermite crater near the NORTH pole in winter night (not for a south pole target). South pole winter or annual minimum not sourced. Earth: 2-m air record 184.0 K (-89.2 C, Vostok, 21 Jul 1983, WMO); satellite snow-surface minimum about 174.6 K (Scambos 2018 GRL; the exact -98.6 C value, its date and the MODIS version are UNVERIFIED).

  What this means for the analogy
  Earth has no lunar-style cold traps, so ice-related analogies do not carry over.

  Does not match | Provisional

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Electrostatic dust. Target: provisional. Earth: no global Earth data.

  What this means for the analogy
  No Earth dataset describes charged dust, so its effects are not covered.

  Does not match | Provisional

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Gravity. Target: 1.62 m/s2 Moon; Mars mean surface gravity 3.73 m/s2 (3.71 m/s2 is the equatorial value). Earth: 9.81 everywhere.

  What this means for the analogy
  Gravity cannot be matched anywhere on Earth. Mobility, dust and structure behave differently on the Moon and Mars.

  Does not match | Provisional

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Line of sight to Earth, GPS. Target: Earth line of sight is an Artemis III site criterion; Earth GPS and Galileo signals are weak but were tracked on the lunar surface once (NASA LuGRE, 3 March 2025). Earth: always available.

  What this means for the analogy
  Earth always has a view of Earth and satellites; the Moon does not.

  Does not match | Provisional

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Oxidants. Target: 0.5-1 wt% perchlorate on Mars. Earth: up to 0.6 wt% in the Atacama.

  What this means for the analogy
  Mars soil oxidants are only partly matched on Earth.

  Partly matches

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Polar lighting. Target: Sun within about 1.5 deg of the horizon all year, 18.6-year cycle. Earth: low Sun in polar spring and autumn only (pole Sun reaches 23.4 deg).

  What this means for the analogy
  Earth has only partial matches to lunar polar lighting, and the score does not use it.

  Partly matches | Provisional

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Radiation and UV. Target: no shielding; lunar surface particle dose about 60 microsieverts per hour. Earth: UV rises about 10 percent per 1000 m of altitude (WHO); ground particle dose more than 200 times lower than on the Moon, not zero.

  What this means for the analogy
  Earth's surface radiation is far lower, so exposure cannot be rehearsed realistically.

  Partly matches | Provisional

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Regolith. Target: unweathered, glass-welded, dry except in cold traps. Earth: weathered, oxidised, damp ash and basalt.

  What this means for the analogy
  Earth soils are weathered and damp; lunar regolith is not.

  Partly matches | Provisional

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Regolith mechanics (grain size, density, abrasiveness). Target: provisional. Earth: no global Earth data.

  What this means for the analogy
  Grain size, density and abrasiveness are not covered by any Earth dataset used.

  Partly matches | Provisional

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Rock chemistry. Target: iron-rich Martian basalt; lunar anorthosite. Earth: basalt 45-52% silica; anorthosite rare.

  What this means for the analogy
  Earth rarely matches lunar or Martian rock chemistry.

  Partly matches

  Source: v1: config/ledger.yaml (dc07b3129af7)

• Sterility. Target: no known life. Earth: 1.7-13.6 million cells per gram in desert cores.

  What this means for the analogy
  Earth deserts carry microbial life; Mars is treated as sterile for planning.

  Does not match

  Source: v1: config/ledger.yaml (dc07b3129af7)

What is not scored

Other known limits

• Both Mars negative controls were gated out, so the control test could not be run for Mars.

  What this means for the analogy
  There is no control result for Mars, so nothing shows whether the Mars ranking separates good from poor matches.

  Not testable | Provisional

  Source: v1: results/mars/validation.json (7df4fe948dda)

• The Moon primary site criterion could not be tested: only 1 primary site could be scored.

  What this means for the analogy
  The Moon ranking has no passed test on known analog sites. Lonar was gated out because its ground is vegetated or farmed.

  Not testable | Provisional

  Source: v1: results/moon/validation.json (68c9ae4d4477)

• Earth slope is not corrected for the map projection, so east-west slopes read too steep toward the equator and too gentle toward the poles.

  What this means for the analogy
  Slope is the heaviest family in the Moon score and is biased by latitude. A latitude band check was run to see whether the ranking holds.

  accepted by v1 authors

  Source: v1: results/moon/validation.json (68c9ae4d4477)

• Only 0.6444 of land tiles have terrain data. Tiles without terrain are never scored.

  What this means for the analogy
  Large parts of the Earth cannot appear in the search because they have no terrain data.

  coverage gap

  Source: v1: data/processed/earth_coverage.json (8a53830089f6)

• Terrain changes source at the northern edge of the main elevation dataset. Roughness steps up across that seam.

  What this means for the analogy
  Scores near and beyond the seam partly reflect a change of data source, not a change of ground.

  accepted by v1 authors

  Source: v1: docs/DECISIONS.md (f25c17dab5b4)

• 11 of the sites v1 used have coordinates that were not confirmed from a primary source.

  What this means for the analogy
  Site results are provisional until the coordinates are confirmed.

  provisional | Provisional

  Source: v1: results/moon/validation.json (68c9ae4d4477)

The v2 run

v2 run not completed

Gate results for v2 are not shown because the run has no result files here. Nothing is filled in.

• PREREG_B: reference-side thermal features (ROLE B)

  2751d789cc0be325ed516e00cf0063d9d1f1c296f0e8137b085565b443832f36

  The recorded hash matches the document. A deviations file is recorded for it.

• PREREG_C: site-level open-data evidence (Role C)

  c7f1db52b339c4fd85137e36b738cfc57863609c8867e78379c350234fd1d734

  The recorded hash matches the document.

Pre-registration

The criteria were written down before results were computed. The hash identifies that record. It does not mean the test was passed.

• mars: 2fa168bea9cdcada26a06a0588c7f1c2ee2baf81e2be72097225ebb16064ed69
• moon: 6bdaaf3a0ece49205985363a026287d34b5492291e33e2bf8b4a159c41961d55
• moon_lat20-40: 6bdaaf3a0ece49205985363a026287d34b5492291e33e2bf8b4a159c41961d55

See the method page for what the screening does.