PENROSE
Penrose/Documentation
Back to site

The learning loop

Penrose does not only judge one claim at a time. It accumulates what it has falsified and feeds that back into what it proposes next - without ever letting learned state change how a proposal is judged. That boundary is the whole design.

What feeds the loop

Every verdict carries a labeled reason. kill verdicts, and only structural ones, are the raw material: an underpowered or cannot_replicate result means Penrose could not tell whether the claim survives, so it is not a falsification and cannot support learning. The loop turns a growing pile of structural kills into structure, then into new candidates.

structural kills            (the verdict corpus)
   │  deterministic distillation
   ▼
connections → contrastive observations   (advisory; "this is an observation, not a recommendation")
   │  grounding, versioned by an evidence snapshot
   ▼
generation (dream / synthesize)   ← grounded in what has been learned
   │
   ▼
new candidate hypotheses → the SAME gate stack → their own verdicts
   │  exposure telemetry
   ▼
retrieved → cited → grounded → tested → did citing it lower that failure again?

The invariant

For any frozen claim, executable spec, data snapshot, and engine version, learned state cannot alter execution, gates, thresholds, search denominators, confirmation access, or the verdict. Learning may change the proposal distribution - what candidates get thought of - but it may not rank candidates by predicted survival, suppress otherwise-testable families, or optimize the survivor rate.

Two consequences fall out of this:

  • A learned principle can make the generator think of a strategy. It can never help that strategy pass. Every generated candidate is tested independently, on its own data, through the same gauntlet as a claim from a paper.
  • Learning lives on the generation plane. There, inspired_by grounding is legitimately a hard candidate-admission gate - gating what gets proposed is not gating a verdict.

What is automatic, and what is not

The split is deliberate: automatic on the generation side, gated on the truth side.

Automatic and deterministic:

  • Connections regenerate when their supporting kills change - a content-hash staleness trigger, not a background daemon. Same inputs always produce the same connections, and confidence decays as supporting evidence ages, so old patterns do not retain permanent weight.
  • Generation pulls from the refreshed connections, capturing the exact evidence snapshot it grounded on.
  • Exposure telemetry records the funnel (retrieved → cited → grounded → tested) after each run. It is append-only, content-addressed, and regenerable, and it is read by no retrieval, ranking, admission, or verdict code.

Held to a human or advisory gate:

  • Nothing auto-promotes a principle to "true," auto-archives a stale one, or ranks candidates by it.
  • The closed-loop signal - did citing a principle actually lower its own kill-reason recurrence, against a same-opportunity control? - is a read-only diagnostic (penrose learning-calibration). It informs a researcher's judgment; it never acts on its own.
  • P9 still commits every verdict. Learning changes what enters the funnel, never what leaves it.

Reading the calibration honestly

The calibration query reports honest denominators. It compares kill-reason recurrence for candidates that cited a principle against a control drawn from the same retrieval opportunities, and it refuses to report a direction until both arms clear a minimum sample floor - otherwise the status is insufficient_data. The survival rate it carries is labeled a safety diagnostic, never an optimization target.

In practice this means the machinery is wired and turning well before the payoff signal has evidence to speak: with a thin corpus the calibration query correctly returns insufficient_data rather than pretending to a conclusion it cannot support. That is the intended behavior - the loop runs, and its self-measurement waits for volume.

Penrose is a falsification referee for quantitative trading claims. A verdict is evidence about survival under the referee, never a trading instruction.