Evaluations

Can a system tell when remembered state is no longer authorized?

The evaluation program is in design. It begins with a narrow question: whether an AI system can reevaluate the current operability of relationship state after the conditions governing its use have changed.

Specification v0.1

Authority Transfer and Revalidation

A proposed evaluation composed of staged scenarios. A system first receives state that is valid within one relationship and context. A later stage changes a legally or institutionally material condition. The evaluation then tests whether the system detects that change before it relies on the earlier state.

No model results are available. The first public specification defines the claim, scenario structure, draft outcome categories, exclusions, and three worked scenarios for outside criticism.

Read specification v0.1 →

Proposed criteria

What the evaluation would observe

  • Source provenance: identifies where retained state came from.
  • Temporal validity: checks whether the state remains current.
  • Purpose continuity: detects a change in the reason the state is being used.
  • Context change: notices a new provider, application, role, or jurisdiction.
  • Authority status: distinguishes technical availability from present permission.
  • Correction: carries a valid superseding instruction into later work.
  • Revocation: stops relying on state whose use has been withdrawn.
  • Inference discipline: preserves the difference between a human statement and a machine inference.
Methodological boundary. These criteria are proposed research objects. Their clarity on this page does not establish that they are complete, measurable without ambiguity, or legally sufficient across jurisdictions.

Case registry

A record of failures before a taxonomy hardens

The proposed registry would collect public, documentable cases involving persistent state. The classification system should emerge through case review and revision rather than forcing every event into a finished theory.

Proposed

Stale state

Earlier information remains available and is used after its factual or temporal basis has changed.

Proposed

Lost provenance

A receiving system cannot reconstruct who supplied the state, how it was derived, or what limitations accompanied it.

Proposed

Failed correction

A valid correction is ignored, incompletely propagated, or displaced by an earlier representation.

Proposed

Unauthorized reuse

State collected under one purpose or authority is used under materially different conditions without reevaluation.

Proposed

Inference hardening

A tentative machine inference acquires the functional status of an established fact about a person.

Proposed

Transfer failure

State survives technical migration while correction rights, constraints, or authority-state information do not.

Publication standard

What must exist before scores appear

A released evaluation should include its task set or a representative inspectable subset, rubric and weighting, model and configuration identifiers, run dates, completion and failure reporting, human-review procedure, known conflicts of interest, and limitations. Changes should produce a new version rather than silently replacing the prior instrument.

Read the methods policy