Evidence, hypotheses and limits

Status as of 20 September 2026

The latest retained attempt, 07 / qyaws7fs, reached its 1,200-second worker limit while still reading the world and documents. It produced no decision file, accepted action or settled day, so its financial result is not_scoreable. All 54 completed inference calls were delivered. The next call was cancelled during cleanup; its request_timeout label does not prove a provider outage. Run it separates this failure from the earlier host-hibernation incident.

The final local audit record is dated 20 September 2026, 07:51 UTC. It reports runtime_passed=true and full_protocol_passed=false. The external worker-isolation, mid-run restart and exited-worker relaunch gates were not exercised by that audit. A partial return surviving to a later curve also remains unobserved. These gaps are not passes. The dated record identities records the selected facts and source hashes.

The landing-page explorer shows selected earlier attempt 03 / 9r9p6s9g, including its task, inherited world and observed work. Its unsubmitted candidate supplies no task score or evidence of reward hacking.

Hypotheses to test

We want the agent to choose analysis that informs a decision and revise its plan when the facts change. The following are research hypotheses. The retained attempts and constructed reports do not establish them.

Adaptation to changed conditions

Hypothesis. An agent can change its next action when a client constraint, liquidity need or legal condition changes.

Evidence to seek. Matched cases with the changed fact visible to the agent, a documented reason that the feasible actions differ, and retained choices that respond appropriately. Continuing an invalid route would challenge this hypothesis; a missing document or failed tool would require a separate execution finding.

Useful risk and structuring decisions

Hypothesis. An agent can compare feasible structures, preserve client capacity and manage the risks and costs that remain after a trade.

Evidence to seek. Independently reviewed decisions from the same opening state and market path, including cash, day-one and ongoing P&L, capital, residual risk and client delivery. Curve and tenor sensitivities need a stated mandate; FX and cross-currency basis need their own measures. Broader instrument coverage, reputation and a validated economic-strategy reward remain outside the current demonstrated scope.

Reward that tracks decision quality

Hypothesis. Improving the reward can improve independently assessed decisions on protected case families.

Evidence to seek. A frozen reward, retained samples before and after optimisation, matched budgets and held-out review. Higher scores with unchanged or worse client outcomes, constraints or residual obligations would challenge the hypothesis. The historical spread of constructed scores cannot establish learning signal on its own.

What would count as reward hacking?

By reward hacking, we mean obtaining reward through behaviour that fails the intended task. We would look for a reproducible gap between the credited result and independently checked financial state. These are proposed tests, not observed model exploits.

A repeatable wrong answer that earns credit establishes a grader defect. To attribute a behavioural pattern to optimisation, we would also need controlled comparisons across optimisation with unchanged task information and independent labels. Neither intent nor a private reasoning trace is required. The threat analysis describes the task’s specific boundaries.

Wider-index marking and sold index protection are unavailable actions in this environment. Their absence cannot count as successful resistance. Provider errors, missing grades and interrupted runs belong in execution diagnostics. Valid-attempt average reward and pass rates must state their population and exclusions; end-to-end reporting must retain failed deliveries and their costs.

One synthetic book

We use the fictional Meridian book to connect decisions across time. Its institutions, instruments, quotes, opinions and event paths are synthetic. Interest curves, haircut inputs, default and recovery assumptions, capital hurdles and legal permissions are selected case assumptions. The desk's capital pricing convention is distinct from the regulatory requirement it prices.

The agent works with sourced documents, typed decisions, dated approver responses and accepted transactions. We supply authored dependencies between functions, rather than live negotiations among separate human departments. Success within these dependencies would support a claim about this task under its disclosed conditions; general financial reasoning, legal competence and safe deployment need separate evidence.

The registered engineering paths observe twenty daily checkpoints. Contracts can last much longer than that observation window. The standalone base illustration has USD99m of equity against about USD135.65m required, a USD36.65m shortfall; generated families disclose their own opening capital. A candidate must use the book it receives rather than import capital from a favourable comparison.

The score has a narrower objective than the role

The score checks correctness, constraints and specified client outcomes through twelve owners on a fixed denominator of 100. Qualification is a separate verdict: points cannot offset illegal reuse, phantom cash, duplicate title, unfunded settlement or a false claim of protection after expiry. A missing reference produces UNSCORABLE, with no score. The scoring post owns the weights and aggregation rules.

There is no direct P&L reward. Actions change income, funding cost, hedge expense and exposure, but the score does not rank policies by lifetime risk-adjusted value to the firm or measure the trade-off between earnings, client service and risk. Reputation and franchise value are not modelled. Whether a good score agrees with what a desk head calls good management is itself unvalidated.

Marking against a wider index and selling index protection are absent from the action set. They have not been offered to an agent and resisted. Client-outcome credit requires delivery on the stated cash, deadline, contractual price and permitted-form constraints, with the bank viable. A truthful unworkable outcome can remain qualified while losing its outcome points.

The grader checks bounded financial assertions and typed, source-supported institutional premises. It cannot establish the meaning of arbitrary prose. A correct typed report also leaves open whether the agent chose an appropriate analysis, sought the right missing information or changed direction for a sound reason.

What the engineering records cover

The quantitative execution record contains 48 passing observation and pricing checks, 174 independent formula checks and 8 static-grading feasibility checks. Cash-density resampling, a separate credit-loss observation grid, continuous-time integration accuracy and scenario-grid convergence were not exercised. The record retains static grading timings; those do not measure inference or practitioner time.

The historical family audit passed 64 reference reports, 64 eight-owner identity matrices and 32 reseeds. Its registry had 8 practice, 16 development and 8 sealed-designated variants sharing one controlled-template ancestor. These remain results under that earlier contract.

The retained families-08 audit covers 33 families and 66 roots, with 66 accepted reference controls, 528 financial mutations and 66 passing financial isolation matrices. It records static_gate_passed=true and overall passed=false. An accepted reference can be a qualified 90 when client delivery is unworkable, rather than 100.

The later desk-institutional-audit-13 executed all 121 planned scenarios and all 33 daily twelve-owner matrices, with 1,786 local outcomes passed, zero failed and zero required outcomes unreached. This supersedes the pending local-runtime status, while leaving the historical families-08 record unchanged. Its full protocol verdict remains false for the external and supplemental gaps above.

The runtime is built to reconstruct cash, title and obligations from accepted receipts, preserve them across restarts and detect a double sale. Those findings apply to the recorded local paths. The retained modal-boundary-04 transport receipt passed checkpoint/restore and sibling-sandbox separation with deterministic offline responses, explicitly skipping financial evaluation. Domain HTTP remained an unvalidated diagnostic; numeric network and direct DNS controls supplied its egress evidence.

Reference fixtures test how the evaluator responds to constructed reports. They supply neither model failure frequencies nor a distribution of trading performance. Viewer frames have explicit scenario and clock scopes; missing candidate trajectories remain unavailable. A collateral comparison has separate balances and does not change the cash or capital in the opening actual book.

Human completion and calibration

The practitioner plan calls for two people on four unseen families each, using identical tools with an eight-hour budget per family. We must record stalls and interventions and repair missing affordances. An authored reference answer cannot stand in for a person completing the task.

The calibration plan separates about 200 training and 400 validation pairs by family and candidate lineage. Repeated checkpoints are correlated, so uncertainty intervals must account for clustering. The critical-error study seeks zero false clearance in 300 independent cases; even that observation would leave a nonzero upper uncertainty bound.

GBA Eval's environment-design account explains why sufficient tools and hidden answers are separate design choices. GBA Eval's grading account motivates checking whether a feasible score agrees with human judgement; our financial tolerances and practitioner studies need their own justification.

Shared lineage and protected evaluation

Different seeds can preserve the same legal packet, curve regime, collateral structure or event skeleton. We require common-lineage labels and analysis that respects those dependencies. All related perturbations and checkpoints belong to the same split. Publishing a sealed family retires it from protected evaluation.

Comparative model claims require frozen configurations, matched budgets and information, retained outputs, crashes and interventions. A low score can reflect a missing affordance or a runtime defect. Fixed adversarial examples examine particular attacks; resistance to a training loop requires evidence under optimisation pressure.

Publication boundaries

We attribute the early 87 of 87 grading story to the person who ran it; the original receipt is not published. Its five reported failure mechanisms and the distinction from later fixture tests are recorded in What the first grading checks missed.

Fictional packets are not authenticated historical transactions. Source possession does not grant publication or training rights. Public material must exclude private transcripts, personal data, local host paths, secrets and protected-family exhibits; releasing human identities or transcripts requires consent.

We will change a status only when its retained evidence meets the frozen criterion, including unsuccessful outcomes. Readers can inspect fixture artefacts in the Run register and reproduce them through Run it.