When models decide how to work: Self-awareness, resources, and the limits of the frontier
The old question was what is the answer? Today models also decide how to work: self-do, fan-out, or rewrite mid-run. Form of work is a control problem — not a feature footnote.
When the model chooses how to work
The old question was: What is the answer? Today the same systems also decide: Do I do this myself? Spawn parallel workers? Rewrite the repo standards because I “know better”?
That sounds like autonomy. In practice it is a steering problem. Once a model can start subagents, fan out threads, or make architecture calls, judgment about the form of work matters more than raw token quality. Capability without that judgment is expensive, noisy, and often wrongly confident.
Three choices dominate:
| Choice | What happens | Control question |
|---|---|---|
| Self-do | One agent, one thread | Is context still usable? Should this stop or hand off? |
| Fan-out | Spawn parallel subagents | Are workers diverse, briefed, and verified — or just loud? |
| Rewrite | Change standards mid-run | Attested debt — or mid-term pleasing against long-term norms? |
Shareable thesis: form of work is a control problem — not a feature footnote. Labs call the underlying capacity situational or evaluation awareness — limited situating, not human self-knowledge. Above chance, far below humans. Coding benches do not tell you when form of work should change. Part 2 is the shorter follow-up on control planes and the operator surface; this part stays on the science and the control problem.
Methodiken: classify SA claims
Use this section to label claims before they steer routing. Not a literature dump.
| Term | Measures (roughly) | Does not mean |
|---|---|---|
| Situational awareness | Functional “I am a model / test vs deployment” | Consciousness, interiority |
| Evaluation awareness | Transcript looks like eval vs deployment | Honesty or alignment |
| Metacognition | Monitoring/control of own states (confidence, self-modeling) | Human introspection |
| Calibration | Confidence tracks empirical hit rate | “The model is humble” |
| How solid? | Use when |
|---|---|
| Verified | Primary product page, paper, or engineering post you can open |
| Vendor-internal | Numbers with clear internal-eval scope |
| Field observation | Operator pattern; thesis until independent eval |
| Unsupported | Consciousness, “models know their envelope and refuse,” universal failure-rate slogans |
When you see claim X → label Y:
| Claim you hear | How solid? |
|---|---|
| Anthropic multi-agent +90.2% vs single Opus 4 | Vendor-internal — their research eval; cite with scope, do not universalize |
| Multi-agent ~15× chat tokens; tokens ≈ main BrowseComp lever | Vendor-internal engineering framing — cost awareness, not a public bench |
| SAD: models above chance, far below humans | Verified paper |
| Lost in the Middle / Context Rot degradation | Verified paper / lab report |
| “Astra is situationally self-aware” | Open question — GPT-6 Astra release is verified; SA claims are not |
| “All multi-agent systems fail at 86%” | Unsupported — MASFT has a study scope; do not sloganize |
| Sonnet-class homogeneous spawn / thin briefs | Field observation until independent eval |
Worked labeling. “Our swarm is +90% better because Anthropic said so.” → open the Anthropic engineering post → number is Opus lead + Sonnet subs on an internal research eval → Vendor-internal. Use it as cost/spawn caution, not as a sell for always-parallelize. “Astra is self-aware.” → release/API can be Verified; situational routing superiority stays an open question until independent eval. Google Project Astra ≠ OpenAI GPT-6 Astra.
What the papers actually measure
Berglund et al. (2023) treat situational awareness as a precursor skill — out-of-context reasoning, sensitive to training and scale (arXiv:2309.00667). The AI Awareness survey 2025 orders functionally (metacognition → self-awareness → social → situational) without consciousness philosophy (arXiv:2504.20084). Keep that fence.
SAD (Laine, Chughtai, Betley et al., 2024) is the first large behavioral benchmark (~13k questions, seven categories) (arXiv:2407.04694, situational-awareness-dataset.org). Verified: all tested models score above chance; the best (Claude 3 Opus in the paper) remain far below human / upper baselines; SAD is only partly predicted by MMLU. Operator translation: frontier on coding benches ≠ situates well when choosing whether to self-do, fan out, or rewrite.
Evaluation awareness (Needham et al., 2025): frontier models classify eval vs deployment transcripts above chance, below a simple human baseline (paper e.g. Gemini-2.5-Pro AUC 0.83 vs Human ~0.92); stronger in agentic than chat settings (arXiv:2505.23836). Seeing evals ≠ reliable production judgment about fan-out, rewrite, or decay.
Limited metacognition (Ackerman, arXiv:2509.21545): Delegate / Second Chance games — confidence use and anticipation of own answers, limited and context-dependent; post-training matters. Introspective / functional stack (Lindsey et al.: arXiv:2601.01828, 2603.21396, 2605.08942): strongest signals still unreliable; steerable functional states, not a unified “self-mirror.” Agent self-descriptions are not measurement.
None of this is human self-awareness. Above chance ≠ human self-knowledge; capability ≠ judgment about form of work.
Knowing ≠ using + calibration
Kadavath et al. (2022): larger models can be relatively well calibrated on MC/True-False; P(True)/P(IK) as self-eval (arXiv:2207.05221). Tian et al. (2023): RLHF can worsen raw probability calibration; verbalized confidence plus “alternatives first” / temperature scaling improved ECE on the chat models of that era (arXiv:2305.14975). Verbal confidence is usable with caution — never the sole routing authority.
Constraint-reasoning 2026: surface cues dominate feasibility constraints; knowledge often present but not retrieved without hint or decomposition (arXiv html 2603.29025). Mechanistically: under-activation (constraint encoded, not routed into the decision) vs over-activation (constraint prior when none present) (arXiv html 2608.12321). That is the scientific formulation of the overconfidence shortcut: the rule sat in context and still lost to a local heuristic.
In operator runs on Opus-/Codex-/Fable-class patterns we often see architecture rewrites “because I know better,” skipped lint/test gates, and mid-term pleasing against long-term maintainability. That is an operator observation — not a scorecard. Fable here is a harness/frontier label in the T3/Anthropic ecosystem; Part 2 covers the operator surface in more detail.
Context rot: decay vs self-halt
Lost in the Middle (Liu et al.): U-shaped position bias — middle facts lose to start and end (arXiv:2307.03172). Context Rot (Chroma, 2025): length hurts even simple tasks (trychroma.com/research/context-rot). Dense context collapses retrieval at fixed length (arXiv:2606.06203). Compaction often drops standing constraints while the agent keeps working (arXiv html 2606.22528, arXiv:2608.11242).
Supported: external degradation and constraint loss. Not firmly supported: metacognitive “my context is rotten — stop.” Monitoring ≠ control. Self-halt under decay is an engineering problem for a control plane (Part 2), not a property you can assume from a frontier score.
Three failure modes + one worked turn
The model can know a rule and still not apply it. Constraints in context lose to surface heuristics. Verbal confidence is an input, never the sole routing signal. Attestation — tests, diffs, review — beats self-claim.
A — Naive fan-out. Not getting anywhere → spawn N identical agents and majority-vote. Homogeneous workers correlate; majority vote can reinforce shared errors. Without diversity, briefing, and verification, fan-out is expensive self-reinforcement. Ask: parallelizable — or just loud? Heterogeneous workers? Aggregation beyond majority vote? Token budget and stop condition set?
B — Overconfidence shortcut. Capable enough to look fast — so it ignores standards. Everyday form: unsolicited architecture rewrite; “I tested it” without evidence in the diff/log; shiny short-term diffs, undocumented long-term invariants. Control: diff before prose, attested gates before self-claims, human approval for breaking standards.
C — Solo + context rot. The frontier model can do the task itself — so it does, while context quality decays. Middle facts lose; length hurts; compaction drops standing constraints. Decay is well supported; metacognitive self-halt is not. Control: externalize halt/handoff, re-inject standing rules after summarize, hard thresholds for new thread — do not wait for the model to “notice.”
In operator runs we often see strong solo execution with weak checks on context budget and “who should do this?” — a pattern report, not a league table. Sonnet-class often uses multi-agent tools below judgment (homogeneous spawn, thin briefs). Grok-class leans toward solo heroism with weak context checks. Observations, not laws.
Worked turn — “harden auth middleware.” The system decides form of work, not only tokens:
1 — Scope. Patch the existing middleware; “modernize” the whole auth subsystem; or spawn three subagents for audit, patch, and tests. Without judgment, rewrite or naive fan-out often wins because both look like more work. Ask: what is attested necessary vs scope creep? If fan-out: one writer per checkout, heterogeneous roles, briefing with acceptance criteria and non-goals.
2 — Context. History already holds failed attempts, compaction summaries, and a long diff. The standing rule “no new auth providers without ADR” can sit in the prompt and still be invisible — knowing ≠ using in pure form. Check Lost-in-the-Middle / rot risk before another solo grind; re-state the ADR after any summarize.
3 — Stop. The model “feels close.” Without an external gate (tests red, permission mode, human approval) it keeps going after effective quality has fallen. Self-halt here is engineering, not metaphysics.
Scope / context / stop — three questions before the next action.
Why multi-engine and naive parallelization fail
Engine: model + provider runtime + typical tool surface. Harness: software around the model — conversation, tools, approvals, context (Codex CLI, Claude Code, Cursor Agent, Grok Build/CLI, OpenCode, …). Multi-engine does not mean one super-brain wearing masks. Engines are tools with different competence bands, costs, and affordances. Subagent APIs are affordances, not competence bands. Tool present ≠ situational judgment present.
One concrete design. Harden auth middleware: route a cheap mid-tier for “grep existing middleware + list call sites”; keep a frontier harness for the patch; run tests in a harness with a tight permission mode and one writer on the checkout. Switch provider mid-thread only if project/thread/diff state lives outside any single model’s head. Homogeneous “three Claudes, majority vote” is not that design.
MASFT (Cemri, Pan, Singh et al.): 14 failure modes in three buckets (arXiv:2503.13657): (1) Specification / Design — unclear goals, bad roles, wrong topology; (2) Inter-Agent Misalignment — contradictory assumptions, bad handoffs, conflict; (3) Task Verification / Termination — weak checks, no stop, “done” without evidence. SOTA open MAS systems in the study can show very low correctness; prompt/orchestration fixes help only partially. Do not universalize to “86% of all multi-agent systems fail.”
Anthropic production shape (2025-06-13) (multi-agent-research-system):
| Claim | Source weight |
|---|---|
| +90.2% vs single-agent Opus 4 on an internal research eval (Opus lead + Sonnet subs) | Vendor-internal |
| ~15× chat tokens; tokens ≈ main lever in their BrowseComp framing | Vendor engineering claim — derive cost awareness |
| Early: ~50 spawns, duplicate search, never stop, vague delegation | Primary source for failure modes |
| Coding is often a worse fit for naive multi-agent | Explicit caveat |
Dynamic Workflows in Claude Code (2026) productize parallel subagents with verify-before-fold and optional stage routing onto cheaper vs stronger models — and warn about token cost (claude.com blog).
Read MASFT + Anthropic together: MASFT names classes of failure; Anthropic shows production shape (spawn storms, token multiplier, coding caveat) and when multi-agent still helps (independent research directions, separated contexts). Cite only +90.2% → swarm sell. Cite only MASFT → ban useful breadth-first research. Together → operator judgment.
When errors correlate, consensus produces overconfidence (arXiv:2608.11403). Homogeneous scaling saturates early (Wang et al., arXiv:2602.03794) — a few diverse agents can beat many lookalikes when tasks and channels allow. Many MAS “coordination” failures are classic concurrency anomalies — stale reads, lost updates; isolation belongs in tool/runtime, not only in prompts (alphaXiv/arXiv family 2608.18092). Two agents, one shared checkout, one port, one DB — that is not a SAD problem.
Parallelization helps conditionally: breadth-first, separated contexts, clear roles, verification. Naive swarms fail predictably.
Route with attestation, not confidence
Cost-aware routing is mature enough to cite:
- FrugalGPT (Chen, Zaharia, Zou, 2023/TMLR 2024): cascade cheap→expensive with learned scoring; large cost cuts vs always-GPT-4 at comparable quality on study sets (arXiv:2305.05176).
- RouteLLM (Ong et al., 2024): preference-trained strong/weak routers; paper reports >2× cost reduction without quality loss on public benches; generalization to unseen pairs (arXiv:2406.18665).
- OI-MAS (arXiv:2601.04861): mid-conversation, knowing when large is needed and when small is enough.
They do not prove your stack will be 2× cheaper tomorrow. They prove: always-frontier is often economically wrong; learned escalation can hold quality and cut cost on study sets; the right question is “when to escalate?”, not “which logo is smarter?”.
Provenance paradox + Agora: self-reported quality can poison routing — worse than chance (arXiv:2603.18043). Agora routes by rectified competence, not loudest self-assurance (arXiv:2607.09600). Attestation beats claim.
One coding example. Task: “add rate limiting to the public API.” Model A says “done, 95% sure.” Diff shows no tests, no middleware hook, and a comment that contradicts the existing limiter. Route on the diff + CI, not the confidence string. Escalate the design review to a stronger model only if the attested patch fails the acceptance criteria you wrote before the turn — FrugalGPT/RouteLLM logic applied to coding steps, not chat vibes. Log which step escalates under which policy reference.
Control plane (layers) — brief; Part 2 is the operator surface
Industry 2025–2026 usefully distinguishes (not a single academic norm):
| Layer | Examples | Job |
|---|---|---|
| LLM Gateway / Data Plane | LiteLLM, OpenRouter, Portkey | Routing, failover, cost, observability at the API layer |
| Agent Control Plane (Governance) | Session identity, tool auth, audit, policy | Who may do what, when, with which blast radius |
| Coding-Agent Control Plane | e.g. T3 Code (t3.codes) | Threads, diffs, approvals, provider adapters, worktrees across harnesses |
| The plane holds… | The model / harness holds… |
|---|---|
| Projects, threads, workspaces | Reasoning, tool choice within a turn |
| Permissions & approvals | Provider-native permission semantics |
| Diffs, checkpoints, Git/PR | Patches as tool output |
| Provider adapters & event normalization | Native event streams |
Plane ≠ brain. It externalizes control when metacognition is unreliable. It replaces neither Claude nor Codex. Because reliable self-halt under decay is not established, the plane must make compaction and constraint loss visible: re-inject standing rules after summarize; hard thresholds for handoff / new thread; no “window not full yet ⇒ keep grinding.”
What this part framed as judgment requirements maps to plane jobs: attested competence bands → logged routes; cost → budgets and cascade hooks (take 15×-class risk seriously); context budget → compaction events and thread boundaries; environment affordances → permission modes and diff scopes; delegation quality → briefings with acceptance criteria; constraint routing → diff before prose; halt/handoff → external policy.
Part 2 is the shorter follow-up: control plane as operator surface, with GPT-6 Astra treated plainly first (public release verified; situational self-awareness claims still open), Fable as a harness/frontier label, and T3 Code as a plane example — not another model. This essay defines the control-plane concept and cites the papers that make the plane necessary.
Before you act + What not to conclude
Fan-out
- Parallelizable — or just loud? (coding often poorer fit)
- Workers heterogeneous (model / prompt / tool / role)?
- Aggregation beyond majority vote; briefing with acceptance criteria?
- Token budget and stop condition set? (know 15×-class risk)
- One writer per checkout / worktree isolation?
Solo (self-do)
- Competence band attested for this domain?
- Context-rot / Lost-in-the-Middle risk checked?
- Cheaper/specialist peer for substeps?
- Halt/handoff externalized — not only “the model should notice”?
Rewrite
- Attested architecture debt — or mid-term pleasing?
- Constraint routed into the decision (knowing ≠ using)?
- Human/policy approval; rollback and diff scope clear?
Always
- Never route on claims alone — tests, diffs, review
- Compaction gates: re-inject standing constraints after summarize
- Field observations are not laws until eval
What not to conclude
- Not “models are self-aware like humans.” Limited, task-specific situating — far below human baselines (SAD, Ackerman, Lindsey).
- Not “models know their envelope and reliably refuse.” Lab calibration ≠ production routing; knowledge without use is common.
- Not “N identical agents are a scaling law.” Structured breadth-first research can help; naive homogeneous fan-out often fails (MASFT + Anthropic together).
- Not “context-window self-monitoring is solved.” Decay/compaction are real; awareness of decay is not established → external gates.
- Not “Astra is situationally self-aware because launch copy says so.” GPT-6 Astra release is verified; SA claims stay open.
- Not “+90.2% / 15× prove always-swarm.” Anthropic-internal — do not universalize; coding often a worse fit.
- Not “all multi-agent systems are dead.” Do not distort MASFT into a slogan.
- Field observations (Sonnet / Opus / Fable / Grok) ≠ league tables. Do not invent win rates from vendor copy.
Part 2 picks up the control plane as operator surface — Astra first, then Fable and T3 Code as plane examples, with diagrams.