The pitch: dark factory, autonomous agents, lights off, no humans, build loops, shipping value 24/7… just sign the check here.
The reality: Token Furnace, a subsystem confidently executing a plan against a world that no longer exists, at a cost compounding faster than the value it creates, with an “Accountability” RACI column nobody filled in.
The dark factory promise succeeds wildly and yet fails three ways, and they stack:
architecture - the loop can’t reason about time or know what it doesn’t know
economics - the bill arrives non-linearly
extraction - the loop runs, outputs flow, and none of it is worth what it cost.
Most dark factory evangelism addresses none of these, majority just relies on N+1 model to solve it all, aka cloud era “infinite compute, 10x cheaper, next cycle” promise. This article attempts to address all three and ends with how you could fix yours in the meantime while that N+1 model is not yet here.
A bit of true steel-man treatment first.
Three arguments for the dark factory that hold:
Human review is mostly theatre. Rubber-stamp approvals, review fatigue, a queue of 200 items before lunch. The honest comparison is not “agent critic vs. perfect human reviewer”; it’s “agent critic vs. the review that actually happens.” That bar is on the floor.
Compound loops work. Critique-revision cycles measurably beat single-shot inference. The gains are real.
The arithmetic is legitimate. Marginal inference cost is deflationary. For high-volume, low-novelty decisions the cycle is simple: triage, validation, classification. The ROI requires no heroic assumptions. It just works.
This covers a good fraction of routine knowledge work. “BorRRing…” is where the actual ROI is. The problem: most valuable work escapes these constraints and, when it escapes, the failure isn’t graceful.
Failure One: The Architecture
Interval blindness
Every agent action has duration. The plan was generated at world state S_0. mid-execution, the world is at S_3. The plan is still internally consistent, however, It is executing against a ghost. This is a bit worse than hallucination, hallucination produces garbage you can spot. Interval blindness produces coherent outputs, rational critic approvals, no merge conflicts, and a plan executing flawlessly against a world that no longer exists - remember: the lights are off.
What LLMs know about time
They somewhat know narrative / sequence, logical / order at thinking modes and none of the physical like duration, friction nor drag. A reasoning model can spot “you can’t deploy before the migration completes or without lazy one”. It cannot spot “this person has never shipped more than 2 containers, so that work estimate carries a 10× uncertainty band.”
The rule: give the model the structure problem. Never the duration problem. Duration is yours - have we all giggled at “this would take you 1 week…” work estimates?
The embedded critic cannot work, LeCun is right about this one
The planner-critic loop assumes the critic can evaluate a plan. Evaluating a plan means predicting its consequences: if we execute this, what happens to the world? That is a world model, and Yann LeCun’s standing critique of autoregressive LLMs lands precisely here. A next-token predictor doesn’t model the world; it models text about the world. Ask it to critique a plan and it produces plausible text about the plan, fluent, structured, confident - not a prediction of what the plan does to actual state.
Follow that one step further and the whole loop unravels: if the model cannot predict the impact of an action, it cannot rank candidate actions by impact either. The embedded critic isn’t a weaker version of evaluation, it is just a scoring method. You’ve built a peer-review process where both reviewers are betting on the same side: “LGTM +1”
This is why external oracles aren’t an optimisation - they’re the only place real evaluation enters the loop. A test suite predicts consequences by executing them. A schema validator, a compiler, a deployment dry-run - all are world models, crude but grounded
This is a distributed systems problem again, right?.. right?
Multi-agent state coordination is not a new problem and it is not an AI problem. It is the oldest problem in distributed systems, and the actor model solved it: no shared state. Each process owns its state exclusively. Nothing reaches into another process’s state. All communication is immutable messages. Now look at what most dark factories actually build. Conversation history, tool results, world-state snapshots ,blah and blah…That’s shared mutable state, just with an LLM and a token meter attached.
The consequence is a CAP trade-off you made without noticing. Nobody discussed nor wrote that decision down - and yet, it was shipped anyway. From here, the failure modes you’ve been told are separate collapse into same failure with different name:
Hallucination - agent’s world model diverged from world state; nothing checks
Interval blindness - State mutated between read and execute
Entropy - ungrounded loop drifts toward noise and signal decays
One mechanism: unmanaged state divergence. And one fix - explicit state topology. Ownership, message contracts, supervision - you have to choose it. Nothing in the framework will.
The empty column
An agent has no track record. Principal-level authority derives from decisions made, consequences absorbed, judgement calibrated across years. An agent’s RACI: Responsible: agent. Accountable: (empty). Consulted: other agent. Informed: nobody, but you may be lucky to be that consumer at the end.
You cannot patch this with one more agent. Accountability is a property of the system boundary, placed there by a human, at design time.
Failure Two: The Economics
The shock
One production data point: identical 2-hour Opus (N-1) automation runs: pre-June 2026 - ~$18, while now - ~$100. If you budgeted on old rates, your AI factory just became a token furnace with a subscription. it is easy to imagine one real factory with a few agents of mixed caliber…
Now, when was the last time you’ve actually reviewed your bespoke cost observability metrics or any AI tool? Have you noticed where the money went: >80% of spend is reading, not writing. Your agents are expensive readers, not expensive writers. Optimise accordingly.
The hidden multipliers
Tokeniser tax. The same JSON/base64/URL-heavy payload costs 6–10× more tokens on Claude than GPT. At thousands of calls a day, you’re paying per brace and per slash. The advertised 1M context is ~500K effective on structured payloads.
Loop tax. Cheapest per token ≠ cheapest per correct output. We all know: the premium model at one loop beats the cheap model at ten, because retries accumulate input tokens.
Infrastructure. Cache or bleed. OSS or open your wallet.
Failure Three: Extraction
The least discussed and most fatal one… Physics already has the perfect joke for it. Fusion power has been “20–30 years away” for seventy years. The reactors work. The plasma ignites. The problem was never producing the reaction - it’s that for decades, every reactor consumed more energy than it returned. Q < 1.
You would be surprised to know, your dark factory has a Q factor too: value extracted divided by cost of the run. I think we would be “2-3 years away” with AI factory…
The demos ignite beautifully. The agents loop, the tokens burn, the outputs flow. And in most deployments, nobody has measured whether Q ever crossed 1. We all are running a AI Floating Point reactor and calling the heat produced - progress. Ask an operator when their factory reaches sustained net-positive output and you’ll get the fusion answer: it’s always one more optimisation away.
A loop that runs is trivial - that’s the plasma. A loop whose outputs are worth more than they cost must produce work that is:
Used, not corrected-then-used. If every output needs a human editor before shipping, you didn’t automate the work — you added a layer.
Trusted, not plausible. The killer isn’t the spectacular crash. It’s the coherent output that’s subtly wrong in ways that surface three steps downstream. Interval blindness, again.
Bounded. Wrong output in a closed task (test suite) is recoverable. Wrong output in an open one (customer comms, prod deploy) may not be.
Agents run 24/7, never get tired, never complain. They also never notice the world changed, never feel the cost of being wrong, and never develop judgement from consequences. Every one of those properties must be engineered in from outside. Nobody puts that on the slide. have you set KPIs for the individual agents inside the factory?
Maxwell’s demon, or: who pays for all this jazz
Physics has a second older joke here too. Maxwell’s demon, anyone? Apparently for free, apparently violating the second law. It took some time to find the catch: the demon pays. Sorting requires information about every molecule, and acquiring, storing, and erasing that information costs more energy than the sorting yields. That book was closed a long time ago, there is no magic, only information-processing cost centre.
Now the AI agent version. One hallucinating agent is one bad molecule — annoying, catchable, bounded. An ensemble of hallucinating agents with distributed state is a gas. Each agent’s world model diverges independently. The errors don’t average out; with shared context they correlate and compound. Extracting reliable work from that gas requires a demon: something with enough information about every agent’s state to sort good outputs from plausible garbage. That demon is your verification layer — your oracles, your checkpoints, your human at the boundary.
The bill still arrives: the information cost of sorting is the irreducible cost of the factory.
So the extraction question reduces to demon economics: what does your sorting cost, and did you count it? Q < 1 again, just stated in information instead of energy.
When It Works: The Time-Collapse Condition
The pattern works in exactly one regime: engineered temporal isolation. World state frozen or fully owned by the agents. Self-contained codebase, fixed test suite, bounded problem. In that regime, the context window is the world — there’s nothing external to drift against, and interval blindness is structurally impossible.
- World state = Frozen / agent-owned, Gated updates
- Critic = External oracle, tests, schemas, lookups
- Blast radius = one or few agents max.
- Cost = Loop-count predictable
The efficiency gains are real — but they come from parallelism and termination logic designed by the human architect at design time, not from agent time-awareness at runtime. The agents experience token budgets and state deltas, not duration. In the closed regime that’s fine. In the open regime it’s catastrophic.
Or in other words, four questions before you build. Any “no” means you’re building a confident failure machine:
Is world state frozen for the run?
Are all mutations local to agent-owned state?
Is the blast radius bounded if the plan executes stale?
Are KPIs set?
Engineering Q > 1
Q = value extracted over cost of the run so it has exactly two levers.
Shrink the denominator. None of this is clever. All of it compounds. This is the unglamorous part, and it starts from the number above: your agents are expensive readers. So attack the reads. Strip tool payloads before they reach context, do not pollute with million tool definitions and skills. Compress prompts from essays into structures; a thousand-token instruction block collapses to under a hundred as a flow diagram, with no loss the model notices. Index the codebase instead of reading it - fetch the ten lines you need, not the file. Route by measured loop count on your own task distribution, not by the rate card: the premium model that one-shots the fix beats the cheap model that retries ten times and re-reads context on every attempt.
Grow the numerator. Value only counts when the output is used, not corrected-then-used — and that requires the demon: external oracles where they exist, gated state updates where they don’t, a human at the boundaries where consequences cross out of the sandbox, and KPIs per agent so you can tell which molecules in the gas are the slow ones.
The unicorn objection. At this point somebody at the back says: “come on, the value is non-linear”. Nobody is pricing a flow here - we’re buying an option. If the factory works, the payoff is 1000×; Q < 1 today is just the premium. Every venture bet is Q < 1 until it isn’t.
Fair. Gambling is a legitimate strategy - fusion itself is a seventy-year tail bet. But gamblers who survive do three things the dark factory crowd skips. They know the price of the ticket, an option has a premium. They set a stop-loss: venture works as a portfolio - price each bet, kill losers fast. And they don’t book the heat as revenue - burn hidden in OPEX.
An option has a premium, a strike, and an expiry. If you can state all three, you are a dark factor investor and Q is your instrument panel while the bet runs. If you can’t, you’re not holding the option. You’re the heat.
Now the constraint that makes this a design problem rather than a checklist: the demon’s cost sits in the denominator. Every oracle run, every checkpoint, every human review is paid input. Over-verify and you kill Q from the other side — a factory where every output gets a full human review is Q < 1 by construction. So the question is never “how much verification” but where: place the demon at the boundaries where blast radius is consequential, and nowhere else.
Fusion’s actual roadmap to Q > 1 was never “more plasma”. It was better confinement and less input power. Same two levers. Same discipline.
The Bottom Line
The lights-off factory needs more engineering than the lights-on one and not less. Removing the humans from the loop doesn’t remove the need for humans to have designed the loop. It removes the safety net that would have caught the plan executing against a dead world.
This week: run one pipeline through the four questions above. Any “no” is unpriced open-world exposure.
This month: instrument tokens per pipeline stage. Reads will dominate. Attack the top of that funnel before touching model selection.
Before your next architecture review: write the agent system’s RACI. If the Accountable column is empty, fix it at design time, because it cannot be fixed at runtime.
The dark factory can produce value. The only question is whether you engineered the extraction or whether you’re running a token fusion reactor with Q < 1.