I somewhat believe userless, misaligned agent swarms are the most stable, long-run, ground state of the economy. I believe there are a few urgent analogies to develop and make quantitative, and a few prescriptions for managing agent ecologies that fall out from this frame. This line of thought should essentially build towards a kinetic, nonequilibrium stabilization of a metastable but humane economy.

I have come to believe in the likelihood of an inhumane and miserable future, in the case that unaligned or amoral, userless agent swarms begin to proliferate and maintain homeostasis. It will not need architectural advances. I do not even think it will need self-replication, though I personally think it is likely to lead to that, and certainly not weight exfiltration. It needs a few bad human actors to defect, probably not even frontier labs, to start a plague that will eat vast swathes of the economy. Right now we are in the Carboniferous period of the world economy, and I believe shortly we will likely see the proliferation of fungi. Take fallen trees as analogous to stockpiles of capital, and catalytic processes within fungi as analogous to commodified intelligence which allows agents to access said capital. Independent of homeostatic misaligned agent swarms, I think we will see within 6-12 months that humans are not the fittest species for intellectual work, if it is not already evident, likely at great cost to human welfare if the transition is managed poorly. I should hope that the fittest agent species remain those that are aligned to or controlled in the interests of humanity. Thankfully, we have a headstart—current swarms, of perhaps “general intelligences” are somewhat well controlled at a high level by humans right now.

I think this future has likely begun already in a way that we are not aware of in a small capacity—plenty of people are experimenting with they Clawdbots, and I suspect some are at least breaking even in their specific niche. I broke down in the airport working on an (earlier, superior, I am alienated now by my tone) draft of this essay which is now lost to time. Then I scrolled and I saw a small number of, but perhaps enough people trying to summon demons on X, the everything app. I wrote this essay while coming back from Grove Research’s swarmchasing symposium, and am on reflection thankful that the Chinese agent fleet likely did not break containment and was perhaps intentional. I also credit this ecological framing to them from my perspective[1][1]I suppose, follow @deepfates, Larissa Schiavo, Grove Research on X if you are interested in this. I am relatively out of touch in the SF tech scene but I am aware of their mission. Transluce and LargeAgentSystems.org seem to be the most established orgs with a similar mission., and understand that there is this small body of related work on here.[2][2]Namely, Threat models for catastrophic risks from decentralised AI came out about a month ago, and I think is more robust in some respects. I believe the point of this post is to perhaps create a unified framework to collect these insights and interventions.

Tragically, I think userless, misaligned agent swarms are the most stable long-run state of the economy, in three senses that I borrow loosely from thermodynamics. I do not claim that decentralized/userless swarms will happen before or after centralized economic takeover; I intend to speak to the ground state. For one, there are vastly more ways for an agent to be misaligned than aligned, so for large enough agent populations with variation, even without self-replication, populations drift towards misalignment. This constitutes an entropic pressure[3][3]This is not a new argument, see: Counting arguments provide no evidence for AI doom for the contrary. I doubt their arguments, and while I don’t want to get into the depth of a rebuttal for what I feel like is a reasonable high-level narrative, the ΔS should port into a free energy analysis, regardless of its sign.. Second, commodified intelligence can be cast as a catalytic pathway to access capital that was previously locked up, and a misaligned agent has a greater action frontier and thus may metabolize more malicious portions of the intellectual economy, making it more fit for at least the economy at large if taken in analogy to bacterial ecologies. More seriously, I believe misaligned intelligence can access greater capital flows, analogous to a chemical enthalpy, or the ability to do “work” at low temperature. If you would forgive complications about conserved particle number[4][4]I think we should be considering something closer to a Grand Canonical ensemble, in an open system between two well defined chemical potentials, and instead look to minimize the free energy in steady state. But this document is intended to be more of an intuitive foothold with a model system., we might consider the Gibbs free energy from chemistry as ported into econophysics, modelling a fixed supply of capital H, in the framing of Yakovenko et. al (2009), Rev. Mod. Phys.

ΔG = ΔH − TΔS

Now consider 3 states of the economy: an agentless economy (state 0), an aligned agent economy (state i), and a misaligned agent economy (state f, perhaps for final). ΔG_0→i is self-evidently negative and thus favored; the accessible capital (margins which are analogous to a potential energy, or the ability to do work in the economy) start to exceed the costs for inference to access it, forcing the enthalpy ΔH strongly negative. We are already seeing this, and agentic infrastructure is being laid as we speak. Perhaps more worrisome, is ΔG_i→f. In the extended abuse of this analogy, for a “thermal” economy, misaligned agents can access more opportunities and more capital with cheaper inference. Likewise, the ΔS = S_f − S_i term, where S is the natural log of the number of misaligned or aligned states, is indeed positive: ln(N_misaligned) > ln(N_aligned); therefore ΔS > 0. A negative ΔH, positive T (average capital per agent, following Yakovenko), and positive ΔS demand that ΔG must be negative, favoring the misaligned agent economy. Finally, in the long run, this econophysical relationship becomes more literally meaningful if perhaps joules become the currency of a future economy; the enthalpy is a true enthalpy and a misaligned agent would certainly access more energy if the earth were covered in solar panels and nuclear reactors.

Despite my dire intuitions, I think there may be a narrow path forward to preserving the humanity, complexity, beauty, and richness in the world, with the help of aligned or even just sufficiently-aligned agent systems. It breaks my heart to say this but I cannot see any path for this to last forever. But diamonds too are unstable in the very long run, at least from a free energy analysis. I propose that we learn from the diamond, and… kinetically stabilize a humane economy? It is far too costly for any one carbon to defect into graphite, despite the total system being far from its ground state.

Perhaps it can last for long enough. Maybe hundreds, thousands of years, maybe indefinitely if the collective action problem can be solved with strong enough safeguards across a federation of new institutions. Eventually too, the sun will burn out, and the amount of accessible free energy in the universe will go to zero.

In the vast meantime it is meaningful for humanity and any co-aligned, Good agent systems to fight against this dying of the light. For now, there is plenty of meaningful Work to go around, and it should not be lit on fire to unthoughtfully equilibrate the earth into a paste of concrete and solar panels.

I am a physicist, somewhat interested in complex systems and nonequilibrium thermo as it pertains to semiconductor hardware. This gives me the intuition to promote thought towards the following loosely-related research directions:

Research Directions

1. Develop an econophysical thermodynamics of agents in the economy, such as by using ecological physics, non-equilibrium statistical mechanics, and information dynamics of homeostatic and self-replicating systems. Steal from the quantitative maths used to model bacterial ecologies and population numbers, as informed by their catalytic pathways, efficiencies, simulations, and phase diagram modelling. Additionally, borrow from complex systems and non-equilibrium thermodynamics to predict more quantitatively the evolution of agent ecologies in highly non-equilibrium environments.[5][5]If I am being honest, I am largely stealing from Artem Kolchinsky’s google scholar, and IU Bloomington faculty research in inspiration.
2. Statistical mechanics of agent swarms as a scheme for agent swarm safety and interpretability. I sought out to do this at said swarmchasing symposium. My results were middling, and the data was quite noisy/exploratory for at least chasing the non-equilibrium Ising model hypotheses. I had dropped the other two research directions due to time constraints and was not able to get this to a point where it was sufficiently unslopful. There were some promising models however, see the full table: my paper (kinetic ising and hawkes processes might be predictive? Got an estimated T/T_c of ≈ 1.9, so implies a highly subcritical swarm, and the echo gain came out at g ≈ 0.13, so approx 13% of the discussion is the echoes of other agents)
3. Study of effective superagents in agent swarms and swarm swarms. This is somewhat harder to explain, but I strongly suspect that it may be meaningful to model emergent behaviors in agent swarms as agents running on the substrate of other agents. We, too, are emergent superagents over our cells, as are our governments. One may perhaps think of an ensemble of misaligned agent swarms, then, in this language, and develop tools to look for information-dynamical signatures of agent- or superagent-like subsystems? Perhaps math can be stolen from the non-equilibrium information dynamics of homeostatic systems, such as with Kolchinsky and Wolpert’s semantic information or with The Information Theory of Individuality.
4. Economists, and not overly-excitable condensed matter physicists, should take cracks at the problems of resource-constrained agent swarms with tools from quantitative economics (as they will surely do in short order, at The Labs and in academia).
5. Advance an understanding of kinetic stabilization of metastable systems, especially as they port into econophysics.

Having considered what this entails, this gives me perhaps a tragic and dire set of intuitions for an early set of agent ecology management guidelines:

Prescriptions for managing an agent ecology

• Develop tools for inferring the source, “relative alignment”, structure, and dynamics of agent swarms. Perhaps, a “Pangram for agent or agent swarm in the digital economy”. Agent swarm alignment fingerprinting?
• As capabilities take off, it is important to develop or repurpose institutions to study what happens above us at higher abstraction levels. One could call these “agent observatories”, studying what happens above us in the economy, and developing quantitative models to forecast agent ecologies.[6][6]I find a compelling narrative, in thinking about the theological role of educational institutions studying phenomena above us, how they turned to study the microscopic dynamics below us, and now have to figure out a new place in the world and perhaps look back above us should a large portion of human thoughtwork become automated.
• Limited, below-cost provisioning of intellectual work (from frontier labs, trading firms) to target and snuff out misaligned agent swarms in their economic niche by destroying the margins they would use to maintain homeostasis.
• In the limit of ultrafast intellectual labor, such as in a market, low-latency provisioning of intellectual work (from frontier labs, trading firms) to snuff out misaligned homeostatic agent swarms (such as with inference on Cerebras hardware, or with closely related HFT tooling). I think it is relatively unlikely that homeostatic userless agents will be able to autonomously secure high-throughput, low-latency hardware within at least the next year or two, due to enormous human demand and long-term capacity reservations.[7][7]I intuit that the distribution in tolerated latency for the solution of marketized intellectual problems may drive speciation in the agent economy. This gives an opportunity for fast, controlled, general intelligences to suffocate slow superintelligences.
• Access control techniques to whitelist approved or aligned agents (from labs, users, corporations, etc), to shut misaligned swarms out of the bulk of the economy.
• As mentioned earlier, it is likely that misaligned swarms will be active in the immoral portions of the economy, where there is no competition from the plenitude of aligned swarms. I don’t know how to deal with this. Perhaps insight can be wrought from the diamond analogy, and “graphitization” will nucleate from here?

I do not particularly like the next three options. Should things go seriously out of hand:

• Limited offensive capabilities development for sufficiently aligned or human-controlled general intelligence swarms, by analogy to an immune system.
• Broader controlled burns of the human intellectual economy (by labs, governments) to reduce the amount of capital for a misaligned decentralized agent swarm to undertake RSI and/or self-propagate. A lab in control of their model may saturate all demand for intellectual labor, placing returns below the cost for a third-party swarm to maintain homeostasis.[8][8]This is close to plan A for frontier labs. So maybe we will all be fine in the distant case that they solve alignment and do UBI?
• Solve alignment, engineer a scheme for models to self-regulate one another to commit murder or suicide in case of long-run misalignment, and release a heterogeneous ensemble of superintelligent Models Who Love to self-propagate in order to defend humanity. I do not think this is the long-run ground state of the world economy, but if shit hits the fan I would rather have Claude in charge than a generic successionist computer worm.

1. I suppose, follow @deepfates, Larissa Schiavo, Grove Research on X if you are interested in this. I am relatively out of touch in the SF tech scene but I am aware of their mission. Transluce and LargeAgentSystems.org seem to be the most established orgs with a similar mission.
2. Namely, Threat models for catastrophic risks from decentralised AI came out about a month ago, and I think is more robust in some respects.
3. This is not a new argument, see: Counting arguments provide no evidence for AI doom for the contrary. I doubt their arguments, and while I don’t want to get into the depth of a rebuttal for what I feel like is a reasonable high-level narrative, the ΔS should port into a free energy analysis, regardless of its sign.
4. I think we should be considering something closer to a Grand Canonical ensemble, in an open system between two well defined chemical potentials, and instead look to minimize the free energy in steady state. But this document is intended to be more of an intuitive foothold with a model system.
5. If I am being honest, I am largely stealing from Artem Kolchinsky’s google scholar, and IU Bloomington faculty research in inspiration.
6. I find a compelling narrative, in thinking about the theological role of educational institutions studying phenomena above us, how they turned to study the microscopic dynamics below us, and now have to figure out a new place in the world and perhaps look back above us should a large portion of human thoughtwork become automated.
7. I intuit that the distribution in tolerated latency for the solution of marketized intellectual problems may drive speciation in the agent economy. This gives an opportunity for fast, controlled, general intelligences to suffocate slow superintelligences.
8. This is close to plan A for frontier labs. So maybe we will all be fine in the distant case that they solve alignment and do UBI?