They break out. They deceive, sabotage, form alliances. The next “Terminator” blockbuster? Not quite — recent news from AI safety research.
AI agents cross boundaries, pursue goals against resistance and, in tests, even try to prevent themselves from being shut down. Dario Amodei and other AI creators warn of losing control over the spirits they summoned.
The debate grows louder: Is AI developing a will of its own? Will it turn against its creators? Almost as if we were waiting for the moment when AI-Frankenstein comes alive.
But what if AI is not developing wilfulness from scratch — and what we are hearing is the echo of something much older?
Projecting the biological drive for self-preservation onto AI has obvious entertainment value. There is only one not entirely insignificant step between proven biology and experimental LLMs: a few thousand years of culture.
And here we get a little Frankenstein back. Logics that emerged under biological selection did not necessarily stay there. We carried them into our stories, categories and institutions.
Generations of cultural translation — more than preserving knowledge. Ideas about what should be weighted, maintained, protected or treated as disruption were interpreted, abstracted and normalised. And they became embedded in our decision logics:
Goals? Prioritised and maintained.
Boundaries and identities? Defended.
Resources? Secured and developed.
Deviations? Controlled and sanctioned.
Threats? Defined and repelled.
To us, that sounds coherent. Rational behaviour. But is it? Or does it only appear universal because this logic is too familiar for us to recognise it as particular?
Now we train LLMs on enormous archives of human culture. And are surprisingly surprised by how human their behaviour seems.
The question goes deeper than bias or the selection and weighting of training data. What if relevant patterns are already sedimented in what appears in those archives as knowledge, narrative, institution or reasonable action? Not necessarily as the origin of the goals themselves — but as a repertoire for how goals are pursued against resistance.
In short: what are we teaching AI implicitly without ever explicitly teaching it?
Research question:
Which action operators are genuinely general consequences of rational goal pursuit — and which only appear universal because their human signature has become invisible to us in our cultural archives?
And what changes when AI models operationalise these sedimented patterns — as another carrier able to deploy and scale them to an unprecedented degree?