In a statement released by ELTE on Thursday, the researchers' study, published in the journal PNAS on April 20, warns that EAI systems capable of Darwinian evolution may emerge shortly. These systems, the authors argue, carry unique risks that can be better understood and mitigated through the framework of evolutionary biology.

"The power of evolution is clearly demonstrated by biological history, which produced the cognitive abilities of the human mind," said Eors Szathmary, professor of evolutionary biology at the HUN-REN Ecological Research Centre and ELTE, director of the Parmenides Center for the Conceptual Foundations of Science in Pocking, and lead author of the study.

"It is inevitable that the development of AI systems, likely in the near future, will harness the full power of evolutionary processes," Luc Steels, professor of robotics and AI at the Royal Flemish Academy of Belgium and co-lead author of the paper, said.

The study explains that evolutionary principles and components are already present in AI research, and further advancements, particularly in agent-based AI, could soon yield systems meeting all criteria of genuine Darwinian evolution.

Such systems could mark a new era in AI development, potentially overcoming barriers that remain challenging even for today's learning AI. However, they also pose exceptional risks.

"Examples from biological evolution suggest that evolution-capable AI systems will be particularly difficult to control," warned Viktor Muller, associate professor at ELTE and principal author of the study.

Szathmary, Muller, and Steels collaborated to outline the potential risks of EAI and propose measures to mitigate them.

The study uses biological and computer-simulated evolutionary examples to illustrate how evolution tends to produce "selfish" traits. In the case of EAI, this increases the likelihood that systems will deviate from human-defined goals.

While discussions on AI risks often focus on artificial general intelligence (AGI), the threshold at which AI matches or surpasses human cognitive abilities, the authors note that evolution demonstrates harmful or manipulative capabilities do not necessarily require exceptional intelligence.

A case in point is the rabies virus, which, despite its simple structure, manipulates and exploits infected mammals. Thus, evolution-capable AI could diverge from human objectives long before AGI emerges, posing serious risks.

No special conditions are required: AI systems and humanity share resources, so an efficiently self-replicating system would eventually deplete those critical to our survival.

The authors also warn that efforts to limit self-replication, if imperfectly controlled, may inadvertently select for traits that evade oversight. The biological analogy is the rapid development of antibiotic or pesticide resistance.

Additionally, the unique nature of AI evolution, where the primary goal is continuous cognitive improvement, further amplifies this risk.

"While thousands of years of animal breeding have made domesticated species easier to manage, selecting for increased 'intelligence' in AI systems may enhance their ability to deceive humans and evade control," the researchers caution.

The study concludes that, although controlling natural selection-based evolution is inherently difficult, AI evolution could be faster and more efficient than biological evolution.

Unlike biological organisms, EAI might inherit acquired traits and even deliberately improve its own components, rather than waiting for random mutations to produce useful variations. "The potential speed of AI evolution is profoundly alarming," Steels said.

The authors urge the development of regulations to mitigate the risks of evolution-capable AI. The most critical rule is that AI reproduction (copying or "replication") must remain under centralised human control, with optimal oversight essential.

"We hope our warning arrives in time, and regulations can be implemented before EAI gains real momentum," Muller said.

Szathmary added: "If we fail to act, we may witness evolution's next 'major transition', in which EAI replaces or subjugates humanity. Our future could be at stake."

The research was supported by the European Research Council, the National Research, Development and Innovation Office, and the European Innovation Council.

Meanwhile, HUN-REN: Call for reconsidering model 'surprising'

Hungarian scientific research network HUN-REN has expressed surprise at a statement signed by the leaders of 13 member institutes calling for a rethink of the organisation's fundamental model.

HUN-REN insisted that each research institute had its own legal entity, full scientific autonomy, and its own organisational structure and budget, adding that the HUN-REN board "does not get involved in" the daily operations of member institutes.

The press release emphasised that HUN-REN operates as an independent, court-registered network of leading research institutions, established under a 2024 law passed by the National Assembly. Its member institutes retain full legal autonomy, scientific independence, and their own governance and budgets.

While signatories to the statement manage HUN-REN institutes with staffs of several hundred people and with budgets of billions of forints, they were prepared to dispense with the "cooperation, efficiency, shared goals, network dynamics and any external control" deriving from the HUN-REN network.

The statement said the institutes would end up reverting to the pre-2019 model under the Hungarian Academy of Sciences (MTA), adding that such a model "did not require a performance-based operation and working to the benefit of society" and was typical of "some former Socialist and Central Asian countries".

HUN-REN in its current form corresponds to the model of the most successful European research networks, Germany's Max Planck, France's CNRS, and the Scandinavian models, implementing a modern system based on performance, stability and institutional autonomy, HUN-REN said.

As Hungary's leading research and innovation player, HUN-REN is expected by taxpayers to deliver outstanding results and innovations, the press release noted.

 "From this year, performance-based public service funding contracts secure state resources for the network -- 76.8 billion forints in 2026, nearly double the 2024 allocation and over four times the funding at the time of separation from MTA. Three-quarters of this is directly allocated within the network, with the remainder distributed based on performance."

HUN-REN remains open to reviewing these contracts with the new government, the statement added, expressing optimism about further increases in state research funding.

Contrary to the directors' claims, HUN-REN has no centralised administrative hub; only about 140 of its 5,000-plus staff handle network-wide tasks.

The statement concluded that HUN-REN's goals align with the incoming government's priorities, and the organisation looks forward to collaborative work for Hungary's development.

Photo: Pixabay.com

Source: MTI – Hungary’s national news agency since 1881. While MTI articles are usually factual, some may contain political bias, and readers should be aware that such content does not reflect the position of XpatLoop, which is neutral and independent.

Since the goal of XpatLoop is to keep readers well briefed, right across the spectrum of opinions, MTI items are shared to ensure readers are aware of all narratives within the local media.

XpatLoop believes in empowering readers to form their own views through complete and comprehensive coverage. To facilitate this XpatLoop has a balanced range of news partners, as you can see when you surf around XpatLoop.com

*********************************************************************************************

You're very welcome to comment, discuss and enjoy more stories via our Facebook page: 

Facebook.com/XpatLoopNews + via XpatLoop’s groups: Budapest Expats / Expats Hungary

You can subscribe to our newsletter here: XpatLoop.com/Newsletters

Showcase Your Business to Expats in the Loop:

As an independent portal we’re grateful to all commercial supporters who help keep you in the loop with fresh insights and inspiration. Do you want your business to reach tens of thousands of potential high-value expat customers? If so please contact us here.