AI, the Snake that eats itself? An introduction to knowledge collapse and how to mitigate it

"When you start to feed a model synthetic stuff you start to get diminishing returns... over-reliance on synthetic data risked 'model collapse'..." — Andrew Duncan, Alan Turing Institute

Proponents hail AI as the Fourth Industrial Revolution, promising greater efficiency, lighter workloads, and relief from mundane tasks. Critics counter that it was uninvited outside tech circles, risks amplifying misinformation and bias, and poses environmental, even civilisational threats. Despite these concerns, AI’s permanence is evident: $1.6 trillion has been invested since 2013, with 2025 investment forecast at $375 billion. Investment shows no sign of slowing, a McKinsey report from April 2025 estimates that $5.2 trillion will be required for data centres by 2030 to meet AI demand alone. The sheer scale of this commitment suggests AI is becoming “too big to fail,” making understanding its impacts, strengths, weaknesses, and liabilities an increasingly essential part of both professional and personal life.

It is estimated that in Europe by 2030 there will be roughly 527 million unique smart phone users, which represents a penetration in excess of 85% of the population. Of these, 80% are forecast to utilise 5G access for the internet.[1]  In the USA the numbers are not too dissimilar, and in 2025 it was estimated that there was an overall national internet penetration of 93.1%.[2] Not only are more people than ever connected to the internet, but there is a growing proliferation of devices and search engines with AI tools in built. Whilst it is possible to deactivate and suspend some in-built AI features, to do so is a complex and unintuitive task that could seem daunting to the average user. Therefore, millions of internet users are all but compelled to utilise toolsets that they may not appreciate the full utility of, or indeed the risks of doing so.

With the rapid growth in the proliferation of AI toolsets, and its equally rapidly developing capabilities, it is increasingly important to be aware of where AI draws its information from. Typically, AI is trained upon data from open-source information on the internet, or where appropriate, closed sources, such as those relating to government work. However, the source of some data is given extra weighting by AI and LLMs, and therefore, more likely to be utilised than other, perhaps more varied sources. This can lead to a potentially limited pool of source data, which is then negatively reinforced by a phenomenon known as Knowledge Collapse, where AI cite increasingly small and repetitive sources, particularly amplified when pulling material produced by other AI.

In this short paper, it will outline what knowledge collapse is, the dangers and implications of it, and a few ways in which you, an AI user, can counter these dangers.

Knowledge collapse is where AI cites from either a singular or small search pool, which is then amplified when other AIs cite from either the condensed version or rework the original source material. Although AIs are generally trained to search from a diverse source pool, some data is highlighted and preferred over others, this can lead to output swung towards and focussed upon the “centre” of distribution.[3] Ironically when researching this paper, there is a prime example of knowledge collapse in that when “Knowledge Collapse” is searched in Google and Bing, 80% of the returns on the first page are AI rehashed and shortened versions of a singular paper, “AI and the Problem of Knowledge Collapse” by Andrew Peterson, inadvertently displaying the highlighted issues of knowledge collapse. AI software is inherently unable to search out challenging forms of information and data because they are not trained to do so, they look for the median and central data. However, in doing so they are at risk of omitting potentially contradictory and outlier data that counters the accepted and trained narrative.

The below (from Shumailov 2023’s work)[4] highlights the degeneration in data quality when AI’s “learn” from themselves. The input below (more commonly known as a prompt) is the original data given to the Generative AI, the below outputs highlight the collapse in the quality of information, with each Gen representing a respective repeat of the prompt but run internally with the AI “Learning” from itself, and other AI produced data. This can lead to a related phenomenon known as model collapse, where the data is sourced, scrubbed and deployed from the previous generation of AI source data.  

Highlighting Data Degradation within the Model Collapse Phenomenon

Input: some started before 1360 — was typically accomplished by a master mason and a small team of itinerant masons, supplemented by local parish labourers, according to Poyntz Wright. But other authors reject this model, suggesting instead that leading architects designed the parish church towers based on early examples of Perpendicular

Outputs: Gen 0: Revival architecture such as St. John’s Cathedral in London. The earliest surviving example of Perpendicular Revival architecture is found in the 18th century Church of Our Lady of Guernsey, which dates from the late 19th century. There are two types of perpendicular churches : those

Gen 1: architecture such as St. Peter’s Basilica in Rome or St. Peter’s Basilica in Buenos Aires. There is no evidence that any of these buildings were built during the reign of Pope Innocent III, but it is possible that they may have been built during the reign of his successor, Pope Innocent

Gen 7: architecture in England. In an interview with The New York Times, Wright said : ” I don ’t think there is anything wrong with me being able to do what I want to do. It just doesn ’t work for me. ” He added : ” I don ’t know if you can call it funny,

Gen 9: architecture. In addition to being home to some of the world’s largest populations of black @-@ tailed jackrabbits, white @-@ tailed jackrabbits, blue @-@ tailed jackrabbits, red @-@ tailed jackrabbits, yellow

Whilst the first 2 generations of output are at least related to the original input, it is apparent that by 7^th there is no bearing on the input, and that by the 9^th it is complete nonsense. This leads to a very real risk of being provided inaccurate or completely useless information when unwittingly utilised data is sourced through these mediums. The risk is further compounded when one accounts for the fact that as of 2024, 40% [5], rather than fresh data sources.[6]

The severe implication of this is further emphasised when one thinks of the language of AI sources. Given the dominance of US tech companies combined with a broader cultural impact, it should come as little surprise that English is the primary language of the internet. However, this is not widely reflective of the demographic nature of spoken languages. Data from Common Crawl, one of the largest public sources of training data, reveals stark inequalities. It contains more than 300 billion webpages spanning 18 years, but English, which is spoken by approximately 1 in 5 people worldwide, dominates with roughly 45% of the content. However, there can be an alarming imbalance between a language’s demographic size and how well that language is represented in online data. For example, Hindi, the third most popular language globally and spoken by about 7.5% of the world’s population[7] accounts for only 0.2% of Common Crawl’s data. Therefore, not only is data sourced from increasingly singular sources, but it is also mono-dimensional linguistically and culturally, leading to repetitive source completely dismissive of information from outside the English speaking world.

Individually all the above issues mentioned, the data repetition, the single source emphasis, mono-cultural domination and model collapse, would be significant. However, when one factors that these are all facets of the same issue, coupled with the ever-increasing unavoidable presence of AI in daily life, means that the problem is quite profound. If developments in AI continue along this trend, we are likely to be subject to an increasingly slim and repetitive pool of data, with all the associated biases inherent within the source data. It risks reducing dissension and amplifies an accepted narrative, as determined by the AI. In turn this risks a dearth of challenging and niche ideas, and could prompt individuals to accept the lazy, and easily sourced AI version of truth, rather than going out of their way to look for greater depth and nuance. A more worrying potential development is the growing spectre of model collapse, and AI users inadvertently reinforcing mistruths or outright nonsense as it is determined that AI has the capability to learn from itself. This is all likely to have the effect on the average user that the main concern is no longer the question posed, but in fact how the question is asked, to best leverage the tools available.

It is worth noting that these challenges, whilst seemingly daunting, are far from insurmountable, and some of the solutions are refreshingly old-fashioned. One of the more effective ways to determine if your information has been subject to knowledge collapse is to look at the cited sources. If no sources are cited or referenced within an article, it is perhaps safe to assume the information being provided is the accepted AI median and narrative. If time is available, then it might be worth delving into the sources or citations to ensure that they themselves are not purely AI constructed material. Each layer the source material is broken down into adds variety and nuance, and when combined with more traditional tools, such as analysis for political narratives or general biases, can be an incredibly effective tool in combating the knowledge collapse phenomenon.

Conclusion

Ultimately it is likely that the most effective way to combat knowledge collapse is to utilise traditional scholarly methods, seek out multiple sources, try to understand challenging and dissenting viewpoints, look for varying content, dive into the deeper reaches of search engines, much like you would search in a dusty corner of a library, seek out and clarify your information. Whilst this is a time intensive solution that is far from suitable from every occasion in these hectic and rapidly changing times, the core message of doing your homework, rather than relying on someone else is reassuringly simple, and transferable across many disciplines.  

Further Reading

AI and the Problem of Knowledge Collapse

Knowledge Collapse in LLMs: When Fluency Survives but Facts Fail under Recursive Synthetic Training

(PDF) Model Dementia: Generated Data Makes Models Forget

References

[1] The Mobile Economy Europe 2025 - The Mobile Economy Accessed 27/11/25 12:01

[2] Digital 2025: The United States Of America — DataReportal – Global Digital Insights Accessed 27/11/25 11:43

[3] AI and Knowledge Collapse accessed 15/12/25 07:49

[4] Shumailov, I.; Shumaylov, Z.; Zhao, Y.; Gal, Y.; Papernot, N.; and Anderson, R. 2023. The curse of recursion: Training on generated data makes models forget

[6] Is AI Silently Erasing Human Knowledge? The Hidden Dangers of Knowledge Collapse | Latest AI & Tech News on Daily Tech accessed 08/12/25 15:31

[7] Statistics of Common Crawl Monthly Archives by commoncrawl Accessed 01/12/25 13:44