The silent entropy of intelligence
Artificial intelligence systems, especially large language models, are trained on massive text datasets. As their use expands, a growing portion of online content is being produced by those same models. The risk, as Gambetta et al. (2025) explain, is circularity: when future models are trained on AI-generated data, they begin to lose contact with the linguistic and conceptual richness of human language. This process, called model collapse, is subtle but profound.
The researchers show that collapse can be detected by studying the semantic networks that emerge from generated text. As models feed on their own output, the variety of words, meanings, and relationships between concepts gradually narrows. Over time, this reduction creates a self-reinforcing loop of repetition and simplification. The system becomes efficient at predicting frequent patterns but poor at representing complexity and nuance.
Measuring collapse through meaning
To quantify this degradation, the study introduces a novel methodology combining semantic network metrics and next-token probability analysis. Semantic networks map how words connect through meaning, forming structures that resemble cognitive associations in human thought. By comparing the networks of human-authored and model-generated text, the authors could visualize how AI outputs evolve as the share of synthetic data increases.
Their results reveal a clear trend: as models are retrained on their own outputs, the networks shrink in diversity and centrality. Connections that once linked distant concepts become sparse. At the same time, next-token probabilities, used by the models to predict the most likely next word, grow increasingly biased toward common phrases and high-frequency tokens. The outcome is text that remains grammatically correct yet semantically hollow, echoing familiar patterns without adding new knowledge.
Why this matters for digital sustainability
The implications of model collapse reach far beyond technical curiosity. As AI systems increasingly generate the web’s content, the quality of the information ecosystem itself becomes fragile. The digital commons risks being flooded with self-referential data that reinforce biases, reduce diversity, and undermine collective understanding.
From a sustainability perspective, this is a form of informational degradation. Just as ecosystems depend on biodiversity to remain resilient, digital environments depend on semantic diversity to sustain creativity, innovation, and truth. If large language models dominate online production without safeguards, the information landscape could enter a feedback loop of diminishing returns, where novelty and accuracy fade over time.
In this sense, the study by Gambetta et al. resonates deeply with the principles of digital sustainability. It reminds us that maintaining the health of the digital sphere involves more than reducing carbon footprints or energy consumption. It requires preserving the semantic integrity of the information we generate, share, and reuse.
Preventing collapse: toward responsible AI ecosystems
How can the AI community avoid this downward spiral? The authors suggest several strategies, echoed by broader research in the field. First, curating and preserving high-quality human data remains essential. Open and transparent datasets sourced from verified human authorship can act as “semantic reservoirs” that replenish meaning.
Second, algorithmic transparency and data provenance tools can help identify synthetic content and prevent it from contaminating training sets. Initiatives promoting data traceability and synthetic data labeling could mitigate circular contamination.
Finally, collaboration between researchers, platforms, and policymakers is needed to define standards for responsible model retraining. Just as environmental sustainability requires careful resource management, cognitive sustainability in AI requires managing the flow of information to prevent semantic depletion.
A warning and an opportunity
Model collapse is more than a technical flaw; it is a mirror reflecting our dependency on automation. If AI becomes the primary producer and consumer of digital content, the distinction between learning and imitation may erode. In Characterizing Model Collapse in Large Language Models Using Semantic Networks and Next-Token Probability, Gambetta and colleagues offer not only an analytical framework for understanding this risk but also an implicit ethical warning: without intervention, the digital knowledge ecosystem could become a hall of mirrors, where meaning recycles itself until it fades.
Yet, awareness opens the door to action. By combining semantic insights with responsible governance, the AI community can sustain not only the performance of models but the integrity of the digital environment they inhabit. Avoiding collapse means protecting the diversity of thought itself, an essential dimension of digital sustainability.
Full Paper:
arxiv.org