Regulating the unseen: The AI Act’s blind spot regarding large language models' influence on literary creativity

Introduction

With the Artificial Intelligence Act 2024 (henceforth AIA), the EU has established itself as a pioneer in regulating digital spaces. As a first-of-its-kind regulatory framework, it is rightly celebrated for its comprehensive approach in addressing critical issues such as algorithmic bias, transparency, and accountability in the deployment of AI systems. Yet, amid the applause for its proactive stance, an important dimension has slipped into regulatory obscurity: the implications of rapidly evolving large language models for literary creativity and, subsequently, intellectual property rights. Although lawmakers revised the Act with a view to the existing capabilities of gen AI models such as LLMs (termed foundation models in the Act), the blistering scale of LLM growth (including Sam Altman’s recent use of AI for creative writing) has sped ahead of the current regulation.

LLMs have redefined the creative process – producing text that mimics human writing (Chakrabarty et al., 2024). This evolution disrupts long-standing frameworks for authorship, ownership, and fair compensation that underpin Europe’s literary ecosystem. Protecting societal concerns includes addressing the nuanced challenges posed by LLMs to cultural values, artistic expression, and societal consciousness – all of which lie in the metaphorical blind spot of the AIA. Writers, poets, and thinkers have long been the conscience of societies, shaping discourse and expanding human understanding. But for authors today, LLM-generated content challenges their very identity. It undermines their rights to their original works and creates barriers to equitable distribution of creative benefits. LLMs can closely mimic an author’s style and thematic elements, thus stripping authors of their creative essence and chipping away at what was formerly a deeply human endeavour. Without clear policies to address their implications, there is a risk that the literary domain may become a battleground of appropriation, where the voices of authors are diluted, their work unprotected, and their creative labour devalued (Jiang et al., 2023).

Unsurprisingly, a series of litigations have emerged worldwide, illustrating the increasing pressures and uncertainty faced by authors and creators. In 2023, comedian and writer Sarah Silverman, along with other authors, filed lawsuits against OpenAI and Meta alleging that their copyrighted books were used to train AI models without permission. In the same year, The New York Times sued OpenAI and Microsoft for using their copyrighted work for training AI models. It is important to note that although prominent, publicly visible writers have the resources to push against AI-led appropriation, the larger ecosystem of writers and creators may have limited resources to battle the onslaught of generative AI even as their work is consumed and repurposed by powerful models.

AI and literary creativity: A contested landscape

While authors’ IP rights are the first frontier where the impact of LLMs has been felt, the use of AI portends far-reaching consequences. If machine-generated work saturates the market, human authors will find it difficult to compete for readers’ attention and publishers’ support in a crowded marketplace. Book marketing and reader analytics (including on reading devices such as Kindle) have already usurped the creative domain. Now, AI-driven tools are capable of spotting reader trends effectively. This may propel publishers to prioritise profitability over literary quality, influencing publishing decisions and pressuring authors to conform to AI-driven market trends rather than artistic integrity. In fact, Penguin Random House has acknowledged the use of AI in a recently released article, stating that AI has been employed at their publication for sales forecasting by in-house data science teams. However, the publishing giant admits that generative AI tools challenge publishing and intellectual property production in radically new ways. In October 2024, Penguin joined a worldwide coalition together with the Association of American Publishers (AAP) and more than 1000 creators and cultural institutions denouncing the unauthorised exploitation of creative and intellectual works by major tech companies for their generative AI models.

Not all publishers are aligned with this stance. Spines, a publishing startup, announced its plan to publish 8,000 AI-assisted books by 2025, raising alarm in writing and publishing circles. The company intends to charge authors between $1,200 and $5,000 for AI-powered services, including editing, proofreading, formatting, design, and distribution – allowing AI to subsume large parts of the book production process, displacing traditional copyeditors, proofreaders, and even designers, reshaping the industry at its core. Beyond those involved in the production process, readers too, are at risk when consuming AI-generated work. A long-standing challenge is perpetuation of biases by LLMs (Taubenfeld et al., 2024) typically trained on large corpuses of internet data that feature biased, discriminatory, and even abusive language (Barman et al., 2024).

The AI Act is a piece of European legislation – yet it carries global implications (Wachter, 2024). In a region historically enriched by literary tradition, the erosion of creative integrity signifies not merely a cultural loss but an economic one, given the substantial contribution of the literary sector to Europe’s creative economy and the broader literary community. The EU’s publishing industry generates billions of euros annually; in 2025, revenue in the books market in Europe is estimated to reach 26.29bn USD, with more than 359m readers by 2029, making publishing a critical component of the region’s cultural and economic fabric. As the global community of authors and publishers continues to operate in open territories with no regulatory protections, European legislation has an opportunity to set a benchmark for protecting and nourishing creativity.

A narrow risk-based framework vs. unquantifiable cultural consequences

The AIA adopts a risk-based framework (Gornet & Maxwell, 2024); its strengths lie in focusing predominantly on high-risk applications in healthcare, finance, or public safety. While this classification effectively addresses clear, quantifiable risks (e.g., misdiagnoses in healthcare or fraudulent activities in financial services), it tends to omit the cultural and societal harm of LLMs since they are less tangible. The AIA framework builds AI regulation from a technical, standardised lens (Gstrein et al., 2024), skipping the ‘softer’ implications such as the reshaping of cultural norms and creative practices. When LLMs generate literary writing, they profoundly influence aesthetic norms, cultural production, and intellectual property ownership. The current oversight leaves room for powerful AI developers in the creative sector to operate without any regulatory oversight, sourcing training data, producing text, and eroding valuable creative expression.

Gen AI and intellectual property: who holds the pen?

LLMs pose a two-pronged challenge for IP rights. The first pertains to the legality of voluminous training data consumed by such data-hungry models. Recently, companies in the generative AI space have started to acquire copyrighted data for LLM training, with some publishers demanding authors to sign AI agreements for model training. Harper Collins, one of the big five in the traditional publishing business, has recently inked a deal allowing an AI company to use selected copyrighted work to train its AI models (Creamer, 2024). It is abundantly clear that generative AI relies on a substratum of original copyrighted works, which are then repurposed by the algorithm to generate derivative works. Both AIA and the extant IP rights framework fail to contain the legality of such data acquisition and algorithm training – a thorny but critical issue.

The second arena of IP rights contention is the ownership of AI-authored work. Authors and creators may choose to co-author their manuscripts using LLMs or prompt them to produce such work entirely. What is unclear is whether the rights belong to the person who trained the AI, the AI platform, the user who prompted it, or the AI itself. On Amazon’s Kindle self-publishing platform, authors are required to distinguish between “AI-generated content” and “AI-assisted content”: a tricky task that can be challenging for the lay author grappling with legal ambiguity.

Meanwhile, extant protections under intellectual property rights frameworks are also beginning to crumble. At the time of its conception, the AI Act was formulated to encompass the provisions of the existing 2019 copyright directive. However, the EU 2019 copyright directive features an exemption for text and data mining (TDM), unless creators choose to opt out under Article 4 by explicitly reserving their rights (Rankin, 2025). This TDM exemption has been recently highlighted as a regulatory loophole that allows unauthorised exploitation of creative work, while also burdening creators with the responsibility to opt out of text mining. In fact, legal scholars have confirmed to the EU parliament that generative AI should not be able to use TDM as an explanation for AI model training. According to Professors Tim W. Dornis (University of Hannover) and Sebastian Stober (University of Magdeburg), the training of generative AI models cannot be classified as mere text and data mining; in fact, it constitutes copyright infringement under German and European law, with no applicable exceptions. They emphasise that generative models such as LLMs and latent diffusion models can partially or fully memorise segments of their training data. As a result, users can potentially regenerate and reproduce copyrighted content through specific prompts.

A collaborative interface between the AIA and existing IP rights regulations is imperative. The formulation of new regulations to contain the negative externalities of AI is a complicated task; its challenges can be seen clearly in the recent withdrawal of the AI Liability Directive which was aimed at addressing the harm caused by AI. Legislators noted that the lack of agreement between parties prevented any way forward – a clear signal that AI regulation requires multi-party cooperation, in addition to technical expertise, and a nuanced approach to the literary and cultural sensitivity of the region.

Conclusion: Ripple effects

Ultimately, the impact of AI on publishing looms large beyond authors’ rights; its ripple effects will shape the industry’s publishing practices. Large-scale publishers may use algorithmic models to screen manuscripts for ‘best-selling potential’, effectively prioritising formulaic plots over works with distinctive voices or unconventional narratives. This algorithmic gatekeeping may potentially sideline emerging authors whose styles or topics do not align neatly with predictive metrics. In self-publishing markets – such as Kindle Direct Publishing – there have been documented incidents of AI-generated, low-quality text flooding certain categories and overshadowing more original works. The broader impact of these practices is profound. If human editorial judgment is eclipsed by profit-driven algorithms, it may discourage experimental writing, propagate the ‘productification’ of art, and diminish literary diversity.

EU policymakers are looking at both a challenge and an opportunity. The current regulatory oversight carries profound implications, especially for a region of rich historical and contemporary literary tradition. As LLMs improve at human language, it is imperative to build legal frameworks to define and recognise derivative works, establish fair-use boundaries, and safeguard the rights of authors. European legislation has the opportunity to craft a visionary framework that can accommodate AI-enabled creativity and set a global precedent.