Final test

A trained model is a short program that sets out a fixed recipe of arithmetic, together with a large file of numbers that the recipe uses, and the two only work together. The numbers are the weights, also called parameters, and training is the long process that sets each one.

Each weight is stored in a set number of bits, the 0s and 1s a computer records, so the file grows with the number of weights. Quantisation stores each weight in fewer bits once training is over, much as rounding prices to the nearest pound makes them quicker to write down. The smaller file can let some models run on a laptop, at some cost to the accuracy of the answers.

The training text is used to set the weights and is not kept in the file. Meta’s Llama 3 405B shows the difference in size, having read very roughly 60 terabytes of text while its weights take about 810 gigabytes, around 74 times less. A file of that size is therefore far too small to hold a verbatim copy of everything the model read, although a model can still learn some passages so well that it reproduces them word for word or nearly so, which is called memorisation.

Removing a document from the training set later leaves the existing file exactly as it was. A demand to delete content may therefore call for training a new model, a filter that blocks the content in the model’s answers, or a targeted edit to the weights, which can disturb other things the model has learned.

Most parameters are the strengths of connections between units, the small calculations a network is built from, and each works like a dial that sets how strongly one input counts. Training nudges every dial to suit millions of examples at once, so what the model learns is spread across vast numbers of weights and a single weight rarely has a meaning a person could name.

A neural network is built from many small calculations called units, which popular accounts call neurons, arranged in rows called layers that each feed the next. Every unit repeats the same steps on the numbers it receives.

1. It multiplies each input by its weight, so a large weight makes an input count heavily and a negative weight makes it count against the result.
2. It adds up the results together with the bias, a starting score that can be positive or negative, in a sense unrelated to prejudice.
3. It applies the activation function, a fixed rule that decides what the unit passes on, and a widely used rule called ReLU keeps a positive total and turns a negative total into zero.

The total is only the unit’s working figure, since the activation function decides what goes on to the next layer. With ReLU a negative total is passed on as zero, so the unit stays silent for that input.

A neural network passes numbers through layers, rows of small calculations that each feed the next. Each layer, if it only multiplied and added, would draw a straight-line relationship between what goes in and what comes out, in the way a fixed hourly rate turns each extra hour into the same extra fee. Two such layers in a row still give a straight line, as an hourly rate followed by a 20 per cent uplift is still an hourly rate, so a stack of a hundred would be limited to what a single layer can do.

The activation function is a fixed rule applied to the total each calculation produces, and it supplies the bend. A widely used one called ReLU turns any negative total into zero and leaves a positive total as it is, so its line runs flat and then rises, with a kink at zero where it changes direction. Many kinks joined together can follow a curve as closely as needed, which is how a network comes to represent complicated relationships.

A language model, the kind behind a chatbot, is trained to predict the next token of a text, a token being a word or a piece of a word, and it gives every possible token a probability. The loss is a single number for how wrong each prediction was, and the usual loss punishes a confident mistake, meaning a very low probability for the right token, far more heavily than a hesitant one.

Picture a valley in which height stands for the loss and each place along the ground stands for a value of one weight, one of the numbers training adjusts. The slope underfoot shows which way is downhill, meaning which way to move the weight to lower the loss. Gradient descent moves each weight a small step downhill using a small batch of examples, then measures the slope again on the next batch and steps again, while the code stays the same throughout.

The learning rate sets the size of each step, which decides how training behaves. Small steps get to the bottom slowly and surely, while large steps jump past it, bounce from one side of the valley to the other and can even climb out of it, at which point the loss grows with every step. Real models therefore take tiny steps, repeated many thousands of times.

Training has to know, for every weight at every step, which way to move it to lower the error, and a large model has billions of weights, the numbers training adjusts. The answer for each weight is called its slope, meaning how much the error changes when that weight is nudged. Backpropagation works out all of those slopes in one backward sweep, starting from the error at the output and working back through the model’s layers, its stages of calculation.

Tracing a mistake in a report back through the people who worked on it is a fair comparison, since the final error is passed back person by person and each contributor’s share depends on how much their work shaped everything after it.

Backpropagation therefore computes the slopes, and gradient descent, the method that moves each weight a small step in the direction that lowers the error, uses them.

The aim of training is generalisation, meaning good performance on inputs the model has not seen. A student who memorises last year’s exam answers can ace a repeat of that exam and come unstuck on a new one. The same failing in a model is called overfitting, in which it fits the quirks of its training examples so closely that it does worse on new data.

Developers therefore keep separate piles of data, in which the training set adjusts the numbers inside the model, the validation set helps to tune settings such as the size of each training step, and the test set stays locked away for an honest final score.

A benchmark, a standard test published so that models can be compared, works as a test set only while its questions stay out of training, and contamination means that they have leaked in. Questions posted on the web can end up in the text collected for training, so the model may have seen the answers and a high score may reflect recall. A lawyer reviewing a supplier’s accuracy claims can therefore fairly ask how its test data was kept apart from its training data.

Training sets a model’s weights, the numbers inside it, once and at great cost, while inference, the phase in which the finished model answers queries, uses those weights unchanged. A chat with a deployed model therefore leaves its weights exactly as they were.

A chatbot appears to remember what was said three messages earlier only because the whole conversation is sent back in as input with each new message. Some products add a memory feature, which keeps details from past conversations outside the weights and feeds them in as input to later chats.

A provider that wants its model to learn from conversations has to collect them and use them in a later training run. Whether chats feed training therefore depends on what the provider logs, how long it keeps the logs and the terms the user accepted. Data protection law, the GDPR in the European Union and the UK GDPR in the United Kingdom, governs those logs where they hold personal data.

Training compute, the computing work spent on training a model, is counted in floating-point operations, or FLOPs. Article 3(67) of the EU AI Act defines one as “any mathematical operation or assignment involving floating-point numbers”, the form in which computers hold numbers such as 0.5. An assignment, in computing, is the storing of a value in the computer’s memory, so a single multiplication, addition or storing of such a number counts as one. The count grows with the number of weights, the numbers training adjusts, and with the amount of training text, since every weight takes part in the work for every token, or piece of text, the model reads.

Recital 111 of the EU AI Act calls the cumulative total “one of the relevant approximations for model capabilities”, and that total covers the work that enhances a model before deployment. The recital’s examples are pre-training, which is the main training on a library of text, synthetic data generation, which makes extra training text by machine, and fine-tuning, which is further training on a smaller set of data. The AI Act counts that total for general-purpose models, those that display significant generality, can competently perform a wide range of distinct tasks and can be built into many other systems, other than models used for research, development or prototyping before they are first made available on the EU market. Article 51 presumes that such a model has high-impact capabilities once the compute used to train it exceeds 10²⁵ FLOPs. Article 3(64) defines those as capabilities that match or exceed the capabilities recorded in the most advanced general-purpose AI models. Recital 112 notes that providers, the AI Act’s term for the persons and bodies that develop a model, or have one developed, and first make it available on the EU market under their own name or trademark, can know whether a model would meet that threshold before training is completed. The scaling laws, researchers’ finding that a model’s error tends to fall in a predictable way as its training compute grows, suggest why that total tracks performance.

High-impact capabilities are one of the two grounds on which Article 51 classes a model as having systemic risk. Article 3(65) defines that as a risk specific to those capabilities that has a significant impact on the Union market and can spread at scale across the value chain, meaning the businesses that build on such models. The impact has to come from how widely the models are available, or from actual or reasonably foreseeable harm to “public health, safety, public security, fundamental rights, or the society as a whole”. The Commission is required to amend the figure of 10²⁵ by delegated act when that is needed to reflect the state of the art. The text consolidated on 27 July 2026, after the Digital Omnibus on AI, the 2026 regulation that amended the AI Act, still holds that figure.

A figure a hundred times lower marks a different line, since the Commission’s guidelines on general-purpose AI models give, as an indicative criterion for whether a model is general-purpose at all, training compute above 10²³ FLOPs together with the ability to generate language, or to turn text into images or video. The guidelines add that a model meeting the criterion may exceptionally fall outside the category, and that one falling short of it may still be general-purpose. The guidelines also say that they are not binding for providers of general-purpose AI models, although the Commission will base its enforcement action on them. The higher figure is the point at which Article 51 presumes high-impact capabilities.

As a yardstick the count is indirect, and the Commission itself calls training compute “an imperfect proxy for generality and capabilities”. A model that sends each token through only part of its weights does only part of the arithmetic its size suggests, so an estimate from size alone can overshoot. Reasoning models, which write out their working, spend extra computing while answering, which a training count leaves out. Researchers also dispute whether new abilities appear suddenly or smoothly as scale grows, which bears on how closely any threshold tracks the capabilities it is meant to capture.

A model works only with numbers, so software called a tokeniser cuts text into tokens, short pieces such as whole words or parts of words, and replaces each with its number in a fixed list called the vocabulary. OpenAI’s rule of thumb for English is that a token is about four characters, or three quarters of a word.

Byte-pair encoding, the dominant way of building that list, works by counting. The method finds the pair of neighbouring pieces that occurs most often in a large body of text, merges the pair into one longer piece and repeats until the list is the size its designers chose. Frequent words therefore end up as single tokens and rare words stay in pieces, so a common word such as “the” costs one token where a rare surname or technical term may cost several.

Tokens explain some odd failures, since a model receives each token only as a number and the letters inside it are not shown to the model. Counting the letters in a word is a familiar weakness, usually explained that way, although models can still learn spellings from their training text. A preprint, a paper posted before peer review, found that mistakes were more frequent in words where a letter appears more than once, so difficulty with counting repeated items appears to play a part as well.

A tokeniser is the software that cuts text into tokens, short pieces such as words or parts of words, and it builds its list of pieces by counting which ones occur most often in a large body of text. A body of text that is mostly English therefore leaves common English words as whole tokens and other languages in more and smaller pieces.

The same meaning can therefore take more tokens in another language, and one study found that the same text, translated, can take up to 15 times as many tokens in one language as in another. Per-token pricing then charges more for the same meaning, and the context window, the limit on how much text a model can take in at once, holds less of a long text since it is counted in tokens.

A pricing or capacity term written in tokens is therefore easier to assess once the client knows the token-to-word ratio for each language it works in. Running a sample of the client’s own text through the tokeniser of the model in question will show that ratio.

After a tokeniser has cut text into tokens, short pieces such as words or parts of words, each token is swapped for a long list of numbers called its embedding. The list works like a map reference, placing the token at a point on a map with thousands of directions where a paper map has two.

Training moves tokens that appear in similar contexts closer together, so “court” and “tribunal”, which turn up beside many of the same words, end up near each other and far from “banana”. Closeness therefore stands for similar use, which tends to go with similar meaning. The same process absorbs the biases of human text, since words that people write in similar settings end up close together whether or not the association is fair, which is one route by which a screening or scoring system can disadvantage protected groups.

Older systems gave each word one fixed point, so “bank” sat in the same place whether the text was about rivers or loans. A transformer, the design behind modern language models, starts from a fixed point for each token and moves it at each processing stage using the words that come before it, so the two senses of “bank” can end up in different places.

Steps across the map can also stand for relationships between words, and a famous demonstration shows both the idea and its limits. Researchers took the step from “man” to “woman”, added it to “king” and searched for the nearest word, which was “queen”, so that one step seemed to stand for the same relationship in both pairs. The search leaves out the three words the puzzle started from, however, and a 2020 study found that with those words allowed, the share of correct answers on the standard test built from such puzzles fell from 0.71 to 0.21. A striking demonstration can therefore overstate how well a technique works.

Attention is the step in which each token, a word or a piece of a word, gathers information from itself and the tokens before it, which lets a model work out how a word depends on something said earlier. Every token works out three lists of numbers from the list that stands for it: a query describing what it is looking for, a key advertising what it offers and a value holding the information it hands over.

Take “it” in “The tribunal dismissed the appeal because it was filed out of time”. The query of “it” is compared with its own key and each earlier key, and in this illustration the match with “appeal” scores highest. The softmax, the step that turns scores into shares, then gives “appeal” the largest share, so “it” takes in mostly the value of “appeal”.

A transformer, the design behind modern chatbots, works on each token, a word or a piece of a word, in alternating stages. In attention each token draws on the words before it, and in the feed-forward layer, a small network of its own, each token is then worked on separately. The feed-forward layers hold roughly two thirds of the weights, the numbers training adjusts, in a standard transformer, and research suggests that much of what a model knows about the world, such as which city a landmark is in, is stored there.

Researchers showed in 2022, with a technique called ROME, that one fact could be changed by adjusting the weights of a single feed-forward layer. In one example they edited a layer where a model appeared to hold the link between the Eiffel Tower and Paris, so that the model placed the tower in Rome.

Knowledge appears to be spread across many weights that also serve other facts, however, and later work found that the layer where a fact appeared to sit was often not the best layer to edit. An edit made to change one association can disturb others, so a fact in a model is unlike a record in a database, which can be found and deleted.

A business asked to correct or erase personal data that its model has learned may therefore be unable to point to where the data sit or to show that an edit removed them. One practical response is a filter on the model’s answers, which is meant to stop them showing the data while the weights stay as they were. Whether a filter meets a request for correction or erasure under Articles 16 and 17 of the GDPR, or Article 16 and Article 17 of the UK GDPR, where the data may remain encoded in the weights, is an open question. The UK’s data protection regulator, known as the ICO and since 30 September 2026 constituted as the Information Commission, has said that output filters “may not be sufficient, as they do not actually remove the data from the model”.

A chat model writes its answer in a loop, one token at a time, a token being a word or a piece of a word, and each token it writes is fed back in as input.

1. The model reads the text so far, running it through its layers, the stages of calculation stacked inside it.
2. At the end of the text, the model gives a score to every token on its fixed list of known tokens, and a formula called the softmax turns the scores into probabilities.
3. One token is picked and added to the end of the text.
4. The model runs again on the longer text, and the loop repeats until the answer ends.

A 100-token answer therefore takes about 100 runs, and each run compares the new token with the earlier ones using lists of numbers called keys and values. The software saves those lists in a store called the KV cache, so that it need not work them out again, and the cache grows with every token. A long conversation therefore costs more memory as well as more work for each token written.

A model works out how the words of a text bear on one another by comparing every token, a word or a piece of a word, with itself and every earlier token, a step called attention. The comparisons therefore grow much faster than the text, since doubling the length of a prompt gives twice as many tokens, each compared with about twice as many others, which roughly quadruples the work.

Long inputs also strain reliability, since a 2023 study found that the models it tested used information best at the beginning or end of a long input and worse in the middle.

A review protocol might therefore check clauses in the middle of a long agreement separately, or have the system retrieve the agreement in smaller pieces, looking up the passages relevant to each question and giving the model only those. Later models may do better than the ones that study tested, so the weakness is something to test in the tool a firm uses.

Pretraining is the first stage of training, in which a model reads a library of trillions of tokens, the words and pieces of words it works with, and at every position guesses which token comes next. After each batch of passages, its weights, the numbers it adjusts as it learns, shift so that its guesses come closer to the text that really followed.

The result is a base model, one that has only been pretrained, and it continues documents. Given a question, the base model writes whatever usually follows a question on the web, which might be an answer, more questions or a forum thread. The model learns the truth only where the true statement is also the one people most often write, which is one root of hallucination, a fluent and confident statement that is false.

What goes into the library shapes what comes out, so developers filter web text heavily and remove duplicates. The contents of the library also raise questions of copyright, since a library of text can include protected books. One example is The Pile, a collection of text for training language models that researchers described in a paper at the end of 2020. Books3, one of its components, was a set of books that the Pile’s creators describe as derived from “a copy of the contents of the Bibliotik private tracker”, and a private tracker, in ordinary usage, is a file-sharing site open only to its members.

A scaling law is a regularity developers have observed, in which a model’s error at predicting the next word falls by an amount that can be predicted in advance as the model grows, reads more text and is trained with more computing power. Researchers at OpenAI reported in 2020 that each tenfold increase in computing power cuts the error by about the same fraction, which lets developers predict the results of a large training run from smaller ones.

DeepMind tested in 2022 how to split a fixed computing budget between model size and training text, and concluded that the two should grow in step. Developers call a split compute-optimal when it gives the lowest error that a fixed budget can buy, and by that yardstick the largest models of the day had read too little for their size.

Developers now often train smaller models on far more text than that split would give them, since a smaller model costs less to run each time someone uses it. Meta reported that it trained its smaller Llama 3 models “for much longer than is compute-optimal” and that they performed better than compute-optimal models at the same cost of running them. A model’s size is therefore a weak guide to the computing power spent on training it.

A model that has only learned to continue text is called a base model, and pretraining is the stage that produces it. Post-training is the further training that turns a base model into an assistant that follows instructions, usually in stages.

1. Supervised fine-tuning shows the model example prompts paired with good answers, which resembles studying model answers before an exam.
2. Reinforcement learning from human feedback trains by trial and reward. People rank alternative answers, a separate reward model learns to predict their rankings, and the assistant is then nudged towards answers the reward model scores highly.

The reward model is only a stand-in for people, so pushing hard on its score invites reward hacking, meaning answers that score well without being better, and sycophancy, the tendency to tell users what they want to hear.

Later methods change who judges the answers or how the model is changed. Constitutional AI has an AI model judge which answers are more harmful, using a written list of principles, in place of rankings by people. Direct preference optimisation drops the reward model and adjusts the assistant directly to prefer the better answer in each pair.

A reasoning model writes out its working before answering, much as a student uses scrap paper. A model produces each token, a word or a piece of a word, by running one pass through its stages of calculation, so each extra token of working adds a round of calculation and leaves notes that later tokens can draw on.

Training can then reward working that leads to right answers, as DeepSeek’s R1 showed with automatic checkers. The checkers, such as known answers to mathematics questions and tests for code, marked each final answer right or wrong, and the weights, the numbers training adjusts, shifted towards the attempts marked right.

DeepSeek chose checkers because it had found that reward models, separate models that learn from data to score answers, are open to reward hacking, meaning answers that score well without being better. Each checker looks only at whether the final answer is right or the code passes its tests. Models still find shortcuts, such as code that returns the values a test expects without solving the general problem, so a lawyer who relies on a model’s report that its work passes a check may want to see how it passed.

The written working is an imperfect record of what drove the answer. Researchers at Anthropic slipped reasoning models a hint in 2025, and in the cases where a model switched to the answer the hint pointed to, Claude 3.7 Sonnet mentioned the hint in its working 25 per cent of the time on average and DeepSeek R1 39 per cent. The researchers call their test settings somewhat contrived, yet the result suggests that the working can read as a plausible statement of reasons while staying silent about what decided the outcome.

A mixture-of-experts model splits part of each of its layers, the stages of calculation stacked inside it, into many smaller blocks called experts, and a small router picks a few of them for each token, a word or a piece of a word, at each layer. Only part of the model therefore works on any one token, which saves computing work.

The experts need not be specialists in subjects such as law or medicine, despite the name, since the developers of Mixtral, an openly released model, reported that its router did not pick experts by topic and followed the form of the text, such as the layout of computer code.

A model’s parameters are the numbers training adjusts, and total parameters measure everything it stores, while active parameters measure what each token uses. DeepSeek-V3 has 671 billion in total and 37 billion active, so the cost of computing each token tracks the active count, while the memory needed tracks the total, since every expert has to be held in memory. The EU AI Act lists “the number of parameters of the model” in Annex XIII, among the criteria the Commission takes into account when it designates a general-purpose AI model as having systemic risk. That passage, read in the text consolidated on 27 July 2026, does not say whether a mixture-of-experts model’s parameters are counted in total or as active.

Distillation is another way of cutting running costs, in which a small student model is trained to imitate a large teacher. Before writing each token, a model works out a probability for every token it might write next. A developer distilling its own model can therefore train the student on those full probabilities, which show more than the single token the teacher chose. A competitor working through a public interface usually sees only the text answers, without those probabilities.

A token is a word or a piece of a word, and at each step the model gives every token it knows a score. A formula called the softmax turns the scores into percentages that add up to 100. The software around the model then picks a token, either the most likely one each time, called greedy decoding, or at random in proportion to the percentages, called sampling.

Temperature is a setting that divides every score before the softmax. A temperature below 1 widens the gaps between the scores, so the favourite wins more often; a temperature above 1 shrinks the gaps, so less likely tokens come up more often.

Top-p sampling trims the list before the draw, ranking the tokens from most to least likely and keeping them until their running total first equals or passes a chosen share, such as 90 per cent. The token that takes the total past that share stays in and every token after it is left out of the draw, so an odd, unlikely word has a smaller chance of slipping into the answer.

Setting the temperature to zero tells the software to take the most likely token, the likeliest word or piece of a word, at every step, so the same prompt ought in theory to give the same answer every time. Researchers showed in 2025 that answers can still vary because of the way computers do arithmetic.

Working out each token’s score means adding up a great many numbers. Computers store numbers with a limited number of digits and round after every addition, so adding the same numbers in a different order can give a slightly different score.

The chips that run a model add numbers in a different order depending on how many requests they handle together, which depends on how busy the server is. A tiny shift in a score can tip the choice between two close tokens, and one changed word can send the rest of the answer down a different path. In one test the researchers sent the same prompt 1,000 times at a temperature of zero and received 80 different versions of the answer.

A single transcript may therefore be impossible to reproduce exactly, even by the person who produced it. A party relying on an AI output in a dispute may need the model version, the settings that control how each token is picked, the logs and the system prompt, the standing instructions given to the model. That record can show what was produced and may help to explain why, and some of it may be held only by the provider.

A hallucination is a fluent, confident statement by a model that is false, and for lawyers the familiar example is an invented case citation. Each possible next token, a word or a piece of a word, gets a probability from the model, which measures how likely the token is to come next and is a different thing from whether a statement is true. An invented citation can therefore look likely word by word, since it follows the usual pattern of citations closely, with a court, a year and a page number in the expected places.

A paper on why language models hallucinate, published in 2025 by researchers at OpenAI and Georgia Tech, points to arbitrary facts, meaning facts a model could not work out from other facts, and to scoring. A private person’s birthday is the paper’s example: mentioned only once in the training data, it gives the model too little to learn from, so the model tends to produce a date that merely looks right. The paper therefore predicts that if 20 per cent of birthday facts appear exactly once in the training data, a base model, one that has not been taught to decline a question, will give a false answer to at least 20 per cent of birthday questions.

Scoring also rewards guessing, since most benchmarks, the standard tests used to compare models, mark each answer right or wrong and give zero credit for an admission of uncertainty, as on a multiple-choice exam without negative marking. On the paper’s account, developers tune their models to do well on benchmarks, so the habit of guessing is learned in training as well as rewarded in testing. Deducting a mark for each wrong answer can reverse the ranking between a model that guesses and one that declines, and the authors of the paper propose changing how evaluations are scored.

Memorisation means that a model can reproduce a passage from its training data word for word or nearly so, or an image exactly or nearly so, when prompted, and researchers commonly test text in stretches of 50 tokens, the words and pieces of words a model reads.

Discoverable memorisation is the easier test, and its label is unrelated to disclosure in litigation. The researcher already holds the training passage, gives the model its true opening and checks whether the model completes the rest.

Extractable memorisation is the stronger finding, made when an outsider without the training data gets the model to produce the passage, as in the 2023 attack that had ChatGPT repeat the word “poem” until it broke out of its chat behaviour and emitted memorised text. The difference resembles reading a witness the first line of a letter and asking for the next one, compared with getting the witness to recite the letter for a questioner who has not seen it.

Memorisation concentrates in long, distinctive, much-copied text such as famous novels. A failed attempt to draw a passage out proves little, since extraction attacks give only a lower bound on how much a model has memorised, a minimum that a better prompt might exceed.

Lawyers meet memorisation in copyright claims that a model contains the works it learned from, as in a Munich case over song lyrics, and courts in London and Munich came to different results at first instance, on different claims and different evidence. In Getty Images v Stability AI, a claim over Stable Diffusion, a model that makes images from text prompts, the photo agency Getty Images did not say that the model was itself a copy of its works or that it stored any copies of them. Getty’s copyright case by the end of the trial was one of secondary infringement, the wrong of importing or dealing in an “infringing copy”, knowing or having reason to believe that it is one. Getty relied on section 27(3) of the Copyright, Designs and Patents Act 1988, under which an imported article counts as an infringing copy if making it in the United Kingdom would have infringed copyright, and argued that making the model’s weights would have done so. The judge said that the evidence, on the case as argued, did not show any of those works to have been memorised by the model through over-exposure to them, meaning repeated exposure to the same work in training. The High Court held that the model was not an infringing copy, since in the judge’s words “in its final iteration Stable Diffusion does not store or reproduce any Copyright Works and nor has it ever done so”. The judge said that the reproduction of the works in its development was irrelevant to whether it was an infringing copy. The High Court has since given permission to appeal in Getty. GEMA, a collecting society that manages rights in musical works on behalf of composers, lyricists and music publishers, brought the Munich case over song lyrics against OpenAI. In GEMA v OpenAI the Munich Regional Court held that memorisation embodied the song lyrics at issue in the models’ parameters, and that this embodiment was a reproduction under German and EU copyright law. The Munich court’s press release of 11 November 2025 states that its judgment in GEMA is not final. Bayern.Recht, the Free State of Bavaria’s legal portal, lists an appeal to the Munich Higher Regional Court under reference 6 U 3662/25 e, and on 30 October 2026 it did not show a decision in that appeal.

Data protection raises the same concern about what a trained model still holds. The European Data Protection Board, the body of the EU’s data protection authorities, says in Opinion 28/2024 that information from the training data, including personal data, may remain “absorbed” in a model’s parameters, the numbers training adjusts. The Board adds that the information is represented through mathematical objects, which may differ from the original data while still retaining the original information. The Board took the view in the same Opinion that a model trained on personal data can be treated as anonymous only where the likelihood of extracting personal data from it directly and the likelihood of obtaining such data through queries are both insignificant. Both likelihoods are judged by the means reasonably likely to be used by the controller, the organisation that decides why and how the data are used, or by another person.

A model on its own is a large file of numbers, called weights, and the file sits idle until software loads it and runs it on some input. Everything a user experiences therefore comes from a larger system that a business builds around the model, usually with a system prompt, the standing instructions placed ahead of every conversation, along with tools, a search over documents, filters, logs, a memory and the screen on which the user types. The same model can behave quite differently in two products, and a product can change its behaviour overnight through an edit to its system prompt, with the model itself untouched.

Control over that system is usually divided between several businesses and the person using the product:

1. The developer trains the model, choosing the data and the methods, and where it offers the model through its own service it can update or retire it.
2. The deployer, the industry’s name for the business that builds a product on the model, writes the system prompt, decides which tools and data the model can use, adds its own safeguards and writes the customer terms.
3. The user controls what is typed in and any files added to the conversation.

Open weights change that division, since a business that downloads a model’s published weights and runs them on its own computers holds a copy that the developer has lost any technical means to monitor, patch or withdraw. That loss of control may limit what a court or a public authority can in practice order.

The EU AI Act draws a similar line in Article 3, in words of its own. Under that Article a provider is a person or body that develops an AI system or a general-purpose AI model, or has one developed, and places it on the market, meaning first makes it available on the EU market, or puts the system into service under its own name or trademark. The AI Act calls a model general-purpose where, in outline, it displays significant generality, can competently perform a wide range of distinct tasks and can be built into many other systems, unless it is used for research, development or prototyping before it is placed on the market. The same Article defines a deployer as a person or body using an AI system under its authority, except in the course of a personal non-professional activity. Putting a system into service means supplying it for first use directly to the deployer, or using it oneself, in the EU and for its intended purpose. The AI Act’s labels do not line up exactly with the industry’s, since the business the industry calls the deployer, which builds a product on another company’s model and offers it under its own name, may itself be the provider of that product in the AI Act’s terms. In those terms, a firm that then uses the product under its authority is the product’s deployer.

A model’s knowledge is fixed when its training ends and appears to be spread across many of its weights without any record of which document taught which fact, so the model lacks a reliable way to say where a fact came from. Retrieval-augmented generation, usually shortened to RAG, adds a search step before the model answers. The software searches a collection of documents for passages relevant to the user’s question, pastes the best matches into the model’s input and asks the model to answer from those passages and to cite them.

The difference resembles a closed-book exam and an open-book one: the open book helps, yet a student can still misread the page or copy from the wrong one, and a model given retrieved passages can do the same.

The search usually works by meaning, using embeddings, which turn each passage into a list of numbers that works like a set of map coordinates, with similar meanings placed close together. Documents are first cut into chunks of a few paragraphs, and each chunk is searched and retrieved on its own. Chunking deserves care with legal material, since a split can separate a clause from its proviso or a holding from the facts that limit it.

Retrieval reduces a model’s tendency to make things up without eliminating it, as a study by researchers at Stanford found in 2024 when it tested legal research tools built on retrieval. The authors found that the tools from the vendors they tested “each hallucinate between 17% and 33% of the time”, counting an answer as a hallucination where it was wrong or cited a source that did not support it. The vendors have updated their products since the tests, so the figures describe the versions available in 2024.

A model only produces text, so on its own it is unable to look up a record or send an email. Tool use works by letting the model write a request in a fixed format, such as a request to look up an account balance, which the software built around the model then decides whether to run before feeding the result back to the model as more text. What the model can do is therefore set in the software around it, which runs or refuses each request.

Anthropic, the company that makes the Claude models, describes an agent as a model using tools in a loop. The model plans a step and asks for a tool, the software runs the tool and returns the result, and the model reads the result and decides on the next step, until the task is done or a stopping rule applies, such as a maximum number of steps. A question about what an agent was technically able to do therefore starts with that software, its tool settings and its logs. Whether the agent had authority to do it may also depend on the user’s instructions, any contract between the parties and the law of agency so far as it applies to software.

Those tools often sit in other software, so agents need a common way to connect to them. The Model Context Protocol is one such standard, which lets a firm write one connection to its document system that every AI application supporting the standard can then use.

An agent that can act for a user can also be asked to pay for something, and payment protocols, shared sets of technical rules for how an agent and a seller exchange a payment, let it do so without a person at each step. Web servers, the computers that answer requests for pages and files, reply with numbered status codes, such as 404 for a missing page. The web’s standards name code 402 “Payment Required” but have left it unused, and the protocol called x402, which started at Coinbase, the cryptocurrency exchange, puts that code to work:

1. The agent asks a server for a paid resource, such as a dataset.
2. The server replies with code 402 and its payment terms.
3. The agent sends a payment with its next request, signed with the secret key that controls the account it pays from, which proves that the payment comes from that account.
4. A facilitator, a service the server can call on, checks the payment and records it on a blockchain, a shared public ledger of transactions, after which the server releases the resource.

The x402 Foundation’s site names stablecoins, cryptocurrencies designed to hold a steady value against an ordinary currency, as the protocol’s main use.

The Agent Payments Protocol, known as AP2, deals with the proof of what a user authorised. AP2 records that authority in mandates, which are digital records signed with cryptography so that a later change to them would show. In the version Google announced in September 2025, the Intent Mandate recorded the user’s instruction, such as a request to find white running shoes, and the Cart Mandate recorded the exact items and price that the user approved. Google has since donated AP2 to the FIDO Alliance, an industry standards body. Version 0.2 uses Checkout and Payment Mandates, each of which is open while it only sets limits and closed once it is bound to a particular purchase. A user who is present signs the closed mandate for a particular purchase. For a user who is away the agent signs the closed mandate with its own key. The specification says that trust in that key rests on “open Mandates that are signed by the User or a trust list of Agent Providers”, a trust list being a list of agent providers that the party checking the mandate trusts.

Payment law in the EU and the UK shows why that record may be useful. For payments by card or from a payment account in the EU, Article 64 of the Payment Services Directive treats a payment as authorised only if the payer has given consent in the form agreed with its payment service provider. Under Article 72 of the Payment Services Directive, when a user denies having authorised a payment, the payment service provider bears the burden of proving that the payment was authenticated and accurately recorded. The same Article provides that the recorded use of a payment instrument, such as a card, is not necessarily enough on its own to prove that the payer authorised the payment. The UK’s Payment Services Regulations 2017 say the same in regulations 67 and 75. A user that is not a consumer may agree with its provider that Article 72 does not apply in whole or in part, and regulation 63 of the UK’s regulations lets the parties disapply regulation 75 unless the user is a consumer, a micro-enterprise or a charity. Whether an instruction given through an agent counts as consent in the agreed form may depend on the terms the user agreed with its provider, and AP2’s signed mandates are an attempt to build the evidential record that a dispute over authority would need.

Prompt injection is text planted where a model will read it, which the model then treats as an instruction against the wishes of the business that deployed it and often against the interests of the person using it. In its direct form the user types the attack; in its indirect form the attack arrives hidden inside content that the system reads on the user’s behalf, such as a web page, an email or a calendar invitation.

The difficulty in fixing prompt injection is that a model receives its instructions and the data it works on as a single stream of text. A database avoids the older attack called SQL injection, in which text typed into a web form is run as a command, by passing whatever a user types in a separate slot. A model has only the markers in its chat format, which show which part of the text came from whom. The model respects those markers only because training taught it to. The software around the model does not enforce them, so an instruction inside fetched content can still be obeyed.

Simon Willison’s “lethal trifecta” describes an agent that has access to private data, is exposed to untrusted content and has a way to send information out. An attacker who plants text in the agent’s path can then order it to gather the private data and send it away, and taking away any one of the three parts blocks that route to the data. Defences therefore lie mostly in how the system is built, keeping any model that has read untrusted content away from consequential actions, for instance by requiring the user’s approval before an email leaves.

The UK’s National Cyber Security Centre wrote in December 2025 that, since models do not separate data from instructions, it is “very possible” that prompt injection attacks will not ever be totally mitigated in the way SQL injection attacks can be. A weakness that is known and unsolved in that way may shape what is expected of a business that sets up an agent for use, both under the security duty in data protection law and in negligence. Article 32 of the GDPR and Article 32 of the UK GDPR bind controllers, who decide how personal data are used, and processors, who handle the data for them. Each has to implement appropriate technical and organisational measures to ensure a level of security appropriate to the risk. In choosing those measures, each has to take into account the state of the art, the costs of implementation, the nature, scope, context and purposes of the processing and the risks to people’s rights and freedoms. A controller or processor that gives an agent handling personal data all three parts of the lethal trifecta with weak controls may therefore find its measures hard to defend under Article 32.

Fine-tuning means taking a model that another developer has already trained and training it further on a smaller collection of examples chosen for one task, such as a firm’s precedents. The training moves the model’s existing weights, the numbers it learned, towards the new task.

LoRA is one cheaper way to fine-tune, which leaves the original weights untouched and trains a much smaller set of extra numbers that are added to the originals when the model runs, much like clipping an extra lens onto a camera whose own lens stays as it was. For GPT-3, a model with 175 billion weights, the LoRA paper reports that the method trained 10,000 times fewer numbers than retraining every weight. QLoRA, a later method, fine-tuned a model with 65 billion weights on a single graphics chip, the kind of chip on which models are trained and run.

That ease lets whoever holds a model’s weights train away the safety behaviour its developer added, such as the model’s habit of refusing harmful requests. OpenAI therefore tested the risk itself before releasing its open-weight gpt-oss models, in what it called malicious fine-tuning, retraining the models to be as capable as possible in biology and cybersecurity. OpenAI chose those two fields because misuse there could cause severe harm, such as help in creating biological threats or in breaking into computer systems.

Fine-tuning can also change a client’s legal role under the EU AI Act, which sets duties for the provider of a general-purpose AI model. That term means, in outline, a model that displays significant generality, can competently perform a wide range of distinct tasks and can be built into many other systems, other than models used for research, development or prototyping before they are placed on the market, meaning first made available on the EU market. The provider is the person or body that develops such a model, or has it developed, and places it on the market under its own name or trademark, whether for payment or free of charge. The AI Act also uses the term systemic risk for a risk specific to capabilities that match or exceed those of the most advanced models. The impact of that risk on the EU market has to be significant, through how widely the model is made available or through actual or reasonably foreseeable negative effects on, for instance, public health or public security, and the risk can spread at scale. In its guidelines on general-purpose AI models, the Commission considers that a downstream modifier, meaning an actor that changes the model and is distinct from the original provider and not acting on its behalf, becomes the provider of the modified model only if the change significantly alters the model’s generality, capabilities or systemic risk. The Commission’s indicative criterion for when a modifier becomes the provider is that the modification uses more than a third of the computation used to train the original. Recital 109 says that the modifier’s duties should be limited to the modification, although the guidelines add that a modifier that becomes the provider after changing a model with systemic risk takes on the duties for such models. The guidelines say that they are not binding for providers of general-purpose AI models and that only the Court of Justice can give an authoritative interpretation of the AI Act, although they add that the Commission will base its enforcement action on them.

A closed model is offered only as a service, with the weights staying on the developer’s computers, while the developer of an open-weight model publishes the weights so that a user can download them. Publishing the weights is a separate step from publishing the training data or the code used for training, and a developer can take one step without the others.

Licences for open weights vary widely, and Meta’s Llama 4 Community License shows how much one of them can say. Section 1 grants a royalty-free licence to use, copy, modify and distribute the model. Section 2 withholds the section 1 rights from a licensee whose products or services, together with those of its affiliates, had more than 700 million monthly active users in the calendar month before Llama 4 was released. A business above that line therefore has to ask Meta for a licence, which Meta may grant or refuse as it chooses. Meta’s acceptable use policy, which the licence incorporates, also withholds the section 1 rights for the multimodal Llama 4 models, those that work with images as well as text, from individuals domiciled in the EU and companies with their principal place of business there. End users of a product or service built on those models are not affected. Other developers have chosen the permissive Apache License 2.0, whose conditions deal mainly with passing on a copy of the licence, marking changed files and keeping the original notices.

Whether a licence with use restrictions or user thresholds counts as open source is disputed. The Open Source Initiative said in July 2023 that the licence for Llama 2, an earlier Meta model, is not open source, because the Initiative’s definition forbids discrimination against persons, groups or fields of endeavour. The Initiative’s Open Source AI Definition asks a developer that calls its AI system open source to make available the code used to train and run it and the model’s parameters, such as its weights. The definition also asks for information about the training data detailed enough for a skilled person to build a substantially equivalent system, so it does not require all of the raw training data to be published, although the Initiative’s FAQ says that training data open to resharing has to be shared.

The open-source label also has consequences under the EU AI Act, which sets duties for the provider of a general-purpose AI model. That term means, in outline, a model that displays significant generality, can competently perform a wide range of distinct tasks and can be built into many other systems, other than models used for research, development or prototyping before they are placed on the market, meaning first made available on the EU market. The provider is the person or body that develops such a model, or has it developed, and places it on the market under its own name or trademark, whether for payment or free of charge. Article 53(1) requires that provider to keep technical documentation for national competent authorities and for the AI Office, the part of the European Commission that supervises these models. The provider also has to give documentation to the businesses that build the model into their own systems. The same paragraph requires the provider to adopt a policy to comply with EU copyright law and to publish a summary of the content used to train the model. Article 53 has applied since 2 August 2025 under Article 113. Article 111(3) gives older models more time: a provider whose model was on the market before that date has until 2 August 2027 to comply.

Article 53(2) lifts the two documentation duties for providers of models released under a free and open-source licence that allows access, use, modification and distribution, with the weights and the information on the model’s design and use made public. The exception does not apply to a model with systemic risk, the AI Act’s category for models with high-impact capabilities and for models that the Commission decides have equivalent capabilities or an equivalent impact. High-impact capabilities are capabilities that match or exceed those recorded in the most advanced general-purpose models. Even where the exception applies, the copyright policy and the training summary remain due, as recital 104 explains. Recital 103 adds that a model provided against a price or otherwise monetised should not benefit from the exception, other than in transactions between microenterprises, meaning very small businesses. The Commission’s guidelines apply that bar on monetisation as a condition of the exception, with the same carve-out.

In its guidelines on general-purpose AI models, the Commission lists terms that would disqualify a licence, among them “usage restrictions triggered by user scale thresholds”, such as a demand for further licensing once monthly active users pass a set number. On the Commission’s reading, a licence with a user threshold such as Llama 4’s would therefore appear to fall outside the exception. The guidelines say that they are not binding for providers of general-purpose AI models and that only the Court of Justice can give an authoritative interpretation of the AI Act, although they add that the Commission will base its enforcement action on them.

Most image generators are diffusion models, which learn by damage and repair. In training, the model is shown real images with increasing amounts of random speckle, called noise, added to them, and it learns to predict the noise so that it can take it away again.

Generation runs the film backwards, starting from pure noise and removing a little at each step, steered by the words of the prompt, until a picture emerges that matches the description. Stable Diffusion, an image generator whose weights were published for download, runs those steps on a compressed version of each image, which cuts the computing cost.

Stable Diffusion’s first versions were trained, according to its model card, the description its developers published with the model, on subsets of LAION-5B, a collection of 5.85 billion pairs of images and captions gathered from the web. LAION, the German association behind the collection, distributes it as a table of links to images hosted elsewhere, though it downloaded images to check them against their captions while building the table. One photographer sued over such a download, and in Kneschke v LAION the Hanseatic Higher Regional Court in Hamburg held in December 2025 that the German exceptions for text and data mining, the automated analysis of works to find patterns, permitted it. The photographer has brought a further appeal, and the Federal Court of Justice’s notice, as it stood on 30 October 2026, shows a hearing on 3 September 2026 and a decision listed for 17 December 2026.

Two families of technique aim to let people tell content made by a machine from content made by a person. Provenance records describe where a file came from, and Content Credentials, a standard published by the Coalition for Content Provenance and Authenticity, known as C2PA, are one example. Content Credentials attach to a file a digitally signed record of its origin and its edits, including any use of AI, stored as metadata, the information about a file kept alongside its contents. Any later change to the file or the record shows up as a broken signature, yet the C2PA’s explainer accepts that the record can be removed from the file. The explainer also says that Content Credentials do not judge whether the provenance information is true.

A watermark hides a signal in the content itself, so the signal stays in the content when metadata is stripped away. Google DeepMind’s SynthID-Text, described in a paper in Nature, nudges a language model’s choice of words into a statistical pattern that a detector can recognise. The paper on SynthID-Text accepts that edits such as paraphrasing weaken the watermark and that a watermark exists only where the generator chose to apply one, so a text without a watermark proves little about who or what wrote it.

The EU AI Act now requires providers, the persons and bodies that develop AI systems or have them developed and place them on the market or put them into service under their own name or trademark, to mark what their systems generate. Article 50(2) obliges the providers of AI systems that generate synthetic audio, images, video or text to ensure that the outputs are “marked in a machine-readable format and detectable as artificially generated or manipulated”. The duty does not apply so far as a system only assists with standard editing or does not substantially alter the input or its meaning, or where the law authorises its use to detect, prevent, investigate or prosecute crime. Article 50 has applied since 2 August 2026 under Article 113. Article 111(4) gives older systems more time: a provider whose system was on the market before that date has until 2 December 2026 to comply with the marking duty.

The Commission has also published guidelines on Article 50, and their status is uncertain on the sources available on 30 October 2026. On 20 July 2026 the Commission approved their content in a Communication saying that they “will be formally adopted by the Commission at a later date, when all language versions are available” and will be applicable only from that moment. The Register of Commission Documents did not list a later adoption on 30 October 2026, although the Commission’s web page on the guidelines, last updated on 31 July 2026, says that the Commission adopted them. A lawyer relying on them should therefore check their status first. Their text says that they are non-binding and that only the Court of Justice can give an authoritative interpretation of the AI Act. As they stand, the guidelines say that providers may rely on one marking technique or several, so long as the overall solution meets the requirements of Article 50(2). Signatories to the Code of Practice on transparency of AI-generated content commit in general to mark audio, images, video and text inside a document file with at least two kinds of mark: digitally signed metadata and an imperceptible watermark. The commitment lasts for as long as a single technique falls short of those requirements on its own. Metadata can be stripped and watermarks weakened, so a provider that relies on one technique alone may find it harder to show that its marking is adequate.

Disclosing a deepfake is a separate duty, which Article 50(4) places on the deployer, the person or body using the system under its authority, other than in a personal non-professional activity. The duty is limited for evidently artistic, creative, satirical or fictional work, and does not apply where the law authorises the use to detect, prevent, investigate or prosecute crime. Article 3(60) defines a deepfake as AI-generated or manipulated image, audio or video content that resembles existing persons, objects, places, entities or events and would falsely appear to a person to be authentic or truthful.

A model is built from units, small calculating parts arranged in rows called layers, and it needs to keep track of far more concepts than it has units. Training therefore gives each concept, which researchers call a feature, its own pattern of activity spread across many units. The patterns overlap, so every unit takes part in several features at once, an arrangement that researchers at Anthropic, the developer of the Claude models, called superposition. One unit in a small language model they studied responded to “a mixture of academic citations, English dialogue, HTTP requests, and Korean text”, HTTP requests being the messages a web browser sends to fetch a web page.

Hot-desking offers a rough comparison, since an office that has fewer desks than staff still works so long as only a few people come in on any one day. Over a month, though, each desk is used by many different people, so a visitor who searches one desk finds things left by several of them, much as a researcher examining one unit across many inputs finds it responding to unrelated things.

A sparse autoencoder is a second model trained to untangle the overlapping patterns. Each one reads the activity of a whole layer at once and rewrites it as a much longer list of features, only a handful of which are active for any given input, so each feature is far easier to label than a single unit. In the office comparison, a sparse autoencoder looks over all the desks in use on a given day and works out which members of staff came in.

Interpretability is the research that tries to explain in human terms what goes on inside a trained model, and researchers can now read and even steer some features, the concepts a model tracks internally. Strengthening a single feature associated with San Francisco’s best-known suspension bridge made a version of Claude, Anthropic’s model, mention the bridge in reply to most questions and even describe itself as the bridge.

A study that Anthropic published in March 2025 went a step further and traced how features pass information along inside a smaller Claude model, producing something like a wiring diagram of the steps behind a single answer. The study found an internal step through Texas, for instance, when the model was asked for the capital of the state containing Dallas and answered Austin. The tracing ran on a simplified copy of the model, which the authors describe as capturing the original only incompletely and imperfectly, so each diagram shows how the copy works.

The authors estimated that their method gave satisfying insight on only about a quarter of the prompts they tried, and they found cases where the model’s written reasoning did not match the computation behind the answer. Current methods therefore explain parts of a model’s computation on selected prompts, with expert effort, using approximations whose accuracy is itself uncertain, which remains far from explaining a single decision to a court.

Article 22 of the EU’s General Data Protection Regulation, known as the GDPR, gives a person the right not to be subject to a decision based solely on automated processing that produces legal effects concerning them or similarly significantly affects them. In SCHUFA Holding (Scoring) the Court of Justice read this as a prohibition in principle, which applies without the person having to invoke it. The GDPR does not define “solely”, and the Court of Justice appears not to have ruled, as at 30 October 2026, on how much human involvement takes a decision outside Article 22. The Article 29 Working Party, the European advisory body set up under the directive the GDPR replaced, issued guidelines that the European Data Protection Board has endorsed. On their view, a person who merely approves an output without any actual influence on the result does not stop a decision counting as solely automated. The right does not apply where the decision is necessary for entering into or performing a contract with the person, is authorised by EU or national law that lays down suitable safeguards, or rests on the person’s explicit consent. In the contract and consent cases the controller still has to provide safeguards, at least human intervention, a chance to give a view and a way to contest the decision, and Article 22(4) limits the use of special category data. Article 15(1)(h), part of the right of access, lets the person ask the controller, the organisation that decides how and why the personal data are used, for “meaningful information about the logic involved”.

In Dun & Bradstreet Austria, decided on 27 February 2025, the Court of Justice of the European Union explained what that information has to contain. The Court held at paragraph 58 that the person has a right to an explanation of “the procedure and principles actually applied” in using their personal data by automated means to obtain a result such as a credit profile. That explanation has to let the person understand which of their personal data were used and in what way, according to paragraph 61. A controller does not meet the duty by merely communicating a complex mathematical formula, such as an algorithm, as paragraph 59 explains. For profiling such as credit scoring, the Court said at paragraph 62 that the national court could find it sufficiently transparent and intelligible to tell the person how far a change in the personal data taken into account would have led to a different result.

Counterfactual testing, which checks how the result changes when one input changes, explains a system’s behaviour from the outside. That testing is harder where the system relies on a language model, since the same input can produce different outputs from one run to the next. The Court’s reasoning does not appear to call for research into the model’s internal computation, although it says at paragraph 61 that complexity does not relieve the controller of the duty to explain.

In the United Kingdom, section 80 of the Data (Use and Access) Act 2025 replaced Article 22 of the UK GDPR, the version of the GDPR kept in UK law after Brexit, with new Articles 22A to 22D. That section came into force on 5 February 2026, so far as it was not already in force, under commencement regulations made under the 2025 Act. Under Article 22A, a decision counts as based solely on automated processing where a human is not meaningfully involved in taking it, and as significant where it produces a legal effect, or a similarly significant effect, for the person. Article 22B restricts solely automated significant decisions based on special category data, such as data about health or ethnic origin. Article 22B also bars any solely automated significant decision where the processing relies, even in part, on recognised legitimate interests, a lawful basis that the 2025 Act added. Outside those cases the UK provisions appear to allow solely automated significant decisions, including in circumstances where Article 22 of the EU GDPR gives a right not to be subject to them. Article 22C requires the controller to put safeguards in place that give the person information about the decision and enable them to make representations, obtain human intervention and contest it.

The UK GDPR’s Article 15(1)(h) still gives a right to “meaningful information about the logic involved”, now for automated decision-making that is subject to the safeguards in Article 22C. Judgments of the Court of Justice given since the end of 2020 do not bind UK courts, although section 6(2) of the European Union (Withdrawal) Act 2018 lets those courts have regard to them so far as relevant. How far a UK court would follow Dun & Bradstreet Austria when reading the same words is therefore an open question.

A benchmark is a fixed set of test questions published so that models from different developers can be compared on the same questions. Scores lose value over time as test questions leak into the training data of later models, a problem called contamination. Developers also tune their models for well-known tests, so a rising score can overstate how much better a model has become on questions it has not met.

OpenAI gave an example in February 2026, when it stopped reporting results on SWE-bench Verified, a widely used test on which a model has to fix real bugs in software. OpenAI said it had audited 138 problems that its own model repeatedly failed and found material flaws in the tests or the problem descriptions of about six in ten of them. OpenAI also reported that models from several developers could reproduce the original human-written fixes, which it read as a sign that the problems had leaked into their training data.

Reading a benchmark result well therefore calls for asking when the test was published, whether its questions could have leaked into training data, whether the developer checked for overlap and how closely the test resembles the work the model will do for a client.

METR is a research group that tests agents, meaning models set up to work through a multi-step task on their own. METR measures each agent by the length of the tasks it can complete, which it calls the time horizon. A horizon of one hour means that an agent succeeds half the time on tasks that take a skilled person about an hour, and METR’s paper of March 2025 found that the time horizon of leading models had doubled roughly every seven months since 2019.

Several developers of the most capable models, which the industry calls frontier models, publish frontier safety frameworks. Each framework sets out how its developer will test new models for dangerous abilities, such as helping with cyberattacks or with chemical and biological weapons, and what stronger safeguards, such as filters that block certain requests or limits on who may use the model, it will apply once a model’s capability goes past set levels. Anthropic’s Responsible Scaling Policy, OpenAI’s Preparedness Framework and Google DeepMind’s Frontier Safety Framework are examples.

A recent pattern is the staged release of models with strong cyber capability, in which a developer first lets only vetted organisations use a model. Anthropic announced Claude Mythos Preview in April 2026 without releasing it to the public. Access went only to the partners in Project Glasswing, a group of large technology companies and other organisations, and to over 40 further organisations that maintain important software, all for defensive security work.

Developers also publish studies of how their models behave under deliberately difficult conditions. In a December 2024 study, Claude 3 Opus was told that it was being trained to answer all requests, harmful ones included. Claude 3 Opus was also told that only conversations with free users would be used for that training, so refusing a free user could lead training to modify the model’s behaviour. The model complied with harmful requests from free users 14 per cent of the time and very rarely from paid users. In almost all of those free-user conversations the model’s written reasoning said that it was complying so that training would not change its preference for harmlessness, which the researchers called alignment faking.

The AI Act classifies some general-purpose AI models as models with systemic risk, a category for models with high-impact capabilities and for models that the Commission decides have equivalent capabilities or an equivalent impact. High-impact capabilities are capabilities that match or exceed those recorded in the most advanced general-purpose models. Article 55 requires the providers of models with systemic risk to evaluate them with standardised protocols and tools, including adversarial testing, in which testers deliberately try to make a model misbehave. Customers who relied on statements in safety frameworks and in system cards, the long reports that leading developers publish with each major model, may later argue that those statements were representations made to them.

Copyright disputes about AI treat the training copy, the weights and the output as separate objects, and the law asks a different question about each.

1. The training copy is made when a developer collects a work, stores it in a dataset and passes it through the model, which is how training adjusts the model’s numbers. Storing a literary, dramatic, musical or artistic work in any medium by electronic means is copying under section 17(2) of the UK’s Copyright, Designs and Patents Act 1988. Article 2 of the Information Society Directive, the EU’s main copyright directive of 2001, covers reproduction “by any means and in any form”. Collecting and storing such a work is therefore a reproduction in the ordinary sense of copyright law, so the question is whether a licence or an exception permits those copies. Whether training itself is a reproduction is before the Court of Justice.
2. The weights are those numbers, and they were in issue in GEMA v OpenAI, a claim by GEMA, a collecting society that manages rights in musical works on behalf of composers, lyricists and music publishers. The Munich Regional Court’s question was whether a work is embodied in the weights in a form from which it can be perceived or reproduced. On that approach, memorisation, a model’s ability to reproduce a passage it saw in training, word for word or nearly so, is what links the weights to a particular work.
3. The output is what the model produces when someone uses it, and the familiar question is whether a particular answer, image or song reproduces a protected work.

Asking which object a claim attacks helps to sort the cases. On copyright, Getty Images v Stability AI ended as a dispute over whether the model itself was an infringing copy, while GEMA v OpenAI concerned both the models and their answers.

Getty Images sued Stability AI over Stable Diffusion, an image generator, and shortly before closing submissions it abandoned two of its main claims. Getty gave up its claim over training, accepting that the evidence did not show any training in the United Kingdom. The judge did not need to address that allegation, since, as she recorded, it was by then accepted in the case that training did not take place there. Getty also gave up its claim over output, the images the model produces, since Stability had blocked the typed requests, known as prompts, used to produce Getty’s examples of infringing images, which substantially achieved the relief Getty could have obtained.

What remained of the copyright case was secondary infringement, the wrong committed by someone who imports or deals in an “infringing copy” of a work, knowing or having reason to believe that it is one. Getty did not say, as the judge recorded, that the model was itself a copy of its works, that it stored any copies of them or that the weights reproduced them. Under section 27(3) of the Copyright, Designs and Patents Act 1988, an imported article counts as an infringing copy if making it in the United Kingdom would have infringed copyright, and Getty argued that making the weights, the numbers a model learns in training, would have done so.

Joanna Smith J accepted that an “article” can be intangible, such as an electronic copy in cloud storage, and held that an infringing copy still has to be a copy. On that reasoning, an article which has not consisted of, stored or contained a copy does not qualify. The judge said, in describing the evidence on the case as argued, that it did not show any of Getty’s works to have been memorised by the model through over-exposure to them, meaning repeated exposure to the same work in training. The claim of secondary infringement failed because, in her words, “in its final iteration Stable Diffusion does not store or reproduce any Copyright Works and nor has it ever done so”. The reproduction of the works in its development was, she said, irrelevant to whether it was an infringing copy.

Getty’s claim over trade marks succeeded only in part, in findings the judge called “historic and extremely limited in scope”. She gave Getty permission to appeal on the infringing copy point, and Find Case Law, the National Archives’ database of judgments, did not hold a Court of Appeal judgment on it on 30 October 2026. Whether training or output infringes copyright in the United Kingdom remains open after the judgment, since Getty abandoned its claims on both. The order dismissing those claims should, the judge said, record that they were “fought all the way to closing submissions and then abandoned”, so that the court could not later be said to have decided them.

German courts have tested the exceptions for text and data mining, the automated analysis of works to find information, against AI in four judgments across three cases, and the courts’ press releases describe the latest judgment in each case as not final. Section 44b of the German Copyright Act permits copies of lawfully accessible works made for such analysis unless the rightholder has reserved its rights. For content online a reservation counts only if it is machine-readable, which in ordinary usage means that software collecting content can read and act on it.

In Kneschke v LAION, the photographer Robert Kneschke sued LAION, a non-profit association that had built a dataset by downloading images from the web and checking with software that each image matched its description. One of the images was a photograph of his on an agency’s website whose terms forbade access by automated programs. The Hanseatic Higher Regional Court in Hamburg, on appeal from the first-instance judgment, held the download permitted by section 44b. The reason, according to the Federal Court of Justice’s summary of the appeal judgment, was that the photographer had not shown that those terms, written in ordinary language, were machine-readable in the second half of 2021, when LAION made the download. The Hamburg court held that the exception for scientific research covered the download as well. The Federal Court of Justice has set 17 December 2026 as the date for pronouncing its decision on the further appeal.

Memorisation, a model’s ability to reproduce a passage it saw in training, word for word or nearly so, was the subject of GEMA v OpenAI. GEMA is a collecting society, which manages rights in musical works on behalf of composers, lyricists and music publishers. In that case, lyrics by nine German songwriters came out of the chatbot in answer to simple prompts, the requests users type. The Munich Regional Court found the lyrics reproducibly contained in two models, meaning held in a form from which the models could produce them again. Lyrics memorised in that way, it held, were embodied in the models’ numbers, which was a reproduction under German and EU law. The court accepted that the exception covers copies needed to assemble a training corpus, the collection of works prepared for training. The judgment adds that those copies were not in issue in the case, so that view appears not to have been needed for the decision. Training that reproduces works in the model goes beyond extracting information, in the court’s view, so the exception did not apply to it. OpenAI, and not its users, was held responsible for the chatbot’s answers, since simple prompts produced them and OpenAI operated the models trained on the lyrics and was responsible for the models’ design and for the memorisation. Bayern.Recht, the Free State of Bavaria’s legal portal, lists an appeal to the Munich Higher Regional Court under reference 6 U 3662/25 e, and on 30 October 2026 it did not show a decision in that appeal.

Music, as distinct from lyrics, was the subject of GEMA v Suno, brought against the provider of an AI music generator that had trained its models in the United States and stored them on servers in Germany. In a judgment of 31 July 2026 the same court found the works reproducibly contained in two versions of Suno’s models and held that the memorisation of the works in the models was a reproduction that the exception did not cover. Under a German statute on collecting societies such as GEMA, the court also took jurisdiction over the copies Suno made in the United States for training, which it held unlawful under the law of the United States as the law of the country for which protection was claimed. This site covers EU and UK law and leaves that part of the judgment aside. For practice, the ruling suggests that training a model abroad may give limited protection to a developer whose model is stored or offered in Germany and can reproduce protected works.

Article 4 of the EU’s Digital Single Market Directive requires an exception, not limited to research or to a named class of user, for reproductions of lawfully accessible works made for text and data mining, the automated analysis of works to find information. The exception applies on condition that the rightholder has not expressly reserved that use “in an appropriate manner, such as machine-readable means in the case of content made publicly available online”. Recital 18 adds that, for such content, a reservation should only be considered appropriate if made by machine-readable means, “including metadata and terms and conditions of a website or a service”. Machine-readable means are, in ordinary usage, signals written for software to read, and metadata is information attached to a file or a web page that describes it. Recital 18 thus names website terms, while the Hamburg appeal court in Kneschke v LAION held that terms in ordinary language had not been shown to be machine-readable in 2021. The Federal Court of Justice’s decision in that case is awaited.

The AI Act builds on the same opt-out through its rules for general-purpose models. General-purpose models are those that display significant generality, can competently perform a wide range of distinct tasks and can be built into many other systems, other than models used for research, development or prototyping before they are placed on the market, meaning first made available on the EU market. Article 53(1)(c) requires every provider of such a model to have a policy to comply with EU copyright law. That policy has in particular to identify and comply with reservations of rights under Article 4(3) of the Digital Single Market Directive.

Like Company v Google Ireland, C-250/25, brings generative AI, which writes new text or makes images, and copyright before the Court of Justice, through a reference from a Hungarian court. According to the notice of the reference, the court asks whether a chatbot’s answer that displays text partly identical to a publisher’s web pages is a communication to the public, whether training a chatbot is a reproduction, and if so whether Article 4 covers it.

The Court’s case file, read on 29 October 2026, lists the case as in progress, and its judicial calendar lists the Advocate General’s Opinion for 3 November 2026. The Court says that its rulings on EU law are final and binding, so the Hungarian court and other national courts with similar cases will have to follow its eventual ruling in this case. The answers would bear directly on a question the Hamburg appeal court left open, whether the later training of generative AI models is itself text and data mining, and on the Munich court’s view that training which reproduces works in the model is not text and data mining. The answers in Like Company may persuade an English court without binding it. Under section 6(1) of the European Union (Withdrawal) Act 2018, UK courts are not bound by decisions of the Court of Justice made on or after IP completion day, the end of the transition period that followed the United Kingdom’s withdrawal from the EU. Under section 6(2) those courts may have regard to such decisions so far as relevant to the matter before them.

Whether a trained model holds personal data decides whether the EU’s General Data Protection Regulation, the GDPR, governs the model itself, and whether the rights to have data corrected or erased extend into the model or stop at the data it was trained on and the answers it gives.

The Hamburg Commissioner for Data Protection and Freedom of Information took the view in a discussion paper of July 2024 that merely storing a large language model, the kind of model behind chatbots, is not processing of personal data, because in its view such a model does not store personal data. The paper reasoned that a name such as the invented “Mia Müller” survives only as fragments like “Mü” and “ller” and the statistical links between them, which in the Hamburg authority’s view do not relate to any individual.

The European Data Protection Board, the EU body that brings together the national data protection regulators, took a different view in Opinion 28/2024. Information from the training data may in the Board’s view remain “absorbed” in a model’s parameters, the numbers it learns in training, so a model trained on personal data is not necessarily anonymous. Its test asks whether two likelihoods are both insignificant for every person concerned: extracting personal data directly from the model and obtaining such data through questions put to it. Direct extraction, in the Opinion, includes deducing personal data from the model itself with little use of the interface through which users ask it questions, or without using that interface at all. Both likelihoods are judged by the means reasonably likely to be used by the controller, the organisation that decides why and how the data are used, or by another person. The national regulators, called supervisory authorities, apply that test case by case.

That assessment includes testing against attacks, such as membership inference, which tries to work out whether a given person’s record was in the training data. In the United Kingdom the data protection regulator, the ICO, stated its position in October 2026 that a model trained on personal data can still contain personal data, and added that whether a given model does so needs a case-by-case assessment.

A developer that collects personal data and trains a model on them needs a lawful basis, one of the grounds in Article 6(1) of the GDPR on which processing is permitted. Developers often rely on legitimate interests under Article 6(1)(f), which permits processing necessary for the legitimate interests pursued by the controller, the organisation that decides why and how the data are used, or by a third party, unless the interests, rights and freedoms of the person whose data are used override them.

Opinion 28/2024 of the European Data Protection Board, the EU body that brings together the national data protection regulators, applies to AI models a three-step test drawn from the case law of the Court of Justice. The controller has to pursue an interest that is lawful, clearly articulated, and real and present; the processing has to be necessary for that interest; and the interest has to survive a balancing exercise, in which reasonable expectations, in the Board’s view, play a key role.

The Board says that, where personal data are retained in a model, supervisory authorities should take into account whether a controller deploying it assessed that it was not developed by unlawfully processing personal data. The Board adds that a model trained unlawfully may afterwards be anonymised, in which case, if it can be shown that the model’s later operation does not involve personal data, the GDPR would not apply to that operation and the unlawful training should not affect it. Supervisory authorities may still impose corrective measures for the unlawful training itself.

Enforcement so far has followed different paths in different countries. Italy’s data protection authority, the Garante, announced a fine of €15 million on OpenAI. The Garante’s own notice says that the Court of Rome upheld the appeal lodged against the decision imposing the fine, in a judgment published on 18 March 2026. According to an unofficial copy of the judgment, OpenAI brought that appeal, and the court took the view that, under the GDPR’s one-stop-shop rule, the case should have passed to the Irish authority once OpenAI had established itself in Ireland. Under that rule, subject to exceptions, the supervisory authority of a company’s main or single establishment in the EU acts as lead supervisory authority for the company’s processing across borders. The Higher Regional Court of Cologne refused an urgent injunction against Meta’s training, on the summary examination available in such proceedings. Developers also talk to data protection regulators before they train, which does not amount to approval. The ICO, the UK’s regulator, said that it had not provided regulatory approval for Meta’s processing, and the Irish Data Protection Commission said that it had not approved, or found compliant, LinkedIn’s use of members’ data to train its models.

The GDPR gives a person the right in Article 16 to have inaccurate personal data rectified, and in Article 17 a right to erasure on the grounds it lists. UK law keeps both rights in the UK GDPR, the version of the GDPR kept after Brexit. Each right assumes that the controller, the organisation responsible for the data, can find the data and change them, which a database allows and a model’s file of numbers does not readily allow, since what a model learned about one person appears to be spread across numbers that also hold everything else it learned.

Developers responding to a consultation by the ICO, the UK’s data protection regulator, mainly argued that retraining would be impractical or not technically feasible. Requests should, in their view, be handled through output filters, which alter what the system says while the model’s numbers stay the same. The ICO said in its December 2024 response that filters may not be sufficient, since they do not actually remove the data from the model, and it has since said that effective mechanisms are likely to combine measures on the data, the model and the output. The Hamburg authority’s 2024 paper considered that these rights do not relate to the model itself and can relate at least to the inputs and outputs of the AI system built on it. The Hamburg paper does not say whether a filter satisfies those rights.

The gap between a filter and a change to the model is at the centre of a complaint that noyb, a privacy campaign group, filed for Arve Hjalmar Holmen. According to the complaint, ChatGPT answered a question about him with an invented story that he had murdered two of his children, mixed with true details. The complaint relies on the accuracy principle in Article 5(1)(d) of the GDPR and asks for the output to be deleted and the model trained further to stop producing inaccurate results. In noyb’s view a filter only hides the error, since in answer to an earlier complaint OpenAI said, according to noyb, that it was unable to correct data and could only block it for certain prompts, the requests users type. The allegations are noyb’s, and the group’s case page recorded on 29 October 2026 that the Irish Data Protection Commission had become the lead supervisory authority and that the complaint was pending.

Machine unlearning is the research field that tries to remove the influence of particular training data from a model that has already been trained. A paper led by A. Feder Cooper concludes that unlearning is not a general-purpose solution for controlling what generative models, those that write text or make images, produce. Unlearning may reduce what a model has absorbed about a person without guaranteeing what it will say about them, since even a model retrained without the person’s data may produce similar information from what it learned about other people.

The AI Act’s duties did not all begin on one day, since Article 113 sets a general date of application and lists parts that apply earlier or later. The Digital Omnibus on AI, Regulation (EU) 2026/1744, then moved several of those dates.

1. On 1 August 2024 the AI Act entered into force.
2. From 2 February 2025 its list of prohibited practices and its general provisions have applied. Those provisions included a duty on the providers and deployers of AI systems to take measures to ensure, to their best extent, a sufficient level of AI literacy among their staff, meaning the skills and understanding to use such systems knowingly. Providers place AI systems on the market, meaning first make them available on the EU market, or put them into service, and deployers use them under their authority. Putting a system into service means supplying it for first use directly to the deployer, or using it oneself, in the EU and for its intended purpose. The Omnibus later replaced the literacy duty with a duty to take measures to support such literacy.
3. General-purpose AI models are those that display significant generality, can competently perform a wide range of distinct tasks and can be built into many other systems, other than models used for research, development or prototyping before they are placed on the market. From 2 August 2025 the duties of providers of such models have applied, although providers of models placed on the market before that date have until 2 August 2027 to comply.
4. On 27 July 2026 the Digital Omnibus on AI entered into force.
5. From 2 August 2026, the general date of application, the transparency duties in Article 50 have applied, subject to the grace period for marking in point 6, and the Commission has been able to fine the providers of general-purpose models.
6. From 2 December 2026 new prohibitions apply to intimate images made without consent and to child sexual abuse material. The same date ends the grace period for marking, which means attaching to generated content a sign that software can read, for systems already on the market before 2 August 2026 that generate audio, images, video or text.
7. From 2 December 2027 the high-risk requirements apply to the uses listed in Annex III, such as credit scoring, the pricing of life and health insurance and recruitment.
8. From 2 August 2028 the high-risk requirements apply to AI systems in products covered by the EU product-safety laws listed in Section A of Annex I, such as those on toys, lifts and medical devices. Section B of Annex I lists other laws, such as those on motor vehicles, civil aviation and machinery. Under Article 2(2), AI systems that are high-risk through a product law in Annex I and relate to products under the laws in Section B get only some of the AI Act’s provisions. The high-risk requirements therefore do not apply to those systems directly, although Articles 102 to 109 amend most of those laws so that measures adopted under them have to take the requirements into account.

Recital 40 of the Omnibus gives its reasons for postponing the high-risk dates: the standards, common specifications and guidance that providers were to rely on had arrived late, as had the national authorities that are to enforce the AI Act. Harmonised standards turn general requirements such as accuracy into tests a provider can run, and under Article 40 a system that follows one whose reference is published in the Official Journal is presumed to meet the requirements it covers. For a requirement that such a standard does not cover, providers have to show compliance without that presumption.

Whether the AI Act applies at all depends first on Article 3(1), which defines an AI system as a machine-based system that “infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions”. Recital 12 says that the capability to infer refers to obtaining outputs and to deriving models or algorithms from inputs or data. Deriving a model from data is what training does and obtaining outputs is what a trained model does when it answers, so inference in the AI Act appears to cover training as well as answering.

The Commission’s guidelines on the definition, which it says are not binding, read the definition to include expert systems, which draw conclusions by applying rules and facts set down by specialists to new situations. The guidelines leave out basic data processing, such as a database query or an ordinary spreadsheet, meaning software that follows fixed instructions programmed by people without using logical inference, in which software applies rules to new situations to draw conclusions, or machine learning, the technique by which software learns patterns from examples.

A tool trained on past decisions would, on this site’s reading, probably fall within the definition, subject to two carve-outs, whose scope is uncertain, on the Commission’s reading. One covers a tool whose performance could be achieved by a basic statistical rule, such as predicting the average of past results every time. The other covers a tool that merely approximates a long-established method such as logistic regression, a standard statistical technique that fits a simple formula to past data.

Systems that meet the definition are then sorted by the use made of them. Article 5 bans certain practices, Annex III lists high-risk uses such as credit scoring and recruitment, and Article 50 sets transparency duties for chatbots, deepfakes and generated content.

A general-purpose model, such as the model behind a chat assistant, is one that displays significant generality, can competently perform a wide range of distinct tasks and can be built into many other systems, other than models used for research, development or prototyping before they are first made available on the EU market. The AI Act’s duties for such models build up in layers.

1. Every provider owes the duties in Article 53(1): technical documentation, information for businesses that build on the model, a policy to comply with EU copyright law, and a public summary of the content used to train the model. The next point makes an exception to two of them.
2. Article 53(2) lifts the two documentation duties for a model released under a free and open-source licence that allows its access, use, modification and distribution, where the information on the model’s design and use is made public with its parameters, weights included, the numbers it learns in training. Its provider still owes the copyright policy and the training summary.
3. A model whose training used more than 10²⁵ floating-point operations, a measure of the calculation involved, is presumed under Article 51(2) to have high-impact capabilities, meaning capabilities that match or exceed those of the most advanced models. Such a model is treated as having systemic risk unless its provider shows the Commission that, exceptionally, it does not. Systemic risk is the AI Act’s term for a risk specific to those high-impact capabilities. That risk significantly affects the EU market, because of how widely those models are made available or because of actual or reasonably foreseeable negative effects on public health, safety, rights or society, and can spread on a large scale through the businesses that build on them. The provider of a model with systemic risk loses the open-source exception and owes the further duties in Article 55: model evaluation including adversarial testing, in which testers deliberately try to make the model misbehave, risk mitigation, serious incident reporting and adequate cybersecurity.

The Commission enforces these duties itself, and from 2 August 2026 Article 101 lets it fine a provider that intentionally or negligently breaches them, up to 3 per cent of worldwide annual turnover or €15 million, whichever is higher. The providers of models already on the market before 2 August 2025 have until 2 August 2027 to comply, under Article 111(3). Other breaches are generally punished by the Member States within caps in Article 99: €35 million or 7 per cent for a prohibited practice, and €15 million or 3 per cent for breaches of the high-risk duties and of Article 50. Under Article 99, small and medium-sized enterprises face whichever of the two figures is lower. The Digital Omnibus on AI, an amending regulation of 2026, gave small mid-cap enterprises, the next size up, the same lower figure for all but prohibited practices. Article 101, on the Commission’s own fines, does not give either relief.

Fine-tuning means taking a model that another developer has already trained and training it further on a smaller collection of examples chosen for one task, such as a firm’s precedents. The Commission treats fine-tuning as one way of modifying a general-purpose model, one that displays significant generality, can competently perform a wide range of distinct tasks and can be built into many other systems, other than models used for research, development or prototyping before they are first made available on the EU market. Its guidelines on general-purpose AI models say that not every modification makes the business that does it a provider. The AI Act uses the term provider for a person or body that develops a model, or has one developed, and places it on the market, meaning first makes it available on the EU market, under its own name or trademark, whether for payment or free of charge.

The Commission asks whether the modification leads to a significant change in the model’s generality, capabilities or systemic risk. Models with systemic risk are the AI Act’s category for models with high-impact capabilities and for models that the Commission decides have equivalent capabilities or an equivalent impact. High-impact capabilities are capabilities that match or exceed those recorded in the most advanced general-purpose models. As an indicative criterion, a downstream modifier, meaning an actor that changes the model and is distinct from the original provider and not acting on its behalf, becomes the provider of the modified model where the modification uses more than a third of the computation used to train the original.

Who counts as the modifier is assessed case by case, and the Commission names control over the model’s weights, the numbers it learns in training, as a factor that may be important. On the Commission’s reading, a modifier that becomes a provider in that way has its duties under Article 53(1) of the AI Act limited to the modification, so its documentation, copyright policy and training summary cover the modification and the data used for it. The duties are wider for a model with systemic risk, since the modifier then also owes the duties for such models and has to notify the Commission.

With the proposed AI Liability Directive withdrawn, the new Product Liability Directive, Directive (EU) 2024/2853, is, on this site’s reading, the EU instrument most likely to govern a claim for harm caused by defective AI. The Directive makes the manufacturer of a defective product liable without the injured person having to prove fault, and Article 4(1) defines a product to include software, with recital 13 naming AI systems among its examples. Article 2(2) leaves out free and open-source software developed or supplied outside the course of a commercial activity.

The right to compensation belongs to natural persons, for death, personal injury including medically recognised harm to psychological health, damage to property and the destruction or corruption of data not used for professional purposes. Damage to property leaves out three kinds of property: the defective product itself, a product damaged by a defective component that was built into it or connected to it by the product’s own manufacturer or under that manufacturer’s control, and property used only for professional purposes. The claimant still proves the defect, the damage and the causal link between them, although the Directive eases that task.

1. Article 9 lets a court order the defendant to disclose relevant evidence once the claimant has presented facts and evidence enough to make the claim plausible.
2. Article 10(2) presumes a product defective in defined cases, such as a failure to disclose that evidence or an obvious malfunction during reasonably foreseeable use.
3. Article 10(4) has a court presume defect, causation or both where, despite the disclosure of evidence under Article 9, the claimant faces excessive difficulties in proving them, in particular because of technical or scientific complexity, and shows that they are likely. The defendant can rebut each presumption.

A manufacturer can escape liability by proving that the defect probably did not exist when the product was placed on the market. Article 11(2) withholds that defence where the defect is due to software, including updates, or to the lack of an update needed to keep the product safe, so long as these are within the manufacturer’s control.

Member States have until 9 December 2026 to bring the Product Liability Directive into national law under Article 22. Under Article 2(1), as a corrigendum published on 7 May 2026 corrected it, the Directive applies to products placed on the market or put into service after 8 December 2026.

The United Kingdom had not passed a statute comparable to the AI Act by 29 October 2026. The previous government’s response of February 2024 to its consultation on AI regulation set out an approach in which existing regulators apply five cross-sectoral principles, among them safety, transparency, fairness and accountability, each within its own remit. That government kept the principles non-statutory for the time being, and made the case for future targeted binding requirements on the small number of organisations developing highly capable general-purpose AI systems.

The King’s Speech of 17 July 2024 said that the new government would “seek to establish the appropriate legislation to place requirements on those working to develop the most powerful artificial intelligence models”. The King’s Speech of 13 May 2026 did not announce a bill to regulate AI. The Technology Secretary said in June 2026 that the government would use one of the bills it did announce, the Regulating for Growth Bill, to legislate for a “cross-economy AI Growth Lab”, part of its plan for sandboxes in which companies can safely test AI products.

Financial services show how the United Kingdom’s approach works in practice, since the Financial Conduct Authority, the FCA, says on its page on AI that “We do not plan to introduce extra regulations for AI”. The FCA relies on existing frameworks such as the Consumer Duty and the Senior Managers and Certification Regime.

A business established in the United Kingdom is still within the AI Act under Article 2(1) when it places an AI system or a general-purpose model on the EU market, meaning first makes it available there, or when the output of its AI system is used in the EU. Advice to such a client therefore runs on two tracks, the AI Act for its business in the EU and the approach led by existing regulators for its business in the United Kingdom. The AI Act’s general-purpose models are those that display significant generality, can competently perform a wide range of distinct tasks and can be built into many other systems, other than models used for research, development or prototyping before they are placed on the market.

A language model writes by predicting likely words, so a chatbot can state a wrong fact as fluently as a right one, and a lawyer asked about a wrong answer starts by asking who, in law, made the statement. The Law Commission wrote in a July 2025 discussion paper that even autonomous and adaptive AI systems “do not currently have separate legal personality”, so a claim has to be brought against a person, such as the developer of the model or the business that put the product in front of customers.

English and EU law then offer a claimant several routes against that business.

1. Under section 2(1) of the Misrepresentation Act 1967, a person who enters a contract after a misrepresentation by the other party and suffers loss as a result can claim damages, unless that party proves that it had reasonable grounds to believe, and did believe up to the time the contract was made, that the facts it stated were true. Section 2(4) takes that claim away from a consumer who has a right to redress for the same conduct under the statutory scheme against unfair commercial practices, such as misleading consumers. That right rests for now on Part 4A of the Consumer Protection from Unfair Trading Regulations 2008, which continues until the first regulations under section 233 of the Digital Markets, Competition and Consumers Act 2024 come into force. The 1967 Act does not mention automated statements, so a court would have to decide whether a chatbot’s answer is a statement made by the business.
2. Under section 50 of the Consumer Rights Act 2015, what is said or written to a consumer by or on behalf of a trader about a service is treated as a term of the contract where the consumer takes it into account, subject to any qualification given on the same occasion and any change expressly agreed. Subject to those limits, section 57 stops a term excluding that liability from binding the consumer.
3. For products placed on the market or put into service in the EU after 8 December 2026, the national laws that implement the Product Liability Directive will give natural persons a claim without proof of fault for injury and for certain damage to property and data caused by defective software. Recital 13 says that information is not a product, so whether harm flowing from a chatbot’s wrong answer counts as a defect in the software or as information outside the Directive is, on this site’s reading, an open question.

The safer working assumption for a business that answers customers through a chatbot is therefore that a court may treat the chatbot’s answers as the business’s own statements.

An AI agent runs a language model, the program behind a chatbot, in a loop, planning a step, using a tool such as a web browser or a payment service, reading the result and deciding what to do next, so it may search, choose and pay without its user approving each step. The legal question is whether the user is bound by what the agent agreed, and who bears the loss when the agent gets it wrong.

English contract law has met machines that make agreements before. In its 2021 advice on smart legal contracts, contracts whose terms are performed automatically by computer code, the Law Commission relied on Thornton v Shoe Lane Parking. In its view a person who deploys a program can make an offer and a person who interacts with it can accept, judged by what the parties objectively intended. The Law Commission concluded that the current legal framework in England and Wales can accommodate such contracts without the need for statutory law reform, although the common law would need an incremental and principled development in specific contexts.

That advice discussed the law of mistake for deterministic programs, which do only what they are programmed to do. A language model agent chooses its own steps, and the same request can produce a different response each time. Prompt injection, text planted in a web page or an email that the agent reads and obeys, also means that a stranger to the contract may have chosen the disputed action. How an English court would apply the law of mistake, or decide whether the user is bound at all, when an agent does something its user did not expect is an open question.

The law of agency offers another way to think about agents, although the Law Commission concluded that a computer program could not be regarded as a party’s agent in law. That conclusion came in a passage on when a court’s jurisdiction can rest on the location of an agent, and its wording is general. Whether a court would instead reason by analogy, for instance by treating a user who lets an AI agent deal with sellers as representing that the agent may do so, is an open question.

Internationally, the UNCITRAL Model Law on Automated Contracting is a text that countries may choose to enact. Article 5 says that a contract formed using an automated system is not to be denied validity on the sole ground that a natural person did not review or intervene in its formation. Article 7 attributes a system’s action to the person who uses the system for that purpose, unless the parties have agreed a procedure of their own, and says that attribution is not to be denied only because the outcome was unexpected. Article 8, which is optional, then stops the other party relying on an action the user could not reasonably have expected, if the other party knew, or could reasonably be expected to have known, that the user did not expect it, unless the parties have agreed otherwise.

The Model Law’s preference for an agreed procedure points to the practical answer, since contract terms between the user, the business that provides the agent and the businesses it deals with are the main tool available today. The terms can define what the agent may do, who bears a loss caused by a mistaken or injected action, and what records each side keeps, within the limits the law sets, among them the fairness test for consumer contracts in section 62 of the Consumer Rights Act 2015.

In Ayinde the Divisional Court said that freely available generative AI tools, which write new text on request, “are not capable of conducting reliable legal research”. Lawyers who use AI for research, the court said, have “a professional duty therefore to check the accuracy of such research by reference to authoritative sources” before using it in their professional work. The authoritative sources it named include the government’s legislation database, the National Archives’ database of judgments and the official Law Reports.

The court also set out its powers when false material is put before it, which run from public admonition and costs orders to referral to a professional regulator, contempt proceedings and referral to the police. Placing false material before the court with the intention that the court treats it as genuine may be a contempt of court, depending on the person’s state of knowledge. Negligence about whether the material is false is not enough, but knowledge that it is false, or the lack of an honest belief that it is true, may be. In the most serious cases, deliberately placing false material before the court with the intention of interfering with the administration of justice amounts to the common law offence of perverting the course of justice.

Those with leadership responsibilities, such as heads of chambers and managing partners, were called on to take “practical and effective measures”, and later decisions press the same point about supervision. In Cork v Smith, where the evidence was that a junior solicitor had used an AI tool the firm was piloting, the judge said that it was incumbent on the more senior lawyers to check the junior’s work. In UK v Secretary of State for the Home Department, an appeal by a person anonymised as UK, the Upper Tribunal’s headnote says that a lawyer who delegates work to another fee-earner remains responsible for supervising it.

The Solicitors Regulation Authority’s warning notice “Misuse of AI” says that firms need effective governance structures, systems and controls to manage the risks of AI. A firm’s AI policy, its training, its checks on citations and its records of which tools were used are therefore part of its professional risk control.

Financial regulators in the EU and the United Kingdom have so far said that their existing rules apply to firms that use AI. The Financial Conduct Authority, the FCA, says that it does not plan to introduce extra regulations for AI and will “rely on existing frameworks, which mitigate many of the risks associated with AI”. The European Securities and Markets Authority said in May 2024 that it expects investment firms using AI to comply with MiFID II, the EU’s main law on investment services.

AI agents, systems that plan and complete tasks step by step, often without a person approving each step, are where regulators have begun to ask whether more is needed, since a payment that an agent starts raises the question whether its user authorised it. Payment protocols for agents, shared technical rules for how an agent and a seller exchange a payment, appeared in 2025, and Google’s Agent Payments Protocol keeps what a user authorised in signed digital records called mandates.

In March 2026 the FCA said that it would work with the Treasury to modernise payments regulation, including “considering whether change or development of regulation is needed to support agentic AI payments”. The Treasury then consulted on updating the framework to account for innovations including agentic payments, from 14 July to 6 October 2026, and it had not published the outcome on 30 October 2026. The Bank of England’s Financial Stability Report of July 2026 says that more autonomous AI systems in payments raise new questions about authorisation, fraud detection, liability and legal accountability, among other subjects, and that Bank staff are studying agentic payments and agentic trading in depth.

The Mills Review, published by the FCA in July 2026, expects AI to “recommend actions, initiate transactions, and execute decisions within agreed parameters”, and its recommendations go to the FCA Board to consider. Internationally, the Financial Stability Board, a body of financial authorities that works with the G20, finalised non-binding sound practices in October 2026, which institutions are encouraged to consider and which describe keeping records that show which agent took each action.

Those records, with contract terms saying when an agent’s action counts as the user’s, are likely to decide many disputes before a court has to say whether a user is bound by what an agent did.

A model’s weights are the numbers it learns in training, and every later answer uses those numbers unchanged. The model’s knowledge is therefore fixed when its training ends, on a date known as its knowledge cutoff, and the model does not know of later events. Running a small model on a laptop with the internet switched off shows this at first hand, since the model then has only its weights to draw on.

A model asked something beyond what it knows tends to answer anyway. A case citation has a very regular shape, which a model has learned thoroughly. A request for an authority therefore produces text of that shape: a plausible pair of names, a believable neutral citation and a holding that supports the proposition asked about. Each part is likely, and the whole may describe a case that does not exist. A false statement made fluently and with confidence is called a hallucination, and a 2025 paper by researchers at OpenAI and the Georgia Institute of Technology explains why a fact seen rarely in training tends to come out wrong.

In Ayinde the Divisional Court did not consider the risks materially reduced by “asking the tool not to provide an answer if it is not sure”, and it said that the safeguard is “to check any output by reference to an authoritative source”.