Artificial intelligence and machine learning techniques are used in video games for a wide variety of applications such as non-player character (NPC) control, procedural content generation (PCG) and deep learning-based content generation. Machine learning is a subset of artificial intelligence that uses historical data to build predictive and analytical models. This is in sharp contrast to traditional methods of artificial intelligence such as search trees and expert systems. Information on machine learning techniques in the field of games is mostly known to public through research projects as most gaming companies choose not to publish specific information about their intellectual property. The most publicly known application of machine learning in games is likely the use of deep learning agents that compete with professional human players in complex strategy games. There has been a significant application of machine learning on games such as Atari/ALE, Doom, Minecraft, StarCraft, and car racing. Other games that did not originally exists as video games, such as chess and Go have also been affected by the machine learning. == Overview of relevant machine learning techniques == === Deep learning === Deep learning is a subset of machine learning which focuses heavily on the use of artificial neural networks (ANN) that learn to solve complex tasks. Deep learning uses multiple layers of ANN and other techniques to progressively extract information from an input. Due to this complex layered approach, deep learning models often require powerful machines to train and run on. ==== Convolutional neural networks ==== Convolutional neural networks (CNN) are specialized ANNs that are often used to analyze image data. These types of networks are able to learn translation invariant patterns, which are patterns that are not dependent on location. CNNs are able to learn these patterns in a hierarchy, meaning that earlier convolutional layers will learn smaller local patterns while later layers will learn larger patterns based on the previous patterns. A CNN's ability to learn visual data has made it a commonly used tool for deep learning in games. === Recurrent neural network === Recurrent neural networks are a type of ANN that are designed to process sequences of data in order, one part at a time rather than all at once. An RNN runs over each part of a sequence, using the current part of the sequence along with memory of previous parts of the current sequence to produce an output. These types of ANN are highly effective at tasks such as speech recognition and other problems that depend heavily on temporal order. There are several types of RNNs with different internal configurations; the basic implementation suffers from a lack of long term memory due to the vanishing gradient problem, thus it is rarely used over newer implementations. ==== Long short-term memory ==== A long short-term memory (LSTM) network is a specific implementation of a RNN that is designed to deal with the vanishing gradient problem seen in simple RNNs, which would lead to them gradually "forgetting" about previous parts of an inputted sequence when calculating the output of a current part. LSTMs solve this problem with the addition of an elaborate system that uses an additional input/output to keep track of long term data. LSTMs have achieved very strong results across various fields, and were used by several monumental deep learning agents in games. === Reinforcement learning === Reinforcement learning is the process of training an agent using rewards and/or punishments. The way an agent is rewarded or punished depends heavily on the problem; such as giving an agent a positive reward for winning a game or a negative one for losing. Reinforcement learning is used heavily in the field of machine learning and can be seen in methods such as Q-learning, policy search, Deep Q-networks and others. It has seen strong performance in both the field of games and robotics. === Neuroevolution === Neuroevolution involves the use of both neural networks and evolutionary algorithms. Instead of using gradient descent like most neural networks, neuroevolution models make use of evolutionary algorithms to update neurons in the network. Researchers claim that this process is less likely to get stuck in a local minimum and is potentially faster than state of the art deep learning techniques. == Deep learning agents == Machine learning agents have been used to take the place of a human player rather than function as NPCs, which are deliberately added into video games as part of designed gameplay. Deep learning agents have achieved impressive results when used in competition with both humans and other artificial intelligence agents. === Chess === Chess is a turn-based strategy game that is considered a difficult AI problem due to the computational complexity of its board space. Similar strategy games are often solved with some form of a Minimax Tree Search. These types of AI agents have been known to beat professional human players, such as the historic 1997 Deep Blue versus Garry Kasparov match. Since then, machine learning agents have shown ever greater success than previous AI agents. === Go === Go is another turn-based strategy game which is considered an even more difficult AI problem than chess. The state space of is Go is around 10^170 possible board states compared to the 10^120 board states for Chess. Prior to recent deep learning models, AI Go agents were only able to play at the level of a human amateur. ==== AlphaGo ==== Google's 2015 AlphaGo was the first AI agent to beat a professional Go player. AlphaGo used a deep learning model to train the weights of a Monte Carlo tree search (MCTS). The deep learning model consisted of 2 ANN, a policy network to predict the probabilities of potential moves by opponents, and a value network to predict the win chance of a given state. The deep learning model allows the agent to explore potential game states more efficiently than a vanilla MCTS. The network were initially trained on games of humans players and then were further trained by games against itself. ==== AlphaGo Zero ==== AlphaGo Zero, another implementation of AlphaGo, was able to train entirely by playing against itself. It was able to quickly train up to the capabilities of the previous agent. === StarCraft series === StarCraft and its sequel StarCraft II are real-time strategy (RTS) video games that have become popular environments for AI research. Blizzard and DeepMind have worked together to release a public StarCraft 2 environment for AI research to be done on. Various deep learning methods have been tested on both games, though most agents usually have trouble outperforming the default AI with cheats enabled or skilled players of the game. ==== Alphastar ==== Alphastar was the first AI agent to beat professional StarCraft 2 players without any in-game advantages. The deep learning network of the agent initially received input from a simplified zoomed out version of the gamestate, but was later updated to play using a camera like other human players. The developers have not publicly released the code or architecture of their model, but have listed several state of the art machine learning techniques such as relational deep reinforcement learning, long short-term memory, auto-regressive policy heads, pointer networks, and centralized value baseline. Alphastar was initially trained with supervised learning, it watched replays of many human games in order to learn basic strategies. It then trained against different versions of itself and was improved through reinforcement learning. The final version was hugely successful, but only trained to play on a specific map in a protoss mirror matchup. === Dota 2 === Dota 2 is a multiplayer online battle arena (MOBA) game. Like other complex games, traditional AI agents have not been able to compete on the same level as professional human player. The only widely published information on AI agents attempted on Dota 2 is OpenAI's deep learning Five agent. ==== OpenAI Five ==== OpenAI Five utilized separate long short-term memory networks to learn each hero. It trained using a reinforcement learning technique known as Proximal Policy Learning running on a system containing 256 GPUs and 128,000 CPU cores. Five trained for months, accumulating 180 years of game experience each day, before facing off with professional players. It was eventually able to beat the 2018 Dota 2 esports champion team in a 2019 series of games. === Planetary Annihilation === Planetary Annihilation is a real-time strategy game which focuses on massive scale war. The developers use ANNs in their default AI agent. === Supreme Commander 2 === Supreme Commander 2 is a real-time strategy (RTS) video game. The game uses Multilayer Perceptrons (MLPs) to control a platoon’s reaction to encountered enemy units. Total of four MLPs are used, one for each platoon type: land, naval
Dynamic texture
Dynamic texture ( sometimes referred to as temporal texture) is the texture with motion which can be found in videos of sea-waves, fire, smoke, wavy trees, etc. Dynamic texture has a spatially repetitive pattern with time-varying visual pattern. Modeling and analyzing dynamic texture is a topic of images processing and pattern recognition in computer vision. Extracting features that describe the dynamic texture can be utilized for tasks of images sequences classification, segmentation, recognition and retrieval. Comparing with texture found within static images, analyzing dynamic texture is a challenging problem. It is important that the extracted features from dynamic texture combine motion and appearance description, and also be invariance to some transformation such as rotation, translation and illumination. == Analysis methods of dynamic texture == The methods of dynamic texture recognition can categorized as follows: Methods based on optical flow: by applying optical flow to the dynamic texture, velocity with direction and magnitude can be detected and used to recognize the dynamic texture. Due to simplicity of its computation, it is currently the most popular method. Methods computing geometric properties: this methods track the surfaces of motion trajectories in spatiotemporal domain. Methods based on local spatiotemporal filtering : this methods analyze the local spatiotemporal patterns and its orientation and energy and employ them as feature used for classification. Methods based on global spatiotemporal transform: this method characterize the motion at different scale using wavelets that can decompose the motion into local and global. Model-based methods : These methods aims at generating a model to describe the motion by a set of parameters. == Applications == - Segmenting the sequence images of natural scenes. This helps on differentiate between streets and grass alongside these streets which could be used in the application of navigations. - Motion detection : Dynamic texture features extracted from footage videos can be exploited to detect abnormal crowd activities. - Video classification: video of natural scenes or other scenes that exhibit dynamic textures. - Video retrieval : Dynamic textures can be employed as a feature retrieve videos that contain, for example, sea-waves, smoke, clouds, wavy trees.
Copyright and artificial intelligence in the United Kingdom
The interaction of artificial intelligence and copyright law has become one of the most contentious tech policy debates in the United Kingdom, centering on whether AI developers should be permitted to train their models on copyrighted material without explicit consent or remuneration. This debate has exposed a deep fracture between the creative industries, which seek to protect their intellectual property from unauthorised commercial exploitation, and tech companies. The academic and library sectors are also impacted, and argue that overly restrictive copyright laws hinder scientific research and the UK's sovereign AI capabilities. In 2024, the UK government proposed a broad text and data mining (TDM) exception to copyright that would have allowed AI companies to use publicly available copyrighted material for training, offering creators only an "opt-out" mechanism, similar to the exception introduced in Europe. This proposal faced intense opposition from across the creative sector. Trade unions representing writers, musicians, performers, and journalists argued that such an exception would effectively expropriate their members' work for the commercial benefit of tech giants. A report from the House of Lords Communications and Digital Committee, warned that generative AI posed a "clear and present danger" to the £124 billion creative economy. The government abandoned the opt-out model in March 2026, opting instead to build a stronger evidence base before pursuing any copyright reform. Conversely, the academic and library sectors have raised significant concerns that the UK's current TDM exception, which is strictly limited to non-commercial research, is too narrow. Universities and research libraries occupy a dual role as both creators of vast datasets and beneficiaries of TDM exceptions. They argue that the current legal framework restricts their ability to computationally analyse the very research they produce, thereby hobbling the UK's "AI for Science" strategy. Advocacy groups have highlighted a "triple payment" problem, wherein publicly funded research is handed over to publishers, who then charge universities substantial subscription fees and demand additional payments for specific TDM licences. This tension is further complicated by the commercial practices of major academic publishers. While publishers often restrict universities from using subscribed databases for AI training, they have simultaneously entered into lucrative, multi-million-dollar licensing agreements to sell access to this academic content to commercial AI developers. Furthermore, academics have accused publishers of actively steering authors away from permissive open-access licences towards more restrictive variants. By doing so, publishers retain the exclusive commercial rights necessary to strike these AI training deals, often without consulting the original authors or offering them any additional remuneration. This dynamic has not only reopened debates within the Open Access movement but has also created complex legal scenarios where publishers, rather than authors, control the terms of copyright litigation against major tech companies. == Training on copyrighted material == The question of whether AI developers should be permitted to train their models on copyrighted material without payment or consent has been one of the most contentious policy debates in the UK AI landscape. In 2024, the then-Conservative government proposed a broad text and data mining (TDM) exception that would have allowed AI companies to use any publicly available copyrighted material for training purposes, with creators able only to "opt out" of having their work used. This proposal provoked intense opposition from writers, musicians, visual artists, publishers, and broadcasters, who argued it would effectively expropriate their intellectual property for the commercial benefit of AI companies. The debate over text and data mining exceptions extends significantly beyond generative AI and the creative industries, implicating a wide range of scientific, industrial, and academic research applications. TDM is a foundational process for analysing large datasets to identify patterns, trends, and correlations, which is heavily utilised in fields such as medical research, climate modelling, and financial services. In the scientific and academic sectors, researchers rely on TDM to process vast amounts of published literature. For example, in biomedical research, TDM is used to accelerate drug discovery, identify new uses for existing medicines, and extract insights from clinical notes and genomic datasets. However, the application of traditional copyright frameworks to scientific literature has been criticised by academics. Researchers argue that scientific writing is intended to convey factual, verifiable information rather than creative originality, and that copyright restrictions on TDM hinder reproducibility, validation, and the advancement of science. The current UK copyright exception for TDM (Section 29A of the Copyright, Designs and Patents Act 1988) is limited strictly to non-commercial research, which creates barriers for public-private research partnerships and commercial scientific development. Beyond academia, non-generative AI and TDM are critical to various industrial and commercial operations. In the financial services sector, TDM is employed to monitor transactions, detect fraud, and analyse market feeds. Other non-generative applications include search engine indexing, plagiarism detection software, and media monitoring. A 2026 report by Public First estimated that 19% of UK businesses use specialised TDM tools, and that a restrictive copyright regime requiring licenses for all copyrighted content could cost the UK economy £220 billion in lost AI-driven GDP growth by 2035 compared to a broad commercial TDM exemption. Industry advocates argue that the lack of a commercial TDM exception in the UK creates legal uncertainty that stifles innovation across these broader, non-generative applications of data analysis. === Tech and AI industry positions === The technology and artificial intelligence industries lobbied for a broad text and data mining (TDM) exception to UK copyright law, arguing that such an exception is essential for the UK to remain globally competitive in AI development. Industry bodies such as techUK have argued that without a TDM exception, the UK risks becoming an "AI taker rather than an AI maker," as developers will relocate training operations to jurisdictions with more permissive copyright regimes, such as the United States, Japan, Singapore, and the European Union. During the UK government's 2024–2025 consultation on copyright and AI, major AI developers and trade associations strongly supported "Option 2" (a broad TDM exception) or "Option 3" (a TDM exception with an opt-out mechanism). OpenAI stated in its consultation response that a broad TDM exception is "necessary to drive AI innovation and investment in the UK," arguing that developers should be permitted to train models on lawfully accessed copies without further distribution. The Computer and Communications Industry Association (CCIA) similarly argued that restricting TDM to non-commercial development would undermine the government's ambitions for the UK tech sector and frustrate partnerships between commercial entities and research institutions. Tech industry advocates have also highlighted the economic implications of copyright policy. According to analysis by the think tank UK Day One, adopting an overly restrictive licensing-only approach could result in the UK economy losing up to £182 billion over 20 years, whereas a broad TDM exception could generate a positive impact of £131.61 billion over the same period. Following the government's March 2026 decision to drop plans for a TDM exception in favour of a market-led licensing approach, techUK's Deputy CEO Antony Walker criticised the move, stating that "copyright material cannot be used for AI development and training without permission" under the current framework, which he argued would push AI model training to the US. === Creative sector and political opposition to text and data mining === In March 2026, the House of Lords Communications and Digital Committee published a report, AI, Copyright and the Creative Industries, which concluded that the creative industries face "a clear and present danger from generative AI" and that it would be "a very poor bet" for the government to weaken copyright protections to attract AI investment. The Committee noted that the creative industries contributed £124 billion to the UK economy in 2023 and employed 2.4 million people, compared to the AI sector's £12 billion GVA and 86,000 employees in 2024. The Committee called on the government to develop a "licensing-first" regime underpinned by mandatory transparency requirements, and to rule out any new commercial TDM exception with an opt-out model. Tra
Karen Hao
Karen Hao (born in the United States c. 1993) is an American journalist and author. Currently a freelancer for publications like The Atlantic and previously a foreign correspondent based in Hong Kong for The Wall Street Journal and senior artificial intelligence editor at the MIT Technology Review, she is best known for her coverage on AI research, technology ethics and the social impact of AI. Hao also co-produced the podcast In Machines We Trust and wrote the newsletter The Algorithm. Previously, she worked at Quartz as a tech reporter and data scientist and was an application engineer at the first startup to spin out of X Development. Hao's writing has also appeared in Mother Jones, Sierra Magazine, The New Republic, and other publications. == Early life and education == Hao is the daughter of Chinese immigrant parents, and grew up in New Jersey. She is a native speaker of both English and Mandarin Chinese. She graduated from The Lawrenceville School in 2011. She then studied at the Massachusetts Institute of Technology (MIT), graduating with a B.S. in mechanical engineering and a minor in energy studies in 2015. == Career == Hao is known in the technology world for her coverage of new AI research findings and their societal and ethical impacts. Her writing has spanned research and issues regarding big tech data privacy, misinformation, deepfakes, facial recognition, and AI healthcare tools. In March 2021, Hao published a piece that uncovered previously unknown information about how attempts to combat misinformation by different teams at Facebook using machine learning were impeded and constantly at odds with Facebook's drive to grow user engagement. Upon its release, leaders at Facebook including Mike Schroepfer and Yann LeCun immediately criticized the piece through Twitter responses. AI researchers and AI ethics experts Timnit Gebru and Margaret Mitchell responded in support of Hao's writing and advocated for more change and improvement for all. Hao also co-produced the podcast In Machines We Trust, which discusses the rise of AI with people developing, researching, and using AI technologies. The podcast won the 2020 Front Page Award in investigative reporting. Hao has occasionally created data visualizations that have been featured in her work at the MIT Technology Review and elsewhere. In 2018, her "What is AI?" flowchart visualization was exhibited as an installation at the Museum of Applied Arts in Vienna. She has been an invited speaker at TEDxGateway, the United Nations Foundation, EmTech, WNPR, and many other conferences and podcasts. Her TEDx talk discussed the importance of democratizing how AI is built. In March 2022, she was hired by The Wall Street Journal to cover China technology and society, while being based in Hong Kong. She left the WSJ in 2023. In May 2025, Hao released the book Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI. The book became a New York Times Bestseller and was named a Book of the Year by the Financial Times. In December 2025, after criticism from readers, Hao issued a correction to her book where she had previously overestimated the water consumption of a data center in Chile compared to the community's water consumption by factor of 1,000, due to an error in a government document. In April 2026 the book won the New York Public Library's Helen Bernstein Book Award for Excellence in Journalism. === Selected awards and honors === 2019 Webby Award nominee for best newsletter, as a writer of The Algorithm 2021 Front Page Award in investigative reporting, as a co-producer for In Machines We Trust 2021 Ambies Award nominee for best knowledge and science podcast, as a co-producer for In Machines We Trust 2021 Webby Award nominee for best technology podcast, as a co-producer for In Machines We Trust 2024 American Humanist Media Award 2025 TIME100 AI, named by TIME magazine as one of the 100 most influential people in artificial intelligence 2026 New York Public Library's Helen Bernstein Book Award for Excellence in Journalism 2026 Whiting Award in Non-fiction
PyTorch
PyTorch is an open-source deep learning library, originally developed by Meta Platforms and currently developed with support from the Linux Foundation. The successor to Torch, PyTorch provides a high-level API that builds upon optimised, low-level implementations of deep learning algorithms and architectures, such as the Transformer, or SGD. Notably, this API simplifies model training and inference to a few lines of code. PyTorch allows for automatic parallelization of training and, internally, implements CUDA bindings that speed training further by leveraging GPU resources. PyTorch utilises the tensor as a fundamental data type, similarly to NumPy. Training is facilitated by a reversed automatic differentiation system, Autograd, that constructs a directed acyclic graph of the operations (and their arguments) executed by a model during its forward pass. With a loss, backpropagation is then undertaken. As of 2025, PyTorch remains one of the most popular deep learning libraries, alongside others such as TensorFlow and Keras. It can be installed using Anaconda package managers. A number of commercial deep learning architectures are built on top of PyTorch, including ChatGPT, Tesla Autopilot, Uber's Pyro, and Hugging Face's Transformers. == History == In 2001, Torch was written and released under a GPL. It was a machine-learning library written in C++ and CUDA, supporting methods including neural networks, support vector machines (SVM), hidden Markov models, etc. Around 2010, it was rewritten by Ronan Collobert, Clement Farabet and Koray Kavuckuoglu. This was known as Torch7 or LuaTorch. This was written so that the backend was in C and the frontend was in Lua. In mid-2016, some developers refactored it to decouple the frontend and the backend, with strong influence from torch-autograd and Chainer. In turn, torch-autograd was influenced by HIPS/autograd. Development on Torch7 ceased in 2018 and was subsumed by the PyTorch project. Meta (formerly known as Facebook) operates both PyTorch and Convolutional Architecture for Fast Feature Embedding (Caffe2), but models defined by the two frameworks were mutually incompatible. The Open Neural Network Exchange (ONNX) project was created by Meta and Microsoft in September 2017 to decouple deep learning frameworks from hardware-specific runtimes, allowing models to be converted between frameworks and optimized for execution providers like NVIDIA’s TensorRT. Caffe2 was merged into PyTorch at the end of March 2018. In September 2022, Meta announced that PyTorch would be governed by the independent PyTorch Foundation, a newly created subsidiary of the Linux Foundation. PyTorch 2.0 was released on 15 March 2023, introducing TorchDynamo, a Python-level compiler that makes code run up to two times faster, along with significant improvements in training and inference performance across major cloud platforms. == PyTorch tensors == PyTorch defines a class called Tensor (torch.Tensor) to store and operate on homogeneous multidimensional rectangular arrays of numbers. PyTorch supports various sub-types of multi-dimensional arrays, or Tensors. PyTorch Tensors are similar to NumPy Arrays, but can also be operated on by a CUDA-capable NVIDIA GPU. PyTorch has also been developing support for other GPU platforms, for example, AMD's ROCm and Apple's Metal Framework. == PyTorch neural networks == PyTorch defines a module called nn (torch.nn) to describe neural networks and to support training. This module offers a comprehensive collection of building blocks for neural networks, including various layers and activation functions, enabling the construction of complex models. Networks are built by inheriting from the torch.nn module and defining the sequence of operations in the forward() function. == PyTorch Serialized File Format == Pytorch can save and load models using its own file format, which is a ZIP64 archive containing the model weights in a Python pickle file, and other information such as the byte order. The file extensions .pt and .pth are commonly used for these files. == Example == The following program shows the low-level functionality of the library with a simple example. The following code block defines a neural network with linear layers using the nn module.
SIP (software)
SIP is an open source software tool used to connect computer programs or libraries written in C or C++ with the scripting language Python. It is an alternative to SWIG. SIP was originally developed in 1998 for PyQt — the Python bindings for the Qt GUI toolkit — but is suitable for generating bindings for any C or C++ library. == Concept == SIP takes a set of specification (.sip) files describing the API and generates the required C++ code. This is then compiled to produce the Python extension modules. A .sip file is essentially the class header file with some things removed (because SIP does not include a full C++ parser) and some things added (because C++ does not always provide enough information about how the API works). For PyQt v4 I use an internal tool (written using PyQt of course) called metasip. This is sort of an IDE for SIP. It uses GCC-XML to parse the latest header files and saves the relevant data, as XML, in a metasip project. metasip then does the equivalent of a diff against the previous version of the API and flags up any changes that need to be looked at. Those changes are then made through the GUI and ticked off the TODO list. Generating the .sip files is just a button click. In my subversion repository, PyQt v4 is basically just a 20M XML file. Updating PyQt v4 for a minor release of Qt v4 is about half an hours work. In terms of how the generated code works then I don't think it's very different from how any other bindings generator works. Python has a very good C API for writing extension modules - it's one of the reasons why so many 3rd party tools have Python bindings. For every C++ class, the SIP generated code creates a corresponding Python class implemented in C. == Notable applications that use SIP == PyQt, a python port of the application framework and widget toolkit Qt QGIS, a free and open-source cross-platform desktop geographic information system (GIS) QtiPlot, a computer program to analyze and visualize scientific data calibre (software), a free and open-source cross-platform e-book manager Veusz, a free and open-source cross-platform program to visualize scientific data
Catastrophic interference
Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to abruptly and drastically forget previously learned information upon learning new information. Neural networks are an important part of the connectionist approach to cognitive science. The issue of catastrophic interference when modeling human memory with connectionist models was originally brought to the attention of the scientific community by research from McCloskey and Cohen (1989), and Ratcliff (1990). It is a radical manifestation of the 'sensitivity-stability' dilemma or the 'stability-plasticity' dilemma. Specifically, these problems refer to the challenge of making an artificial neural network that is sensitive to, but not disrupted by, new information. Lookup tables and connectionist networks lie on the opposite sides of the stability plasticity spectrum. The former remains completely stable in the presence of new information but lacks the ability to generalize, i.e. infer general principles, from new inputs. On the other hand, connectionist networks like the standard backpropagation network can generalize to unseen inputs, but they are sensitive to new information. Backpropagation models can be analogized to human memory insofar as they have a similar ability to generalize, but these networks often exhibit less stability than human memory. Notably, these backpropagation networks are susceptible to catastrophic interference. This is an issue when modelling human memory, because unlike these networks, humans typically do not show catastrophic forgetting. == Discovery == The term catastrophic interference was originally coined by McCloskey and Cohen (1989) but was also brought to the attention of the scientific community by research from Ratcliff (1990). === The Sequential Learning Problem: McCloskey and Cohen (1989) === McCloskey and Cohen (1989) noted the problem of catastrophic interference during two different experiments with backpropagation neural network modelling. Experiment 1: Learning the ones and twos addition facts In their first experiment they trained a standard backpropagation neural network on a single training set consisting of 17 single-digit ones problems (i.e., 1 + 1 through 9 + 1, and 1 + 2 through 1 + 9) until the network could represent and respond properly to all of them. The error between the actual output and the desired output steadily declined across training sessions, which reflected that the network learned to represent the target outputs better across trials. Next, they trained the network on a single training set consisting of 17 single-digit twos problems (i.e., 2 + 1 through 2 + 9, and 1 + 2 through 9 + 2) until the network could represent, respond properly to all of them. They noted that their procedure was similar to how a child would learn their addition facts. Following each learning trial on the twos facts, the network was tested for its knowledge on both the ones and twos addition facts. Like the ones facts, the twos facts were readily learned by the network. However, McCloskey and Cohen noted the network was no longer able to properly answer the ones addition problems even after one learning trial of the twos addition problems. The output pattern produced in response to the ones facts often resembled an output pattern for an incorrect number more closely than the output pattern for a correct number. This is considered to be a drastic amount of error. Furthermore, the problems 2+1 and 1+2, which were included in both training sets, even showed dramatic disruption during the first learning trials of the twos facts. Experiment 2: Replication of Barnes and Underwood (1959) study In their second connectionist model, McCloskey and Cohen attempted to replicate the study on retroactive interference in humans by Barnes and Underwood (1959). They trained the model on A-B and A-C lists and used a context pattern in the input vector (input pattern), to differentiate between the lists. Specifically the network was trained to respond with the right B response when shown the A stimulus and A-B context pattern and to respond with the correct C response when shown the A stimulus and the A-C context pattern. When the model was trained concurrently on the A-B and A-C items then the network readily learned all of the associations correctly. In sequential training the A-B list was trained first, followed by the A-C list. After each presentation of the A-C list, performance was measured for both the A-B and A-C lists. They found that the amount of training on the A-C list in Barnes and Underwood study that lead to 50% correct responses, lead to nearly 0% correct responses by the backpropagation network. Furthermore, they found that the network tended to show responses that looked like the C response pattern when the network was prompted to give the B response pattern. This indicated that the A-C list apparently had overwritten the A-B list. This could be likened to learning the word dog, followed by learning the word stool and then finding that you think of the word stool when presented with the word dog. McCloskey and Cohen tried to reduce interference through a number of manipulations including changing the number of hidden units, changing the value of the learning rate parameter, overtraining on the A-B list, freezing certain connection weights, changing target values 0 and 1 instead 0.1 and 0.9. However, none of these manipulations satisfactorily reduced the catastrophic interference exhibited by the networks. Overall, McCloskey and Cohen (1989) concluded that: at least some interference will occur whenever new learning alters the weights involved in representing old learning the greater the amount of new learning, the greater the disruption in old knowledge interference was catastrophic in the backpropagation networks when learning was sequential but not concurrent === Constraints Imposed by Learning and Forgetting Functions: Ratcliff (1990) === Ratcliff (1990) used multiple sets of backpropagation models applied to standard recognition memory procedures, in which the items were sequentially learned. After inspecting the recognition performance models he found two major problems: Well-learned information was catastrophically forgotten as new information was learned in both small and large backpropagation networks. Even one learning trial with new information resulted in a significant loss of the old information, paralleling the findings of McCloskey and Cohen (1989). Ratcliff also found that the resulting outputs were often a blend of the previous input and the new input. In larger networks, items learned in groups (e.g. AB then CD) were more resistant to forgetting than were items learned singly (e.g. A then B then C...). However, the forgetting for items learned in groups was still large. Adding new hidden units to the network did not reduce interference. Discrimination between the studied items and previously unseen items decreased as the network learned more. This finding contradicts studies on human memory, which indicated that discrimination increases with learning. Ratcliff attempted to alleviate this problem by adding 'response nodes' that would selectively respond to old and new inputs. However, this method did not work as these response nodes would become active for all inputs. A model which used a context pattern also failed to increase discrimination between new and old items. == Proposed solutions == The main cause of catastrophic interference seems to be overlap in the representations at the hidden layer of distributed neural networks. In a distributed representation, each input tends to create changes in the weights of many of the nodes. Catastrophic forgetting occurs because when many of the weights where "knowledge is stored" are changed, it is unlikely for prior knowledge to be kept intact. During sequential learning, the inputs become mixed, with the new inputs being superimposed on top of the old ones. Another way to conceptualize this is by visualizing learning as a movement through a weight space. This weight space can be likened to a spatial representation of all of the possible combinations of weights that the network could possess. When a network first learns to represent a set of patterns, it finds a point in the weight space that allows it to recognize all of those patterns. However, when the network then learns a new set of patterns, it will move to a place in the weight space for which the only concern is the recognition of the new patterns. To recognize both sets of patterns, the network must find a place in the weight space suitable for recognizing both the new and the old patterns. Below are a number of techniques which have empirical support in successfully reducing catastrophic interference in backpropagation neural networks: === Orthogonality === Many of the early techniques in reducing representational overlap involved making either the input vecto