AI Bot Grammar Checker

AI Bot Grammar Checker — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

  • EPages

    EPages

    ePages is an e-commerce software that allows merchants to create and run online shops in the cloud. The number of shops based on ePages is currently 140,000 worldwide. ePages software is regularly updated due to its Software-as-a-Service model. An investor in the company is United Internet, with a 25% stake. ePages focuses upon distributing its products mainly through hosting providers. ePages is headquartered in Hamburg, with additional offices Barcelona, Jena, and Bilbao. == History == The name ePages was used for the first time for software in 1997 to market "Intershop ePages". In 2002, the product line then called Intershop 4 was taken over by ePages GmbH and renamed to ePages. == Features == Depending on the ePages product and packages offered by hosting providers, merchants can sell up to an unlimited number of items. Users can offer their products and services in 15 languages and with all currencies. With ePages, merchants can use web marketing tools; e.g. newsletters, coupons or social media plug-ins for social commerce.

    Read more →
  • Google Brain

    Google Brain

    Google Brain was a deep learning artificial intelligence research team that served as the sole AI branch of Google before being incorporated under the newer umbrella of Google AI, a research division at Google dedicated to artificial intelligence. Formed in 2011, it combined open-ended machine learning research with information systems and large-scale computing resources. It created tools such as TensorFlow, which allow neural networks to be used by the public, and multiple internal AI research projects, and aimed to create research opportunities in machine learning and natural language processing. It was merged into former Google sister company DeepMind to form Google DeepMind in April 2023. == History == The Google Brain project began in 2011 as a part-time research collaboration between Google fellow Jeff Dean and Google Researcher Greg Corrado. Google Brain started as a Google X project and became so successful that it was graduated back to Google: Astro Teller has said that Google Brain paid for the entire cost of Google X. In June 2012, The New York Times reported that a cluster of 16,000 processors in 1,000 computers dedicated to mimicking some aspects of human brain activity had successfully trained itself to recognize a cat based on 10 million digital images taken from YouTube videos. The story was also covered by National Public Radio (NPR). In March 2013, Google hired Geoffrey Hinton, a leading researcher in the deep learning field, and acquired the company DNNResearch Inc. headed by Hinton. Hinton said that he would be dividing his future time between his university research and his work at Google. In April 2023, Google Brain merged with Google sister company DeepMind to form Google DeepMind, as part of the company's continued efforts to accelerate work on AI. == Team and location == Google Brain was initially established by Google Fellow Jeff Dean and visiting Stanford professor Andrew Ng. In 2014, the team included Jeff Dean, Quoc V. Le, Ilya Sutskever, Alex Krizhevsky, Samy Bengio, and Vincent Vanhoucke. In 2017, team members included Anelia Angelova, Samy Bengio, Greg Corrado, George Dahl, Michael Isard, Anjuli Kannan, Hugo Larochelle, Chris Olah, Benoit Steiner, Vincent Vanhoucke, Vijay Vasudevan, and Fernanda Viegas. Chris Lattner, who created Apple's programming language Swift and then ran Tesla's autonomy team for six months, joined Google Brain's team in August 2017. Lattner left the team in January 2020 and joined SiFive. As of 2021, Google Brain was led by Jeff Dean, Geoffrey Hinton, and Zoubin Ghahramani. Other members include Katherine Heller, Pi-Chuan Chang, Ian Simon, Jean-Philippe Vert, Nevena Lazic, Anelia Angelova, Lukasz Kaiser, Carrie Jun Cai, Eric Breck, Ruoming Pang, Carlos Riquelme, Hugo Larochelle, and David Ha. Samy Bengio left the team in April 2021, and Zoubin Ghahramani took on his responsibilities. Google Research includes Google Brain and is based in Mountain View. It also has satellite groups in Accra, Amsterdam, Atlanta, Beijing, Berlin, Cambridge, Israel, Los Angeles, London, Montreal, Munich, New York City, Paris, Pittsburgh, Princeton, San Francisco, Seattle, Tokyo, Toronto, and Zurich. == Projects == === Artificial-intelligence-devised encryption system === In October 2016, Google Brain designed an experiment to determine that neural networks are capable of learning secure symmetric encryption. In this experiment, three neural networks were created: Alice, Bob and Eve. Adhering to the idea of a generative adversarial network (GAN), the goal of the experiment was for Alice to send an encrypted message to Bob that Bob could decrypt, but the adversary, Eve, could not. Alice and Bob maintained an advantage over Eve, in that they shared a key used for encryption and decryption. In doing so, Google Brain demonstrated the capability of neural networks to learn secure encryption. === Image enhancement === In February 2017, Google Brain determined a probabilistic method for converting pictures with 8x8 resolution to a resolution of 32x32. The method built upon an already existing probabilistic model called pixelCNN to generate pixel translations. The proposed software utilizes two neural networks to make approximations for the pixel makeup of translated images. The first network, known as the "conditioning network," downsizes high-resolution images to 8x8 and attempts to create mappings from the original 8x8 image to these higher-resolution ones. The other network, known as the "prior network," uses the mappings from the previous network to add more detail to the original image. The resulting translated image is not the same image in higher resolution, but rather a 32x32 resolution estimation based on other existing high-resolution images. Google Brain's results indicate the possibility for neural networks to enhance images. === Google Translate === The Google Brain contributed to the Google Translate project by employing a new deep learning system that combines artificial neural networks with vast databases of multilingual texts. In September 2016, Google Neural Machine Translation (GNMT) was launched, an end-to-end learning framework, able to learn from a large number of examples. Previously, Google Translate's Phrase-Based Machine Translation (PBMT) approach would statistically analyze word by word and try to match corresponding words in other languages without considering the surrounding phrases in the sentence. But rather than choosing a replacement for each individual word in the desired language, GNMT evaluates word segments in the context of the rest of the sentence to choose more accurate replacements. Compared to older PBMT models, the GNMT model scored a 24% improvement in similarity to human translation, with a 60% reduction in errors. The GNMT has also shown significant improvement for notoriously difficult translations, like Chinese to English. While the introduction of the GNMT has increased the quality of Google Translate's translations for the pilot languages, it was very difficult to create such improvements for all of its 103 languages. Addressing this problem, the Google Brain Team was able to develop a Multilingual GNMT system, which extended the previous one by enabling translations between multiple languages. Furthermore, it allows for Zero-Shot Translations, which are translations between two languages that the system has never explicitly seen before. Google announced that Google Translate can now also translate without transcribing, using neural networks. This means that it is possible to translate speech in one language directly into text in another language, without first transcribing it to text. According to the Researchers at Google Brain, this intermediate step can be avoided using neural networks. In order for the system to learn this, they exposed it to many hours of Spanish audio together with the corresponding English text. The different layers of neural networks, replicating the human brain, were able to link the corresponding parts and subsequently manipulate the audio waveform until it was transformed to English text. Another drawback of the GNMT model is that it causes the time of translation to increase exponentially with the number of words in the sentence. This caused the Google Brain Team to add 2000 more processors to ensure the new translation process would still be fast and reliable. === Robotics === Aiming to improve traditional robotics control algorithms where new skills of a robot need to be hand-programmed, robotics researchers at Google Brain are developing machine learning techniques to allow robots to learn new skills on their own. They also attempt to develop ways for information sharing between robots so that robots can learn from each other during their learning process, also known as cloud robotics. As a result, Google has launched the Google Cloud Robotics Platform for developers in 2019, an effort to combine robotics, AI, and the cloud to enable efficient robotic automation through cloud-connected collaborative robots. Robotics research at Google Brain has focused mostly on improving and applying deep learning algorithms to enable robots to complete tasks by learning from experience, simulation, human demonstrations, and/or visual representations. For example, Google Brain researchers showed that robots can learn to pick and throw rigid objects into selected boxes by experimenting in an environment without being pre-programmed to do so. In another research, researchers trained robots to learn behaviors such as pouring liquid from a cup; robots learned from videos of human demonstrations recorded from multiple viewpoints. Google Brain researchers have collaborated with other companies and academic institutions on robotics research. In 2016, the Google Brain Team collaborated with researchers at X in a research on learning hand-eye coordination for robotic grasping. Their method allowed real-time robot control for grasping novel objec

    Read more →
  • Knowledge value chain

    Knowledge value chain

    A knowledge value chain is a sequence of intellectual tasks by which knowledge workers build their employer's unique competitive advantage and/or social and environmental benefit. As an example, the components of a research and development project form a knowledge value chain. Productivity improvements in a knowledge value chain may come from knowledge integration in its original sense of data systems consolidation. Improvements also flow from the knowledge integration that occurs when knowledge management techniques are applied to the continuous improvement of a business process or processes. The term first started coming into common use around 1999, appearing in management-related talks and papers. It was registered as a trademark in 2004 by TW Powell Co., a Manhattan company. Knowledge value chain processes Knowledge acquisition Knowledge storage Knowledge dissemination Knowledge application

    Read more →
  • Cleverpath AION Business Rules Expert

    Cleverpath AION Business Rules Expert

    Cleverpath AION Business Rules Expert (formerly Platinum AIONDS, and before that Trinzic AIONDS, and originally Aion) is an expert system and Business rules engine owned by Computer Associates by 2000. == History == The product was created around 1986 as "Aion" by the Aion company. In its initial release Aion was multi-platform and continues to be deliverable to the PC, Unixs, and Mainframe computer's. In addition it ties in seamlessly with a variety of databases including Oracle, Microsoft SQL Server, and ODBC. Aion was founded by Harry Reinstein, Larry Cohn, Garry Hallee, Scott Grinis, and others. From Scott Grinis's bio: Scott founded Aion, a company that developed expert systems and whose advanced inference engine and object technology were used by financial services and insurance firms to develop risk-scoring and underwriting applications. Harry Reinstein was quoted as saying: “Our biggest competitor was not AICorp, it was COBOL” Trinzic owned AION by 1993. A reference in a 1993 announcement indicates that Trinzic's formation was the result of a merger (paraphased): Trinzic set three development initiatives shortly after its formation from the merger of Aion Corp. and AICorp. The other initiatives -- adding SQL extensions to Aion/DS and evaluating the unbundling of some of that product's object-oriented programming capabilities -- are still active. Writing in 1993 Judith Hodges and Deborah Melewski give the date for the merger: Two rival artificial intelligence software vendors -- AICorp, Inc. and Aion Corp. -- merged in September 1992 to form Trinzic Corp. As part of the merger, redundant jobs were eliminated (20% of the combined work force), leaving a total work force of 245 employees worldwide. The new firm also boasted a combined installed base of more than 1,200 sites representing more than 10,000 software licenses. Although in the merger, technically AICorp bought Aion, as AICorp was a public company and Aion was still private, the reality was that Aion's leadership and technology subsumed AICorp's. Jim Gagnard, the CEO of Aion, became CEO of Trinzic and AICorp's flagship product, KBMS, was discontinued, while the Aion Development System continued to be enhanced and KBMS customers were assisted in converting to AIONDS, under the continued technical leadership of Garry Hallee and Scott Grinis. On August 1, 1994 Trinzic released version 6.4 of AIONDS saying, in part: Trinzic Corp., Palo Alto, Calif., has unveiled The Aion Development System (AionDS) Version 6.4, an upgrade to the company's development environment for building business process automation applications. Version 6.4 provides a visual development environment for Microsoft Windows or OS/2 PM applications using business rules. Trinzic was acquired by PLATINUM Technologies in 1995 which retained at least some of Trinzic's acquisitions Platinum Technologies was acquired by Computer Associates in 1999. CA changed the system's name to CA Aion Business Rules Expert" on or before 2009. It is currently (June 2011) at Release 11 on a wide range of supported platforms. == Applications using Aion == Aion has been used in a variety of industries including Energy, Insurance, Military, Aviation, and Banking. At one point an Aion expert system application written by Covia, LLC existed to do airport gate assignment. Colossus, a computer program, developed by Computer Sciences Corporation is the insurance industry’s leading expert system for assisting adjusters in the evaluation of bodily injury claims (aka "pain and suffering"). Colossus helps adjusters reduce variance in payouts on similar bodily injury claims through objective use of industry standard rules.

    Read more →
  • Resilience week

    Resilience week

    Resilience week is an annual symposium established to enable cross-disciplinary and role based discussions to advance strategies and research that engenders resilience in critical infrastructure systems and communities. Damaging storms, cyber attack and the interconnection of critical infrastructure systems can lead to cascading events that not only affect local but also across regions. However, many of these interdependencies are not easily recognized and obscure and complicate the mitigation of risk. The purpose of the symposia series is hence to facilitate best practice in managing critical infrastructure risks, by bringing together businesses, government and researchers. == Background == Originally organized in 2008 as a focus on the new research area of resilient control systems, including the disciplinary areas of control system, cyber-security, cognitive psychology and any number of critical infrastructure domains. Resilience has long been recognized as an area that requires not only the contributions of multiple disciplines or multidisciplinary participation, but interdisciplinary interaction where there is a common language and familiarity of the contributors to what other disciplines (and roles) contribute. The resulting interactions developed by Resilience Week and associated activities are intended to culture this sharing environment as a safe zone for inclusion; more importantly, an environment that lends to developing the new science and practice. As the attributes of resilience are complex, the contributions and topics for the event have included both the disciplinary and the project considerations, in keynotes, panels and research presentations. Keynotes have included senior leadership in the Department of Energy, Department of Defense, Department of Homeland Security, the National Science Foundation, and other agencies in addition to National Academy and professional organization fellows and senior industry leaders. Project panels and research presentations include emergent topics in resilience to climate change, cyber attack, damaging storms and the energy assurance. Topics Areas of focus have included: Control Systems Cyber Systems Cognitive Systems Communications Systems Communities and Infrastructure Project Focus Areas have included: Dependencies and Interdependencies Cyber Resilience for Operating Technology Commercializing Research and Development Building Critical Infrastructure Resilience through Distributed Energy Resources Energy Equity and Community Resilience Proceedings are developed for each year of the event, documenting the diversity of the research and engagements within these topical areas. == Impacts for the future == Since its inception, the Resilience Week community has evolved from one that primarily included only university researchers to one that includes many government laboratories, universities and private industries in the US and internationally. This type of collaboration forms a feedback loop that informs the research with the current needs and hones best practices. The future of the event is to further advance discussions that advance investment, recognize priorities and expedite technologies and tools to proactively address our energy future, in light of the natural and manmade challenges, and rationalizing the complex relationships that exist in critical infrastructure.

    Read more →
  • LipNet

    LipNet

    LipNet is a deep neural network for audio-visual speech recognition (ASVR). It was created by University of Oxford researchers Yannis Assael, Brendan Shillingford, Shimon Whiteson, and Nando de Freitas. The researchers stated that could match mouth movements to text with 93 percent accuracy, though it was criticized for its test using a limited dataset of words and grammar. It was used in Nvidia's autonomous "backseat driver" prototype Co-Pilot.

    Read more →
  • Reason maintenance

    Reason maintenance

    Reason maintenance is a knowledge representation approach to efficient handling of inferred information that is explicitly stored. Reason maintenance distinguishes between base facts, which can be defeated, and derived facts. As such it differs from belief revision which, in its basic form, assumes that all facts are equally important. Reason maintenance was originally developed as a technique for implementing problem solvers. It encompasses a variety of techniques that share a common architecture: two components—a reasoner and a reason maintenance system—communicate with each other via an interface. The reasoner uses the reason maintenance system to record its inferences and justifications of ("reasons" for) the inferences. The reasoner also informs the reason maintenance system which are the currently valid base facts (assumptions). The reason maintenance system uses the information to compute the truth value of the stored derived facts and to restore consistency if an inconsistency is derived. == Truth maintenance system == A truth maintenance system, or TMS, is a knowledge representation method for representing both beliefs and their dependencies and an algorithm called the "truth maintenance algorithm" that manipulates and maintains the dependencies. The name truth maintenance is due to the ability of these systems to restore consistency. A truth maintenance system maintains consistency between old believed knowledge and current believed knowledge in the knowledge base (KB) through revision. If the current believed statements contradict the knowledge in the KB, then the KB is updated with the new knowledge. It may happen that the same data will again be believed, and the previous knowledge will be required in the KB. If the previous data are not present, but may be required for new inference. But if the previous knowledge was in the KB, then no retracing of the same knowledge is needed. The use of TMS avoids such retracing; it keeps track of the contradictory data with the help of a dependency record. This record reflects the retractions and additions which makes the inference engine (IE) aware of its current belief set. == Algorithm == Each statement having at least one valid justification is made a part of the current belief set. When a contradiction is found, the statement(s) responsible for the contradiction are identified and the records are appropriately updated. This process is called dependency-directed backtracking. The TMS algorithm maintains the records in the form of a dependency network. Each node in the network is an entry in the KB (a premise, antecedent, or inference rule etc.) Each arc of the network represent the inference steps through which the node was derived. A premise is a fundamental belief which is assumed to be true. They do not need justifications. The set of premises are the basis from which justifications for all other nodes will be derived. == Justification == There are two types of justification for a node. They are: Support list [SL] Conditional proof (CP) == Examples == Many kinds of truth maintenance systems exist. Two major types are single-context and multi-context truth maintenance. In single context systems, consistency is maintained among all facts in memory (KB) and relates to the notion of consistency found in classical logic. Multi-context systems support paraconsistency by allowing consistency to be relevant to a subset of facts in memory, a context, according to the history of logical inference. This is achieved by tagging each fact or deduction with its logical history. Multi-agent truth maintenance systems perform truth maintenance across multiple memories, often located on different machines. de Kleer's assumption-based truth maintenance system (ATMS, 1986) was utilized in systems based upon KEE on the Lisp Machine. The first multi-agent TMS was created by Mason and Johnson. It was a multi-context system. Bridgeland and Huhns created the first single-context multi-agent system.

    Read more →
  • Computer Science Ontology

    Computer Science Ontology

    The Computer Science Ontology (CSO) is an automatically generated taxonomy of research topics in the field of Computer Science. It was produced by the Open University in collaboration with Springer Nature by running an information extraction system over a large corpus of scientific articles. Several branches were manually improved by domain experts. The current version (CSO 3.2) includes about 14K research topics and 160K semantic relationships. CSO is available in OWL, Turtle, and N-Triples. It is aligned with several other knowledge graphs, including DBpedia, Wikidata, YAGO, Freebase, and Cyc. New versions of CSO are regularly released on the CSO Portal. CSO is mostly used to characterise scientific papers and other documents according to their research areas, in order to enable different kinds of analytics. The CSO Classifier is an open-source python tool for automatically annotating documents with CSO. == Applications == Recommender Systems. Computing the semantic similarity of documents. Extracting metadata from video lecture subtitles. Performing bibliometrics analysis.

    Read more →
  • Backend as a service

    Backend as a service

    Backend as a service (BaaS), sometimes also referred to as mobile backend as a service (MBaaS), is a service for providing web app and mobile app developers with a way to easily build a backend to their frontend applications. Features available include user management, push notifications, and integration with social networking services. These services are provided via the use of custom software development kits (SDKs) and application programming interfaces (APIs). BaaS is a relatively recent development in cloud computing, with most BaaS startups dating from 2011 or later. Some of the most popular service providers are AWS Amplify and Firebase. == Purpose == Web and mobile apps require a similar set of features on the backend, including notification service, integration with social networks, and cloud storage. Each of these services has its own API that must be individually incorporated into an app, a process that can be time-consuming and complicated for app developers. BaaS providers form a bridge between the frontend of an application and various cloud-based backends via a unified API and SDK. Providing a consistent way to manage backend data means that developers do not need to redevelop their own backend for each of the services that their apps need to access, potentially saving both time and money. Although similar to other cloud-computing business models, such as serverless computing, software as a service (SaaS), infrastructure as a service (IaaS), and platform as a service (PaaS), BaaS is distinct from these other services in that it specifically addresses the cloud-computing needs of web and mobile app developers by providing a unified means of connecting their apps to cloud services. == Features == BaaS providers offer different set of features and backend tools. Some of the most common features include: Database management. Most BaaS solutions provide SQL and/or NoSQL database management services for applications. Developers can store their app data without deploying and managing databases themselves. BaaS usually provides client SDKs, REST and GraphQL APIs for the frontend to interact with databases. File storage. BaaS providers often offer storage solutions for media files, user uploads, and other binary data. Applications can upload, download, and delete files through provided SDKs and APIs. Authentication and authorization. Some BaaS offer authentication and authorization services that allow developers to easily manage app users. This includes user sign-up, login, password reset, social media login integration through OAuth, user group and permission management etc. Notification service. Some BaaS providers such as Firebase and AWS Amplify have notification services that can send custom emails to users and push native notifications on mobile platforms. This is especially useful for applications that need to send messages, alerts, and reminders. Cloud functions. Some BaaS allow developers to deploy and run serverless functions. The functions are usually stateless and can be triggered by various ways including HTTP requests, SDK invocation, background server events, and cloud scheduled executions. Different providers offer runtime support for different languages, some of the popular languages are JavaScript/TypeScript (Node.js, Deno), Python, Java/Kotlin. Cloud functions extend the potential and flexibility of BaaS by allowing developers to write custom functionalities for their apps, working in a way similar to a traditional REST API backend framework. Usage analytics. Analytics data about application usage is often included in BaaS. This allows developers to monitor user behaviors and make decisions correspondingly in marketing strategies and performance optimizations. UI design. Some BaaS providers, such as AWS Amplify and Backendless, offer user interface designing tools that help developers design the frontend UI of web and mobile apps. While this may be useful for small teams and individual developers, UI design assistance may not be conventional in BaaS as it goes beyond the scope of backend infrastructure. Real-Time. Real-time features in a BaaS platform ensure that data updates and synchronizations occur instantly across all clients, making changes immediately visible to users. This is crucial for applications like live chat and collaborative tools, using technologies like WebSockets to maintain continuous server-client connections. == Service providers == BaaS providers have a broad focus, providing SDKs and APIs that work for app development on multiple platforms with different technology stacks, such as JavaScript (for Web apps), Flutter, Java/Kotlin (for Android apps), Swift/Objective-C (for iOS/MacOS/WatchOS/TvOS apps), .NET (for Windows) and others. BaaS providers also come in different types, suiting developers of different needs. === Cloud-based BaaS === Most BaaS providers host backend platforms on their cloud servers. They also manage the infrastructure, security, and scalability of the platforms. Developers can access the backend services via a web interface or the provided APIs. Some examples of cloud-based BaaS include Firebase (hosted on Google Cloud Platform), AWS Amplify (hosted on Amazon Web Services), and Microsoft Azure Mobile Apps (hosted on Microsoft Azure). === Self-hosted BaaS === Self-hosted BaaS allow developers to host backend on their own servers, providing more flexibility and potential to customization compared to cloud-based BaaS, which often is more difficult to migrate from. However, developers are also in charge of managing the infrastructure, security, and scalability of their servers. === Mobile BaaS === Mobile backend as a service (MBaaS) is a type of BaaS specifically for applications deployed in mobile systems. While some references use MBaaS interchangeably for BaaS, BaaS can have a wider variety of support such as for web apps and desktop apps. == Business model == BaaS providers generate revenue from their services in various ways, often using a freemium model. Under this model, a client receives a certain number of free active users or API calls per month, and pays a fee for each user or call over this limit. Alternatively, clients can pay a set fee for a package which allows for a greater number of calls or active users per month. There are also flat fee plans that make the pricing more predictable. Some of the providers offer the unlimited API calls inside their free plan offerings. Another business model that has been used by a lot of BaaS providers is PAYG (pay as you go), which has a flexible cost based on developers' usage of database, storage, bandwidth, function calls, user numbers etc.

    Read more →
  • Simulation decomposition

    Simulation decomposition

    SimDec, or Simulation decomposition, is a hybrid uncertainty and sensitivity analysis method, for visually examining the relationships between the output and input variables of a computational model. SimDec maps multivariable scenarios onto the distribution of the model output. This visual analytics approach exposes the underlying nature of the model behavior, including its nonlinear and multivariate interaction effects. SimDec can be used in any range of science, engineering, and social domains. Existing applications include business and environmental issues. == Method == SimDec operates on Monte Carlo simulation (or measured) data where both output and input values are recorded. At least one thousand observations (or simulated iterations) are typically recommended to preserve the readability of the resulting histograms. An outline of the decomposition algorithm, which is readily available in multiple programming languages, proceeds as follows: Select the input variables for decomposition. One can use sensitivity indices (see variance-based sensitivity analysis) to define the most influential variables for decomposition or choose them manually according to the decision-problem context (for example, only those input variables that the decision-maker can act upon). Two to three input variables, ordered by decreasing value of their sensitivity indices, usually provide the most meaningful decomposition results. Divide the inputs into states. The numeric ranges of the inputs are split into several intervals with an equal number of observations in each. For categorical variables, the categories represent states. Form scenarios. All combinations of states of the selected input variables produce unique scenarios or subsets of the data. For example, if the range of X2 is divided into low, medium and high, and X3 takes values of 1 or 2, six scenarios are formed: (i) X2 low & X3 = 1, (ii) X2 low & X3 = 2, (iii) X2 medium & X3 = 1, (iv) X2 medium & X3 = 2, (v) X2 high & X3 = 1, and (vi) X2 high & X3 = 2. Assign scenarios to each output value. The simulation data is used to define the scenario index for each simulation run. For example, if an X2 value falls into the low state and X3 is equal to 2, the corresponding scenario, defined in Step 3, is (ii). Color-code the output distribution. When all output values are assigned scenario indices, they are plotted as series in a stacked histogram, visually separated by color-coding. For ease of visual perception, the states of the most influential input variable are assigned distinct colors, and all the remaining partitions take shades of those colors (see Figure). All of these steps can be run automatically on the given data using the open-source SimDec packages currently available in Python, R, Julia, and Matlab. A SimDec template in Excel runs a Monte Carlo simulation of a spreadsheet model but possesses only a manual option for input selection. == How to read SimDec == === Histogram === Histogram is an approximate representation of the distribution of numerical data. Its horizontal axis shows the range of the variable of interest, and its vertical axis denotes count, also called frequency, or, if divided by the total number of data points, probability. The distribution alone can supply only limited information about the data – its minimum, maximum, and shape (where the most of data occurs). === Judging the importance of inputs === If an input variable has no effect on the output, its states (e.g., low & high) would lie on top of each other on the SimDec histogram, occupying fully overlapping ranges of the output. If an input variable has a strong effect and explains most of the variance of the output, the border between its states on the SimDec histogram would be vertical. Such visualization has an important decision-making implication – e.g., if the high state of X can be achieved, it would guarantee a certain range of Y. All cases in-between with low-to-strong effects would show a diagonal border between the states. The less they overlap, the larger the effect of X on Y. While the horizontal displacement of sub-distributions on the SimDec histogram is the key to interpreting the results, the vertical disposition of sub-distributions is just a technical matter of the order of plotting the series of the stacked histogram. === Exploring the interaction of inputs === When two or more input variables are used for decomposition, it becomes possible to examine their joint effects. A schematic visualization portrays how different types of joint effects of input variables on the output appear on SimDec visualization. Understanding the nature of interaction effects in a computational model and its behavior in general is crucial for effective decision-making. == Limitations == The SimDec method has several limitations: It is based on Monte Carlo simulation and thus requires running a computational model a thousand of times or more. To models that take hours to evaluate once, it would be impossible to use SimDec (unless a supercomputer and/or large of time are available). SimDec is based on a histogram, thus, for binary or categorical output variables, the visualization would be very limited (e.g., only a few bins). The more input variables one selects for the decomposition, the less readable the histogram becomes. Only cases with two and three input variables are presented in.

    Read more →
  • Jensen Huang

    Jensen Huang

    Jen-Hsun "Jensen" Huang (Chinese: 黃仁勳; Wade–Giles: Huáng Jén-hsūn; Tâi-lô: N̂g Jîn-hun; born February 17, 1963) is a Taiwanese and American business executive and electrical engineer who is the founder, president, and CEO of Nvidia, the world's most valuable company. As of 2026, Forbes estimates his net worth at over US$200 billion, making him the seventh-wealthiest individual in the world. The son of Taiwanese immigrants, Huang spent his childhood in Taiwan and Thailand before moving to the United States, where he was a student in Kentucky and Oregon. After earning a master's degree from Stanford University, Huang launched Nvidia in 1993 from a Denny's restaurant in San Jose, California, at age 30 and has remained its president and CEO ever since. He led the company out of near-bankruptcy during the 1990s and oversaw its expansion into GPU production, high-performance computing, and artificial intelligence (AI). Under Huang, Nvidia experienced rapid growth during the AI boom, becoming the first company to reach a market capitalization of over $5 trillion in October 2025. In 2021 and 2024, Time magazine included Huang in their list of the most influential people. In 2025, he was named as one of the "Architects of AI" for Time's Person of the Year. == Early life and education == Huang was born in Taipei, Taiwan, on February 17, 1963, and moved to the southern city of Tainan as a child. He is the younger of two sons of Huang Hsing-tai, a chemical engineer at an oil refinery, and Lo Tsai-hsiu, a schoolteacher. They were a middle-class Taiwanese family that relocated often, and were native speakers of Taiwanese Hokkien. Each day, Jensen's mother randomly selected 10 words from the dictionary to teach her sons English. When he was five years old, Huang's family moved to Thailand to support his father's refinery career and remained there for approximately four years. He attended Ruamrudee International School while in Bangkok. In the late 1960s, Hsing-tai traveled from Taiwan to New York City to train under an air conditioning company and, after returning home, resolved to send his sons to the United States. At age nine, Jensen, despite not yet being able to speak English fluently, was sent by his parents to live in the United States. He and his older brother moved in 1973 to live with an uncle in Tacoma, Washington, escaping widespread social unrest in Thailand. Both Huang's aunt and uncle were recent immigrants to Washington state; they accidentally enrolled him and his brother in the Oneida Baptist Institute, a religious reform academy in Kentucky for troubled youth, mistakenly believing it to be a prestigious boarding school. In order to afford the academy's tuition, Jensen's parents sold nearly all their possessions. When he was 10 years old, Huang lived with his older brother in the Oneida boys' dormitory. Each student was expected to work every day, and his brother was assigned to perform manual labor on a nearby tobacco farm. Because he was too young to attend classes at the reform academy, Huang was educated at a separate public school—the Oneida Elementary school in Oneida, Kentucky—arriving as "an undersized Asian immigrant with long hair and heavily accented English" and was frequently bullied and beaten. In Oneida, Huang cleaned toilets every day, learned to play table-tennis, joined the swimming team, and appeared in Sports Illustrated at age 14. He taught his illiterate roommate, a "17-year-old covered in tattoos and knife scars," how to read in exchange for being taught how to bench press. In 2002, Huang said he remembered his life in Kentucky "more vividly than just about any other". Two years after Huang arrived in Oneida, his parents moved to the United States and settled in Beaverton, Oregon, after which the brothers withdrew from school in Kentucky to live back with them. As a teenager, Huang attended Aloha High School in Aloha, Oregon, where he excelled academically. He skipped two grades, graduated at age 16, and became a nationally ranked table-tennis player in addition to being a member of its mathematics, computer, and science clubs. In 1977, the school purchased an Apple II computer. Huang used the machine to play Super Star Trek, a text-based game, and to program in BASIC, creating his own version of Snake. Beginning at age 15, Huang got his first job working the graveyard shift at a local Denny's restaurant as a dishwasher, busboy, and waiter from 1978 to 1983. After high school, he chose to enroll at Oregon State University due to its low in-state tuition. He studied electrical engineering and graduated in 1984 with a bachelor's degree with highest honors. Huang later recalled, "I was the youngest kid in school, in class" and the only student who "looked like a child". Years later, while working as a microchip designer in Silicon Valley, he concurrently pursued graduate night classes at Stanford University, where he earned a master's degree in electrical engineering in 1992. == AMD and LSI Logic == After graduating from college, Huang was a microchip designer in Silicon Valley. He was recruited for positions at Texas Instruments, Advanced Micro Devices (AMD), and LSI Logic, ultimately choosing the California-based AMD due to already being familiar with the company. Huang designed AMD microprocessors while simultaneously attending Stanford and raising his two children. However, when he heard of new chip design processes at LSI Logic, Huang left AMD to assume a role as a technical officer at the LSI Corporation, working under a startup company, Sun Microsystems, where he met engineers Chris Malachowsky and Curtis Priem. LSI was in contract with Sun Microsystems and had introduced Huang to Malachowsky and Priem, who were working on a new graphics accelerator card. While the three produced the card's manufacturing process, the relationship between Malachowsky and Priem became strained as the two disputed the chip's design, leading to infighting; according to Malachowsky, they "broke every tool that LSI Logic had in their standard portfolio". In 1989, Huang, Malachowsky, and Priem finalized the accelerator, which they called the "GX graphics engine". GX was a widespread financial success; the sales of the graphics engine contributed to Sun Microsystem's revenue increasing from $262 million in 1987 to $656 million in 1990, and Huang was promoted to be the director of LSI's CoreWare, a division that manufactured chips for hardware vendors. == Nvidia == === Founding (1993) === When business began to slow for Sun Microsystems after 1990, Huang, along with Priem and Malachowsky, each resigned their jobs to pursue a venture together in making graphics chips for PC games. They initially named their new company "NVision" until Huang suggested that the company be named "Nvidia" based on the Latin word invidia, as Priem wanted competitors to turn "green with envy". They eventually dropped the "i" to honor the NV1 chip that they were then developing. The three met frequently in 1992 at a Denny's roadside diner in East San Jose to formulate a business plan. Huang chose for them to meet at Denny's due to his prior work experience at the restaurant chain and because it was "quieter than home and had cheap coffee". The three founded the company during one meeting at a breakfast booth at the diner. To formally incorporate the company, Huang found a lawyer, James Gaither of Cooley Godward, who demanded the $200 in cash in Huang's pockets to capitalize the company. After that meeting, Huang went back to Priem and Malachowsky to ask each of them for $200 for their respective shares of the company, which meant that Nvidia's initial capital was $600. On April 5, 1993, Huang personally signed Nvidia's original articles of incorporation into effect. Although he left LSI, Huang remained in good standing with the company and was able to secure funding for Nvidia from LSI's CEO, Wilfred Corrigan, who introduced Huang to venture capitalist Don Valentine. An account cited how Huang's presentation pitch went badly. Valentine, the leader of Sequoia Capital, chose to invest in Nvidia through Corrigan's support, as did Sutter Hill Ventures. The funding enabled Nvidia to begin development efforts toward its first chip and to begin paying wages for its employees. By the first day of operation, Huang was made Nvidia's president and CEO. Even though Huang, at age 30, was younger than Priem and Malachowsky, both Priem and Malachowsky believed that he was prepared to be CEO. According to Priem, "we basically deferred to Jensen on day one" and told Huang, "you're in charge of running the company—all the stuff Chris and I don't know how to do". === President and CEO (1993–present) === As of 2024, Huang has been Nvidia's chief executive for over three decades, a tenure described by The Wall Street Journal as "almost unheard of in fast-moving Silicon Valley". He owns 3.6% of Nvidia's stock, which went public in 1999. He earned US$24.6 million as CEO i

    Read more →
  • DAYDREAMER

    DAYDREAMER

    DAYDREAMER is a goal-based agent and cognitive architecture developed at the University of California, Los Angeles by Erik T. Mueller and Michael G. Dyer beginning in 1983. The system models the human stream of thought and how it is triggered and directed by emotions, simulating human daydreaming. Taking situational descriptions as input, DAYDREAMER produces English-language daydreams as output and encodes new daydreams, plans, and planning strategies for later reuse. The program comprises five components: a scenario generator based on relaxed planning, a dynamic episodic memory, a collection of personal goals and control goals, an emotion component, and domain knowledge of interpersonal relations and everyday occurrences. The source code was released under a free software license in 2015. == History == Erik Mueller began DAYDREAMER in 1983 while he was a doctoral student in the Artificial Intelligence Laboratory of the Computer Science Department at the University of California, Los Angeles, studying under Michael G. Dyer. Initial development of the project was supported by a grant from the W. M. Keck Foundation with matching funds from the UCLA School of Engineering and Applied Sciences. Additionally, Mueller was supported by an Atlantic Richfield Doctoral Fellowship and Dyer by an IBM Faculty Development Award. The first published descriptions of the program appeared in 1985 at the Ninth International Joint Conference on Artificial Intelligence in Los Angeles and at the Seventh Annual Conference of the Cognitive Science Society in Irvine. Work on the program continued, and a book, Daydreaming in Humans and Machines, was published by Ablex Publishing in 1990. The program was implemented on top of GATE, a knowledge-representation and inference substrate developed by Mueller and Uri Zernik at UCLA, and was originally written in T, a dialect of Scheme. In 2015, Mueller released the DAYDREAMER source code, version 3.5, a Common Lisp rewrite of the original T implementation, on GitHub under the GNU General Public License version 2. The release comprised approximately 12,000 lines of Common Lisp code, along with the GATE knowledge-representation substrate on which DAYDREAMER had originally been built. == Architecture == The program operates in two modes. In daydreaming mode it daydreams continuously until interrupted, while performance mode allows it to demonstrate behavior it has learned through daydreaming. === Emotion and control goals === Emotions and daydreaming form a feedback loop for DAYDREAMER. Emotions activate goals that produce daydreams, and the resulting daydreams modify existing emotions and trigger new ones, which prompt subsequent daydreaming. Recall of a goal success produces a positive emotion whereas recall of a goal failure produces a negative emotion. Emotions activate a set of goals, called control goals, which direct the course of a daydream. The program has four control goals. "Rationalization" generates reasons why an unsatisfactory outcome is in fact acceptable, in order to reduce a negative emotion and maintain self-esteem. "Revenge" is activated by anger when a failure is caused by another and reduces negative emotion through imagined retaliation. "Failure/success reversal" imagines alternative scenarios in which a failure was prevented or a success did not occur as a means of learning planning strategies for future situations. "Preparation" generates hypothetical future scenarios in order to rehearse plans and actions for events that have not yet occurred. === Scenario generator and relaxed planning === The scenario generator produces the sequence of events that make up a daydream. It operates under multiple, often conflicting personal goals rather than pursuing a single goal, applies relaxation rules that permit the generation of non-realistic scenarios, and it draws on episodic memory of past experiences both as subject matter and as a source of planning knowledge. The personal goals that guide the scenario generator include health, food, sex, friendship, love, possessions, self-esteem, social esteem, enjoyment, and achievement. These goals are organized into a goal tree that specifies their relative importance at any given time. Relaxation rules allow the program to set aside its ordinary constraints when generating a scenario. The four constraints that may be relaxed are the behavior of others, the daydreamer's own attributes, physical constraints, and social constraints. The degree of relaxation varies with the active control goal. For example a failure-reversal goal aimed at alternatives uses a low level of relaxation, whereas a revenge goal aimed at a retaliation uses a high level. === Episodic memory and analogy === DAYDREAMER's episodic memory stores its personal and vicarious experiences along with the daydreams it generates. The memory is described as dynamic because it is continually modified during daydreaming such that previously daydreamed episodes become available alongside real ones. As it daydreams, the program indexes daydreams, future plans or actions, and planning strategies into memory. Episodes are organized and retrieved using surface-level similarities, emotions, abstract themes, and Plot Units which are abstract configurations of positive and negative outcomes developed by Wendy Lehnert. A recalled episode is adapted to the current situation through analogy, which requires less effort than generating an equivalent scenario from scratch. == Sample output == In the sample experience from the source code, called LOVERS1, DAYDREAMER begins from an initial situation in which it has a job, is not romantically involved, and is at home. Starting in daydreaming mode, it activates a top-level goal to be in a romantic relationship because it is not currently in one, and a positive motivating emotion of interest becomes associated with that goal. The program then activates a goal to be entertained and pursues seeing a film as a way to achieve it. Facts asserted into memory are converted to English and produced as output, such as "I want to be going out with someone" and "I have to go see a movie". == Reception and influence == DAYDREAMER has been cited in research on computational models of creativity, emotion, and narrative. Linda Wills and Janet Kolodner cite the program as an example of work on opportunism in their study of serendipitous recognition in design. Joseph Bates, A. Bryan Loyall, and W. Scott Reilly of the Carnegie Mellon Oz Project cite DAYDREAMER among prior work in their description of an architecture combining action, emotion, and social behavior. Rafael Pérez y Pérez, Ricardo Sosa, and Christian Lemaitre cite Mueller's DAYDREAMER as one of the few computer models at the time to model daydreaming during the creative process. Jichen Zhu and D. Fox Harrell likewise cite the program in their work on imagining and agency in generative interactive narrative.

    Read more →
  • Closest point method

    Closest point method

    The closest point method (CPM) is an embedding method for solving partial differential equations on surfaces. The closest point method uses standard numerical approaches such as finite differences, finite element or spectral methods in order to solve the embedding partial differential equation (PDE) which is equal to the original PDE on the surface. The solution is computed in a band surrounding the surface in order to be computationally efficient. In order to extend the data off the surface, the closest point method uses a closest point representation. This representation extends function values to be constant along directions normal to the surface. == Definitions == Closest Point function: Given a surface S , c p ( x ) {\displaystyle {\mathcal {S}},cp(\mathbf {x} )} refers to a (possibly non-unique) point belonging to S {\displaystyle {\mathcal {S}}} , which is closest to x {\displaystyle \mathbf {x} } [SE]. Closest point extension: Let S {\displaystyle {\mathcal {S}}} , be a smooth surface in R d {\displaystyle \mathbb {R} ^{d}} . The closest point extension of a function u : S → R {\displaystyle u:{\mathcal {S}}\rightarrow \mathbb {R} } , to a neighborhood Ω {\displaystyle \Omega } of S {\displaystyle {\mathcal {S}}} , is the function v : Ω → R {\displaystyle v:\Omega \rightarrow \mathbb {R} } , defined by v ( x ) = u ( c p ( x ) ) {\displaystyle v(\mathbf {x} )=u(cp(\mathbf {x} ))} . == Closest point method == Initialization consists of these steps [EW]: If it is not already given, a closest point representation of the surface is constructed. A computational domain is chosen. Typically this is a band around the surface. Replace surface gradients by standard gradients in R 3 {\displaystyle \mathbb {R} ^{3}} . Solution is initialized by extending the initial surface data on to the computational domain using the closest point function. After initialization, alternate between the following two steps: Using the closest point function, extend the solution off the surface to the computational domain. Compute the solution to the embedding PDE on a Cartesian mesh in the computational domain for one time step. == Banding == The surface PDE is extended into R 3 {\displaystyle \mathbb {R} ^{3}} however it is only necessary to solve this new PDE near the surface. Hence, we solve the PDE in a band surrounding the surface for efficient computational purposes. Ω c x : ‖ x − c p ( x ) ‖ 2 ≤ λ {\displaystyle \Omega _{c}{x:\|x-cp(x)\|_{2}\leq \lambda }} where λ {\displaystyle \lambda } is the bandwidth. == Example: Heat equation on a circle == Using initial profile u S ( θ , t ) = sin ⁡ ( θ ) {\displaystyle u_{S}(\theta ,t)=\sin(\theta )} leads to the solution u S ( θ , t ) = exp ⁡ ( − t ) sin ⁡ ( θ ) {\displaystyle u_{S}(\theta ,t)=\exp(-t)\sin(\theta )} for the heat equation. Forward Euler time-stepping is used with relation Δ t = 0.1 Δ x 2 {\displaystyle \Delta t=0.1\Delta x^{2}} and degree-four interpolation polynomials for the interpolations. Second-order centered differences are used for the spatial discretization. The CPM results in the expected second order error in the solution u {\displaystyle u} . == Applications == The closest point method can be applied to various PDEs on surfaces. Reaction–diffusion problems on point clouds [RD], eigenvalue problems [EV], and level set equations [LS] are a few examples.

    Read more →
  • Mojo (programming language)

    Mojo (programming language)

    Mojo is an in-development proprietary programming language based on Python available for Linux and macOS. Mojo aims to combine the usability of a high-level programming language, specifically Python, with the performance of a system programming language such as C++, Rust, and Zig. As of October 2025, the Mojo compiler is closed source with an open source standard library. Modular, the company behind Mojo, has stated an intent to open source the Mojo language, committing to open-source Mojo in "fall 2026". Mojo builds on the Multi-Level Intermediate Representation (MLIR) compiler software framework, instead of directly on the lower level LLVM compiler framework like many languages such as Julia, Swift, C++, and Rust. MLIR is a newer compiler framework that allows Mojo to exploit higher level compiler passes unavailable in LLVM alone, and allows Mojo to compile down and target more than only central processing units (CPUs), including producing code that can run on graphics processing units (GPUs), Tensor Processing Units (TPUs), application-specific integrated circuits (ASICs) and other accelerators. It can also often more effectively use certain types of CPU optimizations directly, like single instruction, multiple data (SIMD) with minor intervention by a developer, as occurs in many other languages. According to Jeremy Howard of fast.ai, Mojo can be seen as "syntax sugar for MLIR" and for that reason Mojo is well optimized for applications like artificial intelligence (AI). == Origin and development history == The Mojo programming language was created by Modular Inc, which was founded by Chris Lattner, the original architect of the Swift programming language and LLVM, and Tim Davis, a former Google employee. The intention behind Mojo is to bridge the gap between Python’s ease of use and the fast performance required for cutting-edge AI applications. According to public change logs, Mojo development goes back to 2022. In May 2023, the first publicly testable version was made available online via a hosted playground. By September 2023 Mojo was available for local download for Linux and by October 2023 it was also made available for download on Apple's macOS. In March 2024, Modular open sourced the Mojo standard library and started accepting community contributions under the Apache 2.0 license. == Features == Mojo was created for an easy transition from Python. The language has syntax similar to Python's, with inferred static typing, and allows users to import Python modules. It uses LLVM and MLIR as its compilation backend. The language also intends to add a foreign function interface to call C/C++ and Python code. The language is not source-compatible with Python 3, only providing a subset of its syntax, e.g. missing the global keyword, list and dictionary comprehensions, and support for classes. Further, Mojo also adds features that enable performant low-level programming: fn for creating typed, compiled functions and "struct" for memory-optimized alternatives to classes. Mojo structs support methods, fields, operator overloading, and decorators. The language also provides a borrow checker, an influence from Rust. Mojo def functions use value semantics by default (functions receive a copy of all arguments and any modifications are not visible outside the function), while Python functions use reference semantics (functions receive a reference on their arguments and any modification of a mutable argument inside the function is visible outside). The language is not currently open source, but it is planned to be made open source in the future. Modular has since committed to open-sourcing the Mojo language in "fall 2026". == Programming examples == In Mojo, functions can be declared using both fn (for performant functions) or def (for Python compatibility). Basic arithmetic operations in Mojo with a def function: and with an fn function: The manner in which Mojo employs var and let for mutable and immutable variable declarations respectively mirrors the syntax found in Swift. In Swift, var is used for mutable variables, while let is designated for constants or immutable variables. Variable declaration and usage in Mojo:

    Read more →
  • Catastrophic interference

    Catastrophic interference

    Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to abruptly and drastically forget previously learned information upon learning new information. Neural networks are an important part of the connectionist approach to cognitive science. The issue of catastrophic interference when modeling human memory with connectionist models was originally brought to the attention of the scientific community by research from McCloskey and Cohen (1989), and Ratcliff (1990). It is a radical manifestation of the 'sensitivity-stability' dilemma or the 'stability-plasticity' dilemma. Specifically, these problems refer to the challenge of making an artificial neural network that is sensitive to, but not disrupted by, new information. Lookup tables and connectionist networks lie on the opposite sides of the stability plasticity spectrum. The former remains completely stable in the presence of new information but lacks the ability to generalize, i.e. infer general principles, from new inputs. On the other hand, connectionist networks like the standard backpropagation network can generalize to unseen inputs, but they are sensitive to new information. Backpropagation models can be analogized to human memory insofar as they have a similar ability to generalize, but these networks often exhibit less stability than human memory. Notably, these backpropagation networks are susceptible to catastrophic interference. This is an issue when modelling human memory, because unlike these networks, humans typically do not show catastrophic forgetting. == Discovery == The term catastrophic interference was originally coined by McCloskey and Cohen (1989) but was also brought to the attention of the scientific community by research from Ratcliff (1990). === The Sequential Learning Problem: McCloskey and Cohen (1989) === McCloskey and Cohen (1989) noted the problem of catastrophic interference during two different experiments with backpropagation neural network modelling. Experiment 1: Learning the ones and twos addition facts In their first experiment they trained a standard backpropagation neural network on a single training set consisting of 17 single-digit ones problems (i.e., 1 + 1 through 9 + 1, and 1 + 2 through 1 + 9) until the network could represent and respond properly to all of them. The error between the actual output and the desired output steadily declined across training sessions, which reflected that the network learned to represent the target outputs better across trials. Next, they trained the network on a single training set consisting of 17 single-digit twos problems (i.e., 2 + 1 through 2 + 9, and 1 + 2 through 9 + 2) until the network could represent, respond properly to all of them. They noted that their procedure was similar to how a child would learn their addition facts. Following each learning trial on the twos facts, the network was tested for its knowledge on both the ones and twos addition facts. Like the ones facts, the twos facts were readily learned by the network. However, McCloskey and Cohen noted the network was no longer able to properly answer the ones addition problems even after one learning trial of the twos addition problems. The output pattern produced in response to the ones facts often resembled an output pattern for an incorrect number more closely than the output pattern for a correct number. This is considered to be a drastic amount of error. Furthermore, the problems 2+1 and 1+2, which were included in both training sets, even showed dramatic disruption during the first learning trials of the twos facts. Experiment 2: Replication of Barnes and Underwood (1959) study In their second connectionist model, McCloskey and Cohen attempted to replicate the study on retroactive interference in humans by Barnes and Underwood (1959). They trained the model on A-B and A-C lists and used a context pattern in the input vector (input pattern), to differentiate between the lists. Specifically the network was trained to respond with the right B response when shown the A stimulus and A-B context pattern and to respond with the correct C response when shown the A stimulus and the A-C context pattern. When the model was trained concurrently on the A-B and A-C items then the network readily learned all of the associations correctly. In sequential training the A-B list was trained first, followed by the A-C list. After each presentation of the A-C list, performance was measured for both the A-B and A-C lists. They found that the amount of training on the A-C list in Barnes and Underwood study that lead to 50% correct responses, lead to nearly 0% correct responses by the backpropagation network. Furthermore, they found that the network tended to show responses that looked like the C response pattern when the network was prompted to give the B response pattern. This indicated that the A-C list apparently had overwritten the A-B list. This could be likened to learning the word dog, followed by learning the word stool and then finding that you think of the word stool when presented with the word dog. McCloskey and Cohen tried to reduce interference through a number of manipulations including changing the number of hidden units, changing the value of the learning rate parameter, overtraining on the A-B list, freezing certain connection weights, changing target values 0 and 1 instead 0.1 and 0.9. However, none of these manipulations satisfactorily reduced the catastrophic interference exhibited by the networks. Overall, McCloskey and Cohen (1989) concluded that: at least some interference will occur whenever new learning alters the weights involved in representing old learning the greater the amount of new learning, the greater the disruption in old knowledge interference was catastrophic in the backpropagation networks when learning was sequential but not concurrent === Constraints Imposed by Learning and Forgetting Functions: Ratcliff (1990) === Ratcliff (1990) used multiple sets of backpropagation models applied to standard recognition memory procedures, in which the items were sequentially learned. After inspecting the recognition performance models he found two major problems: Well-learned information was catastrophically forgotten as new information was learned in both small and large backpropagation networks. Even one learning trial with new information resulted in a significant loss of the old information, paralleling the findings of McCloskey and Cohen (1989). Ratcliff also found that the resulting outputs were often a blend of the previous input and the new input. In larger networks, items learned in groups (e.g. AB then CD) were more resistant to forgetting than were items learned singly (e.g. A then B then C...). However, the forgetting for items learned in groups was still large. Adding new hidden units to the network did not reduce interference. Discrimination between the studied items and previously unseen items decreased as the network learned more. This finding contradicts studies on human memory, which indicated that discrimination increases with learning. Ratcliff attempted to alleviate this problem by adding 'response nodes' that would selectively respond to old and new inputs. However, this method did not work as these response nodes would become active for all inputs. A model which used a context pattern also failed to increase discrimination between new and old items. == Proposed solutions == The main cause of catastrophic interference seems to be overlap in the representations at the hidden layer of distributed neural networks. In a distributed representation, each input tends to create changes in the weights of many of the nodes. Catastrophic forgetting occurs because when many of the weights where "knowledge is stored" are changed, it is unlikely for prior knowledge to be kept intact. During sequential learning, the inputs become mixed, with the new inputs being superimposed on top of the old ones. Another way to conceptualize this is by visualizing learning as a movement through a weight space. This weight space can be likened to a spatial representation of all of the possible combinations of weights that the network could possess. When a network first learns to represent a set of patterns, it finds a point in the weight space that allows it to recognize all of those patterns. However, when the network then learns a new set of patterns, it will move to a place in the weight space for which the only concern is the recognition of the new patterns. To recognize both sets of patterns, the network must find a place in the weight space suitable for recognizing both the new and the old patterns. Below are a number of techniques which have empirical support in successfully reducing catastrophic interference in backpropagation neural networks: === Orthogonality === Many of the early techniques in reducing representational overlap involved making either the input vecto

    Read more →