AI Email Edit

AI Email Edit — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

  • ALL-IN-1

    ALL-IN-1

    ALL-IN-1 was an office automation product developed and sold by Digital Equipment Corporation in the 1980s. It was one of the first purchasable off the shelf electronic mail products. It was later known as Office Server V3.2 for OpenVMS Alpha and OpenVMS VAX systems before being discontinued. == Overview == ALL-IN-1 was advertised as an office automation system including functionality in Electronic Messaging, Word Processing and Time Management. It offered an application development platform and customization capabilities that ranged from scripting to code-level integration. ALL-IN-1 was designed and developed by Skip Walter, John Churin and Marty Skinner from Digital Equipment Corporation who began work in 1977. Sheila Chance was hired as the software engineering manager in 1981. The first version of the software, called CP/OSS, the Charlotte Package of Office System Services, named after the location of the developers, was released in May 1982. In 1983, the product was renamed ALL-IN-1 and the Charlotte group continued to develop versions 1.1 through 1.3. Digital then made the decision to move most of the development activity to its central engineering facility in Reading, United Kingdom, where a group there took responsibility for the product from version 2.0 (released in field test in 1984 and to customers in 1985) onward. The Charlotte group continued to work on the Time Management subsystem until version 2.3 and other contributions were made from groups based in Sophia Antipolis, France (System for Customization Management and the integration with VAX Notes), Reading (Message Router and MAILbus), and Nashua, New Hampshire (FMS). ALL-IN-1 V3.0 introduced shared file cabinets and the File Cabinet Server (FCS) to lay the foundation for an eventual integration with TeamLinks, Digital's PC office client. Previous integrations with PCs included PC ALL-IN-1, a DOS-based product introduced in 1989 that never proved popular with customers. Bob Wyman was the first product manager. He oversaw the growth of the product culminating in over $2 billion per year in revenue and market leadership in the proprietary office automation sector. Other consultants from Digital Equipment Corporation involved include Frank Nicodem, Donald Vickers and Tony Redmond.

    Read more →
  • Rohit Chadda

    Rohit Chadda

    Rohit Chadda (born 26 August 1982) is an Indian investment banker and entrepreneur, who is the President & COO of Times Network. He leads the tech business portfolio and AI transformation of Times Group covering verticals like media tech, OTT, fintech, health tech, edu tech, ecommerce, gaming and sports. Previously, CEO of the digital business at Essel Group (Zee Entertainment, Zee Media and DNA), he was the co-founder of online food ordering platform Foodpanda. He is also the founder of omni-channel digital payments platform PayLo. He has been attributed for the turnaround of Zee Digital driving 4x growth in 2 years and bringing Zee's digital business to the second position on ComScore from ninth position making Zee the second largest digital media group in India. He has been featured among Top Tech CEOs of the decade (2010–2020) in India and was featured among Fortune 40 under 40 in 2015. == Education and early career == Chadda graduated from Delhi Technological University (formerly Delhi College of Engineering) with a degree in computer engineering and worked as a software engineer for Computer Sciences Corporation. In 2007 he joined Indian Institute of Management Calcutta to do his MBA after which he worked at Merrill Lynch as an investment banker in United Kingdom. He took an internal transfer to India in 2011. == Career == === Foodpanda === Chadda began his career in 2012 when he co-founded foodpanda. foodpanda expanded to around 40 countries before being bought by Delivery Hero. Before foodpanda got popular, he joked that he delivered pizza for a living. foodpanda had raised a total investment of over US$300 million till 2015. Chadda in the middle of 2015 stepped down from day-to-day responsibilities at Foodpanda to launch his digital payments startup. Foodpanda was acquired by its global competitor Delivery Hero in 2016. === Paylo === In 2015, he launched an omni-channel digital payments platform PayLo which acquired the in-restaurant payments app Ruplee in March 2016 for an undisclosed sum. PayLo was successful in the wake of demonetisation in India and expanded pan-India before being acquired by Immortal Technologies. Chadda believes that execution is more important than the idea to make a startup successful and the key challenge for experienced professionals to work in a startup environment is to unlearn what they have previously learned. PayLo acquired Ruplee before being itself acquired by Immortal Technologies. === Zee Group === Chadda took over as CEO of digital publishing of Zee Group in May 2019. Since 2017, he had led global product and strategy for Zee Group launching ZEE5, the flagship OTT of Zee Entertainment, across 170+ countries. Since June 2019, Zee Digital, the online arm of the Zee group, has registered the highest growth year-on-year among the top media publishers in India. Times Internet Limited, Network 18 Group, and India Today Group have grown by 45%, 21%, and 22% respectively from June 2020 over June 2019 while Zee Digital witnessed a growth of 123% over the same period. Zee Digital achieved its first milestone in September 2019 by crossing 100 million unique monthly visitors and was ranked 6th in the news and information category on ComScore India rankings at the time. Later in the month of March 2020 it crossed 150 million unique monthly visitors mark moving to 4th position. Further in May 2020 Zee Digital moved to 3rd position by crossing 185 million unique monthly visitors mark before finally ranking 2nd position in June 2020 in the ComScore rankings among all digital media groups in India. Chadda has led the transformation of the business of Zee Digital by scaling it to over 200 million users from 60 million users making it the second-largest digital media group in India. He attributes the growth from rank 9 to rank 2 in one year to the data and technology driven approach to content and the focus on vernacular languages. During his tenure, Zee Digital launched 8 new brand websites and 3 new languages to expand the product portfolio to 20 brands and 12 languages. During the US elections in November 2020, Zee Digital launched the English global news channel WION through a digital first approach across Asia Pacific, Middle East, UK and North America. Chadda launched Zee's UGC short video platform HiPi in the midst of the TikTok ban in India. Hipi was first launched within ZEE5 app ecosystem to capitalise on the reach of the OTT platform. After the success of the POC, he launched a standalone app for HiPi. HiPi is a short video platform that provides a complete video creation ecosystem along with news avenues of monetisation to content creators. He plans to use Zee's network reach of 600 million broadcast viewers and 300 million digital users to get creators on HiPi. HiPi launched India's first digital star hunt to allow users to audition for ZEE5 original shows through the short video platform. === Times Group === Chadda took over as President & COO of Times Network in September 2022. Leading the digital transformation of the group Chadda launched 11 new products in 18 months expanding the group's presence to various verticals in the tech business like fintech, health tech, edu tech, auto tech, OTT, ecommerce and gaming while extending the news vertical into business news, tech news and various vernacular languages. Within 4 months of his stint, in January 2023 he launched the digital platform for ET Now, targeting Gen Z, early jobbers and first time investors and laying the foundation for the fintech expansion for the brand. Since then, the product has expended to Hindi language targeting the larger Indian audience through the launch of ET Now Swadesh and further expanding to fintech business by launching ET Now Advisor, a distribution business focussing to upselling of cards, loans etc. to consumers by educating them and enabling them to make the right choices. ET Now reached 10 million users within the first 20 days of launch and became the No.1 business news channel on YouTube with 200 million views in April and May 2024. Expanding to health-tech, he launched AI powered daily health companion Health & Me in the presence of actor & fitness enthusiast Milind Soman. Chadda unveiled the auto-tech platform for Times Drive together with Union Minister of Road Transport and Highways, Nitin Gadkari showcasing the AI assisted platform that helps consumers make the right decisions when it comes to their automotive needs. In order to expand the group's presence into tech and gaming, Chadda acquired India's largest and most popular tech magazine Digit along with their digital platforms Digit.in and Skoar.gg in June 2024. Within a year, he was able to turnaround Digit's business with Digit.in becoming the No.1 Tech news platform in India in April 2025. Times Network launched college discovery platform unilist.in to enable students and parents search for the right course and institute for their higher education needs. With a focus on sports and gaming, Chadda launched India's first Inter-college esports championship under the brand of SKOAR College Gaming Championship. Times Network launched its OTT app Times Play under his leadership. The platform expanded its presence in the US through a partnership with Sling TV. He launched Pickleball Now which is the World's first TV channel focussed on the sport of Pickleball covering tournaments and leagues across the World. The channel has presence on TV and digital platforms and is being distributed to global markets through partnerships with BOTIM, Distro TV, Yupp TV and Rumble. In India, the channel is available on Jio TV, Jio TV+, Airtel Xtream Play, OTT Play, Dailyhunt. Times Group has launched India's Official Pickleball League affiliated with Indian Pickleball Association and Global Pickelball Federation which shall also be streamed live on Pickleball Now from 1st to 7th Dec 2025. === Investing and speaking === Chadda is a mentor at Esselerator, a Startup accelerator by Subhash Chandra Foundation. Esselerator is an initiative by Subhash Chandra, a billionaire Media baron, to promote and support tech entrepreneurs in domains like Media, Fintech and Education. Its powered by TiE Mumbai. Chadda is an angel investor in multiple technology startups like online school aggregator platform SchoolForSure.com. In 2019, he spoke at DPS to students on starting a business. At the time he remained CEO of Zee group's digital business division. == Philanthropy == Chadda organised a £1 mliion charity bike ride in aid of the British Asian Trust which saw participation by the Prince of Wales. Chadda presented the Prince of Wales with a cycling vest, which was said to be for his grandchildren. Chadda supports a non-profit organisation Mukkamaar founded by Bollywood actress Ishita Sharma that works towards fighting crime against women by teaching free self defence to young girls. He is helping the organisation launch their digital program through a WhatsApp-based chatbot. == A

    Read more →
  • Partial-order planning

    Partial-order planning

    Partial-order planning is an approach to automated planning that maintains a partial ordering between actions and only commits ordering between actions when forced to, that is, ordering of actions is partial. Also this planning doesn't specify which action will come out first when two actions are processed. By contrast, total-order planning maintains a total ordering between all actions at every stage of planning. Given a problem in which some sequence of actions is needed to achieve a goal, a partial-order plan specifies all actions that must be taken, but specifies an ordering between actions only where needed. Consider the following situation: a person must travel from the start to the end of an obstacle course. The course is composed of a bridge, a see-saw, and a swing-set. The bridge must be traversed before the see-saw and swing-set are reachable. Once reachable, the see-saw and swing-set can be traversed in any order, after which the end is reachable. In a partial-order plan, ordering between these obstacles is specified only when needed. The bridge must be traversed first. Second, either the see-saw or swing-set can be traversed. Third, the remaining obstacle can be traversed. Then the end can be traversed. Partial-order planning relies upon the principle of least commitment for its efficiency. == Partial-order plan == A partial-order plan or partial plan is a plan which specifies all actions that must be taken, but only specifies the order between actions when needed. It is the result of a partial-order planner. A partial-order plan consists of four components: A set of actions (also known as operators). A partial order for the actions. It specifies the conditions about the order of some actions. A set of causal links. It specifies which actions meet which preconditions of other actions. Alternatively, a set of bindings between the variables in actions. A set of open preconditions. It specifies which preconditions are not fulfilled by any action in the partial-order plan. To keep the possible orders of the actions as open as possible, the set of order conditions and causal links must be as small as possible. A plan is a solution if the set of open preconditions is empty. A linearization of a partial order plan is a total order plan derived from the particular partial order plan; in other words, both order plans consist of the same actions, with the order in the linearization being a linear extension of the partial order in the original partial order plan. === Example === For example, a plan for baking a cake might start: go to the store get eggs; get flour; get milk pay for all goods go to the kitchen This is a partial plan because the order for finding eggs, flour and milk is not specified, the agent can wander around the store reactively accumulating all the items on its shopping list until the list is complete. == Partial-order planner == A partial-order planner is an algorithm or program which will construct a partial-order plan and search for a solution. The input is the problem description, consisting of descriptions of the initial state, the goal and possible actions. The problem can be interpreted as a search problem where the set of possible partial-order plans is the search space. The initial state would be the plan with the open preconditions equal to the goal conditions. The final state would be any plan with no open preconditions, i.e. a solution. The initial state is the starting conditions, and can be thought of as the preconditions to the task at hand. For a task of setting the table, the initial state could be a clear table. The goal is simply the final action that needs to be accomplished, for example setting the table. The operators of the algorithm are the actions by which the task is accomplished. For this example there may be two operators: lay (tablecloth), and place (glasses, plates, and silverware). === Plan space === The plan space of the algorithm is constrained between its start and finish. The algorithm starts, producing the initial state and finishes when all parts of the goal have been achieved. In the setting a table example, two types of actions exist that must be addressed: the put-out and lay operators. Four unsolved operators also exist: Action 1, lay-tablecloth, Action 2, Put-out (plates), Action 3, Put-out (silverware), and Action 4, Put-out (glasses). However, a threat arises if Action 2, 3, or 4 comes before Action 1. This threat is that the precondition to the start of the algorithm will be unsatisfied as the table will no longer be clear. Thus, constraints exist that must be added to the algorithm that force Actions 2, 3, and 4 to come after Action 1. Once these steps are completed, the algorithm will finish and the goal will have been completed. === Threats === As seen in the algorithm presented above, partial-order planning can encounter certain threats, meaning orderings that threaten to break connected actions, thus potentially destroying the entire plan. There are two ways to resolve threats: Promotion Demotion Promotion orders the possible threat after the connection it threatens. Demotion orders the possible threat before the connection it threatens. Partial-order planning algorithms are known for being both sound and complete, with sound being defined as the total ordering of the algorithm, and complete being defined as the capability to find a solution, given that a solution does in fact exist. == Partial-order vs. total-order planning == Partial-order planning is the opposite of total-order planning, in which actions are sequenced all at once and for the entirety of the task at hand. The question arises when one has two competing processes, which one is better? Anthony Barret and Daniel Weld have argued in their 1993 book, that partial-order planning is superior to total-order planning, as it is faster and thus more efficient. They tested this theory using Korf’s taxonomy of subgoal collections, in which they found that partial-order planning performs better because it produces more trivial serializability than total-order planning. Trivial serializability facilitates a planner’s ability to perform quickly when dealing with goals that contain subgoals. Planners perform more slowly when dealing with laboriously serializable or nonserializable subgoals. The determining factor that makes a subgoal trivially or laboriously serializable is the search space of different plans. They found that partial-order planning is more adept at finding the quickest path, and is therefore the more efficient of these two main types of planning. == The Sussman anomaly == Partial-order plans are known to easily and optimally solve the Sussman anomaly. Using this type of incremental planning system solves this problem quickly and efficiently. This was a result of partial-order planning that solidified its place as an efficient planning system. == Disadvantages to partial-order planning == One drawback of this type of planning system is that it requires a lot more computational power for each node. This higher per-node cost occurs because the algorithm for partial-order planning is more complex than others. This has important artificial intelligence implications. When coding a robot to do a certain task, the creator needs to take into account how much energy is needed. Though a partial-order plan may be quicker it may not be worth the energy cost for the robot. The creator must be aware of and weigh these two options to build an efficient robot.

    Read more →
  • Information Processing Language

    Information Processing Language

    Information Processing Language (IPL) is a programming language created by Allen Newell, Cliff Shaw, and Herbert A. Simon at RAND Corporation and the Carnegie Institute of Technology about 1956. Newell had the job of language specifier-application programmer, Shaw was the system programmer, and Simon had the job of application programmer-user. IPL included features to facilitate AI programming, specifically problem solving. such as lists, dynamic memory allocation, data types, recursion, functions as arguments, generators, and cooperative multitasking. IPL also introduced the concepts of symbol processing and list processing. Unfortunately, all of these innovations were cast in a difficult assembly-language style. Nonetheless, IPL-V (the only public version of IPL) ran on many computers through the mid 1960s. == Basics of IPL == An IPL computer has: A set of symbols. All symbols are addresses, and name cells. Unlike symbols in later languages, symbols consist of a character followed by a number, and are written H1, A29, 9–7, 9–100. Cell names beginning with a letter are regional, and are absolute addresses. Cell names beginning with "9-" are local, and are meaningful within the context of a single list. One list's 9-1 is independent of another list's 9–1. Other symbols (e.g., pure numbers) are internal. A set of cells. Lists are made from several cells including mutual references. Cells have several fields: P, a 3-bit field used for an operation code when the cell is used as an instruction, and unused when the cell is data. Q, a 3-valued field used for indirect reference when the cell is used as an instruction, and unused when the cell is data. SYMB, a symbol used as the value in the cell. A set of primitive processes, which would be termed primitive functions in modern languages. The data structure of IPL is the list, but lists are more intricate structures than in many languages. A list consists of a singly linked sequence of symbols, as might be expected—plus some description lists, which are subsidiary singly linked lists interpreted as alternating attribute names and values. IPL provides primitives to access and mutate attribute value by name. The description lists are given local names (of the form 9–1). So, a list named L1 containing the symbols S4 and S5, and described by associating value V1 to attribute A1 and V2 to A2, would be stored as follows. 0 indicates the end of a list; the cell names 100, 101, etc. are automatically generated internal symbols whose values are irrelevant. These cells can be scattered throughout memory; only L1, which uses a regional name that must be globally known, needs to reside in a specific place. IPL is an assembly language for manipulating lists. It has a few cells which are used as special-purpose registers. H1, for example, is the program counter. The SYMB field of H1 is the name of the current instruction. However, H1 is interpreted as a list; the LINK of H1 is, in modern terms, a pointer to the beginning of the call stack. For example, subroutine calls push the SYMB of H1 onto this stack. H2 is the free-list. Procedures which need to allocate memory grab cells off of H2; procedures which are finished with memory put it on H2. On entry to a function, the list of parameters is given in H0; on exit, the results should be returned in H0. Many procedures return a Boolean result indicating success or failure, which is put in H5. Ten cells, W0-W9, are reserved for public working storage. Procedures are "morally bound" (to quote the CACM article) to save and restore the values of these cells. There are eight instructions, based on the values of P: subroutine call, push/pop S to H0; push/pop the symbol in S to the list attached to S; copy value to S; conditional branch. In these instructions, S is the target. S is either the value of the SYMB field if Q=0, the symbol in the cell named by SYMB if Q=1, or the symbol in the cell named by the symbol in the cell named by SYMB if Q=2. In all cases but conditional branch, the LINK field of the cell tells which instruction to execute next. IPL has a library of some 150 basic operations. These include such operations as: Test symbols for equality Find, set, or erase an attribute of a list Locate the next symbol in a list; insert a symbol in a list; erase or copy an entire list Arithmetic operations (on symbol names) Manipulation of symbols; e.g., test if a symbol denotes an integer, or make a symbol local I/O operations "Generators", which correspond to iterators and filters in functional programming. For example, a generator may accept a list of numbers and produce the list of their squares. Generators could accept suitably designed functions—strictly, the addresses of code of suitably designed functions—as arguments. == History == IPL was first utilized to demonstrate that the theorems in Principia Mathematica which were proven laboriously by hand, by Bertrand Russell and Alfred North Whitehead, could in fact be proven by computation. According to Simon's autobiography Models of My Life, this application was originally developed first by hand simulation, using his children as the computing elements, while writing on and holding up note cards as the registers which contained the state variables of the program. IPL was used to implement several early artificial intelligence programs, also by the same authors: the Logic Theorist (1956), the General Problem Solver (1957), and their computer chess program NSS (1958). Several versions of IPL were created: IPL-I (never implemented), IPL-II (1957 for JOHNNIAC), IPL-III (existed briefly), IPL-IV, IPL-V (1958, for IBM 650, IBM 704, IBM 7090, Philco model 212, many others. Widely used). IPL-VI was a proposal for an IPL hardware. A co-processor “IPL-VC” for the CDC 3600 at Argonne National Libraries was developed which could run IPL-V commands. It was used to implement another checker-playing program. This hardware implementation did not improve running times sufficiently to “compete favorably with a language more directly oriented to the structure of present-day machines”. IPL was soon displaced by Lisp, which had much more powerful features, a simpler syntax, and the benefit of automatic garbage collection. == Legacy to computer programming == IPL arguably introduced several programming language features: List manipulation—but only lists of atoms, not general lists Property lists—but only when attached to other lists Higher-order functions—while assembly programming had always allowed computing with the addresses of functions, IPL was an early attempt to generalize this property of assembly language in a principled way Computation with symbols—though symbols have a restricted form in IPL (letter followed by number) Virtual machine Many of these features were generalized, rationalized, and incorporated into Lisp and from there into many other programming languages during the next several decades.

    Read more →
  • AltStore

    AltStore

    AltStore is an alternative app store for the iOS and iPadOS[1] mobile operating systems, which allows users to download applications that are not available on the App Store, most commonly tweaked apps, jailbreak apps, and apps including paid apps on the app store. It was publicly announced on September 25, 2019, and launched on September 28. == History == Riley Testut is an American developer who began to work on AltStore after Apple declined to allow his Nintendo emulator Delta on the App Store. Since Xcode allowed him to temporarily install his Delta app to his iOS device for 7 days of testing, he created AltStore in 2019 to replicate this functionality, which could be extended to other .ipa files. As of 2022, AltStore had been downloaded 1.5 million times. In the following years, AltStore expanded beyond its initial sideloading functionality. The platform was founded by Testut, with Shane Gill later joining as co-founder. AltStore was initially supported through Patreon contributions from its user community, and later saw increased adoption following regulatory developments in the European Union that enabled broader third-party app distribution. The project has also been involved in notable industry collaborations, including a partnership with Epic Games. == Features == AltStore exploits a loophole in the Xcode developer platform, which allows developers to sideload their own apps which they are working on without needing to jailbreak. Sideloaded apps are signed like a developer project for testing and will expire after 7 days with a free account or one year with a paid developer account, by which they will need to be refreshed or reinstalled.

    Read more →
  • Is-a

    Is-a

    In knowledge representation, ontology components and ontology engineering, including for object-oriented programming and design, is-a (also written as is_a or is a) is a subsumptive relationship between abstractions (e.g., types, classes), wherein one class A is a subclass of another class B (and so B is a superclass of A). In other words, type A is a subtype of type B when A's specification implies B's specification. That is, any object (or class) that satisfies A's specification also satisfies B's specification, because B's specification is weaker. For example, a cat 'is a[n]' animal, but not vice versa. All cats are animals, but not all animals are cats. Behaviour that is relevant to all animals is defined on an animal class, whereas behaviour that is relevant only for cats is defined in a cat class. By defining the cat class as 'extending' the animal class, all cats 'inherit' the behaviour defined for animals, without the need to explicitly code that behaviour for cats. == Related concepts == The is-a relationship is to be contrasted with the has-a (has_a or has a) relationship between types (classes); confusing the relations has-a and is-a is a common error when designing a model (e.g., a computer program) of the real-world relationship between an object and its subordinate. The is-a relationship may also be contrasted with the instance-of relationship between objects (instances) and types (classes): see Type–token distinction. To summarize the relations, there are: hyperonym–hyponym (supertype/superclass–subtype/subclass) relations between types (classes) defining a taxonomic hierarchy, where for a subsumption relation: a hyponym (subtype, subclass) has a type-of (is-a) relationship with its hyperonym (supertype, superclass); holonym–meronym (whole/entity/container–part/constituent/member) relations between types (classes) defining a possessive hierarchy, where for an aggregation (i.e. without ownership) relation: a holonym (whole) has a has-a relationship with its meronym (part), for a composition (i.e. with ownership) relation: a meronym (constituent) has a part-of relationship with its holonym (entity), for a containment relation: a meronym (member) has a member-of relationship with its holonym (container); concept–object (type–token) relations between types (classes) and objects (instances), where a token (object) has an instance-of relationship with its type (class).

    Read more →
  • Leabra

    Leabra

    Leabra stands for local, error-driven and associative, biologically realistic algorithm. It is a model of learning which is a balance between Hebbian and error-driven learning with other network-derived characteristics. This model is used to mathematically predict outcomes based on inputs and previous learning influences. Leabra is heavily influenced by and contributes to neural network designs and models, including emergent. == Background == It is the default algorithm in emergent (successor of PDP++) when making a new project, and is extensively used in various simulations. Hebbian learning is performed using conditional principal components analysis (CPCA) algorithm with correction factor for sparse expected activity levels. Error-driven learning is performed using GeneRec, which is a generalization of the recirculation algorithm, and approximates Almeida–Pineda recurrent backpropagation. The symmetric, midpoint version of GeneRec is used, which is equivalent to the contrastive Hebbian learning algorithm (CHL). See O'Reilly (1996; Neural Computation) for more details. The activation function is a point-neuron approximation with both discrete spiking and continuous rate-code output. Layer or unit-group level inhibition can be computed directly using a k-winners-take-all (KWTA) function, producing sparse distributed representations. A feedforward and feedback (FFFB) form of inhibition has now replaced the KWTA form of inhibition. FFFB inhibition can be efficiently implemented by using the average excitatory input and activity levels in a given layer. The net input is computed as an average, not a sum, over connections, based on normalized, sigmoidally transformed weight values, which are subject to scaling on a connection-group level to alter relative contributions. Automatic scaling is performed to compensate for differences in expected activity level in the different projections. Documentation about this algorithm can be found in the book "Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain" published by MIT press. and in the Emergent Documentation Archived 2009-04-16 at the Wayback Machine == Overview of the leabra algorithm == The pseudocode for Leabra is given here, showing exactly how the pieces of the algorithm described in more detail in the subsequent sections fit together. Iterate over minus and plus phases of settling for each event. o At start of settling, for all units: - Initialize all state variables (activation, v_m, etc.). - Apply external patterns (clamp input in minus, input & output in plus). - Compute net input scaling terms (constants, computed here so network can be dynamically altered). - Optimization: compute net input once from all static activations (e.g., hard-clamped external inputs). o During each cycle of settling, for all non-clamped units: - Compute excitatory netinput (g_e(t), aka eta_j or net) -- sender-based optimization by ignoring inactives. - Compute kWTA inhibition for each layer, based on g_i^Q: Sort units into two groups based on g_i^Q: top k and remaining k+1 -> n. If basic, find k and k+1th highest If avg-based, compute avg of 1 -> k & k+1 -> n. Set inhibitory conductance g_i from g^Q_k and g^Q_k+1 - Compute point-neuron activation combining excitatory input and inhibition o After settling, for all units, record final settling activations as either minus or plus phase (y^-_j or y^+_j). After both phases update the weights (based on linear current weight values), for all connections: o Compute error-driven weight changes with CHL with soft weight bounding o Compute Hebbian weight changes with CPCA from plus-phase activations o Compute net weight change as weighted sum of error-driven and Hebbian o Increment the weights according to net weight change. == Implementations == Emergent Archived 2015-10-03 at the Wayback Machine is the original implementation of Leabra; its most recent implementation is written in Go. It was written chiefly by Dr. O'Reilly, but professional software engineers were recently hired to improve the existing codebase. This is the fastest implementation, suitable for constructing large networks. Although emergent has a graphical user interface, it is very complex and has a steep learning curve. If you want to understand the algorithm in detail, it will be easier to read non-optimized code. For this purpose, check out the MATLAB version. There is also an R version available, that can be easily installed via install.packages("leabRa") in R and has a short introduction to how the package is used. The MATLAB and R versions are not suited for constructing very large networks, but they can be installed quickly and (with some programming background) are easy to use. Furthermore, they can also be adapted easily. == Special algorithms == Temporal differences and general dopamine modulation. Temporal differences (TD) is widely used as a model of midbrain dopaminergic firing. Primary value learned value (PVLV). PVLV simulates behavioral and neural data on Pavlovian conditioning and the midbrain dopaminergic neurons that fire in proportion to unexpected rewards (an alternative to TD). Prefrontal cortex basal ganglia working memory (PBWM). PBWM uses PVLV to train prefrontal cortex working memory updating system, based on the biology of the prefrontal cortex and basal ganglia.

    Read more →
  • Chris Olah

    Chris Olah

    Christopher Olah (born 1992 or 1993) is a Canadian machine learning researcher and a co-founder of Anthropic. He is known for his work on neural network interpretability, particularly mechanistic interpretability, and for research and tools that visualise internal representations in neural networks. In 2025, Forbes reported he had become a billionaire due to his ownership in Anthropic. == Early life and education == Olah was born in Canada. According to Wired, he left university at age 18 without earning a degree and later received a Thiel Fellowship, which supported him in pursuing independent work. == Career == Olah has worked on interpretability research at Google Brain, OpenAI, and Anthropic. Time called him one of the pioneers of mechanistic interpretability and noted that he pursued this research line first at Google, then at OpenAI, and later at Anthropic, which he co-founded. Wired reported that Olah was involved in neural network visualisation work including DeepDream in 2015, as part of efforts to better understand what neural networks learn. Later coverage linked him to more structured interpretability approaches such as "activation atlases". The Verge covered activation atlases as a collaboration between Google and OpenAI researchers to help inspect neural network representations. At Anthropic, Olah has been identified in major press coverage as leading interpretability work aimed at mapping internal "features" in large language models and relating interpretability findings to AI safety. Quanta Magazine has also quoted Olah in reporting on interpretability and the internal structure of modern language models. Time included Olah in its TIME100 AI list in 2024. === Vatican address on AI ethics === On May 25, 2026, Olah spoke at the Vatican during the official presentation of Magnifica Humanitas, the first encyclical of Pope Leo XIV, which addresses artificial intelligence and human dignity. Olah said AI could lead to large-scale displacement of human labor and exacerbate global inequality. He said the commercial and geopolitical incentives driving frontier AI labs often conflict with the public good, and described AI systems as "grown" rather than strictly engineered. Olah called for external moral oversight from religious institutions, scholars, and civil society to hold the technology sector accountable.

    Read more →
  • Collateral freedom

    Collateral freedom

    Collateral freedom is an anti-censorship strategy that attempts to make it economically prohibitive for censors to block content on the Internet. This is achieved by hosting content on cloud services that are considered by censors to be "too important to block", and then using encryption to prevent censors from identifying requests for censored information that is hosted among other content, forcing censors to either allow access to the censored information or take down entire services.

    Read more →
  • The Synthetic Party

    The Synthetic Party

    Det Syntetiske Parti (English: The Synthetic Party) is a political party driven by artificial intelligence (AI), founded in May 2022 in Denmark. The party aims to represent non-voters and fringe political parties while raising awareness of AI's societal role and exploring how it can be integrated into democratic processes. == Founder == The founder and continuous party secretary is Asker Bryld Staunæs, a philosopher from Aarhus University and a conceptual artist. == Main goal == The political goals have been machine learned from texts by Danish fringe parties since 1970 and represent the 20 percent of Danes who do not vote in the election. The party is synthetic; as such, many of the policies, such as universal basic income, can be contradictory to one another. == International collaborations == The Synthetic Party has signed bilateral collaboration agreements with the Finnish AI Party and AI Party (Japan) concerning the development of a global project created around artificial intelligence and politics These collaborations were expanded during the exhibition-event Synthetic Summit (28 February – 13 April 2025) at Kunsthal Aarhus, curated by Computer Lars (Asker Bryld Staunæs) on behalf of The Synthetic Party. The summit staged parliamentary scenography, performances, and computer sculptures, and invited both the public and policymakers to encounter an international line-up of AI parties and virtual politicians. Aarhus University described the event as part of Staunæs's PhD research, positioning it as an international top-meeting of virtual politicians. Participants included the Japanese AI Party, the Swedish AI Party, the Finnish AI Party, Parker Politics (New Zealand), Lex AI (Brazil), the Simiyya collective (Egypt/Sweden), the Synthetic Party (Denmark), and Wiktoria Cukt 2.0 (Poland). As part of the summit, the one-day AI World Congress was held on 1 March 2025, structured as a performative assembly where each group participated through both machinic agents and human delegates. Sessions were chaired by participating parties, with Computer Lars delivering the opening presentation. Throughout the day, contributions were synthesized into a common record using a shared AI system. The congress concluded with the adoption of the Synthetic Summit Resolution, a collectively authored treaty of algorithmic governance. Signatories included Floor Kist and Nick Gerritsen (Parker Politics), Michihito Matsuda (Japanese AI Party), Emma Bexell (Swedish AI Party), Samee Haapa (Finnish AI Party), Pedro Markun (Lex AI), Kristian T. Madsen and Michael Birkebæk Jensen (NextGen Democracy / DemAI), Asker Bryld Staunæs, Benjamin Asger Krog Møller, Caroline Sofie Axelsson, Life with Artificials (The Synthetic Party), and Piotr Wyrzykowski (Wiktoria Cukt 2.0).

    Read more →
  • Elon Musk

    Elon Musk

    Elon Reeve Musk ( EE-lon; born June 28, 1971) is a businessman and former public official known for his leadership of Tesla and SpaceX. Musk has been the wealthiest person in the world since 2025; as of June 2026, Forbes estimates his net worth to be US$834 billion. Born into the wealthy Musk family in Pretoria, South Africa, Musk emigrated in 1989 to Canada; he has Canadian citizenship since his mother was born there. He received bachelor's degrees in 1997 from the University of Pennsylvania before moving to California to pursue business ventures. In 1995, Musk co-founded the software company Zip2. Following its sale in 1999, he co-founded X.com, an online payment company that later merged to form PayPal, which was acquired by eBay in 2002. Musk also became an American citizen in 2002. In 2002, Musk founded the space technology company SpaceX, becoming its CEO and chief engineer; the company has since led innovations in reusable rockets and commercial spaceflight. Musk joined the automaker Tesla as an early investor in 2004 and became its CEO and product architect in 2008; it has since become a leader in electric vehicles. In 2015, he co-founded OpenAI to advance artificial intelligence (AI) research, but later left; growing discontent with the organization's direction and leadership in the AI boom in the 2020s led him to establish xAI, which became a subsidiary of SpaceX in 2026. In 2022, he acquired the social network Twitter, implementing significant changes, and rebranding it as X in 2023. His other businesses include the neurotechnology company Neuralink, which he co-founded in 2016, and the tunneling company the Boring Company, which he founded in 2017. In November 2025, Tesla approved a pay package worth $1 trillion for Musk, which he is to receive over 10 years if he meets specific goals. Musk is a supporter of global far-right politics, figures, and political parties. He was the largest donor in the 2024 U.S. presidential election, where he supported Donald Trump. After Trump was inaugurated as president in January 2025, Musk served as Senior Advisor to the President and as the de facto head of the Department of Government Efficiency (DOGE). Shortly before a public feud with Trump, Musk left the Trump administration in May 2025 and returned to managing his companies. Musk's political activities, statements and views have made him a polarizing figure. He has been criticized for making unscientific and misleading statements, including spreading COVID-19 misinformation, promoting conspiracy theories, and affirming antisemitic, racist, and transphobic comments. His acquisition of Twitter was controversial due to a subsequent increase in hate speech and the spread of misinformation on the service, following his pledge to decrease censorship. His role in the second Trump administration attracted public backlash, particularly in response to DOGE. == Early life and education == Elon Reeve Musk was born on June 28, 1971, in Pretoria, South Africa's administrative capital. He is of British and Pennsylvania Dutch ancestry. His mother, Maye (née Haldeman), is a model and dietitian born in Saskatchewan, Canada, and raised in South Africa. Musk therefore holds both South African and Canadian citizenship from birth. His father, Errol Musk, is a South African electromechanical engineer, pilot, sailor, consultant, emerald dealer, and property developer, who partly owned a rental lodge at Timbavati Private Nature Reserve. His maternal grandfather, Joshua N. Haldeman, who died in a plane crash when Elon was a toddler, was an American-born Canadian chiropractor, aviator and political activist in the Technocracy movement who moved to South Africa in 1950. Haldeman's anti-government, anti-democratic and conspiracist views, which included the promotion of far-right antisemitic conspiracy theories, "fanatical" support of apartheid, and according to Errol Musk, support of Nazism, have been suggested as an influence on Elon. During his childhood, Elon was told stories by his grandmother of Haldeman's travels and exploits, and Elon has suggested that all of Haldeman's descendants have his "desire for adventure, exploration – doing crazy things". Elon has a younger brother, Kimbal, a younger sister, Tosca, and four paternal half-siblings. Musk was baptized as a child in the Anglican Church of Southern Africa. The Musk family was wealthy during Elon's youth. Despite both Elon and Errol previously stating that Errol was a part owner of a Zambian emerald mine, in 2023, Errol recounted that the deal he made was to receive "a portion of the emeralds produced at three small mines". Errol was elected to the Pretoria City Council as a representative of the anti-apartheid Progressive Party and has said that his children shared their father's dislike of apartheid. After his parents divorced in 1979, Elon, aged around 9, chose to live with his father because he had an Encyclopædia Britannica set and a computer. Elon later regretted his decision and became estranged from his father. Elon has recounted trips to a wilderness school that he described as a "paramilitary Lord of the Flies" where "bullying was a virtue" and children were encouraged to fight over rations. In one incident, after an altercation with a fellow pupil, Elon was thrown down concrete steps and beaten severely, leading to him being hospitalized for his injuries. Elon described his father berating him after he was discharged from the hospital. Errol denied berating Elon and claimed, "The [other] boy had just lost his father to suicide, and Elon had called him stupid. Elon had a tendency to call people stupid. How could I possibly blame that child?" Elon was an enthusiastic reader of books, and had attributed his success in part to having read The Lord of the Rings, the Foundation series, and The Hitchhiker's Guide to the Galaxy. At age ten, he developed an interest in computing and video games, teaching himself how to program from the VIC-20 user manual. At age twelve, Elon sold his BASIC-based game Blastar to PC and Office Technology magazine for approximately $500 (equivalent to $1,600 in 2025). === Education === Musk attended Waterkloof House Preparatory School, Bryanston High School, and then Pretoria Boys High School, where he graduated. Musk was a decent but unexceptional student, earning a 61/100 in Afrikaans and a B on his senior math certification. Musk applied for a Canadian passport through his Canadian-born mother to avoid South Africa's mandatory military service, which would have forced him to participate in the apartheid regime, as well as to ease his path to immigration to the United States. While waiting for his application to be processed, he attended the University of Pretoria for five months. Musk arrived in Canada in June 1989, connected with a second cousin in Saskatchewan, and worked odd jobs, including at a farm and a lumber mill. In 1990, he entered Queen's University in Kingston, Ontario. Two years later, he transferred to the University of Pennsylvania, where he studied until 1995. Although Musk has said that he earned his degrees in 1995, the University of Pennsylvania did not award them until 1997 – a Bachelor of Arts in physics and a Bachelor of Science in economics from the university's Wharton School. He reportedly hosted large, ticketed house parties to help pay for tuition, and wrote a business plan for an electronic book-scanning service similar to Google Books. In 1994, Musk held two internships in Silicon Valley: one at energy storage startup Pinnacle Research Institute, which investigated electrolytic supercapacitors for energy storage, and another at Palo Alto–based startup Rocket Science Games. In 1995, he was accepted to a graduate program in materials science at Stanford University, but did not enroll. Musk decided to join the Internet boom of the 1990s, applying for a job at Netscape, to which he reportedly never received a response. The Washington Post reported that Musk lacked legal authorization to remain and work in the United States after failing to enroll at Stanford. In response, Musk said he was allowed to work at that time and that his student visa transitioned to an H1-B. According to numerous former business associates and shareholders, Musk said he was on a student visa at the time. == Business career == === Zip2 === In 1995, Musk, his brother Kimbal, and Greg Kouri founded the web software company Zip2 with funding from a group of angel investors. They housed the venture at a small rented office in Palo Alto. Replying to Rolling Stone, Musk denounced the notion that they started their company with funds borrowed from Elon's father Errol Musk, but in a tweet, he recognized that his father contributed 10% of a later funding round. The company developed and marketed an Internet city guide for the newspaper publishing industry, with maps, directions, and yellow pages. According to Musk, "The website was up during the day and I was coding it

    Read more →
  • Shane Legg

    Shane Legg

    Shane Legg (born 1973 or 1974) is a machine learning researcher and entrepreneur. With Demis Hassabis and Mustafa Suleyman, he cofounded DeepMind Technologies (later bought by Google and now called Google DeepMind), and works there as the chief AGI scientist. He is also known for his academic work on artificial general intelligence, including his thesis supervised by Marcus Hutter. == Early life and education == Legg attended Rotorua Lakes High School in Rotorua, on New Zealand's North Island. He completed his undergraduate studies at Waikato University in 1996. Also in 1996, he obtained his MSc degree with a thesis entitled "Solomonoff Induction", with Cristian S. Calude at the University of Auckland. == Research interests == In the early 2000s, Legg re-introduced and popularized with Ben Goertzel the term "artificial general intelligence" (AGI), to describe an AI that can do practically any cognitive task a human can do. At that time, talking about AGI "would put you on the lunatic fringe". Legg is known for his concern of existential risk from AI, highlighted in 2011 in an interview on LessWrong and in 2023 he signed the statement on AI risk of extinction. == Career == Before his PhD and before cofounding DeepMind, Shane Legg worked at "a number of software development positions at private companies", including the "big data firm Adaptive Intelligence" and the startup WebMind founded by Ben Goertzel. === Research === Legg later obtained a PhD at the Dalle Molle Institute for Artificial Intelligence Research (IDSIA), a joint research institute of USI Università della Svizzera italiana and SUPSI. He worked on theoretical models of super intelligent machines (AIXI) with Marcus Hutter, and completed in 2008 his doctoral thesis entitled "Machine Super Intelligence". He then went on to complete a postdoctoral fellowship in finance at USI, and began a further fellowship at University College London's Gatsby Computational Neuroscience Unit. === DeepMind === Demis Hassabis and Shane Legg first met in 2009 at University College London, where Legg was a postdoctoral researcher. In 2010, Legg cofounded the start-up DeepMind Technologies along with Demis Hassabis and Mustafa Suleyman. DeepMind Technologies was bought in 2014 by Google. After the merge with Google Brain in 2023, the company is now known as Google DeepMind. According to a 2017 article, a significant part of his job as the chief scientist was to supervise recruitment, to decide where DeepMind should focus its efforts, and to lead DeepMind's AI safety work. As of July 2023, Legg works at Google DeepMind as the Chief AGI Scientist. == Awards and honors == Legg was awarded the $10,000 prize of the Singularity Institute for Artificial Intelligence for his PhD done in 2008. Legg was appointed Commander of the Order of the British Empire (CBE) in the 2019 Birthday Honours for services to the science and technology sector and to investment.

    Read more →
  • Round-trip engineering

    Round-trip engineering

    Round-trip engineering (RTE) in the context of model-driven architecture is a functionality of software development tools that synchronizes two or more related software artifacts, such as, source code, models, configuration files, documentation, etc. between each other. The need for round-trip engineering arises when the same information is present in multiple artifacts and when an inconsistency may arise in case some artifacts are updated. For example, some piece of information was added to/changed in only one artifact (source code) and, as a result, it became missing in/inconsistent with the other artifacts (in models). == Overview == Round-trip engineering is closely related to traditional software engineering disciplines: forward engineering (creating software from specifications), reverse engineering (creating specifications from existing software), and reengineering (understanding existing software and modifying it). Round-trip engineering is often wrongly defined as simply supporting both forward and reverse engineering. In fact, the key characteristic of round-trip engineering that distinguishes it from forward and reverse engineering is the ability to synchronize existing artifacts that evolved concurrently by incrementally updating each artifact to reflect changes made to the other artifacts. Furthermore, forward engineering can be seen as a special instance of RTE in which only the specification is present and reverse engineering can be seen as a special instance of RTE in which only the software is present. Many reengineering activities can also be understood as RTE when the software is updated to reflect changes made to the previously reverse engineered specification. === Types === Various books describe two types of RTE: partial or uni-directional RTE: changes made to a higher level representation of a code and model are reflected in lower level, but not otherwise; the latter might be allowed, but with limitations that may not affect higher-level abstractions full or bi-directional RTE: regardless of changes, both higher and lower-level code and model representations are synchronized if any of them altered === Auto synchronization === Another characteristic of round-trip engineering is automatic update of the artifacts in response to automatically detected inconsistencies. In that sense, it is different from forward- and reverse engineering which can be both manual (traditionally) and automatic (via automatic generation or analysis of the artifacts). The automatic update can be either instantaneous or on-demand. In instantaneous RTE, all related artifacts are immediately updated after each change made to one of them. In on-demand RTE, authors of the artifacts may concurrently update the artifacts (even in a distributed setting) and at some point choose to execute matching to identify inconsistencies and choose to propagate some of them and reconcile potential conflicts. === Iterative approach === Round trip engineering may involve an iterative development process. After you have synchronized your model with revised code, you are still free to choose the best way to work – make further modifications to the code or make changes to your model. You can synchronize in either direction at any time and you can repeat the cycle as many times as necessary. == Software == Many commercial tools and research prototypes support this form of RTE; a 2007 book lists Rational Rose, Together, ESS-Model, BlueJ, and Fujaba among those capable, with Fujaba said to be capable to also identify design patterns. == Limitations == A 2005 book on Visual Studio notes for instance that a common problem in RTE tools is that the model reversed is not the same as the original one, unless the tools are aided by leaving laborious annotations in the source code. The behavioral parts of UML impose even more challenges for RTE. Usually, UML class diagrams are supported to some degree; however, certain UML concepts, such as associations and containment do not have straightforward representations in many programming languages which limits the usability of the created code and accuracy of code analysis/reverse engineering (e.g., containment is hard to recognize in the code). A more tractable form of round-trip engineering is implemented in the context of framework application programming interfaces (APIs), whereby a model describing the usage of a framework API by an application is synchronized with that application's code. In this setting, the API prescribes all correct ways the framework can be used in applications, which allows precise and complete detection of API usages in the code as well as creation of useful code implementing correct API usages. Two prominent RTE implementations in this category are framework-specific modeling languages and Spring Roo (Java). Round-trip engineering is critical for maintaining consistency among multiple models and between the models and the code in Object Management Group's (OMG) Model-driven architecture. OMG proposed the QVT (query/view/transformation) standard to handle model transformations required for MDA. To date, a few implementations of the standard have been created. (Need to present practical experiences with MDA in relation to RTE). == Controversies == === Code generation controversy === Code generation (forward-engineering) from models means that the user abstractly models solutions, which are connoted by some model data, and then an automated tool derives from the models parts or all of the source code for the software system. In some tools, the user can provide a skeleton of the program source code, in the form of a source code template where predefined tokens are then replaced with program source code parts during the code generation process. UML (if used for MDA) diagrams specification was criticized for lack the detail which is needed to contain the same information as is covered with the program source. Some developers even claim that "the Code is the design". == Disadvantages == There is a serious risk that the generated code will rapidly differ from the model or that the reverse-engineered model will lose its reflection on the code or a mix of these two problems as result of cycled reengineering efforts. Regarding behavioral/dynamic part of UML for features like statechart diagram there is no equivalents in programming languages. Their translation during code-generation will result in common programming statement (.e.g if,switch,enum) being either missing or misinterpreted. If edited and imported back may result in different or incomplete model. The same goes for code snippets used for code generation stage for the pattern-implementation and user-specific logic: intermixed they may not be easily reverse-engineered back. There is also general lack of advanced tooling for modelling that are comparable to that of modern IDEs (for testing, debugging, navigation, etc.) for general-purpose programming languages and domain-specific languages. == Examples in software engineering == Perhaps the most common form of round-trip engineering is synchronization between UML (Unified Modeling Language) models and the corresponding source code and entity–relationship diagrams in data modelling and database modelling. Round-trip engineering based on Unified Modeling Language (UML) needs three basic tools for software development: Source Code Editor; UML Editor for the Attributes and Methods; Visualisation of UML structure

    Read more →
  • Catastrophic interference

    Catastrophic interference

    Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to abruptly and drastically forget previously learned information upon learning new information. Neural networks are an important part of the connectionist approach to cognitive science. The issue of catastrophic interference when modeling human memory with connectionist models was originally brought to the attention of the scientific community by research from McCloskey and Cohen (1989), and Ratcliff (1990). It is a radical manifestation of the 'sensitivity-stability' dilemma or the 'stability-plasticity' dilemma. Specifically, these problems refer to the challenge of making an artificial neural network that is sensitive to, but not disrupted by, new information. Lookup tables and connectionist networks lie on the opposite sides of the stability plasticity spectrum. The former remains completely stable in the presence of new information but lacks the ability to generalize, i.e. infer general principles, from new inputs. On the other hand, connectionist networks like the standard backpropagation network can generalize to unseen inputs, but they are sensitive to new information. Backpropagation models can be analogized to human memory insofar as they have a similar ability to generalize, but these networks often exhibit less stability than human memory. Notably, these backpropagation networks are susceptible to catastrophic interference. This is an issue when modelling human memory, because unlike these networks, humans typically do not show catastrophic forgetting. == Discovery == The term catastrophic interference was originally coined by McCloskey and Cohen (1989) but was also brought to the attention of the scientific community by research from Ratcliff (1990). === The Sequential Learning Problem: McCloskey and Cohen (1989) === McCloskey and Cohen (1989) noted the problem of catastrophic interference during two different experiments with backpropagation neural network modelling. Experiment 1: Learning the ones and twos addition facts In their first experiment they trained a standard backpropagation neural network on a single training set consisting of 17 single-digit ones problems (i.e., 1 + 1 through 9 + 1, and 1 + 2 through 1 + 9) until the network could represent and respond properly to all of them. The error between the actual output and the desired output steadily declined across training sessions, which reflected that the network learned to represent the target outputs better across trials. Next, they trained the network on a single training set consisting of 17 single-digit twos problems (i.e., 2 + 1 through 2 + 9, and 1 + 2 through 9 + 2) until the network could represent, respond properly to all of them. They noted that their procedure was similar to how a child would learn their addition facts. Following each learning trial on the twos facts, the network was tested for its knowledge on both the ones and twos addition facts. Like the ones facts, the twos facts were readily learned by the network. However, McCloskey and Cohen noted the network was no longer able to properly answer the ones addition problems even after one learning trial of the twos addition problems. The output pattern produced in response to the ones facts often resembled an output pattern for an incorrect number more closely than the output pattern for a correct number. This is considered to be a drastic amount of error. Furthermore, the problems 2+1 and 1+2, which were included in both training sets, even showed dramatic disruption during the first learning trials of the twos facts. Experiment 2: Replication of Barnes and Underwood (1959) study In their second connectionist model, McCloskey and Cohen attempted to replicate the study on retroactive interference in humans by Barnes and Underwood (1959). They trained the model on A-B and A-C lists and used a context pattern in the input vector (input pattern), to differentiate between the lists. Specifically the network was trained to respond with the right B response when shown the A stimulus and A-B context pattern and to respond with the correct C response when shown the A stimulus and the A-C context pattern. When the model was trained concurrently on the A-B and A-C items then the network readily learned all of the associations correctly. In sequential training the A-B list was trained first, followed by the A-C list. After each presentation of the A-C list, performance was measured for both the A-B and A-C lists. They found that the amount of training on the A-C list in Barnes and Underwood study that lead to 50% correct responses, lead to nearly 0% correct responses by the backpropagation network. Furthermore, they found that the network tended to show responses that looked like the C response pattern when the network was prompted to give the B response pattern. This indicated that the A-C list apparently had overwritten the A-B list. This could be likened to learning the word dog, followed by learning the word stool and then finding that you think of the word stool when presented with the word dog. McCloskey and Cohen tried to reduce interference through a number of manipulations including changing the number of hidden units, changing the value of the learning rate parameter, overtraining on the A-B list, freezing certain connection weights, changing target values 0 and 1 instead 0.1 and 0.9. However, none of these manipulations satisfactorily reduced the catastrophic interference exhibited by the networks. Overall, McCloskey and Cohen (1989) concluded that: at least some interference will occur whenever new learning alters the weights involved in representing old learning the greater the amount of new learning, the greater the disruption in old knowledge interference was catastrophic in the backpropagation networks when learning was sequential but not concurrent === Constraints Imposed by Learning and Forgetting Functions: Ratcliff (1990) === Ratcliff (1990) used multiple sets of backpropagation models applied to standard recognition memory procedures, in which the items were sequentially learned. After inspecting the recognition performance models he found two major problems: Well-learned information was catastrophically forgotten as new information was learned in both small and large backpropagation networks. Even one learning trial with new information resulted in a significant loss of the old information, paralleling the findings of McCloskey and Cohen (1989). Ratcliff also found that the resulting outputs were often a blend of the previous input and the new input. In larger networks, items learned in groups (e.g. AB then CD) were more resistant to forgetting than were items learned singly (e.g. A then B then C...). However, the forgetting for items learned in groups was still large. Adding new hidden units to the network did not reduce interference. Discrimination between the studied items and previously unseen items decreased as the network learned more. This finding contradicts studies on human memory, which indicated that discrimination increases with learning. Ratcliff attempted to alleviate this problem by adding 'response nodes' that would selectively respond to old and new inputs. However, this method did not work as these response nodes would become active for all inputs. A model which used a context pattern also failed to increase discrimination between new and old items. == Proposed solutions == The main cause of catastrophic interference seems to be overlap in the representations at the hidden layer of distributed neural networks. In a distributed representation, each input tends to create changes in the weights of many of the nodes. Catastrophic forgetting occurs because when many of the weights where "knowledge is stored" are changed, it is unlikely for prior knowledge to be kept intact. During sequential learning, the inputs become mixed, with the new inputs being superimposed on top of the old ones. Another way to conceptualize this is by visualizing learning as a movement through a weight space. This weight space can be likened to a spatial representation of all of the possible combinations of weights that the network could possess. When a network first learns to represent a set of patterns, it finds a point in the weight space that allows it to recognize all of those patterns. However, when the network then learns a new set of patterns, it will move to a place in the weight space for which the only concern is the recognition of the new patterns. To recognize both sets of patterns, the network must find a place in the weight space suitable for recognizing both the new and the old patterns. Below are a number of techniques which have empirical support in successfully reducing catastrophic interference in backpropagation neural networks: === Orthogonality === Many of the early techniques in reducing representational overlap involved making either the input vecto

    Read more →
  • Sentential decision diagram

    Sentential decision diagram

    In artificial intelligence, a sentential decision diagram (SDD) is a type of knowledge representation used in knowledge compilation to represent Boolean functions. SDDs can be viewed as a generalization of the influential ordered binary decision diagram (OBDD) representation, by allowing decisions on multiple variables at once. Like OBDDs, SDDs allow for tractable Boolean operations, while being exponentially more succinct. For this reason, they have become an important representation in knowledge compilation. == Properties == SDDs are defined with respect to a generalization of variable ordering known as a variable tree (vtree). Provided that they satisfy additional properties known as compression and trimming (which are analogous to ROBDDs), SDDs are a canonical representation of Boolean functions; that is, they are unique given a vtree. Like OBDDs, they allow for operations such as conjunction, disjunction and negation to be computed directly on the representation in polynomial time, while being potentially more compact. They also allow for polynomial-time model counting. SDDs are known to be exponentially more succinct than OBDDs. == Applications == SDDs are used as a compilation target for probabilistic logic programs by the ProbLog 2 system since they support tractable (weighted) model counting as well as tractable negation, conjunction and disjunction while being more succinct than BDDs. SDDs have also been extended to model probability distributions, in which context they are known as probabilistic sentential decision diagrams (PSDD).

    Read more →