A color space is a specific organization of colors. In combination with color profiling supported by various physical devices, it supports reproducible representations of color – whether such representation entails an analog or a digital representation. A color space may be arbitrary, i.e. with physically realized colors assigned to a set of physical color swatches with corresponding assigned color names (including discrete numbers in – for example – the Pantone collection), or structured with mathematical rigor (as with the NCS System, Adobe RGB and sRGB). A "color space" is a useful conceptual tool for understanding the color capabilities of a particular device or digital file. When trying to reproduce color on another device, color spaces can show whether shadow/highlight detail and color saturation can be retained, and by how much either will be compromised. A "color model" is an abstract mathematical model describing the way colors can be represented as tuples of numbers (e.g. triples in RGB or quadruples in CMYK); however, a color model with no associated mapping function to an absolute color space is a more or less arbitrary color system with no connection to any globally understood system of color interpretation. Adding a specific mapping function between a color model and a reference color space establishes within the reference color space a definite "footprint", known as a gamut, and for a given color model, this defines a color space. For example, Adobe RGB and sRGB are two different absolute color spaces, both based on the RGB color model. When defining a color space, the usual reference standard is the CIELAB or CIEXYZ color spaces, which were specifically designed to encompass all colors the average human can see. Since "color space" identifies a particular combination of the color model and the mapping function, the word is often used informally to identify a color model. However, even though identifying a color space automatically identifies the associated color model, this usage is incorrect in a strict sense. For example, although several specific color spaces are based on the RGB color model, there is no such thing as the singular RGB color space. == History == In 1802, Thomas Young postulated the existence of three types of photoreceptors (now known as cone cells) in the eye, each of which was sensitive to a particular range of visible light. Hermann von Helmholtz developed the Young–Helmholtz theory further in 1850: that the three types of cone photoreceptors could be classified as short-preferring (blue), middle-preferring (green), and long-preferring (red), according to their response to the wavelengths of light striking the retina. The relative strengths of the signals detected by the three types of cones are interpreted by the brain as a visible color. But it is not clear that they thought of colors as being points in color space. The color-space concept was likely due to Hermann Grassmann, who developed it in two stages. First, he developed the idea of vector space, which allowed the algebraic representation of geometric concepts in n-dimensional space. Fearnley-Sander (1979) describes Grassmann's foundation of linear algebra as follows: The definition of a linear space (vector space)... became widely known around 1920, when Hermann Weyl and others published formal definitions. In fact, such a definition had been given thirty years previously by Peano, who was thoroughly acquainted with Grassmann's mathematical work. Grassmann did not put down a formal definition—the language was not available—but there is no doubt that he had the concept. With this conceptual background, in 1853, Grassmann published a theory of how colors mix; it and its three color laws are still taught, as Grassmann's law. As noted first by Grassmann... the light set has the structure of a cone in the infinite-dimensional linear space. As a result, a quotient set (with respect to metamerism) of the light cone inherits the conical structure, which allows color to be represented as a convex cone in the 3- D linear space, which is referred to as the color cone. == Examples == Colors can be created in printing with color spaces based on the CMYK color model, using the subtractive primary colors of pigment (cyan, magenta, yellow, and key [black]). To create a three-dimensional representation of a given color space, we can assign the amount of magenta color to the representation's X axis, the amount of cyan to its Y axis, and the amount of yellow to its Z axis. The resulting 3-D space provides a unique position for every possible color that can be created by combining those three pigments. Colors can be created on computer monitors with color spaces based on the RGB color model, using the additive primary colors (red, green, and blue). A three-dimensional representation would assign each of the three colors to the X, Y, and Z axes. Colors generated on a given monitor will be limited by the reproduction medium, such as the phosphor (in a CRT monitor) or filters and backlight (LCD monitor). Another way of creating colors on a monitor is with an HSL or HSV color model, based on hue, saturation, brightness (value/lightness). With such a model, the variables are assigned to cylindrical coordinates. Many color spaces can be represented as three-dimensional values in this manner, but some have more, or fewer dimensions, and some, such as Pantone, cannot be represented in this way at all. == Conversion == Color space conversion is the translation of the representation of a color from one basis to another. This typically occurs in the context of converting an image that is represented in one color space to another color space, the goal being to make the translated image look as similar as possible to the original. == RGB density == The RGB color model is implemented in different ways, depending on the capabilities of the system used. The most common incarnation in general use as of 2021 is the 24-bit implementation, with 8 bits, or 256 discrete levels of color per channel. Any color space based on such a 24-bit RGB model is thus limited to a range of 256×256×256 ≈ 16.7 million colors. Some implementations use 16 bits per component for 48 bits total, resulting in the same gamut with a larger number of distinct colors. This is especially important when working with wide-gamut color spaces (where most of the more common colors are located relatively close together), or when a large number of digital filtering algorithms are used consecutively. The same principle applies for any color space based on the same color model, but implemented at different bit depths. == Lists == CIE 1931 XYZ color space was one of the first attempts to produce a color space based on measurements of human color perception (earlier efforts were by James Clerk Maxwell, König & Dieterici, and Abney at Imperial College) and it is the basis for almost all other color spaces. The CIERGB color space is a linearly-related companion of CIE XYZ. Additional derivatives of CIE XYZ include the CIELUV, CIEUVW, and CIELAB. === Generic === RGB uses additive color mixing, because it describes what kind of light needs to be emitted to produce a given color. RGB stores individual values for red, green and blue. RGBA is RGB with an additional channel, alpha, to indicate transparency. Common color spaces based on the RGB model include sRGB, Adobe RGB, ProPhoto RGB, scRGB, and CIE RGB. CMYK uses subtractive color mixing used in the printing process, because it describes what kind of inks need to be applied so the light reflected from the substrate and through the inks produces a given color. One starts with a white substrate (canvas, page, etc.), and uses ink to subtract color from white to create an image. CMYK stores ink values for cyan, magenta, yellow and black. There are many CMYK color spaces for different sets of inks, substrates, and press characteristics (which change the dot gain or transfer function for each ink and thus change the appearance). YIQ was formerly used in NTSC (North America, Japan and elsewhere) television broadcasts for historical reasons. This system stores a luma value roughly analogous to (and sometimes incorrectly identified as) luminance, along with two chroma values as approximate representations of the relative amounts of blue and red in the color. It is similar to the YUV scheme used in most video capture systems and in PAL (Australia, Europe, except France, which uses SECAM) television, except that the YIQ color space is rotated 33° with respect to the YUV color space and the color axes are swapped. The YDbDr scheme used by SECAM television is rotated in another way. YPbPr is a scaled version of YUV. It is most commonly seen in its digital form, YCbCr, used widely in video and image compression schemes such as MPEG and JPEG. xvYCC is an international digital video color space standard published by the IEC (IEC 61966-2-4). It is based on the ITU BT.601 and BT.709
Semantic analysis (machine learning)
In machine learning, semantic analysis of a text corpus is the task of building structures that approximate concepts from a large set of documents. It generally does not involve prior semantic understanding of the documents. Semantic analysis strategies include: Metalanguages based on first-order logic, which can analyze the speech of humans. Understanding the semantics of a text is symbol grounding: if language is grounded, it is equal to recognizing a machine-readable meaning. For the restricted domain of spatial analysis, a computer-based language understanding system was demonstrated. Latent semantic analysis (LSA), a class of techniques where documents are represented as vectors in a term space. A prominent example is probabilistic latent semantic analysis (PLSA). Latent Dirichlet allocation, which involves attributing document terms to topics. n-grams and hidden Markov models, which work by representing the term stream as a Markov chain, in which each term is derived from preceding terms. == Stochastic semantic analysis ==
Demis Hassabis
Sir Demis Hassabis (/ˈdɛ.mɪs/ DE-mis /hɑːˈsɑː.bis/ hah-SAH-bees; born Dimitrios Hassapis, Greek: Δημήτριος Χασάπης, 27 July 1976) is a British artificial intelligence (AI) researcher and entrepreneur. He is the chief executive officer and co-founder of Google DeepMind and Isomorphic Labs, and a UK Government AI Adviser. In 2024, Hassabis and John M. Jumper were jointly awarded the Nobel Prize in Chemistry for their AI research contributions to protein structure prediction. Hassabis is a Fellow of the Royal Society and has won awards for his research efforts, including the Breakthrough Prize, the Canada Gairdner International Award and the Lasker Award. He was appointed a CBE in 2017, and knighted in 2024 for his work on AI. He was also listed among the Time 100 most influential people in the world in 2017 and 2025, and was one of the "Architects of AI" collectively chosen as Time's 2025 Person of the Year. == Early life and education == Hassabis was born to Costas and Angela Hassapis. His father is a Greek Cypriot and his mother is a Chinese Singaporean. Demis grew up in North London. His original surname was "Hassapis" (Greek: Χασάπης), meaning "butcher" in Greek, but he later, according to Ingo Althöfer, "executed a point mutation by changing ‘p’ to ‘b’". One of his younger brothers still carries the original surname. In his early career, he was a video game AI programmer and designer, and an expert board games player. A child prodigy in chess from the age of four, when he first learnt chess by watching his father playing against his uncle, Hassabis reached master standard at the age of 13 with an Elo rating of 2300 and captained many of the England junior chess teams. He represented the University of Cambridge in the Oxford–Cambridge varsity chess matches of 1995, 1996 and 1997, winning a half blue. He first got interested in technology after buying his first computer in 1984, a ZX Spectrum 48K, funded from chess winnings. He taught himself how to program from books. He subsequently wrote his first AI program on a Commodore Amiga to play the reversi board game. Between 1988 and 1990, Hassabis was educated at Queen Elizabeth's School, Barnet, a boys' grammar school in North London. He was subsequently home-schooled by his parents for a year, before studying at the comprehensive school of Christ's College in East Finchley. He completed his A-level exams two years early at 16. === Bullfrog Productions === Asked by Cambridge University to take a gap year owing to his young age, Hassabis began his computer games career at Bullfrog Productions after entering an Amiga Power "Win-a-job-at-Bullfrog" competition. He began by playtesting on Syndicate and then at 17 co-designing and lead-programming on the 1994 game Theme Park, with the game's designer Peter Molyneux. Theme Park, a simulation video game, sold several million copies and inspired a whole genre of simulation sandbox games. Despite being offered a seven-figure sum to remain in the games industry, he turned it down. He earned enough from his gap year to pay his own way through university. === University of Cambridge === Hassabis left Bullfrog to study at Queens' College of the University of Cambridge, where he completed the Computer Science Tripos and graduated in 1997 with a double first. == Career and research == === Lionhead === After graduating from Cambridge, Hassabis worked at Lionhead Studios. Games designer Peter Molyneux, with whom Hassabis had worked at Bullfrog Productions, had recently founded the company. At Lionhead, Hassabis worked as lead AI programmer on the 2001 god game Black & White. === Elixir Studios === Hassabis left Lionhead in 1998 to found Elixir Studios, a London-based independent games developer, signing publishing deals with Eidos Interactive, Vivendi Universal and Microsoft. In addition to managing the company, Hassabis served as executive designer of the games Republic: The Revolution and Evil Genius. Each received BAFTA nominations for their interactive music scores, created by James Hannigan. The release of Elixir's first game, Republic: The Revolution, a highly ambitious and unusual political simulation game, was delayed due to its huge scope, which involved an AI simulation of the workings of an entire fictional country. The final game was reduced from its original vision and greeted with lukewarm reviews, receiving a Metacritic score of 62/100. Evil Genius, a tongue-in-cheek Austin Powers parody, fared much better with a score of 75/100. In April 2005 the intellectual property and technology rights were sold to various publishers and the studio was closed. === Neuroscience research === Following Elixir Studios, Hassabis returned to academia to obtain his PhD in cognitive neuroscience from UCL Queen Square Institute of Neurology in 2009 supervised by Eleanor Maguire. He sought to find inspiration in the human brain for new AI algorithms. He continued his neuroscience and artificial intelligence research as a visiting scientist jointly at Massachusetts Institute of Technology (MIT), in the lab of Tomaso Poggio, and Harvard University, before earning a Henry Wellcome postdoctoral research fellowship to the Gatsby Computational Neuroscience Unit at UCL in 2009 working with Peter Dayan. Working in the field of imagination, memory, and amnesia, he co-authored several influential papers published in Nature, Science, Neuron, and PNAS. His very first academic work, published in PNAS, was a landmark paper that showed systematically for the first time that patients with damage to their hippocampus, known to cause amnesia, were also unable to imagine themselves in new experiences. The finding established a link between the constructive process of imagination and the reconstructive process of episodic memory recall. Based on this work and a follow-up functional magnetic resonance imaging (fMRI) study, Hassabis developed a new theoretical account of the episodic memory system identifying scene construction, the generation and online maintenance of a complex and coherent scene, as a key process underlying both memory recall and imagination. This work received widespread coverage in the mainstream media and was listed in the top 10 scientific breakthroughs of the year by the journal Science. He later generalised these ideas to advance the notion of a 'simulation engine of the mind' whose role it was to imagine events and scenarios to aid with better planning. === DeepMind === Hassabis is the CEO and co-founder of DeepMind, a machine learning AI startup, founded in London in 2010 with Shane Legg and Mustafa Suleyman. Hassabis met Legg when both were postdocs at the Gatsby Computational Neuroscience Unit, and he and Suleyman had been friends through family. Hassabis also recruited his university friend and Elixir partner David Silver. DeepMind's mission is to "solve intelligence" and then use intelligence "to solve everything else". More concretely, DeepMind aims to combine insights from systems neuroscience with new developments in machine learning and computing hardware to unlock increasingly powerful general-purpose learning algorithms that will work towards the creation of an artificial general intelligence (AGI). The company has focused on training learning algorithms to master games, and in December 2013 it announced that it had made a pioneering breakthrough by training an algorithm called a Deep Q-Network (DQN) to play Atari games at a superhuman level by using only the raw pixels on the screen as inputs. DeepMind's early investors included several high-profile tech entrepreneurs. In 2014, Google purchased DeepMind for £400 million. Although most of the company has remained an independent entity based in London, DeepMind Health has since been directly incorporated into Google Health. Since the Google acquisition, the company has notched up a number of significant achievements, perhaps the most notable being the creation of AlphaGo, a program that defeated world champion Lee Sedol at the complex game of Go. Go had been considered a holy grail of AI, for its high number of possible board positions and resistance to existing programming techniques. However, AlphaGo beat European champion Fan Hui 5–0 in October 2015 before winning 4–1 against former world champion Lee Sedol in March 2016 and winning 3–0 against the world's top-ranked player Ke Jie in 2017. Additional DeepMind accomplishments include creating a neural Turing machine, reducing the energy used by the cooling systems in Google's data centres by 40%, and advancing research on AI safety. DeepMind has also been responsible for technical advances in machine learning, having produced a number of award-winning papers. In particular, the company has made significant advances in deep learning and reinforcement learning, and pioneered the field of deep reinforcement learning which combines these two methods. Hassabis has predicted that artificial intelligence will be "one of the most beneficial techn
Leabra
Leabra stands for local, error-driven and associative, biologically realistic algorithm. It is a model of learning which is a balance between Hebbian and error-driven learning with other network-derived characteristics. This model is used to mathematically predict outcomes based on inputs and previous learning influences. Leabra is heavily influenced by and contributes to neural network designs and models, including emergent. == Background == It is the default algorithm in emergent (successor of PDP++) when making a new project, and is extensively used in various simulations. Hebbian learning is performed using conditional principal components analysis (CPCA) algorithm with correction factor for sparse expected activity levels. Error-driven learning is performed using GeneRec, which is a generalization of the recirculation algorithm, and approximates Almeida–Pineda recurrent backpropagation. The symmetric, midpoint version of GeneRec is used, which is equivalent to the contrastive Hebbian learning algorithm (CHL). See O'Reilly (1996; Neural Computation) for more details. The activation function is a point-neuron approximation with both discrete spiking and continuous rate-code output. Layer or unit-group level inhibition can be computed directly using a k-winners-take-all (KWTA) function, producing sparse distributed representations. A feedforward and feedback (FFFB) form of inhibition has now replaced the KWTA form of inhibition. FFFB inhibition can be efficiently implemented by using the average excitatory input and activity levels in a given layer. The net input is computed as an average, not a sum, over connections, based on normalized, sigmoidally transformed weight values, which are subject to scaling on a connection-group level to alter relative contributions. Automatic scaling is performed to compensate for differences in expected activity level in the different projections. Documentation about this algorithm can be found in the book "Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain" published by MIT press. and in the Emergent Documentation Archived 2009-04-16 at the Wayback Machine == Overview of the leabra algorithm == The pseudocode for Leabra is given here, showing exactly how the pieces of the algorithm described in more detail in the subsequent sections fit together. Iterate over minus and plus phases of settling for each event. o At start of settling, for all units: - Initialize all state variables (activation, v_m, etc.). - Apply external patterns (clamp input in minus, input & output in plus). - Compute net input scaling terms (constants, computed here so network can be dynamically altered). - Optimization: compute net input once from all static activations (e.g., hard-clamped external inputs). o During each cycle of settling, for all non-clamped units: - Compute excitatory netinput (g_e(t), aka eta_j or net) -- sender-based optimization by ignoring inactives. - Compute kWTA inhibition for each layer, based on g_i^Q: Sort units into two groups based on g_i^Q: top k and remaining k+1 -> n. If basic, find k and k+1th highest If avg-based, compute avg of 1 -> k & k+1 -> n. Set inhibitory conductance g_i from g^Q_k and g^Q_k+1 - Compute point-neuron activation combining excitatory input and inhibition o After settling, for all units, record final settling activations as either minus or plus phase (y^-_j or y^+_j). After both phases update the weights (based on linear current weight values), for all connections: o Compute error-driven weight changes with CHL with soft weight bounding o Compute Hebbian weight changes with CPCA from plus-phase activations o Compute net weight change as weighted sum of error-driven and Hebbian o Increment the weights according to net weight change. == Implementations == Emergent Archived 2015-10-03 at the Wayback Machine is the original implementation of Leabra; its most recent implementation is written in Go. It was written chiefly by Dr. O'Reilly, but professional software engineers were recently hired to improve the existing codebase. This is the fastest implementation, suitable for constructing large networks. Although emergent has a graphical user interface, it is very complex and has a steep learning curve. If you want to understand the algorithm in detail, it will be easier to read non-optimized code. For this purpose, check out the MATLAB version. There is also an R version available, that can be easily installed via install.packages("leabRa") in R and has a short introduction to how the package is used. The MATLAB and R versions are not suited for constructing very large networks, but they can be installed quickly and (with some programming background) are easy to use. Furthermore, they can also be adapted easily. == Special algorithms == Temporal differences and general dopamine modulation. Temporal differences (TD) is widely used as a model of midbrain dopaminergic firing. Primary value learned value (PVLV). PVLV simulates behavioral and neural data on Pavlovian conditioning and the midbrain dopaminergic neurons that fire in proportion to unexpected rewards (an alternative to TD). Prefrontal cortex basal ganglia working memory (PBWM). PBWM uses PVLV to train prefrontal cortex working memory updating system, based on the biology of the prefrontal cortex and basal ganglia.
Procedural reasoning system
In artificial intelligence, a procedural reasoning system (PRS) is a framework for constructing real-time reasoning systems that can perform complex tasks in dynamic environments. It is based on the notion of a rational agent or intelligent agent using the belief–desire–intention software model. A user application is predominately defined, and provided to a PRS system is a set of knowledge areas. Each knowledge area is a piece of procedural knowledge that specifies how to do something, e.g., how to navigate down a corridor, or how to plan a path (in contrast with robotic architectures where the programmer just provides a model of what the states of the world are and how the agent's primitive actions affect them). Such a program, together with a PRS interpreter, is used to control the agent. The interpreter is responsible for maintaining beliefs about the world state, choosing which goals to attempt to achieve next, and choosing which knowledge area to apply in the current situation. How exactly these operations are performed might depend on domain-specific meta-level knowledge areas. Unlike traditional AI planning systems that generate a complete plan at the beginning, and replan if unexpected things happen, PRS interleaves planning and doing actions in the world. At any point, the system might only have a partially specified plan for the future. PRS is based on the BDI or belief–desire–intention framework for intelligent agents. Beliefs consist of what the agent believes to be true about the current state of the world, desires consist of the agent's goals, and intentions consist of the agent's current plans for achieving those goals. Furthermore, each of these three components is typically explicitly represented somewhere within the memory of the PRS agent at runtime, which is in contrast to purely reactive systems, such as the subsumption architecture. == History == The PRS concept was developed by the Artificial Intelligence Center at SRI International during the 1980s, by many workers including Michael Georgeff, Amy L. Lansky, and François Félix Ingrand. Their framework was responsible for exploiting and popularizing the BDI model in software for control of an intelligent agent. The seminal application of the framework was a fault detection system for the reaction control system of the NASA Space Shuttle Discovery. Development on this PRS continued at the Australian Artificial Intelligence Institute through to the late 1990s, which led to the development of a C++ implementation and extension called dMARS. == Architecture == The system architecture of SRI's PRS includes the following components: Database for beliefs about the world, represented using first order predicate calculus. Goals to be realized by the system as conditions over an interval of time on internal and external state descriptions (desires). Knowledge areas (KAs) or plans that define sequences of low-level actions toward achieving a goal in specific situations. Intentions that include those KAs that have been selected for current and eventual execution. Interpreter or inference mechanism that manages the system. == Features == SRI's PRS was developed for embedded application in dynamic and real-time environments. As such it specifically addressed the limitations of other contemporary control and reasoning architectures like expert systems and the blackboard system. The following define the general requirements for the development of their PRS: asynchronous event handling guaranteed reaction and response types procedural representation of knowledge handling of multiple problems reactive and goal-directed behavior focus of attention reflective reasoning capabilities continuous embedded operation handling of incomplete or inaccurate data handling of transients modeling delayed feedback operator control == Applications == The seminal application of SRI's PRS was a monitoring and fault detection system for the reaction control system (RCS) on the NASA space shuttle. The RCS provides propulsive forces from a collection of jet thrusters and controls altitude of the space shuttle. A PRS-based fault diagnostic system was developed and tested using a simulator. It included over 100 KAs and over 25 meta level KAs. RCS specific KAs were written by space shuttle mission controllers. It was implemented on the Symbolics 3600 Series LISP machine and used multiple communicating instances of PRS. The system maintained over 1000 facts about the RCS, over 650 facts for the forward RCS alone and half of which are updated continuously during the mission. A version of the PRS was used to monitor the reaction control system on the Space Shuttle Discovery. PRS was tested on Shakey the robot including navigational and simulated jet malfunction scenarios based on the space shuttle. Later applications included a network management monitor called the Interactive Real-time Telecommunications Network Management System (IRTNMS) for Telecom Australia. == Extensions == The following list the major implementations and extensions of the PRS architecture. UM-PRS OpenPRS (formerly C-PRS and Propice) AgentSpeak Distributed multi-agent reasoning system (dMARS) GORITE JAM JACK Intelligent Agents SRI Procedural Agent Realization Kit (SPARK) PRS-CL
Kernel density estimation
In statistics, kernel density estimation (KDE) is the application of kernel smoothing for probability density estimation, i.e., a non-parametric method to estimate the probability density function of a random variable based on kernels as weights. KDE answers a fundamental data smoothing problem where inferences about the population are made based on a finite data sample. In some fields such as signal processing and econometrics it is also termed the Parzen–Rosenblatt window method, after Emanuel Parzen and Murray Rosenblatt, who are usually credited with independently creating it in its current form. One of the famous applications of kernel density estimation is in estimating the class-conditional marginal densities of data when using a naive Bayes classifier, which can improve its prediction accuracy. == Definition == Let x = ( x 1 , x 2 , x 3 , . . . ) {\displaystyle \mathbf {x} =\left(x_{1},x_{2},x_{3},...\right)} be independent and identically distributed samples drawn from some univariate distribution with an unknown density f at any given point x. We are interested in estimating the shape of this function f. Its kernel density estimator is f ^ h ( x ) = 1 n ∑ i = 1 n K h ( x − x i ) = 1 n h ∑ i = 1 n K ( x − x i h ) , {\displaystyle {\hat {f}}_{h}(x)={\frac {1}{n}}\sum _{i=1}^{n}K_{h}(x-x_{i})={\frac {1}{nh}}\sum _{i=1}^{n}K{\left({\frac {x-x_{i}}{h}}\right)},} where K is the kernel — a non-negative function — and h > 0 is a smoothing parameter called the bandwidth or simply width. A kernel with subscript h is called the scaled kernel and defined as Kh(x) = 1/h K(x/h). Intuitively one wants to choose h as small as the data will allow; however, there is always a trade-off between the bias of the estimator and its variance. The choice of bandwidth is discussed in more detail below. A range of kernel functions are commonly used: uniform, triangular, biweight, triweight, Epanechnikov (parabolic), normal, and others. The Epanechnikov kernel is optimal in a mean square error sense, though the loss of efficiency is small for the kernels listed previously. Due to its convenient mathematical properties, the normal kernel is often used, which means K(x) = ϕ(x), where ϕ is the standard normal density function. The kernel density estimator then becomes f ^ h ( x ) = 1 n ∑ i = 1 n 1 h 2 π exp ( − ( x − x i ) 2 2 h 2 ) , {\displaystyle {\hat {f}}_{h}(x)={\frac {1}{n}}\sum _{i=1}^{n}{\frac {1}{h{\sqrt {2\pi }}}}\exp \left({\frac {-(x-x_{i})^{2}}{2h^{2}}}\right),} where h {\displaystyle h} is the standard deviation of the sample x {\displaystyle \mathbf {x} } . The construction of a kernel density estimate finds interpretations in fields outside of density estimation. For example, in thermodynamics, this is equivalent to the amount of heat generated when heat kernels (the fundamental solution to the heat equation) are placed at each data point locations xi. Similar methods are used to construct discrete Laplace operators on point clouds for manifold learning (e.g. diffusion map). == Example == Kernel density estimates are closely related to histograms, but can be endowed with properties such as smoothness or continuity by using a suitable kernel. The diagram below based on these 6 data points illustrates this relationship: For the histogram, first, the horizontal axis is divided into sub-intervals or bins which cover the range of the data: In this case, six bins each of width 2. Whenever a data point falls inside this interval, a box of height 1/12 is placed there. If more than one data point falls inside the same bin, the boxes are stacked on top of each other. For the kernel density estimate, normal kernels with a standard deviation of 1.5 (indicated by the red dashed lines) are placed on each of the data points xi. The kernels are summed to make the kernel density estimate (solid blue curve). The smoothness of the kernel density estimate (compared to the discreteness of the histogram) illustrates how kernel density estimates converge faster to the true underlying density for continuous random variables. == Bandwidth selection == The bandwidth of the kernel is a free parameter which exhibits a strong influence on the resulting estimate. To illustrate its effect, we take a simulated random sample from the standard normal distribution (plotted at the blue spikes in the rug plot on the horizontal axis). The grey curve is the true density (a normal density with mean 0 and variance 1). In comparison, the red curve is undersmoothed since it contains too many spurious data artifacts arising from using a bandwidth h = 0.05, which is too small. The green curve is oversmoothed since using the bandwidth h = 2 obscures much of the underlying structure. The black curve with a bandwidth of h = 0.337 is considered to be optimally smoothed since its density estimate is close to the true density. An extreme situation is encountered in the limit h → 0 {\displaystyle h\to 0} (no smoothing), where the estimate is a sum of n delta functions centered at the coordinates of analyzed samples. In the other extreme limit h → ∞ {\displaystyle h\to \infty } the estimate retains the shape of the used kernel, centered on the mean of the samples (completely smooth). The most common optimality criterion used to select this parameter is the expected L2 risk function, also termed the mean integrated squared error: MISE ( h ) = E [ ∫ ( f ^ h ( x ) − f ( x ) ) 2 d x ] {\displaystyle \operatorname {MISE} (h)=\operatorname {E} \!\left[\int \!{\left({\hat {f}}\!_{h}(x)-f(x)\right)}^{2}dx\right]} Under weak assumptions on f and K, (f is the, generally unknown, real density function), MISE ( h ) = AMISE ( h ) + o ( ( n h ) − 1 + h 4 ) {\displaystyle \operatorname {MISE} (h)=\operatorname {AMISE} (h)+{\mathcal {o}}{\left((nh)^{-1}+h^{4}\right)}} where o is the little o notation, and n the sample size (as above). The AMISE is the asymptotic MISE, i. e. the two leading terms, AMISE ( h ) = R ( K ) n h + 1 4 m 2 ( K ) 2 h 4 R ( f ″ ) {\displaystyle \operatorname {AMISE} (h)={\frac {R(K)}{nh}}+{\frac {1}{4}}m_{2}(K)^{2}h^{4}R(f'')} where R ( g ) = ∫ g ( x ) 2 d x {\textstyle R(g)=\int g(x)^{2}\,dx} for a function g, m 2 ( K ) = ∫ x 2 K ( x ) d x {\textstyle m_{2}(K)=\int x^{2}K(x)\,dx} and f ″ {\displaystyle f''} is the second derivative of f {\displaystyle f} and K {\displaystyle K} is the kernel. The minimum of this AMISE is the solution to this differential equation ∂ ∂ h AMISE ( h ) = − R ( K ) n h 2 + m 2 ( K ) 2 h 3 R ( f ″ ) = 0 {\displaystyle {\frac {\partial }{\partial h}}\operatorname {AMISE} (h)=-{\frac {R(K)}{nh^{2}}}+m_{2}(K)^{2}h^{3}R(f'')=0} or h AMISE = R ( K ) 1 / 5 m 2 ( K ) 2 / 5 R ( f ″ ) 1 / 5 n − 1 / 5 = C n − 1 / 5 {\displaystyle h_{\operatorname {AMISE} }={\frac {R(K)^{1/5}}{m_{2}(K)^{2/5}R(f'')^{1/5}}}n^{-1/5}=Cn^{-1/5}} Neither the AMISE nor the hAMISE formulas can be used directly since they involve the unknown density function f {\displaystyle f} or its second derivative f ″ {\displaystyle f''} . To overcome that difficulty, a variety of automatic, data-based methods have been developed to select the bandwidth. Several review studies have been undertaken to compare their efficacies, with the general consensus that the plug-in selectors and cross validation selectors are the most useful over a wide range of data sets. Substituting any bandwidth h which has the same asymptotic order n−1/5 as hAMISE into the AMISE gives that AMISE(h) = O(n−4/5), where O is the big O notation. It can be shown that, under weak assumptions, there cannot exist a non-parametric estimator that converges at a faster rate than the kernel estimator. Note that the n−4/5 rate is slower than the typical n−1 convergence rate of parametric methods. If the bandwidth is not held fixed, but is varied depending upon the location of either the estimate (balloon estimator) or the samples (pointwise estimator), this produces a particularly powerful method termed adaptive or variable bandwidth kernel density estimation. Bandwidth selection for kernel density estimation of heavy-tailed distributions is relatively difficult. === A rule-of-thumb bandwidth estimator === If Gaussian basis functions are used to approximate univariate data, and the underlying density being estimated is Gaussian, the optimal choice for h (that is, the bandwidth that minimises the mean integrated squared error) is: h = ( 4 σ ^ 5 3 n ) 1 / 5 ≈ 1.06 σ ^ n − 1 / 5 , {\displaystyle h={\left({\frac {4{\hat {\sigma }}^{5}}{3n}}\right)}^{1/5}\approx 1.06\,{\hat {\sigma }}\,n^{-1/5},} An h {\displaystyle h} value is considered more robust when it improves the fit for long-tailed and skewed distributions or for bimodal mixture distributions. This is often done empirically by replacing the standard deviation σ ^ {\displaystyle {\hat {\sigma }}} by the parameter A {\displaystyle A} below: A = min ( σ ^ , I Q R 1.34 ) {\displaystyle A=\min \left({\hat {\sigma }},{\frac {\mathrm {IQR} }{1.34}}\right)} where IQR is the
Logico-linguistic modeling
Logico-linguistic modeling is a method for building knowledge-based systems with a learning capability using conceptual models from soft systems methodology, modal predicate logic, and logic programming languages such as Prolog. == Overview == Logico-linguistic modeling is a six-stage method developed primarily for building knowledge-based systems (KBS), but it also has application in manual decision support systems and information source analysis. Logico-linguistic models have a superficial similarity to John F. Sowa's conceptual graphs; both use bubble style diagrams, both are concerned with concepts, both can be expressed in logic and both can be used in artificial intelligence. However, logico-linguistic models are very different in both logical form and in their method of construction. Logico-linguistic modeling was developed in order to solve theoretical problems found in the soft systems method for information system design. The main thrust of the research into has been to show how soft systems methodology (SSM), a method of systems analysis, can be extended into artificial intelligence. == Background == SSM employs three modeling devices i.e. rich pictures, root definitions, and conceptual models of human activity systems. The root definitions and conceptual models are built by stakeholders themselves in an iterative debate organized by a facilitator. The strengths of this method lie, firstly, in its flexibility, the fact that it can address any problem situation, and, secondly, in the fact that the solution belongs to the people in the organization and is not imposed by an outside analyst. Information requirements analysis (IRA) took the basic SSM method a stage further and showed how the conceptual models could be developed into a detailed information system design. IRA calls for the addition of two modeling devices: "Information Categories", which show the required information inputs and outputs from the activities identified in an expanded conceptual model; and the "Maltese Cross", a matrix which shows the inputs and outputs from the information categories and shows where new information processing procedures are required. A completed Maltese Cross is sufficient for the detailed design of a transaction processing system. The initial impetus to the development of logico-linguistic modeling was a concern with the theoretical problem of how an information system can have a connection to the physical world. This is a problem in both IRA and more established methods (such as SSADM) because none base their information system design on models of the physical world. IRA designs are based on a notional conceptual model and SSADM is based on models of the movement of documents. The solution to these problems provided a formula that was not limited to the design of transaction processing systems but could be used for the design of KBS with learning capability. == The six stages of logico-linguistic modeling == The logico-linguistic modeling method comprises six stages. === 1. Systems analysis === In the first stage logico-linguistic modeling uses SSM for systems analysis. This stage seeks to structure the problem in the client organization by identifying stakeholders, modelling organizational objectives and discussing possible solutions. At this stage it not assumed that a KBS will be a solution and logico-linguistic modeling often produces solutions that do not require a computerized KBS. Expert systems tend to capture the expertise, of individuals in different organizations, on the same topic. By contrast a KBS, produced by logico-linguistic modeling, seeks to capture the expertise of individuals in the same organization on different topics. The emphasis is on the elicitation of organizational or group knowledge rather than individual experts. In logico-linguistic modeling the stakeholders become the experts. The end point of this stage is an SSM style conceptual models such as figure 1. === 2. Language creation === According to the theory behind logico-linguistic modeling the SSM conceptual model building process is a Wittgensteinian language-game in which the stakeholders build a language to describe the problem situation. The logico-linguistic model expresses this language as a set of definitions, see figure 2. === 3. Knowledge elicitation === After the model of the language has been built putative knowledge about the real world can be added by the stakeholders. Traditional SSM conceptual models contain only one logical connective (a necessary condition). In order to represent causal sequences, "sufficient conditions" and "necessary and sufficient conditions" are also required. In logico-linguistic modeling this deficiency is remedied by two addition types of connective. The outcome of stage three is an empirical model, see figure 3. === 4. Knowledge representation === Modal predicate logic (a combination of modal logic and predicate logic) is used as the formal method of knowledge representation. The connectives from the language model are logically true (indicated by the "L" modal operator) and connective added at the knowledge elicitation stage are possibility true (indicated by the "M" modal operator). Before proceeding to stage 5, the models are expressed in logical formulae. === 5. Computer code === Formulae in predicate logic translate easily into the Prolog artificial intelligence language. The modality is expressed by two different types of Prolog rules. Rules taken from the language creation stage of model building process are treated as incorrigible. While rules from the knowledge elicitation stage are marked as hypothetical rules. The system is not confined to decision support but has a built in learning capability. === 6. Verification === A knowledge based system built using this method verifies itself. Verification takes place when the KBS is used by the clients. It is an ongoing process that continues throughout the life of the system. If the stakeholder beliefs about the real world are mistaken this will be brought out by the addition of Prolog facts that conflict with the hypothetical rules. It operates in accordance to the classic principle of falsifiability found in the philosophy of science == Applications == === Knowledge-based computer systems === Logico-linguistic modeling has been used to produce fully operational computerized knowledge based systems, such as one for the management of diabetes patients in a hospital out-patients department. === Manual decision support === In other projects the need to move into Prolog was considered unnecessary because the printed logico-linguistic models provided an easy-to-use guide to decision making. For example, a system for mortgage loan approval === Information source analysis === In some cases a KBS could not be built because the organization did not have all the knowledge needed to support all their activities. In these cases logico-linguistic modeling showed shortcomings in the supply of information and where more was needed. For example, a planning department in a telecoms company == Criticism == While logico-linguistic modeling overcomes the problems found in SSM's transition from conceptual model to computer code, it does so at the expense of increased stakeholder constructed model complexity. The benefits of this complexity are questionable and this modeling method may be much harder to use than other methods. This contention has been exemplified by subsequent research. An attempt by researchers to model buying decisions across twelve companies using logico-linguistic modeling required simplification of the models and removal of the modal elements.