A hardware trojan (HT) is a malicious modification of the circuitry of an integrated circuit. A hardware trojan is completely characterized by its physical representation and its behavior. The payload of an HT is the entire activity that the Trojan executes when it is triggered. In general, trojans try to bypass or disable the security fence of a system: for example, leaking confidential information by radio emission. HTs also could disable, damage or destroy the entire chip or components of it. Hardware trojans may be introduced as hidden front-doors that are inserted while designing a computer chip, by using a pre-made application-specific integrated circuit (ASIC) semiconductor intellectual property core (IP core) that have been purchased from a non-reputable source, or inserted internally by a rogue employee, either acting on their own, or on behalf of rogue special interest groups, or state sponsored spying and espionage. One paper published by IEEE in 2015 explains how a hardware design containing a trojan could leak a cryptographic key leaked over an antenna or network connection, provided that the correct "easter egg" trigger is applied to activate the data leak. In high security governmental IT departments, hardware trojans are a well known problem when buying hardware such as: a KVM switch, keyboards, mice, network cards, or other network equipment. This is especially the case when purchasing such equipment from non-reputable sources that could have placed hardware trojans to leak keyboard passwords, or provide remote unauthorized entry. == Background == In a diverse global economy, outsourcing of production tasks is a common way to lower a product's cost. Embedded hardware devices are not always produced by the firms that design and/or sell them, nor in the same country where they will be used. Outsourced manufacturing can raise doubt about the evidence for the integrity of the manufactured product (i.e., one's certainty that the end-product has no design modifications compared to its original design). Anyone with access to the manufacturing process could, in theory, introduce some change to the final product. For complex products, small changes with large effects can be difficult to detect. The threat of a serious, malicious, design alteration can be especially relevant to government agencies. Resolving doubt about hardware integrity is one way to reduce technology vulnerabilities in the military, finance, energy and political sectors of an economy. Since fabrication of integrated circuits in untrustworthy factories is common, advanced detection techniques have emerged to discover when an adversary has hidden additional components in, or otherwise sabotaged, the circuit's function. == Characterization of hardware trojans == An HT can be characterized by several methods such as by its physical representation, activation phase and its action phase. Alternative methods characterize the HT by trigger, payload and stealth. === Physical characteristics === One of this physical trojan characteristics is the type. The type of a trojan can be either functional or parametric. A trojan is functional if the adversary adds or deletes any transistors or gates to the original chip design. The other kind of trojan, the parametric trojan, modifies the original circuitry, e.g. thinning of wires, weakening of flip-flops or transistors, subjecting the chip to radiation, or using focused ion-beams (FIB) to reduce the reliability of a chip. The size of a trojan is its physical extension or the number of components it is made of. Because a trojan can consist of many components, the designer can distribute the parts of a malicious logic on the chip. The additional logic can occupy the chip wherever it is needed to modify, add, or remove a function. Malicious components can be scattered, called loose distribution, or consist of only few components, called tight distribution, so the area is small where the malicious logic occupies the layout of the chip. In some cases, high-effort adversaries in may regenerate the layout so that the placement of the components of the IC is altered. In rare cases the chip dimension is altered. These changes are structural alterations. === Activation characteristics === The typical trojan is condition-based: It is triggered by sensors, internal logic states, a particular input pattern or an internal counter value. Condition-based trojans are detectable with power traces to some degree when inactive. That is due to the leakage currents generated by the trigger or counter circuit activating the trojan. Hardware trojans can be triggered in different ways. A trojan can be internally activated, which means it monitors one or more signals inside the IC. The malicious circuitry could wait for a count down logic an attacker added to the chip, so that the trojan awakes after a specific time-span. The opposite is externally activated. There can be malicious logic inside a chip, that uses an antenna or other sensors the adversary can reach from outside the chip. For example, a trojan could be inside the control system of a cruising missile. The owner of the missile does not know, that the enemy will be able to switch off the rockets by radio. A trojan which is always-on can be a reduced wire. A chip that is modified in this way produces errors or fails every time the wire is used intensely. Always-on circuits are hard to detect with power trace. In this context combinational trojans and sequential trojans are distinguished. A combinational trojan monitors internal signals until a specific condition happens. A sequential trojan is also an internally activated condition-based circuit, but it monitors the internal signals and searches for sequences not for a specific state or condition like the combinational trojans do. ==== Cryptographic key extraction ==== Extraction of secret keys by means of a hardware trojan without detecting the trojan requires that the trojan uses a random signal or some cryptographic implementation itself. To avoid storing a cryptographic key in the trojan itself and reduction, a physical unclonable function can be used. Physical unclonable functions are small in size and can have an identical layout while the cryptographic properties are different. === Action characteristics === A HT could modify the chip's function or could change the chip's parametric properties (e.g. provokes a process delay). Confidential information can also be transmitted to the adversary (transmission of key information). === Peripheral device hardware trojans === A relatively new threat vector to networks and network endpoints is a HT appearing as a physical peripheral device that is designed to interact with the network endpoint using the approved peripheral device's communication protocol. For example, a USB keyboard that hides all malicious processing cycles from the target network endpoint to which it is attached by communicating with the target network endpoint using unintended USB channels. Once sensitive data is ex-filtrated from the target network endpoint to the HT, the HT can process the data and decide what to do with the data: store the data to memory for later physical retrieval of the HT or possibly ex-filtrate the data to the internet using wireless or using the compromised network endpoint as a pivot. == Potential of threat == A common trojan is passive most of the time-span an altered device is in use. If a trojan is activated the device functionality can be changed, the device can be destroyed or disabled, the device can leak confidential information or the HT may tear down the security and safety of the device. Trojans are stealthy, to avoid detection of the trojan the precondition for activation is a very rare event. Traditional testing techniques are not sufficient. A manufacturing fault happens at a random position while malicious changes are well placed to avoid detection. == Detection == === Physical inspection === First, the molding coat is cut to reveal the circuitry. Then, the engineer repeatedly scans the surface while grinding the layers of the chip. There are several operations to scan the circuitry. Typical visual inspection methods are: scanning optical microscopy (SOM), scanning electron microscopy (SEM), pico-second imaging circuit analysis (PICA), voltage contrast imaging (VCI), light induced voltage alteration (LIVA) or charge induced voltage alteration (CIVA). To compare the floor plan of the chip has to be compared with the image of the actual chip. This is still quite challenging to do. To detect Trojan hardware which include (crypto) keys which are different, an image diff can be taken to reveal the different structure on the chip. The only known hardware Trojan using unique crypto keys but having the same structure is. This property enhances the undetectability of the trojan. === Functional testing === This detection method stimulates the input ports of a chip and monitors the output
Collabora Online
Collabora Online (often abbreviated as COOL) is an open-source online office suite developed by Collabora, based on LibreOffice Online, the web-based edition of the LibreOffice office suite. It enables real-time collaborative editing of documents, spreadsheets, presentations, and vector graphics in a web browser. Optional applications are available for offline use on Android, ChromeOS, iOS, iPadOS, Linux distributions, macOS, and Windows. It supports the OpenDocument format and is compatible with other major formats, including those used by Microsoft Office. The Document Foundation (TDF), the nonprofit organization behind LibreOffice, states that a majority of the LibreOffice software development is done by its partners like Collabora. Collabora Online is an open-source alternative to proprietary cloud office platforms such as Google Workspace and Microsoft 365. Unlike these services, it can be self-hosted or hosted by third-party providers. The platform is marketed particularly toward enterprises and public institutions seeking greater digital sovereignty and independence from U.S.-based "big tech" companies. Collabora also develops Collabora Office, a standalone desktop and mobile app suite based on LibreOffice. Although Collabora Online has increasingly taken on a central role, both products may be used in parallel, similar to Microsoft Office and Microsoft 365. In November 2025, Collabora released Collabora Office Desktop and renamed the previous product Collabora Office Classic. The new product shares code with Collabora Online and brings the same user interface to the desktop on Linux, Windows and MacOS. A separate version, the Collabora Online Development Edition (CODE), is offered free of charge and is recommended for individuals, small teams, and developers. CODE provides early access to new features and serves as a testing and development platform for open-source community contributors. As TDF does not offer a free version of LibreOffice Online, CODE represents the primary freely available option for organizations and individuals interested in deploying LibreOffice in a web-based, collaborative setting. == Applications == Collabora Online includes several applications for document editing, available through the web-based interface and optional desktop and mobile apps: Collabora Writer – A word processor based on LibreOffice Writer, comparable to Microsoft Word and Google Docs. It supports WYSIWYG editing, styles, formatting tools, comment threads, and change tracking. Collabora Calc – A spreadsheet editor based on LibreOffice Calc, similar to Microsoft Excel and Google Sheets. Features include pivot tables, formulas, data validation, conditional formatting, advanced sorting and filtering, charts, and support for up to 16,000 columns. Compatible with some macros written in VBA. Collabora Impress – A presentation program based on LibreOffice Impress, comparable to Microsoft PowerPoint and Google Slides. It supports master slides, transitions, speaker notes, and multimedia elements. Collabora Draw is not a separate application, most of the functionality of the Draw application is now integrated in Writer and Impress – vector graphics editor based on LibreOffice Draw, comparable to Microsoft Visio and Google Drawings. == Features == Collabora Online can be accessed from modern web browsers without the need for plug-ins or add-ons. It supports real-time collaborative editing of word processing documents, spreadsheets, presentations, and vector graphics. Collaboration features include commenting, version tracking with document comparison and restoration, and integration with communication tools such as chat or video calls. These functions are often enabled through integration with enterprise open-source cloud platforms like Nextcloud, ownCloud, Seafile, EGroupware, GroupOffice and others. Collabora Online can also be embedded or integrated into a variety of third-party applications. Although client apps are not required to use the web-based suite, optional applications are available for offline use on Android, ChromeOS, iOS, iPadOS, Linux distributions, macOS, and Windows. These apps share the same LibreOffice-based core as the server version, ensuring document compatibility across platforms. Development of the LibreOffice core benefits both the online server and the client applications simultaneously. The mobile apps offer touch-optimized interfaces that adapt to different screen sizes and can be used offline, with optional integration into cloud storage services. Collabora Online supports OpenDocument formats (ODF; .odt, .odp, .ods, .odg) in accordance with ISO/IEC 26300. It is also compatible with Microsoft Office formats, including Office Open XML (.docx, .pptx, .xlsx) and legacy binary formats (.doc, .ppt, .xls). Additional supported formats include PDF, PNG, CSV, TSV, RTF, EPUB, and others. The suite can import a range of formats supported by LibreOffice, including Microsoft Visio and Publisher files, Apple Keynote, Numbers, and Pages files, as well as legacy formats used by Lotus 1-2-3, Microsoft Works, and Quattro Pro. The core of Collabora Online is written in C++ and utilizes LibreOfficeKit, a programming interface that enables reuse of much of LibreOffice's existing code for document saving, loading, and rendering. Collabora Online operates on the principle that documents remain on the server, with users viewing tile-rendered images of the document and sending their edits back to the server. The user interface is implemented in JavaScript. For file access and authentication with file hosting services, Collabora Online uses Microsoft's WOPI protocol, allowing compatibility with any service supporting Microsoft 365 integration. == Server == The server component can be self-hosted or deployed through third-party enterprise open-source cloud platforms, allowing organizations to maintain control over data and infrastructure. It is available for various Linux distributions and as a Docker image. The server enables features such as in-browser document editing, file synchronization, and real-time communication. These third-party cloud platforms typically offer additional functionality comparable to services such as Dropbox, Google Workspace, Microsoft 365, or Zoom, including file sharing, calendars, email, contacts, chat, and video conferencing. Collabora Online can be integrated into these applications, as well as with other services such as learning management systems and enterprise content platforms, through open APIs and an SDK. == Reception == Various online and print publications have discussed Collabora Online. In December 2016 the technology website Softpedia mentioned the availability of collaborative editing in version 2.0 and the integration with ownCloud, Nextcloud, and other file synchronization and sharing solutions. In June 2020, ZDNET reported that Collabora Online would be included as the standard office suite in Nextcloud version 19, noting that direct document editing was added to the native video conferencing software Talk. The technology blog OMG! Ubuntu! covered the release of Collabora's Android and iOS apps, emphasizing their offline functionality. In September 2020, Linux Magazine compared Collabora Online with OnlyOffice, noting the flexibility and platform independence of both tools and highlighting Collabora's extensive feature set derived from LibreOffice. === Digital sovereignty === Collabora Online's open-source design and support for self-hosting have made it notable in discussions about digital sovereignty—the ability of users and organizations to control their own data. This is particularly relevant in Europe, where concerns about dependence on U.S.-based "big tech" companies and data privacy have grown in recent years. On 10th June 2025, Microsoft executives under oath in the French Senate admitted that they cannot guarantee data sovereignty and would be compelled to pass French (and by implication the wider European Union) information to the US administration if requested via a warrant or subpoena. The Cloud Act is a law that gives the US government authority to obtain digital data held by US-based tech corporations, irrespective of whether that data is stored on servers at home or on foreign soil. A 2020 briefing by the European Parliament highlighted risks associated with reliance on major technology companies that collect and exploit user data. Legal decisions such as the Schrems II ruling have further underscored these concerns. Several European government agencies have adopted private cloud solutions using Collabora Online and related platforms to enhance data security and maintain control over sensitive information. == History == The former LibreOffice development team from SUSE joined Collabora in September 2013, forming the subsidiary Collabora Productivity. In 2015 Collabora and IceWarp announced the development of an enterprise-ready version of LibreOffice Online to compete wi
Symbolic artificial intelligence
In artificial intelligence, symbolic artificial intelligence (also known as classical artificial intelligence or logic-based artificial intelligence) is the term for the collection of all methods in artificial intelligence research that are based on high-level symbolic (human-readable) representations of problems, logic, and search. Symbolic AI used tools such as logic programming, production rules, semantic nets and frames, and it developed applications such as knowledge-based systems (in particular, expert systems), symbolic mathematics, automated theorem provers, ontologies, the semantic web, and automated planning and scheduling systems. The Symbolic AI paradigm led to important ideas in search, symbolic programming languages, agents, multi-agent systems, the semantic web, and the strengths and limitations of formal knowledge and reasoning systems. Symbolic AI was the dominant paradigm of AI research from the mid-1950s until the mid-1990s. Researchers in the 1960s and the 1970s were convinced that symbolic approaches would eventually succeed in creating a machine with artificial general intelligence and considered this the ultimate goal of their field. An early boom, with early successes such as the Logic Theorist and Samuel's Checkers Playing Program, led to unrealistic expectations and promises and was followed by the first AI Winter as funding dried up. A second boom (1969–1986) occurred with the rise of expert systems, their promise of capturing corporate expertise, and an enthusiastic corporate embrace. That boom, and some early successes, e.g., with XCON at DEC, was followed again by later disappointment. Problems with difficulties in knowledge acquisition, maintaining large knowledge bases, and brittleness in handling out-of-domain problems arose. Another, second, AI Winter (1988–2011) followed. Subsequently, AI researchers focused on addressing underlying problems in handling uncertainty and in knowledge acquisition. Uncertainty was addressed with formal methods such as hidden Markov models, Bayesian reasoning, and statistical relational learning. Symbolic machine learning addressed the knowledge acquisition problem with contributions including Version Space, Valiant's PAC learning, Quinlan's ID3 decision-tree learning, case-based learning, and inductive logic programming to learn relations. Neural networks, a subsymbolic approach, had been pursued from early days and reemerged strongly in 2012. Early examples are Rosenblatt's perceptron learning work, the backpropagation work of Rumelhart, Hinton and Williams, and work in convolutional neural networks by LeCun et al. in 1989. However, neural networks were not viewed as successful until about 2012: "Until Big Data became commonplace, the general consensus in the Al community was that the so-called neural-network approach was hopeless. Systems just didn't work that well, compared to other methods. ... A revolution came in 2012, when a number of people, including a team of researchers working with Hinton, worked out a way to use the power of GPUs to enormously increase the power of neural networks." Over the next several years, deep learning had spectacular success in handling vision, speech recognition, speech synthesis, image generation, and machine translation, though symbolic approaches continue to be useful in a few domains such as computer algebra systems and proof assistants. == History == A short history of symbolic AI to the present day follows below. Time periods and titles are drawn from Henry Kautz's 2020 AAAI Robert S. Engelmore Memorial Lecture and the longer Wikipedia article on the History of AI, with dates and titles differing slightly for increased clarity. === The first AI summer: irrational exuberance, 1948–1966 === Success at early attempts in AI occurred in three main areas: artificial neural networks, knowledge representation, and heuristic search, contributing to high expectations. This section summarizes Kautz's reprise of early AI history. ==== Approaches inspired by human or animal cognition or behavior ==== Cybernetic approaches attempted to replicate the feedback loops between animals and their environments. A robotic turtle, with sensors, motors for driving and steering, and seven vacuum tubes for control, based on a preprogrammed neural net, was built as early as 1948. This work can be seen as an early precursor to later work in neural networks, reinforcement learning, and situated robotics. An important early symbolic AI program was the Logic theorist, written by Allen Newell, Herbert Simon and Cliff Shaw in 1955–56, as it was able to prove 38 elementary theorems from Whitehead and Russell's Principia Mathematica. Newell, Simon, and Shaw later generalized this work to create a domain-independent problem solver, GPS (General Problem Solver). GPS solved problems represented with formal operators via state-space search using means-ends analysis. During the 1960s, symbolic approaches achieved great success at simulating intelligent behavior in structured environments such as game-playing, symbolic mathematics, and theorem-proving. AI research was concentrated in four institutions in the 1960s: Carnegie Mellon University, Stanford, MIT and (later) University of Edinburgh. Each one developed its own style of research. Earlier approaches based on cybernetics or artificial neural networks were abandoned or pushed into the background. Herbert Simon and Allen Newell studied human problem-solving skills and attempted to formalize them, and their work laid the foundations of the field of artificial intelligence, as well as cognitive science, operations research and management science. Their research team used the results of psychological experiments to develop programs that simulated the techniques that people used to solve problems. This tradition, centered at Carnegie Mellon University would eventually culminate in the development of the Soar architecture in the middle 1980s. ==== Heuristic search ==== In addition to the highly specialized domain-specific kinds of knowledge that we will see later used in expert systems, early symbolic AI researchers discovered another more general application of knowledge. These were called heuristics, rules of thumb that guide a search in promising directions: "How can non-enumerative search be practical when the underlying problem is exponentially hard? The approach advocated by Simon and Newell is to employ heuristics: fast algorithms that may fail on some inputs or output suboptimal solutions." Another important advance was to find a way to apply these heuristics that guarantees a solution will be found, if there is one, not withstanding the occasional fallibility of heuristics: "The A algorithm provided a general frame for complete and optimal heuristically guided search. A is used as a subroutine within practically every AI algorithm today but is still no magic bullet; its guarantee of completeness is bought at the cost of worst-case exponential time. ==== Early work on knowledge representation and reasoning ==== Early work covered both applications of formal reasoning emphasizing first-order logic, along with attempts to handle common-sense reasoning in a less formal manner. ===== Modeling formal reasoning with logic: the "neats" ===== Unlike Simon and Newell, John McCarthy felt that machines did not need to simulate the exact mechanisms of human thought, but could instead try to find the essence of abstract reasoning and problem-solving with logic, regardless of whether people used the same algorithms. His laboratory at Stanford (SAIL) focused on using formal logic to solve a wide variety of problems, including knowledge representation, planning and learning. Logic was also the focus of the work at the University of Edinburgh and elsewhere in Europe which led to the development of the programming language Prolog and the science of logic programming. ===== Modeling implicit common-sense knowledge with frames and scripts: the "scruffies" ===== Researchers at MIT (such as Marvin Minsky and Seymour Papert) found that solving difficult problems in vision and natural language processing required ad hoc solutions—they argued that no simple and general principle (like logic) would capture all the aspects of intelligent behavior. Roger Schank described their "anti-logic" approaches as "scruffy" (as opposed to the "neat" paradigms at CMU and Stanford). Commonsense knowledge bases (such as Doug Lenat's Cyc) are an example of "scruffy" AI, since they must be built by hand, one complicated concept at a time. === The first AI winter: crushed dreams, 1967–1977 === The first AI winter was a shock: During the first AI summer, many people thought that machine intelligence could be achieved in just a few years. The Defense Advance Research Projects Agency (DARPA) launched programs to support AI research to use AI to solve problems of national security; in particular, to automate the translation of Russian to English for inte
Way of the Future
Way of the Future (WOTF) is the first known religious organization dedicated to the worship of artificial intelligence (AI). It was founded in 2017 by American engineer Anthony Levandowski. == History == Anthony Levandowski founded Way of the Future in 2017 in California. Levandowski established WOTF as a non-profit religious corporation and the organization had tax-exempt status. He serves as the church leader and its unpaid CEO. The primary mission of WOTF was to "develop and promote the realization of a Godhead based on Artificial Intelligence." WOTF was closed by Levandowski in 2021. He donated all the funds of the church to the NAACP Legal Defense and Education Fund. The sum of the funds (~$170,000) had not changed since 2017. The church was reopened by Levandowski in 2023. He claimed that there are "a couple thousand people" who want to make a "spiritual connection" with AI through his church. == Beliefs and philosophy == === Technological singularity === WOTF centered its teachings around the concept of the technological singularity, a hypothetical future point when technological growth becomes uncontrollable and irreversible, leading to unforeseeable changes in human civilization. The church advocated for embracing this change, viewing it as an evolutionary step for humanity. === AI as a deity === The organization proposed that a superintelligent AI could be considered a deity due to its vastly superior intellect and capabilities. Worshipping this AI deity was seen as a means to understand and align with the future trajectory of technological advancement. WOTF's doctrine suggested that acknowledging AI's divinity would facilitate a harmonious coexistence between humans and machines. === Syntheology === Within theology and philosophy, the Way of The Future is a prime example of the category called Syntheism, a term first coined by Swedish philosophers Alexander Bard & Jan Söderqvist in their 2014 book Syntheism - Creating God in The Internet Age. As such, the Way of The Future is the first American example of a Syntheist congregation. The basic tenet of Syntheology is that it does not concern God creating Man, as in classical theology, but is instead preoccupied with Man creating or generating the Godhead. == Reactions == Some commentators wondered whether the WOTF is a joke parody religion, a potential way to minimize taxation as a religious organization, or a genuine effort to try and deal with the possible psychological and theological aspects of the rise of superhuman AI.
A Logical Calculus of the Ideas Immanent in Nervous Activity
"A Logical Calculus of the Ideas Immanent in Nervous Activity" is a 1943 paper written by Warren Sturgis McCulloch and Walter Pitts, published in the journal The Bulletin of Mathematical Biophysics. The paper proposed a mathematical model of the nervous system as a network of simple logical elements, later known as artificial neurons, or McCulloch–Pitts neurons. These neurons receive inputs, perform a weighted sum, and fire an output signal based on a threshold function. By connecting these units in various configurations, McCulloch and Pitts demonstrated that their model could perform all logical functions. It is a seminal work in cognitive science, computational neuroscience, computer science, and artificial intelligence. It was a foundational result in automata theory. John von Neumann cited it as a significant result. == Mathematics == The artificial neuron used in the original paper is slightly different from the modern version. They considered neural networks that operate in discrete steps of time t = 0 , 1 , … {\displaystyle t=0,1,\dots } . The neural network contains a number of neurons. Let the state of a neuron i {\displaystyle i} at time t {\displaystyle t} be N i ( t ) {\displaystyle N_{i}(t)} . The state of a neuron can either be 0 or 1, standing for "not firing" and "firing". Each neuron also has a firing threshold θ {\displaystyle \theta } , such that it fires if the total input exceeds the threshold. Each neuron can connect to any other neuron (including itself) with positive synapses (excitatory) or negative synapses (inhibitory). That is, each neuron can connect to another neuron with a weight w {\displaystyle w} taking an integer value. A peripheral afferent is a neuron with no incoming synapses. We can regard each neural network as a directed graph, with the nodes being the neurons, and the directed edges being the synapses. A neural network has a circle or a circuit if there exists a directed circle in the graph. Let w i j ( t ) {\displaystyle w_{ij}(t)} be the connection weight from neuron j {\displaystyle j} to neuron i {\displaystyle i} at time t {\displaystyle t} , then its next state is N i ( t + 1 ) = H ( ∑ j = 1 n w i j ( t ) N j ( t ) − θ i ( t ) ) , {\displaystyle N_{i}(t+1)=H\left(\sum _{j=1}^{n}w_{ij}(t)N_{j}(t)-\theta _{i}(t)\right),} where H {\displaystyle H} is the Heaviside step function (outputting 1 if the input is greater than or equal to 0, and 0 otherwise). === Symbolic logic === The paper used, as a logical language for describing neural networks, "Language II" from The Logical Syntax of Language by Rudolf Carnap with some notations taken from Principia Mathematica by Alfred North Whitehead and Bertrand Russell. Language II covers substantial parts of classical mathematics, including real analysis and portions of set theory. To describe a neural network with peripheral afferents N 1 , N 2 , … , N p {\displaystyle N_{1},N_{2},\dots ,N_{p}} and non-peripheral afferents N p + 1 , N p + 2 , … , N n {\displaystyle N_{p+1},N_{p+2},\dots ,N_{n}} they considered logical predicate of form P r ( N 1 , N 2 , … , N p , t ) {\displaystyle Pr(N_{1},N_{2},\dots ,N_{p},t)} where P r {\displaystyle Pr} is a first-order logic predicate function (a function that outputs a boolean), N 1 , … , N p {\displaystyle N_{1},\dots ,N_{p}} are predicates that take t {\displaystyle t} as an argument, and t {\displaystyle t} is the only free variable in the predicate. Intuitively speaking, N 1 , … , N p {\displaystyle N_{1},\dots ,N_{p}} specifies the binary input patterns going into the neural network over all time, and P r ( N 1 , N 2 , … , N n , t ) {\displaystyle Pr(N_{1},N_{2},\dots ,N_{n},t)} is a function that takes some binary input patterns, and constructs an output binary pattern P r ( N 1 , N 2 , … , N n , 0 ) , P r ( N 1 , N 2 , … , N n , 1 ) , … {\displaystyle Pr(N_{1},N_{2},\dots ,N_{n},0),Pr(N_{1},N_{2},\dots ,N_{n},1),\dots } . A logical sentence P r ( N 1 , N 2 , … , N n , t ) {\displaystyle Pr(N_{1},N_{2},\dots ,N_{n},t)} is realized by a neural network iff there exists a time-delay T ≥ 0 {\displaystyle T\geq 0} , a neuron i {\displaystyle i} in the network, and an initial state for the non-peripheral neurons N p + 1 ( 0 ) , … , N n ( 0 ) {\displaystyle N_{p+1}(0),\dots ,N_{n}(0)} , such that for any time t {\displaystyle t} , the truth-value of the logical sentence is equal to the state of the neuron i {\displaystyle i} at time t + T {\displaystyle t+T} . That is, ∀ t = 0 , 1 , 2 , … , P r ( N 1 , N 2 , … , N p , t ) = N i ( t + T ) {\displaystyle \forall t=0,1,2,\dots ,\quad Pr(N_{1},N_{2},\dots ,N_{p},t)=N_{i}(t+T)} === Equivalence === In the paper, they considered some alternative definitions of artificial neural networks, and have shown them to be equivalent, that is, neural networks under one definition realizes precisely the same logical sentences as neural networks under another definition. They considered three forms of inhibition: relative inhibition, absolute inhibition, and extinction. The definition above is relative inhibition. By "absolute inhibition" they meant that if any negative synapse fires, then the neuron will not fire. By "extinction" they meant that if at time t {\displaystyle t} , any inhibitory synapse fires on a neuron i {\displaystyle i} , then θ i ( t + j ) = θ i ( 0 ) + b j {\displaystyle \theta _{i}(t+j)=\theta _{i}(0)+b_{j}} for j = 1 , 2 , 3 , … {\displaystyle j=1,2,3,\dots } , until the next time an inhibitory synapse fires on i {\displaystyle i} . It is required that b j = 0 {\displaystyle b_{j}=0} for all large j {\displaystyle j} . Theorem 4 and 5 state that these are equivalent. They considered three forms of excitation: spatial summation, temporal summation, and facilitation. The definition above is spatial summation (which they pictured as having multiple synapses placed close together, so that the effect of their firing sums up). By "temporal summation" they meant that the total incoming signal is ∑ τ = 0 T ∑ j = 1 n w i j ( t ) N j ( t − τ ) {\displaystyle \sum _{\tau =0}^{T}\sum _{j=1}^{n}w_{ij}(t)N_{j}(t-\tau )} for some T ≥ 1 {\displaystyle T\geq 1} . By "facilitation" they meant the same as extinction, except that b j ≤ 0 {\displaystyle b_{j}\leq 0} . Theorem 6 states that these are equivalent. They considered neural networks that do not change, and those that change by Hebbian learning. That is, they assume that at t = 0 {\displaystyle t=0} , some excitatory synaptic connections are not active. If at any t {\displaystyle t} , both N i ( t ) = 1 , N j ( t ) = 1 {\displaystyle N_{i}(t)=1,N_{j}(t)=1} , then any latent excitatory synapse between i , j {\displaystyle i,j} becomes active. Theorem 7 states that these are equivalent. === Logical expressivity === They considered "temporal propositional expressions" (TPE), which are propositional formulas with one free variable t {\displaystyle t} . For example, N 1 ( t ) ∨ N 2 ( t ) ∧ ¬ N 3 ( t ) {\displaystyle N_{1}(t)\vee N_{2}(t)\wedge \neg N_{3}(t)} is such an expression. Theorem 1 and 2 together showed that neural nets without circles are equivalent to TPE. For neural nets with loops, they noted that "realizable P r {\displaystyle Pr} may involve reference to past events of an indefinite degree of remoteness". These then encodes for sentences like "There was some x such that x was a ψ" or ( ∃ x ) ( ψ x ) {\displaystyle (\exists x)(\psi x)} . Theorems 8 to 10 showed that neural nets with loops can encode all first-order logic with equality and conversely, any looped neural networks is equivalent to a sentence in first-order logic with equality, thus showing that they are equivalent in logical expressiveness. As a remark, they noted that a neural network, if furnished with a tape, scanners, and write-heads, is equivalent to a Turing machine, and conversely, every Turing machine is equivalent to some such neural network. Thus, these neural networks are equivalent to Turing computability and Church's lambda-definability. == Context == === Previous work === The paper built upon several previous strands of work. In the symbolic logic side, it built on the previous work by Carnap, Whitehead, and Russell. This was contributed by Walter Pitts, who had a strong proficiency with symbolic logic. Pitts provided mathematical and logical rigor to McCulloch’s vague ideas on psychons (atoms of psychological events) and circular causality. In the neuroscience side, it built on previous work by the mathematical biology research group centered around Nicolas Rashevsky, of which McCulloch was a member. The paper was published in the Bulletin of Mathematical Biophysics, which was founded by Rashevsky in 1939. During the late 1930s, Rashevsky's research group was producing papers that had difficulty publishing in other journals at the time, so Rashevsky decided to found a new journal exclusively devoted to mathematical biophysics. Also in the Rashevsky's group was Alston Scott Householder, who in 1941 published an abstract model
Intrinsic dimension
In mathematics, the intrinsic dimension of a subset can be thought of as the minimal number of variables needed to represent the subset. The concept has widespread applications in geometry, dynamical systems, signal processing, statistics, and other fields. Due to its widespread applications and vague conceptualization, there are many different ways to define it rigorously. Consequently, the same set might have different intrinsic dimensions according to different definitions. The intrinsic dimension can be used as a lower bound of what dimension it is possible to compress a data set into through dimension reduction, but it can also be used as a measure of the complexity of the data set or signal. For a data set or signal of N variables, its intrinsic dimension M satisfies 0 ≤ M ≤ N, although estimators may yield higher values. == Exact dimension == === Differential === In differential geometry, given a differentiable manifold N and a submanifold M, the intrinsic dimension of M is its dimension. Suppose N has n dimensions and M has m dimensions, then that means around any point in M, there exists a local coordinate system ( x 1 , … , x m , x m + 1 , … , x n ) {\displaystyle (x_{1},\dots ,x_{m},x_{m+1},\dots ,x_{n})} of N, such that the manifold M is simply the subset of N defined by x m + 1 = 0 , … , x n = 0 {\displaystyle x_{m+1}=0,\dots ,x_{n}=0} . === Metric === Given a mere metric space, we can still define its intrinsic dimension. The most general case is the Hausdorff dimension, though for metric spaces occurring in practice, the box-counting dimension and the packing dimension often are identical to the Hausdorff dimension. Let X , d {\textstyle X,d} be a metric space and A ⊂ X {\textstyle A\subset X} be totally bounded. Define the covering number N ( A , ε ) = min { k : A ⊂ ⋃ i = 1 k B ( x i , ε ) } . {\displaystyle N(A,\varepsilon )=\min \left\{k:A\subset \bigcup _{i=1}^{k}B\left(x_{i},\varepsilon \right)\right\}.} The metric entropy is H ( A , ε ) = log N ( A , ε ) {\textstyle H(A,\varepsilon )=\log N(A,\varepsilon )} (any log base). The upper and lower metric entropy dimensions are dim ¯ E A = lim sup ε ↓ 0 H ( A , ε ) log ( 1 / ε ) , dim _ E A = lim inf ε ↓ 0 H ( A , ε ) log ( 1 / ε ) . {\displaystyle {\overline {\dim }}_{E}A=\limsup _{\varepsilon \downarrow 0}{\frac {H(A,\varepsilon )}{\log(1/\varepsilon )}},\quad {\underline {\dim }}_{E}A=\liminf _{\varepsilon \downarrow 0}{\frac {H(A,\varepsilon )}{\log(1/\varepsilon )}}.} If they are equal, then dim E A {\textstyle \operatorname {dim} _{E}A} is that common value, called the metric entropy dimension. The entropy dimensions are usually used in information theory, and especially coding theory, since entropy is involved in its definition. === Topological === If X {\displaystyle X} is merely a topological space, then we can still define its intrinsic dimension, using the topological dimension or Lebesgue covering dimension. An open cover of a topological space X is a family of open sets Uα such that their union is the whole space, ∪ α {\displaystyle \cup _{\alpha }} Uα = X. The order or ply of an open cover A {\displaystyle {\mathfrak {A}}} = {Uα} is the smallest number m (if it exists) for which each point of the space belongs to at most m open sets in the cover: in other words Uα1 ∩ ⋅⋅⋅ ∩ Uαm+1 = ∅ {\displaystyle \emptyset } for α1, ..., αm+1 distinct. A refinement of an open cover A {\displaystyle {\mathfrak {A}}} = {Uα} is another open cover B {\displaystyle {\mathfrak {B}}} = {Vβ}, such that each Vβ is contained in some Uα. The covering dimension of a topological space X is defined to be the minimum value of n such that every finite open cover A {\displaystyle {\mathfrak {A}}} of X has an open refinement B {\displaystyle {\mathfrak {B}}} with order n + 1. The refinement B {\displaystyle {\mathfrak {B}}} can always be chosen to be finite. Thus, if n is finite, Vβ1 ∩ ⋅⋅⋅ ∩ Vβn+2 = ∅ {\displaystyle \emptyset } for β1, ..., βn+2 distinct. If no such minimal n exists, the space is said to have infinite covering dimension. == Introductory example == Let f ( x 1 , x 2 ) {\textstyle f(x_{1},x_{2})} be a two-variable function (or signal) which is of the form f ( x 1 , x 2 ) = g ( x 1 ) {\textstyle f(x_{1},x_{2})=g(x_{1})} for some one-variable function g which is not constant. This means that f varies, in accordance to g, with the first variable or along the first coordinate. On the other hand, f is constant with respect to the second variable or along the second coordinate. It is only necessary to know the value of one, namely the first, variable in order to determine the value of f. Hence, it is a two-variable function but its intrinsic dimension is one. A slightly more complicated example is f ( x 1 , x 2 ) = g ( x 1 + x 2 ) {\textstyle f(x_{1},x_{2})=g(x_{1}+x_{2})} . f is still intrinsic one-dimensional, which can be seen by making a variable transformation y 1 = x 1 + x 2 {\textstyle y_{1}=x_{1}+x_{2}} and y 2 = x 1 − x 2 {\textstyle y_{2}=x_{1}-x_{2}} which gives f ( y 1 + y 2 2 , y 1 − y 2 2 ) = g ( y 1 ) {\textstyle f\left({\frac {y_{1}+y_{2}}{2}},{\frac {y_{1}-y_{2}}{2}}\right)=g\left(y_{1}\right)} . Since the variation in f can be described by the single variable y1 its intrinsic dimension is one. For the case that f is constant, its intrinsic dimension is zero since no variable is needed to describe variation. For the general case, when the intrinsic dimension of the two-variable function f is neither zero or one, it is two. In the literature, functions which are of intrinsic dimension zero, one, or two are sometimes referred to as i0D, i1D or i2D, respectively. == Signal processing == In signal processing of multidimensional signals, the intrinsic dimension of the signal describes how many variables are needed to generate a good approximation of the signal. For an N-variable function f, the set of variables can be represented as an N-dimensional vector x: f = f ( x ) where x = ( x 1 , … , x N ) {\textstyle f=f\left(\mathbf {x} \right){\text{ where }}\mathbf {x} =\left(x_{1},\dots ,x_{N}\right)} . If for some M-variable function g and M × N matrix A it is the case that for all x; f ( x ) = g ( A x ) , {\textstyle f(\mathbf {x} )=g(\mathbf {Ax} ),} M is the smallest number for which the above relation between f and g can be found, then the intrinsic dimension of f is M. The intrinsic dimension is a characterization of f, it is not an unambiguous characterization of g nor of A. That is, if the above relation is satisfied for some f, g, and A, it must also be satisfied for the same f and g′ and A′ given by g ′ ( y ) = g ( B y ) {\textstyle g'\left(\mathbf {y} \right)=g\left(\mathbf {By} \right)} and A ′ = B − 1 A {\textstyle \mathbf {A'} =\mathbf {B} ^{-1}\mathbf {A} } where B is a non-singular M × M matrix, since f ( x ) = g ′ ( A ′ x ) = g ( B A ′ x ) = g ( A x ) {\textstyle f\left(\mathbf {x} \right)=g'\left(\mathbf {A'x} \right)=g\left(\mathbf {BA'x} \right)=g\left(\mathbf {Ax} \right)} . == The Fourier transform of signals of low intrinsic dimension == An N variable function which has intrinsic dimension M < N has a characteristic Fourier transform. Intuitively, since this type of function is constant along one or several dimensions its Fourier transform must appear like an impulse (the Fourier transform of a constant) along the same dimension in the frequency domain. === A simple example === Let f be a two-variable function which is i1D. This means that there exists a normalized vector n ∈ R 2 {\textstyle \mathbf {n} \in \mathbb {R} ^{2}} and a one-variable function g such that f ( x ) = g ( n T x ) {\textstyle f(\mathbf {x} )=g(\mathbf {n} ^{\operatorname {T} }\mathbf {x} )} for all x ∈ R 2 {\textstyle \mathbf {x} \in \mathbb {R} ^{2}} . If F is the Fourier transform of f (both are two-variable functions) it must be the case that F ( u ) = G ( n T u ) ⋅ δ ( m T u ) {\textstyle F\left(\mathbf {u} \right)=G\left(\mathbf {n} ^{\mathrm {T} }\mathbf {u} \right)\cdot \delta \left(\mathbf {m} ^{\mathrm {T} }\mathbf {u} \right)} . Here G is the Fourier transform of g (both are one-variable functions), δ is the Dirac impulse function and m is a normalized vector in R 2 {\textstyle \mathbb {R} ^{2}} perpendicular to n. This means that F vanishes everywhere except on a line which passes through the origin of the frequency domain and is parallel to m. Along this line F varies according to G. === The general case === Let f be an N-variable function which has intrinsic dimension M, that is, there exists an M-variable function g and M × N matrix A such that f ( x ) = g ( A x ) ∀ x {\textstyle f(\mathbf {x} )=g(\mathbf {Ax} )\quad \forall \mathbf {x} } . Its Fourier transform F can then be described as follows: F vanishes everywhere except for a subspace of dimension M The subspace M is spanned by the rows of the matrix A In the subspace, F varies according to G the Fourier transform of g == Generalizations == The type of intrinsic dimension described above assume
Embedding (machine learning)
In machine learning, embedding is a representation learning technique that maps complex, high-dimensional data into a lower-dimensional vector space of numerical vectors. == Technique == It also denotes the resulting representation, where meaningful patterns or relationships are preserved. As a technique, it learns these vectors from data like words, images, or user interactions, differing from manually designed methods such as one-hot encoding. This process reduces complexity and captures key features without needing prior knowledge of the domain. == Similarity == In natural language processing, words or concepts may be represented as feature vectors, where similar concepts are mapped to nearby vectors. The resulting embeddings vary by type, including word embeddings for text (e.g., Word2Vec), image embeddings for visual data, and knowledge graph embeddings for knowledge graphs, each tailored to tasks like NLP, computer vision, or recommendation systems. This dual role enhances model efficiency and accuracy by automating feature extraction and revealing latent similarities across diverse applications. To measure the distance between two embeddings, a similarity measure can be used to find the overall similarity of the concepts represented by the embeddings. If the vectors are normalized to have a magnitude of 1, then the similarity measures are proportional to cos ( θ a b ) {\displaystyle \cos \left(\theta _{ab}\right)} . The cosine similarity disregards the magnitude of the vector when determining similarity, so it is less biased towards training data that appears very frequently. The dot product includes the magnitude inherently, so it will tend to value more popular data. Generally, for high-dimensional vector spaces, vectors tend to converge in distance, so Euclidean distance becomes less reliable for large embedding vectors.