Reservoir computing is a framework for computation derived from recurrent neural network theory that maps input signals into higher dimensional computational spaces through the dynamics of a fixed, non-linear system called a reservoir. After the input signal is fed into the reservoir, which is treated as a "black box," a simple readout mechanism is trained to read the state of the reservoir and map it to the desired output. The first key benefit of this framework is that training is performed only at the readout stage, as the reservoir dynamics are fixed. The second is that the computational power of naturally available systems, both classical and quantum mechanical, can be used to reduce the effective computational cost. == History == The first examples of reservoir neural networks demonstrated that randomly connected recurrent neural networks could be used for sensorimotor sequence learning, and simple forms of interval and speech discrimination. In these early models the memory in the network took the form of both short-term synaptic plasticity and activity mediated by recurrent connections. In other early reservoir neural network models the memory of the recent stimulus history was provided solely by the recurrent activity. Overall, the general concept of reservoir computing stems from the use of recursive connections within neural networks to create a complex dynamical system. It is a generalisation of earlier neural network architectures such as recurrent neural networks, liquid-state machines and echo-state networks. Reservoir computing also extends to physical systems that are not networks in the classical sense, but rather continuous systems in space and/or time: e.g. a literal "bucket of water" can serve as a reservoir that performs computations on inputs given as perturbations of the surface. The resultant complexity of such recurrent neural networks was found to be useful in solving a variety of problems including language processing and dynamic system modeling. However, training of recurrent neural networks is challenging and computationally expensive. Reservoir computing reduces those training-related challenges by fixing the dynamics of the reservoir and only training the linear output layer. A large variety of nonlinear dynamical systems can serve as a reservoir that performs computations. In recent years semiconductor lasers have attracted considerable interest as computation can be fast and energy efficient compared to electrical components. Recent advances in both AI and quantum information theory have given rise to the concept of quantum neural networks. These hold promise in quantum information processing, which is challenging to classical networks, but can also find application in solving classical problems. In 2018, a physical realization of a quantum reservoir computing architecture was demonstrated in the form of nuclear spins within a molecular solid. However, the nuclear spin experiments in did not demonstrate quantum reservoir computing per se as they did not involve processing of sequential data. Rather the data were vector inputs, which makes this more accurately a demonstration of quantum implementation of a random kitchen sink algorithm (also going by the name of extreme learning machines in some communities). In 2019, another possible implementation of quantum reservoir processors was proposed in the form of two-dimensional fermionic lattices. In 2020, realization of reservoir computing on gate-based quantum computers was proposed and demonstrated on cloud-based IBM superconducting near-term quantum computers. Reservoir computers have been used for time-series analysis purposes. In particular, some of their usages involve chaotic time-series prediction, separation of chaotic signals, and link inference of networks from their dynamics. == Classical reservoir computing == === Reservoir === The 'reservoir' in reservoir computing is the internal structure of the computer, and must have two properties: it must be made up of individual, non-linear units, and it must be capable of storing information. The non-linearity describes the response of each unit to input, which is what allows reservoir computers to solve complex problems. Reservoirs are able to store information by connecting the units in recurrent loops, where the previous input affects the next response. The change in reaction due to the past allows the computers to be trained to complete specific tasks. Reservoirs can be virtual or physical. Virtual reservoirs are typically randomly generated and are designed like neural networks. Virtual reservoirs can be designed to have non-linearity and recurrent loops, but, unlike neural networks, the connections between units are randomized and remain unchanged throughout computation. Physical reservoirs are possible because of the inherent non-linearity of certain natural systems. The interaction between ripples on the surface of water contains the nonlinear dynamics required in reservoir creation, and a pattern recognition RC was developed by first inputting ripples with electric motors then recording and analyzing the ripples in the readout. === Readout === The readout is a neural network layer that performs a linear transformation on the output of the reservoir. The weights of the readout layer are trained by analyzing the spatiotemporal patterns of the reservoir after excitation by known inputs, and by utilizing a training method such as a linear regression or a Ridge regression. As its implementation depends on spatiotemporal reservoir patterns, the details of readout methods are tailored to each type of reservoir. For example, the readout for a reservoir computer using a container of liquid as its reservoir might entail observing spatiotemporal patterns on the surface of the liquid. === Types === ==== Context reverberation network ==== An early example of reservoir computing was the context reverberation network. In this architecture, an input layer feeds into a high dimensional dynamical system which is read out by a trainable single-layer perceptron. Two kinds of dynamical system were described: a recurrent neural network with fixed random weights, and a continuous reaction–diffusion system inspired by Alan Turing's model of morphogenesis. At the trainable layer, the perceptron associates current inputs with the signals that reverberate in the dynamical system; the latter were said to provide a dynamic "context" for the inputs. In the language of later work, the reaction–diffusion system served as the reservoir. ==== Echo state network ==== The tree echo state network (TreeESN) model represents a generalization of the reservoir computing framework to tree structured data. ==== Liquid-state machine ==== Chaotic liquid state machine The liquid (i.e. reservoir) of a chaotic liquid state machine (CLSM), or chaotic reservoir, is made from chaotic spiking neurons but which stabilize their activity by settling to a single hypothesis that describes the trained inputs of the machine. This is in contrast to general types of reservoirs that don't stabilize. The liquid stabilization occurs via synaptic plasticity and chaos control that govern neural connections inside the liquid. CLSM showed promising results in learning sensitive time series data. ==== Nonlinear transient computation ==== This type of information processing is most relevant when time-dependent input signals depart from the mechanism's internal dynamics. These departures cause transients or temporary altercations which are represented in the device's output. ==== Deep reservoir computing ==== The extension of the reservoir computing framework towards deep learning, with the introduction of deep reservoir computing and of the deep echo state network (DeepESN) model allows to develop efficiently trained models for hierarchical processing of temporal data, at the same time enabling the investigation on the inherent role of layered composition in recurrent neural networks. == Quantum reservoir computing == Quantum reservoir computing may use the nonlinear nature of quantum mechanical interactions or processes to form the characteristic nonlinear reservoirs but may also be done with linear reservoirs when the injection of the input to the reservoir creates the nonlinearity. The marriage of machine learning and quantum devices is leading to the emergence of quantum neuromorphic computing as a new research area. === Types === ==== Gaussian states of interacting quantum harmonic oscillators ==== Gaussian states are a paradigmatic class of states of continuous variable quantum systems. Although they can nowadays be created and manipulated in, e.g, state-of-the-art optical platforms, naturally robust to decoherence, it is well-known that they are not sufficient for, e.g., universal quantum computing because transformations that preserve the Gaussian nature of a state are linear. Normally, linear dynamics would not be sufficient for nontrivial reser
Shell Control Box
Shell Control Box (SCB) is a network security appliance that controls privileged access to remote IT systems, records activities in replayable audit trails, and prevents malicious actions. For example, it records as a system administrator updates a file server or a third-party network operator configures a router. The recorded audit trails can be replayed like a movie to review the events as they occurred. The content of the audit trails is indexed to make searching for events and automatic reporting possible. SCB is a Linux-based device developed by Balabit. It is an application level proxy gateway. In 2017, Balabit changed the name of the product to Privileged Session Management (PSM) and repositioned it as the core module of its Privileged Access Management solution. == Main Features == Balabit’s Privileged Session Management (PSM), Shell Control Box (SCB) is a device that controls, monitors, and audits remote administrative access to servers and network devices. It is a tool to oversee system administrators by controlling the encrypted connections used for administration. PSM (SCB) has full control over the SSH, RDP, Telnet, TN3270, TN5250, Citrix ICA, and VNC connections, providing a framework (with solid boundaries) for the work of the administrators. === Gateway Authentication === PSM (SCB) acts as an authentication gateway, enforcing strong authentication before users access IT assets. PSM can also integrate to user directories (for example, a Microsoft Active Directory) to resolve the group memberships of the users who access the protected servers. Credentials for accessing the server are retrieved transparently from PSM’s credential store or a third-party password management system by PSM impersonating the authenticated user. This automatic password retrieval protects the confidentiality of passwords as users can never access them. === Access Control === PSM controls and audits privileged access over the most wide-spread protocols such as SSH, RDP, or HTTP(s). The detailed access management helps to control who can access what and when on servers. It is also possible to control advanced features of the protocols, like the type of channels permitted. For example, unneeded channels like file transfer or file sharing can be disabled, reducing the security risk on the server. With PSM policies for privileged access can be enforced in one single system. === 4-eyes Authorization === To avoid accidental misconfiguration and other human errors, PSM supports the 4-eyes authorization principle. This is achieved by requiring an authorizer to allow administrators to access the server. The authorizer also has the possibility to monitor – and terminate - the session of the administrator in real-time, as if they were watching the same screen. === Real-time Monitoring and Session Termination === PSM can monitor the network traffic in real time, and execute various actions if a certain pattern (for example, a suspicious command, window title or text) appears on the screen. PSM can also detect specific patterns such as credit card numbers. In case of detecting a suspicious user action, PSM can send an e-mail alert or immediately terminate the connection. For example, PSM can block the connection before a destructive administrator command, such as the „rm” comes into effect. === Session Recording === PSM makes user activities traceable by recording them in tamper-proof and confidential audit trails. It records the selected sessions into encrypted, timestamped, and digitally signed audit trails. Audit trails can be browsed online, or followed real-time to monitor the activities of the users. PSM replays the recorded sessions just like a movie – actions of the users can be seen exactly as they appeared on their monitor. The Balabit Desktop Player enables fast forwarding during replays, searching for events (for example, typed commands or pressing Enter) and texts seen by the user. In the case of any problems (database manipulation, unexpected shutdown, etc.) the circumstances of the event are readily available in the trails, thus the cause of the incident can be identified. In addition to recording audit trails, transferred files can be also recorded and extracted for further analysis.
Arabic Ontology
Arabic Ontology is a website offering linguistic ontology services for the Arabic language which can be used like the online site WordNet. Users can use Arabic Ontology to classify or clarify the concepts and meanings of Arabic terms. == Ontology Structure == The ontology structure (i.e., data model) is similar to WordNet's structure. Each concept in the database is given a unique concept identifier (URI), informally described by a gloss, and lexicalized by one or more synonymous lemma terms. Each term-concept pair is called a sense, and is given a SenseID. A set of senses is called synset. Concepts and senses are described by further attributes such as era and area — to specify example usage and ontological analysis. Semantic relations are defined between concepts. Some important entities are included in the ontology, such as individual countries and bodies of water. These individuals are given separate IndividualIDs and linked with their concepts through the InstanceOf relation. == Mappings to other resources == Concepts in the Arabic Ontology are mapped to synsets in WordNet, as well as to BFO and DOLCE. Terms used in the Arabic Ontology are mapped to lemmas in the LDC's SAMA database. == Applications == Arabic Ontology can be used in many application domains, such as: Information retrieval, to enrich queries (e.g., in search engines) and improve the quality of the results, i.e. meaningful search rather than string-matching search; Machine translation and word-sense disambiguation, by finding the exact mapping of concepts across languages, especially that the Arabic ontology is also mapped to the WordNet; Data Integration and interoperability in which the Arabic ontology can be used as a semantic reference to link databases and information systems; Semantic Web and Web 3.0, by using the Arabic ontology as a semantic reference to disambiguate the meanings used in websites; among many other applications. == URLs Design == The URLs in the Arabic Ontology are designed according to the W3C's Best Practices for Publishing Linked Data, as described in the following URL schemes. This allows one to also explore the whole database like exploring a graph: Ontology Concept: Each concept in the Arabic Ontology has a ConceptID and can be accessed using: https://{domain}/concept/{ConceptID | Term}. In case of a term, the set of concepts that this term lexicalizes are all retrieved. In case of a ConceptID, the concept and its direct subtypes are retrieved, e.g. https://ontology.birzeit.edu/concept/293198 Semantic relations: Relationships between concepts can be accessed using these schemes: (i) the URL: https:// {domain}/concept/{RelationName}/{ConceptID} allows retrieval of relationships among ontology concepts. (ii) the URL: https://{domain}/lexicalconcept/{RelationName}/{lexicalConceptID} allows retrieval of relations between lexical concepts. For example, https://ontology.birzeit.edu/concept/instances/293121 retrieves the instances of the concept 293121. The relations that are currently used in our database are: {subtypes, type, instances, parts, related, similar, equivalent}.
Jais (language model)
Jais is an open-source large language model launched in August 2023. Developed as a collaboration between Emirati AI company G42, the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), and US-based Cerebras Systems, Jais was designed to produce high-quality Arabic text and was also trained on English data. The model's creation was motivated by the underrepresentation of the Arabic language in the field of generative artificial intelligence. It aims to provide a more culturally and linguistically accurate model for the world's 400 million Arabic speakers. Its name is a reference to Jebel Jais, the highest mountain in the UAE. == Background and development == Jais was developed in response to the limited availability of advanced generative artificial intelligence models for the Arabic language, despite it being spoken by over 400 million people. Existing models were often trained on limited or low-quality Arabic web content, resulting in poor performance. The project represents a significant investment by the United Arab Emirates in the field of AI as part of its national strategy. The model was created through a partnership between Inception (now Core42), a subsidiary of the Abu Dhabi-based AI company G42; the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI); and Cerebras Systems, a US company specializing in AI hardware. The model is named after Jebel Jais, the highest peak in the UAE. == Training == The initial version of Jais released in August 2023 had 13 billion parameters. In November 2023, Core42 released Jais 30B, an improved version with 30 billion parameters. Both models were trained on a subset of the Cerebras Condor Galaxy 1 supercomputer. The training dataset consisted of a mix of Arabic, English, and computer code. According to Timothy Baldwin, a professor of natural language processing at MBZUAI, training the model on a diverse Arabic dataset allows it to switch between dialects. == Features == Jais is designed to generate text in both English and Arabic. The project has also released instruction-tuned "Chat" variants for both the 13B and 30B models, which are specifically optimized for conversational applications. Additional functionality for working with images, graphs, and tabular data is planned for future releases.
Bigram
A bigram or digram is a sequence of two adjacent elements from a string of tokens, which are typically letters, syllables, or words. A bigram is an n-gram for n=2. The frequency distribution of every bigram in a string is commonly used for simple statistical analysis of text in many applications, including in computational linguistics, cryptography, and speech recognition. Gappy bigrams or skipping bigrams are word pairs which allow gaps (perhaps avoiding connecting words, or allowing some simulation of dependencies, as in a dependency grammar). == Applications == Bigrams, along with other n-grams, are used in most successful language models for speech recognition. Bigram frequency attacks can be used in cryptography to solve cryptograms. See frequency analysis. Bigram frequency is one approach to statistical language identification. Some activities in logology or recreational linguistics involve bigrams. These include attempts to find English words beginning with every possible bigram, or words containing a string of repeated bigrams, such as logogogue. == Bigram frequency in the English language == The frequency of the most common letter bigrams in a large English corpus is: th 3.56% of 1.17% io 0.83% he 3.07% ed 1.17% le 0.83% in 2.43% is 1.13% ve 0.83% er 2.05% it 1.12% co 0.79% an 1.99% al 1.09% me 0.79% re 1.85% ar 1.07% de 0.76% on 1.76% st 1.05% hi 0.76% at 1.49% to 1.05% ri 0.73% en 1.45% nt 1.04% ro 0.73% nd 1.35% ng 0.95% ic 0.70% ti 1.34% se 0.93% ne 0.69% es 1.34% ha 0.93% ea 0.69% or 1.28% as 0.87% ra 0.69% te 1.20% ou 0.87% ce 0.65%
KE Software
KE Software is a formerly Australian-owned computer software company based in Manchester, United Kingdom, which specialises in collection management programs for museums, galleries and archives. The Axiell Group acquired the firm in 2014. == History == KE Software had its origins in investigations into electronic systems for managing natural science collections conducted in the late 1970s under a joint program of the University of Melbourne, the then National Museum of Victoria and the Australian Museum, which led to the development of the Titan Database in 1984. Much of the credit for the development of the project was due to the work of Martin Hallett of the Museum of Victoria which evolved into Textpress, and by 2000, the KE EMu database program. KE Software was bought by Axiell in 2014 and the team merged with the Axiell staff. Axiell continues to sell and support EMu. == Products == The firm has two main products: the Ke EMu Electronic Museum management system, a collections management system for museums; and Vitalware Vital Records Management System. The first version of Ke EMu was launched in 1997 and uses the Texpress database engine with client/server architecture on a Windows or Unix/Linux server. Ke Emu is consistent with the Dublin Core / Darwin Core standards for archive and museum catalogue metadata. "The company’s clients include the three largest museums in the world.: == KE EMu == KE EMu is considered one of the more effective and purpose-designed museum cataloguing programs. particularly in the creation of public interfaces to museum catalogue data. KE EMu was further developed in 1997 as a multilingual platform, which has been utilised in bilingual institutions such as the Canadian Museum of Civilisation. Subsequently this evolved into Texpress and KE EMu (standing for Electronic MUseum) in 2000, which is "now used across the world in natural science museums with huge collections'". KE EMu is used by a large number of museums and galleries around the world, including the Smithsonian Anthropological Collection, American Museum of Natural HistoryVancouver Art Gallery, New York Botanical Garden, the University of Chicago Research Archives, the University of Pennsylvania Museum in Philadelphia, the National Museum of Australia, the Australian Museum, Museum of Victoria, University of Melbourne Archives, and the Alexander Turnbull Library, National Library of New Zealand. There are over 300 clients, and more than 5000 users of the EMu software worldwide. The program has been described as providing "...comprehensive museum management (collection management plus other administrative needs for a museum), workflow and project management, flexible metadata, various stats and metrics, and comprehensive web interface with support for mobile devices and kiosks" == KE Vitalware == The firm's vitalware software is used by a number of governments and commercial organisations for managing and accessing large data sets, such as the birth records of the Trinidad and Tobago Registrar General, the Government of Anguilla, Ministry for Infrastructure, Communications, Utility and Housing, and the Mississippi Department of Information Technology Services. == Further development == A specialist tracking component for KE EMu has been developed by Forbes Hawkins of Museum Victoria. This enables locations to be barcoded, and data to be updated as items are moved around the stores, or between venues, display, laboratories and other locations. This system has been considered by Museums around the world. The company has been working with Australian government agencies to digitize birth deaths and marriage registers in order to cross match identity data. The program has also been used for managing the Australian Plant Disease Database and the Australian Plant Pest Database as the program "...has several features that have proven to be invaluable for a plant disease database".
Rhetorical structure theory
Rhetorical structure theory (RST) is a theory of text organization that describes relations that hold between parts of text. It was originally developed by William Mann, Sandra Thompson, Christian M. I. M. Matthiessen and others at the University of Southern California's Information Sciences Institute (ISI) and defined in a 1988 paper. The theory was developed as part of studies of computer-based text generation. Natural language processing researchers later began using RST in automatic summarization and other applications. It explains coherence by postulating a hierarchical, connected structure of texts, which are labeled using a small, predefined inventory of relation types - for example, one part of a text may provide an elaboration on another part, provide background or specify a cause for another. In the 2000s, following the release of the first large-scale dataset implementing the theory, the RST Discourse Treebank (RST-DT), Daniel Marcu demonstrated the feasibility of practical applications of RST to discourse parsing and summarization at ISI. Originally limited to written text, subsequent work in the 2010s expanded RST to spoken language analysis, and the framework has been applied to a variety of languages including Farsi, German, Mandarin Chinese, Russian and Spanish. Following the introduction of Transformers, LLMs have been applied to automatic RST parsing, with results approaching human performance on parsing text in English. == Rhetorical relations == Rhetorical relations, also called coherence or discourse relations, are paratactic (coordinate) or hypotactic (subordinate) relations that hold across two or more text spans. The logical arrangement of relations in a text contributes to its coherence by connecting different propositions in a relational structure. RST using rhetorical relations provides a systematic way for an analyst to analyze the underlying intention of a text. The analysis is usually built by reading the text and constructing a tree using the relations. The following example is a title and summary, appearing at the top of an article in Scientific American magazine (adapted from Ramachandran and Anstis, 1986). The original text, broken into numbered units, is: [Title:] The Perception of Apparent Motion [Abstract:] When the motion of an intermittently seen object is ambiguous the visual system resolves confusion by applying some tricks that reflect a built-in knowledge of properties of the physical world. In the figure, the numbers 1-5 show the corresponding units from the text above. Unit 5 provides an "elaboration" on unit 4, and therefore constitutes a less prominent satellite of unit 4, which acts as a nucleus for the relation. Units 4-5 form a relation "Means", explaining the means by which the visual system resolves confusion. Unit 3 is the Central Discourse Unit (CDU) of the text, since all units point to it directly or indirectly. Similarly units 1 and 2 form "preparation" and "circumstance" relations relative to their nuclei. Groups of units which serve as a satellite or nucleus together are called complex discourse units, and always span a set of adjacent EDUs. == Nuclearity in discourse == RST establishes two different types of units. Nuclei are considered as the most important parts of text whereas satellites contribute to the nuclei and are secondary. Nucleus contains basic information and satellite contains additional information about nucleus. The satellite is often incomprehensible without nucleus, whereas a text where satellites have been deleted can be understood to a certain extent. == Hierarchy in the analysis == RST relations are applied recursively in a text, until all units in that text are constituents in an RST relation. The result of such analyses is that RST structure are typically represented as trees, with one top level relation that encompasses other relations at lower levels. == Why RST? == From linguistic point of view, RST proposes a different view of text organization than most linguistic theories. RST points to a tight relation between relations and coherence in text From a computational point of view, it provides a characterization of text relations that has been implemented in different systems and for applications as text generation and summarization. == In design rationale == Computer scientists Ana Cristina Bicharra Garcia and Clarisse Sieckenius de Souz have used RST as the basis of a design rationale system called ADD+. In ADD+, RST is used as the basis for the rhetorical organization of a knowledge base, in a way comparable to other knowledge representation systems such as issue-based information system (IBIS). Similarly, RST has been used in representation schemes for argumentation.