Query rewriting is a typically automatic transformation that takes a set of database tables, views, and/or queries, usually indices, often gathered data and query statistics, and other metadata, and yields a set of different queries, which produce the same results but execute with better performance (for example, faster, or with lower memory use). Query rewriting can be based on relational algebra or an extension thereof (e.g. multiset relational algebra with sorting, aggregation and three-valued predicates i.e. NULLs as in the case of SQL). The equivalence rules of relational algebra are exploited, in other words, different query structures and orderings can be mathematically proven to yield the same result. For example, filtering on fields A and B, or cross joining R and S can be done in any order, but there can be a performance difference. Multiple operations may be combined, and operation orders may be altered. The result of query rewriting may not be at the same abstraction level or application programming interface (API) as the original set of queries (though often is). For example, the input queries may be in relational algebra or SQL, and the rewritten queries may be closer to the physical representation of the data, e.g. array operations. Query rewriting can also involve materialization of views and other subqueries; operations that may or may not be available to the API user. The query rewriting transformation can be aided by creating indices from which the optimizer can choose (some database systems create their own indexes if deemed useful), mandating the use of specific indices, creating materialized and/or denormalized views, or helping a database system gather statistics on the data and query use, as the optimality depends on patterns in data and typical query usage. Query rewriting may be rule based or optimizer based. Some sources discuss query rewriting as a distinct step prior to optimization, operating at the level of the user accessible algebra API (e.g. SQL). There are other, largely unrelated concepts also named similarly, for example, query rewriting by search engines.
Message queuing service
A message queueing service is a message-oriented middleware or MOM deployed in a compute cloud using software as a service model. Service subscribers access queues and or topics to exchange data using point-to-point or publish and subscribe patterns. It's important to differentiate between event-driven and message-driven (aka queue driven) services: Event-driven services (e.g. AWS SNS) are decoupled from their consumers. Whereas queue / message driven services (e.g. AWS SQS) are coupled with their consumers. Message queues can be a good buffer to handle spiky workloads but they have a finite capacity. According to Gregor Hohpe, message queues require proper mechanisms (aka flow controls) to avoid filling the queue beyond its manageable capacity and to keep the system stable. == Ordering Guarantees in Message Queues == Amazon SQS FIFO and Azure Service Bus sessions are queue-based messaging systems that provide ordering guarantees within a message group or session attempt but do not necessarily guarantee ordered delivery in cases of retries or failures. In SQS FIFO, messages in the same message group are processed in order, with subsequent messages held until the preceding message is successfully processed or moved to the dead-letter queue (DLQ). Once a message is placed in the DLQ, it is no longer retried, creating a gap in the sequence. However, the remaining messages continue to be delivered in order. Azure Service Bus sessions function similarly by maintaining ordering within a session, provided a single consumer processes messages sequentially. The implementation differs from SQS FIFO but follows the same fundamental ordering principle. In contrast, Apache Kafka is a distributed log-based messaging system that guarantees ordering within individual partitions rather than across the entire topic. Unlike queue-based systems, Kafka retains messages in a durable, append-only log, allowing multiple consumers to read at different offsets. Kafka uses manual offset management, giving consumers control over retries and failure handling. If a consumer fails to process a message, it can delay committing the offset, preventing further progress in that partition while other partitions remain unaffected. This partition-based design enables fault isolation and parallel processing while allowing ordering to be maintained within partitions, depending on consumer handling. == Vendors == Apache Kafka Apache Kafka is a distributed system consisting of servers that store and forward messages between producer client and consumer applications. IBM MQ IBM MQ offers a managed service that can be used on IBM Cloud and Amazon Web Services. Microsoft Azure Service Bus Service Bus offers queues, topics & subscriptions, and rules/actions in order to support publish-subscribe, temporal decoupling, and load balancing scenarios. Azure Service Bus is built on AMQP allowing any existing AMQP 1.0 client stack to interact with Service Bus directly or via existing .Net, Java, Node, and Python clients. Standard and Premium tiers allow for pay as you go or isolated resources at massive scale. Oracle Messaging Cloud Service This service provides a messaging solution for applications for asynchronous communication and is influenced by the Java Message Service (JMS) API specification. Any application platform that understands HTTP can also use Oracle Messaging Cloud Service through the REST interface. For Java applications, Oracle Messaging Cloud Service provides a Java library that implements and extends the JMS 1.1 interface. The Java library implements the JMS API by acting as a client of the REST API. Amazon Simple Queue Service Supports messages natively up to 256K, or up to 2GB by transmitting payload via S3. Highly scalable, durable and resilient. Provides loose-FIFO and 'at least once' delivery in order to provide massive scale. Supports REST API and optional Java Message Service client. Low latency. Utilizes Amazon Web Services. IronMQ Supports messages up to 64k; guarantees order; guarantees once only delivery; no delays retrieving messages. Supports REST API and beanstalkd open source protocol. Runs on multiple clouds including AWS and Rackspace. Scaling must be managed by user. RabbitMQ RabbitMQ is a reliable and mature messaging and streaming broker, which is easy to deploy on cloud environments, on-premises, and on your local machine. Supports AMQP, STOMP, MQTT StormMQ Open platform supports messages up to 50Mb. Uses AMQP to avoid vendor lock-in and provide language neutrality. Locate-It Option allows customers to audit the location of their data at all times and satisfy data protection principles. AnypointMQ An enterprise multi-tenant, cloud messaging service that performs advanced asynchronous messaging scenarios between applications. Anypoint MQ is fully integrated with Anypoint Platform, offering role based access control, client application management, and connectors.
Ni1000
The Ni1000 is an artificial neural network chip developed by Nestor Corporation and Intel, developed in the 1990s. It is Intel's second-generation neural network chip, but the first all-digital chip. The chip is aimed at image analysis applications– containing more than 3 million transistors – and can analyze 40,000 patterns per second. Prototypes running Nestor's OCR software in 1994 were capable of recognizing around 100 handwritten characters per second. The development was funded with money from DARPA and Office of Naval Research.
Error-driven learning
In reinforcement learning, error-driven learning is a method for adjusting a model's (intelligent agent's) parameters based on the difference between its output results and the ground truth. These models stand out as they depend on environmental feedback, rather than explicit labels or categories. They are based on the idea that language acquisition involves the minimization of the prediction error (MPSE). By leveraging these prediction errors, the models consistently refine expectations and decrease computational complexity. Typically, these algorithms are operated by the GeneRec algorithm. Error-driven learning has widespread applications in cognitive sciences and computer vision. These methods have also found successful application in natural language processing (NLP), including areas like part-of-speech tagging, parsing, named entity recognition (NER), machine translation (MT), speech recognition (SR), and dialogue systems. == Formal Definition == Error-driven learning models are ones that rely on the feedback of prediction errors to adjust the expectations or parameters of a model. The key components of error-driven learning include the following: A set S {\displaystyle S} of states representing the different situations that the learner can encounter. A set A {\displaystyle A} of actions that the learner can take in each state. A prediction function P ( s , a ) {\displaystyle P(s,a)} that gives the learner's current prediction of the outcome of taking action a {\displaystyle a} in state s {\displaystyle s} . An error function E ( o , p ) {\displaystyle E(o,p)} that compares the actual outcome o {\displaystyle o} with the prediction p {\displaystyle p} and produces an error value. An update rule U ( p , e ) {\displaystyle U(p,e)} that adjusts the prediction p {\displaystyle p} in light of the error e {\displaystyle e} . == Algorithms == Error-driven learning algorithms refer to a category of reinforcement learning algorithms that leverage the disparity between the real output and the expected output of a system to regulate the system's parameters. Typically applied in supervised learning, these algorithms are provided with a collection of input-output pairs to facilitate the process of generalization. The widely utilized error backpropagation learning algorithm is known as GeneRec, a generalized recirculation algorithm primarily employed for gene prediction in DNA sequences. Many other error-driven learning algorithms are derived from alternative versions of GeneRec. == Applications == === Cognitive science === Simpler error-driven learning models effectively capture complex human cognitive phenomena and anticipate elusive behaviors. They provide a flexible mechanism for modeling the brain's learning process, encompassing perception, attention, memory, and decision-making. By using errors as guiding signals, these algorithms adeptly adapt to changing environmental demands and objectives, capturing statistical regularities and structure. Furthermore, cognitive science has led to the creation of new error-driven learning algorithms that are both biologically acceptable and computationally efficient. These algorithms, including deep belief networks, spiking neural networks, and reservoir computing, follow the principles and constraints of the brain and nervous system. Their primary aim is to capture the emergent properties and dynamics of neural circuits and systems. === Computer vision === Computer vision is a complex task that involves understanding and interpreting visual data, such as images or videos. In the context of error-driven learning, the computer vision model learns from the mistakes it makes during the interpretation process. When an error is encountered, the model updates its internal parameters to avoid making the same mistake in the future. This repeated process of learning from errors helps improve the model's performance over time. For NLP to do well at computer vision, it employs deep learning techniques. This form of computer vision is sometimes called neural computer vision (NCV), since it makes use of neural networks. NCV therefore interprets visual data based on a statistical, trial and error approach and can deal with context and other subtleties of visual data. === Natural Language Processing === ==== Part-of-speech tagging ==== Part-of-speech (POS) tagging is a crucial component in Natural Language Processing (NLP). It helps resolve human language ambiguity at different analysis levels. In addition, its output (tagged data) can be used in various applications of NLP such as information extraction, information retrieval, question Answering, speech eecognition, text-to-speech conversion, partial parsing, and grammar correction. ==== Parsing ==== Parsing in NLP involves breaking down a text into smaller pieces (phrases) based on grammar rules. If a sentence cannot be parsed, it may contain grammatical errors. In the context of error-driven learning, the parser learns from the mistakes it makes during the parsing process. When an error is encountered, the parser updates its internal model to avoid making the same mistake in the future. This iterative process of learning from errors helps improve the parser's performance over time. In conclusion, error-driven learning plays a crucial role in improving the accuracy and efficiency of NLP parsers by allowing them to learn from their mistakes and adapt their internal models accordingly. ==== Named entity recognition (NER) ==== NER is the task of identifying and classifying entities (such as persons, locations, organizations, etc.) in a text. Error-driven learning can help the model learn from its false positives and false negatives and improve its recall and precision on (NER). In the context of error-driven learning, the significance of NER is quite profound. Traditional sequence labeling methods identify nested entities layer by layer. If an error occurs in the recognition of an inner entity, it can lead to incorrect identification of the outer entity, leading to a problem known as error propagation of nested entities. This is where the role of NER becomes crucial in error-driven learning. By accurately recognizing and classifying entities, it can help minimize these errors and improve the overall accuracy of the learning process. Furthermore, deep learning-based NER methods have shown to be more accurate as they are capable of assembling words, enabling them to understand the semantic and syntactic relationship between various words better. ==== Machine translation ==== Machine translation is a complex task that involves converting text from one language to another. In the context of error-driven learning, the machine translation model learns from the mistakes it makes during the translation process. When an error is encountered, the model updates its internal parameters to avoid making the same mistake in the future. This iterative process of learning from errors helps improve the model's performance over time. ==== Speech recognition ==== Speech recognition is a complex task that involves converting spoken language into written text. In the context of error-driven learning, the speech recognition model learns from the mistakes it makes during the recognition process. When an error is encountered, the model updates its internal parameters to avoid making the same mistake in the future. This iterative process of learning from errors helps improve the model's performance over time. ==== Dialogue systems ==== Dialogue systems are a popular NLP task as they have promising real-life applications. They are also complicated tasks since many NLP tasks deserving study are involved. In the context of error-driven learning, the dialogue system learns from the mistakes it makes during the dialogue process. When an error is encountered, the model updates its internal parameters to avoid making the same mistake in the future. This iterative process of learning from errors helps improve the model's performance over time. == Advantages == Error-driven learning has several advantages over other types of machine learning algorithms: They can learn from feedback and correct their mistakes, which makes them adaptive and robust to noise and changes in the data. They can handle large and high-dimensional data sets, as they do not require explicit feature engineering or prior knowledge of the data distribution. They can achieve high accuracy and performance, as they can learn complex and nonlinear relationships between the input and the output. == Limitations == Although error driven learning has its advantages, their algorithms also have the following limitations: They can suffer from overfitting, which means that they memorize the training data and fail to generalize to new and unseen data. This can be mitigated by using regularization techniques, such as adding a penalty term to the loss function, or reducing the complexity of the model. They can be sensitive to the choice of
UIMA
UIMA ( yoo-EE-mə), short for Unstructured Information Management Architecture, is an OASIS standard for content analytics, originally developed at IBM. It provides a component software architecture for the development, discovery, composition, and deployment of multi-modal analytics for the analysis of unstructured information and integration with search technologies. == Structure == The UIMA architecture can be thought of in four dimensions: It specifies component interfaces in an analytics pipeline. It describes a set of design patterns. It suggests two data representations: an in-memory representation of annotations for high-performance analytics and an XML representation of annotations for integration with remote web services. It suggests development roles allowing tools to be used by users with diverse skills. == Implementations and uses == Apache UIMA, a reference implementation of UIMA, is maintained by the Apache Software Foundation. UIMA is used in a number of software projects: IBM Research's Watson uses UIMA for analyzing unstructured data. The Clinical Text Analysis and Knowledge Extraction System (Apache cTAKES) is a UIMA-based system for information extraction from medical records. DKPro Core is a collection of reusable UIMA components for general-purpose natural language processing.
Joseph Stanislaus Ostoja-Kotkowski
Joseph Stanislaus Ostoja-Kotkowski AM, FRSA (also known as J. S. Ostoja-Kotkowski, Ostoja and Stan Ostoja-Kotkowski; 28 December 1922 – 2 April 1994) was best known for his ground-breaking work in chromasonics, laser kinetics and 'sound and image' productions. He earned recognition in Australia and overseas for his pioneering work in laser sound and image technology. His work included painting (instrumental in developing geometric art in Australia), photography, film-making, theatre design, fabric design, murals, kinetic and static sculpture, stained glass, vitreous enamel murals, op-collages, computer graphics, and laser art. Ostoja flourished between 1940 and 1994. Ostoja's films are still being exhibited. == Biography == Joseph Stanislaus Ostoja-Kotkowski was born in Golub, Poland, on 28 December 1922, descending from an old noble family that was part of the Clan of Ostoja. He studied drawing under Olgierd Vetesco in Przasnysz from 1940-1945. After winning a scholarship, he completed his studies at the Düsseldorf Academy of Fine Arts in Germany in 1949. In 1950 Ostoja migrated to Australia, arriving in Melbourne where he supported himself with work as a labourer. He enrolled at the Victorian School of Fine Arts National Gallery School under Alan Sumner and William Dargie 1950-1955 and there introduced the new abstract expression of Europe both to lecturers and students. He settled in the Adelaide Hills, South Australia, on the Booth estate at Stirling, living under the patronage of the Booth family for over 40 years (Freya Booth, the wife of Edward Stirling Booth, was a daughter of the artist Sir Hans Heysen). His first one-man exhibition was also in South Australia at the Royal Society of Arts, Adelaide. In 1956 Ostoja met and collaborated with Ian Davidson in the production of the short film Five South Australian Artists, and became involved in stage and theatre set design. He co-produced several experimental films again with Ian Davidson, including The Quest of Time in 1957 Ostoja's work in abstract expression began to receive accolades. He won the Cornell Prize for the canvas Form in Landscape. He started to design sets for theatre and dance including for Six Characters in Search of an Author by Luigi Pirandello (1957); the South Australian production of Samuel Beckett's Waiting for Godot (1958); Gaetano Donizetti's Elixir of Love, with novel light settings and modulations, for the Elder Conservatorium of the University of Adelaide which used his techniques for their Opera Workshops (1959); for The Egg; and for two performances of the South Australian Ballet Theatre with light/colour abstract presentations (1959). 1960 This year he designed sets for a new opera group which would eventually grow into the South Australian Opera Company. Among other theatrical events, he designed and executed the scenery for Moon on a Rainbow Shawl by Errol John, and The Teahouse of the August Moon by John Patrick, (a production by the University of Adelaide Theatre Guild). He received artistic satisfaction but little financial reward for these efforts. In this year also, he staged a visual production on the theme of Orpheus, using dance, music and voice with several projectors. This was the first attempt at quadraphonic sound in Australia, working in collaboration with Derek Jolly, who provided the sound and projection equipment. It was also the first demonstration of "Chromasonics" - the science of translating sound into visual images. Ostoja then designed innovative "abstracted" scenery for a production of The Marriage of Figaro and Benjamin Britten's The Turn of the Screw. 1961 Ostoja designed the sets for the controversial South Australian production of Patrick White's The Ham Funeral - also Alan Seymour's Swamp Creatures, both performed by the University of Adelaide Theatre Guild. He designed and constructed six stained glass windows for the Refectory at the University of Adelaide. In this period Ostoja designed special lights and gauzes for difficult effects required in an ambitious production of the opera Don Carlos by the Opera Workshop, for the Elder Conservatorium. 1962 Ostoja designed and built sets for the production of J.B, by Archibald MacLeish, for the second Adelaide Festival of Arts. He exhibited vitreous enamel works in Melbourne's Argus Gallery. Max Harris, in The Bulletin of 20 October 1962, praised Ostoja's sets for My Cousin from Fiji in Union Theatre, Adelaide, and his technique of rear screen projections as later adopted throughout Australia. 1963 Ostoja continued to develop Multi-Image projections, demonstrating for the first time in Australia the concept later to be known as 'audio-visuals!'. Ostoja gave Sir Herbert Read, the art critic, a personal viewing of one of his visual presentations. At Christmas, in the Elder Conservatorium, collaborating again with Derek Jolly, Ostoja gave what was probably the world's first "visual concert", using special projectors and incorporating music, colours and shapes. 1964 With fellow Adelaide artist John Dallwitz, Ostoja co-designed the first of several experimental dance and stage productions in the Adelaide Festival of Arts Sound and Image. The production featured Adelaide dancer Elizabeth_Cameron_Dalman. Also for the Adelaide Festival of Arts of that year, he designed the largest light mosaic ever staged up to that time, upon the facade of an 11-storey building. Ostoja was invited to New Zealand, and exhibited the first electronically generated images in Australia in Melbourne, at the Argus Gallery. His design for the 50-foot (15 m) bas-relief mural for the new B.P. building in Melbourne was the subject of a film which won the "Blue Ribbon" Award in the American Film Festival in New York. 1965 Ostoja designed and made the first light kinetic mural in Australia, and continued to evolve theatrical works using multi-screen and Multi-projector techniques. The Production of Jean Genet's The Balcony was very controversial. With Elizabeth Dalman, Ostoja produced new dance forms for Melbourne Television. He introduced Op Art to Australia, both at South Yarra Gallery in Melbourne, and Gallery A in Sydney. 1966 With John Dallwitz, Ostoja was invited by the Adelaide Festival of Arts to present more experimental theatre, Sound and image 1966. This highly acclaimed production incorporated Australian poetry into the sound, electronic music, and visual images and featured the dancer Antonio Rodrigues. The architect Robin Boyd commissioned Ostoja to design two large Op murals for the Australian Pavilion entrance at the Expo 67. Ostoja was awarded a Churchill Fellowship, which enabled him to have extensive world travel, comparing art and technology in many countries. He began to work with language, contemporary poetry and prose, and computers. 1967 John Dallwitz and Ostoja presented Sound and Image at the Festival of Perth. In Berne, Switzerland, Ostoja received the "Excellence F.I.A.P." Award for innovative photography. 1968 At the Adelaide Festival of Arts, Ostoja and John Dallwitz collaborated again to stage Sound and Image. This was the first theatre production in the world to use a laser beam. It also included the first science fiction play (The Veldt by Ray Bradbury) performed in Australia. Ostoja's theatre methods were increasingly attracting the attention of critics to how plays were staged. "Chromasonics", developed and introduced by Ostoja, was now being used extensively in the entertainment industry. 1969 Ostoja staged Krzysztof Penderecki's St. Luke Passion, a controversial, contemporary religious work. The South Australian The Advertiser wrote an extensive critique of Ostoja's work. Robin Boyd commissioned Ostoja to build a "Chromasonic" exhibit located in the Space Tube at the Australian Pavilion for Expo '70 in Osaka. 1970 Ostoja presented an Australian Aboriginal Dreamtime theme in his "Sound and Image" theatre, working with leading contemporary figures in poetry, music and dance. This was the first production of its kind in Australia, and appeared after the Festival in Melbourne, Sydney, Canberra and Perth. Ostoja's Space Scape mural, sixty feet long by ten feet high, won the Australia-wide competition for a mural for Adelaide Airport. His 120 feet (37 m) high 'light and sound' structure for the Adelaide Festival was the first of its kind in the world. 1971 Ostoja awarded a Creative Arts Fellowship at the Australian National University, Canberra. His 18-month stay resulted in the design and building of a "Chromasonics unit-laser", a 100 feet (30 m) Chromasonic tower, and a world premiere of a Synchronos concert. 1972 With Don Burrows and Don Banks, Ostoja presented Synchronos 72, where one could "hear the colours and see the sounds". Ostoja added Cymatics, developed during the Fellowship, to his workshop repertoire. He was invited to exhibit his photography in the National Gallery, Melbourne. 1973 Ostoja received a Fellowship from the Australian American Education Associatio
Moral graph
In graph theory, a moral graph is used to find the equivalent undirected form of a directed acyclic graph. It is a key step of the junction tree algorithm, used in belief propagation on graphical models. The moralized counterpart of a directed acyclic graph is formed by adding edges between all pairs of non-adjacent nodes that have a common child, and then making all edges in the graph undirected. Equivalently, a moral graph of a directed acyclic graph G is an undirected graph in which each node of the original G is now connected to its Markov blanket. The name stems from the fact that, in a moral graph, two nodes that have a common child are required to be married by sharing an edge. Moralization may also be applied to mixed graphs, called in this context "chain graphs". In a chain graph, a connected component of the undirected subgraph is called a chain. Moralization adds an undirected edge between any two vertices that both have outgoing edges to the same chain, and then forgets the orientation of the directed edges of the graph. == Weakly recursively simplicial == A graph is weakly recursively simplicial if it has a simplicial vertex and the subgraph after removing a simplicial vertex and some edges (possibly none) between its neighbours is weakly recursively simplicial. A graph is moral if and only if it is weakly recursively simplicial. A chordal graph (a.k.a., recursive simplicial) is a special case of weakly recursively simplicial when no edge is removed during the elimination process. Therefore, a chordal graph is also moral. But a moral graph is not necessarily chordal. == Recognising moral graphs == Unlike chordal graphs that can be recognised in polynomial time, Verma & Pearl (1993) proved that deciding whether or not a graph is moral is NP-complete.