AI Art Quora

AI Art Quora — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Text Retrieval Conference

  The Text REtrieval Conference (TREC) is an ongoing series of workshops focusing on a list of different information retrieval (IR) research areas, or tracks. It is co-sponsored by the National Institute of Standards and Technology (NIST) and the Intelligence Advanced Research Projects Activity (part of the office of the Director of National Intelligence), and began in 1992 as part of the TIPSTER Text program. Its purpose is to support and encourage research within the information retrieval community by providing the infrastructure necessary for large-scale evaluation of text retrieval methodologies and to increase the speed of lab-to-product transfer of technology. TREC's evaluation protocols have improved many search technologies. A 2010 study estimated that "without TREC, U.S. Internet users would have spent up to 3.15 billion additional hours using web search engines between 1999 and 2009." Hal Varian the Chief Economist at Google wrote that "The TREC data revitalized research on information retrieval. Having a standard, widely available, and carefully constructed set of data laid the groundwork for further innovation in this field." Each track has a challenge wherein NIST provides participating groups with data sets and test problems. Depending on track, test problems might be questions, topics, or target extractable features. Uniform scoring is performed so the systems can be fairly evaluated. After evaluation of the results, a workshop provides a place for participants to collect together thoughts and ideas and present current and future research work.Text Retrieval Conference started in 1992, funded by DARPA (US Defense Advanced Research Project) and run by NIST. Its purpose was to support research within the information retrieval community by providing the infrastructure necessary for large-scale evaluation of text retrieval methodologies. == Goals == Encourage retrieval search based on large text collections Increase communication among industry, academia, and government by creating an open forum for the exchange of research ideas Speed the transfer of technology from research labs into commercial products by demonstrating substantial improvements retrieval methodologies on real world problems To increase the availability of appropriate evaluation techniques for use by industry and academia including development of new evaluation techniques more applicable to current systems TREC is overseen by a program committee consisting of representatives from government, industry, and academia. For each TREC, NIST provide a set of documents and questions. Participants run their own retrieval system on the data and return to NIST a list of retrieved top-ranked documents. NIST pools the individual result judges the retrieved documents for correctness and evaluates the results. The TREC cycle ends with a workshop that is a forum for participants to share their experiences. == Relevance judgments in TREC == TREC defines relevance as: "If you were writing a report on the subject of the topic and would use the information contained in the document in the report, then the document is relevant." Most TREC retrieval tasks use binary relevance: a document is either relevant or not relevant. Some TREC tasks use graded relevance, capturing multiple degrees of relevance. Most TREC collections are too large to perform complete relevance assessment; for these collections it is impossible to calculate the absolute recall for each query. To decide which documents to assess, TREC usually uses a method call pooling. In this method, the top-ranked n documents from each contributing run are aggregated, and the resulting document set is judged completely. == Various TRECs == In 1992 TREC-1 was held at NIST. The first conference attracted 28 groups of researchers from academia and industry. It demonstrated a wide range of different approaches to the retrieval of text from large document collections .Finally TREC1 revealed the facts that automatic construction of queries from natural language query statements seems to work. Techniques based on natural language processing were no better no worse than those based on vector or probabilistic approach. TREC2 Took place in August 1993. 31 group of researchers participated in this. Two types of retrieval were examined. Retrieval using an ‘ad hoc’ query and retrieval using a ‘routing' query In TREC-3 a small group experiments worked with Spanish language collection and others dealt with interactive query formulation in multiple databases TREC-4 they made even shorter to investigate the problems with very short user statements TREC-5 includes both short and long versions of the topics with the goal of carrying out deeper investigation into which types of techniques work well on various lengths of topics In TREC-6 Three new tracks speech, cross language, high precision information retrieval were introduced. The goal of cross language information retrieval is to facilitate research on system that are able to retrieve relevant document regardless of language of the source document TREC-7 contained seven tracks out of which two were new Query track and very large corpus track. The goal of the query track was to create a large query collection TREC-8 contain seven tracks out of which two –question answering and web tracks were new. The objective of QA query is to explore the possibilities of providing answers to specific natural language queries TREC-9 Includes seven tracks In TREC-10 Video tracks introduced Video tracks design to promote research in content based retrieval from digital video In TREC-11 Novelty tracks introduced. The goal of novelty track is to investigate systems abilities to locate relevant and new information within the ranked set of documents returned by a traditional document retrieval system TREC-12 held in 2003 added three new tracks; Genome track, robust retrieval track, HARD (Highly Accurate Retrieval from Documents) == Tracks == === Current tracks === New tracks are added as new research needs are identified, this list is current for TREC 2018. CENTRE Track – Goal: run in parallel CLEF 2018, NTCIR-14, TREC 2018 to develop and tune an IR reproducibility evaluation protocol (new track for 2018). Common Core Track – Goal: an ad hoc search task over news documents. Complex Answer Retrieval (CAR) – Goal: to develop systems capable of answering complex information needs by collating information from an entire corpus. Incident Streams Track – Goal: to research technologies to automatically process social media streams during emergency situations (new track for TREC 2018). The News Track – Goal: partnership with The Washington Post to develop test collections in news environment (new for 2018). Precision Medicine Track – Goal: a specialization of the Clinical Decision Support track to focus on linking oncology patient data to clinical trials. Real-Time Summarization Track (RTS) – Goal: to explore techniques for real-time update summaries from social media streams. === Past tracks === Chemical Track – Goal: to develop and evaluate technology for large scale search in chemistry-related documents, including academic papers and patents, to better meet the needs of professional searchers, and specifically patent searchers and chemists. Clinical Decision Support Track – Goal: to investigate techniques for linking medical cases to information relevant for patient care Contextual Suggestion Track – Goal: to investigate search techniques for complex information needs that are highly dependent on context and user interests. Crowdsourcing Track – Goal: to provide a collaborative venue for exploring crowdsourcing methods both for evaluating search and for performing search tasks. Genomics Track – Goal: to study the retrieval of genomic data, not just gene sequences but also supporting documentation such as research papers, lab reports, etc. Last ran on TREC 2007. Dynamic Domain Track – Goal: to investigate domain-specific search algorithms that adapt to the dynamic information needs of professional users as they explore in complex domains. Enterprise Track – Goal: to study search over the data of an organization to complete some task. Last ran on TREC 2008. Entity Track – Goal: to perform entity-related search on Web data. These search tasks (such as finding entities and properties of entities) address common information needs that are not that well modeled as ad hoc document search. Cross-Language Track – Goal: to investigate the ability of retrieval systems to find documents topically regardless of source language. After 1999, this track spun off into CLEF. FedWeb Track – Goal: to select best resources to forward a query to, and merge the results so that most relevant are on the top. Federated Web Search Track – Goal: to investigate techniques for the selection and combination of search results from a large number of real on-line web search services. Filtering Track – Goal: to binarily decide retrieval of new

  Read more →
• 

  P4-metric

  The P4 metric (also known as FS or Symmetric F ) enables performance evaluation of a binary classifier. The P4 metric is calculated from precision, recall, specificity, and NPV (negative predictive value). The definition of the P4 metric is similar to that of the F1 metric, however the P4 metric definition addresses criticisms leveled against the definition of the F1 metric. The definition of the P4 metric may, therefore, be understood as an extension of the F1 metric. Like the other known metrics, the P4 metric is a function of: TP (true positives), TN (true negatives), FP (false positives), FN (false negatives). == Justification == The key concept of the P4 metric is to leverage the four key conditional probabilities: P ( + ∣ C + ) {\displaystyle P(+\mid C{+})} — the probability that the sample is positive, provided the classifier result was positive. P ( C + ∣ + ) {\displaystyle P(C{+}\mid +)} — the probability that the classifier result will be positive, provided the sample is positive. P ( C − ∣ − ) {\displaystyle P(C{-}\mid -)} — the probability that the classifier result will be negative, provided the sample is negative. P ( − ∣ C − ) {\displaystyle P(-\mid C{-})} — the probability the sample is negative, provided the classifier result was negative. The main assumption behind this metric is that all the probabilities mentioned above are close to 1 for a properly designed binary classifier. Indeed, P 4 = 1 {\displaystyle \mathrm {P} _{4}=1} if, and only if, all of the probabilities above are equal to 1. Another important feature is that P 4 {\displaystyle \mathrm {P} _{4}} tends to zero any of the above probabilities tend to zero. == Definition == P4 is defined as a harmonic mean of four key conditional probabilities: P 4 = 4 1 P ( + ∣ C + ) + 1 P ( C + ∣ + ) + 1 P ( C − ∣ − ) + 1 P ( − ∣ C − ) = 4 1 p r e c i s i o n + 1 r e c a l l + 1 s p e c i f i c i t y + 1 N P V . {\displaystyle \mathrm {P} _{4}={\frac {4}{{\frac {1}{P(+\mid C{+})}}+{\frac {1}{P(C{+}\mid +)}}+{\frac {1}{P(C{-}\mid -)}}+{\frac {1}{P(-\mid C{-})}}}}={\frac {4}{{\frac {1}{\mathit {precision}}}+{\frac {1}{\mathit {recall}}}+{\frac {1}{\mathit {specificity}}}+{\frac {1}{\mathit {NPV}}}}}.} In terms of TP,TN,FP,FN it can be calculated as follows: P 4 = 4 ⋅ T P ⋅ T N 4 ⋅ T P ⋅ T N + ( T P + T N ) ⋅ ( F P + F N ) . {\displaystyle \mathrm {P} _{4}={\frac {4\cdot \mathrm {TP} \cdot \mathrm {TN} }{4\cdot \mathrm {TP} \cdot \mathrm {TN} +(\mathrm {TP} +\mathrm {TN} )\cdot (\mathrm {FP} +\mathrm {FN} )}}.} == Evaluation of the binary classifier performance == Evaluating the performance of binary classifiers is a multidisciplinary concept. It spans from the evaluation of medical tests, psychiatric tests to machine learning classifiers from a variety of fields. Thus, many of the metrics in use exist under several names, some defined independently. == Properties of P4 metric == Symmetry — contrasting to the F1 metric, P4 is symmetrical. It means - it does not change its value when dataset labeling is changed - positives named negatives and negatives named positives. Range: P 4 ∈ [ 0 , 1 ] {\displaystyle \mathrm {P} _{4}\in [0,1]} . Achieving P 4 ≈ 1 {\displaystyle \mathrm {P} _{4}\approx 1} requires all the key four conditional probabilities being close to 1. For P 4 ≈ 0 {\displaystyle \mathrm {P} _{4}\approx 0} it is sufficient that one of the key four conditional probabilities is close to 0. == Examples, comparing with the other metrics == Dependency table for selected metrics ("true" means depends, "false" - does not depend): Metrics that do not depend on a given probability are prone to misrepresentation when the probability approaches 0. === Example 1: Rare disease detection test === Let us consider a medical test used to detect a rare disease. Suppose a population size of 100000 and 0.05% of the population is infected. Further suppose the following test performance: 95% of all positive individuals are classified correctly (TPR=0.95) and 95% of all negative individuals are classified correctly (TNR=0.95). In such a case, due to high population imbalance and in spite of having high test accuracy (0.95), the probability that an individual who has been classified as positive is in fact positive is very low: P ( + ∣ C + ) = 0.0095. {\displaystyle P(+\mid C{+})=0.0095.} We can observe how this low probability is reflected in some of the metrics: P 4 = 0.0370 {\displaystyle \mathrm {P} _{4}=0.0370} , F 1 = 0.0188 {\displaystyle \mathrm {F} _{1}=0.0188} , J = 0.9100 {\displaystyle \mathrm {J} =\mathbf {0.9100} } (Informedness / Youden index), M K = 0.0095 {\displaystyle \mathrm {MK} =0.0095} (Markedness). === Example 2: Image recognition — cats vs dogs === Consider the problem of training a neural network based image classifier with only two types of images: those containing dogs (labeled as 0) and those containing cats (labeled as 1). Thus, the goal is to distinguish between the cats and dogs. Suppose that the classifier overpredicts in favour of cats ("positive" samples): 99.99% of cats are classified correctly and only 1% of dogs are classified correctly. Further, suppose that the image dataset consists of 100000 images, 90% of which are pictures of cats and 10% are pictures of dogs. In this situation, the probability that the picture containing dog will be classified correctly is pretty low: P ( C − | − ) = 0.01. {\displaystyle P(C-|-)=0.01.} Not all metrics are notice this low probability: P 4 = 0.0388 {\displaystyle \mathrm {P} _{4}=0.0388} , F 1 = 0.9478 {\displaystyle \mathrm {F} _{1}=\mathbf {0.9478} } , J = 0.0099 {\displaystyle \mathrm {J} =0.0099} (Informedness / Youden index), M K = 0.8183 {\displaystyle \mathrm {MK} =\mathbf {0.8183} } (Markedness).

  Read more →
• 

  Restricted Boltzmann machine

  A restricted Boltzmann machine (RBM) (also called a restricted Sherrington–Kirkpatrick model with external field or restricted stochastic Ising–Lenz–Little model) is a generative stochastic artificial neural network that can learn a probability distribution over its set of inputs. RBMs were initially proposed under the name Harmonium by Paul Smolensky in 1986, and rose to prominence after Geoffrey Hinton and collaborators used fast learning algorithms for them in the mid-2000s. RBMs have found applications in dimensionality reduction, classification, collaborative filtering, feature learning, topic modelling, immunology, and even many‑body quantum mechanics. They can be trained in either supervised or unsupervised ways, depending on the task. As their name implies, RBMs are a variant of Boltzmann machines, with the restriction that their neurons must form a bipartite graph: a pair of nodes from each of the two groups of units (commonly referred to as the "visible" and "hidden" units respectively) may have a symmetric connection between them; and there are no connections between nodes within a group. By contrast, "unrestricted" Boltzmann machines may have connections between hidden units. This restriction allows for more efficient training algorithms than are available for the general class of Boltzmann machines, in particular the gradient-based contrastive divergence algorithm. Restricted Boltzmann machines can also be used in deep learning networks. In particular, deep belief networks can be formed by "stacking" RBMs and optionally fine-tuning the resulting deep network with gradient descent and backpropagation. == Structure == The standard type of RBM has binary-valued (Boolean) hidden and visible units, and consists of a matrix of weights W {\displaystyle W} of size m × n {\displaystyle m\times n} . Each weight element ( w i , j ) {\displaystyle (w_{i,j})} of the matrix is associated with the connection between the visible (input) unit v i {\displaystyle v_{i}} and the hidden unit h j {\displaystyle h_{j}} . In addition, there are bias weights (offsets) a i {\displaystyle a_{i}} for v i {\displaystyle v_{i}} and b j {\displaystyle b_{j}} for h j {\displaystyle h_{j}} . Given the weights and biases, the energy of a configuration (pair of Boolean vectors) (v,h) is defined as E ( v , h ) = − ∑ i a i v i − ∑ j b j h j − ∑ i ∑ j v i w i , j h j {\displaystyle E(v,h)=-\sum _{i}a_{i}v_{i}-\sum _{j}b_{j}h_{j}-\sum _{i}\sum _{j}v_{i}w_{i,j}h_{j}} or, in matrix notation, E ( v , h ) = − a T v − b T h − v T W h . {\displaystyle E(v,h)=-a^{\mathrm {T} }v-b^{\mathrm {T} }h-v^{\mathrm {T} }Wh.} This energy function is analogous to that of a Hopfield network. As with general Boltzmann machines, the joint probability distribution for the visible and hidden vectors is defined in terms of the energy function as follows, P ( v , h ) = 1 Z e − E ( v , h ) {\displaystyle P(v,h)={\frac {1}{Z}}e^{-E(v,h)}} where Z {\displaystyle Z} is a partition function defined as the sum of e − E ( v , h ) {\displaystyle e^{-E(v,h)}} over all possible configurations, which can be interpreted as a normalizing constant to ensure that the probabilities sum to 1. The marginal probability of a visible vector is the sum of P ( v , h ) {\displaystyle P(v,h)} over all possible hidden layer configurations, P ( v ) = 1 Z ∑ { h } e − E ( v , h ) {\displaystyle P(v)={\frac {1}{Z}}\sum _{\{h\}}e^{-E(v,h)}} , and vice versa. Since the underlying graph structure of the RBM is bipartite (meaning there are no intra-layer connections), the hidden unit activations are mutually independent given the visible unit activations. Conversely, the visible unit activations are mutually independent given the hidden unit activations. That is, for m visible units and n hidden units, the conditional probability of a configuration of the visible units v, given a configuration of the hidden units h, is P ( v | h ) = ∏ i = 1 m P ( v i | h ) {\displaystyle P(v|h)=\prod _{i=1}^{m}P(v_{i}|h)} . Conversely, the conditional probability of h given v is P ( h | v ) = ∏ j = 1 n P ( h j | v ) {\displaystyle P(h|v)=\prod _{j=1}^{n}P(h_{j}|v)} . The individual activation probabilities are given by P ( h j = 1 | v ) = σ ( b j + ∑ i = 1 m w i , j v i ) {\displaystyle P(h_{j}=1|v)=\sigma \left(b_{j}+\sum _{i=1}^{m}w_{i,j}v_{i}\right)} and P ( v i = 1 | h ) = σ ( a i + ∑ j = 1 n w i , j h j ) {\displaystyle \,P(v_{i}=1|h)=\sigma \left(a_{i}+\sum _{j=1}^{n}w_{i,j}h_{j}\right)} where σ {\displaystyle \sigma } denotes the logistic sigmoid. The visible units of Restricted Boltzmann Machine can be multinomial, although the hidden units are Bernoulli. In this case, the logistic function for visible units is replaced by the softmax function P ( v i k = 1 | h ) = exp ⁡ ( a i k + Σ j W i j k h j ) Σ k ′ = 1 K exp ⁡ ( a i k ′ + Σ j W i j k ′ h j ) {\displaystyle P(v_{i}^{k}=1|h)={\frac {\exp(a_{i}^{k}+\Sigma _{j}W_{ij}^{k}h_{j})}{\Sigma _{k'=1}^{K}\exp(a_{i}^{k'}+\Sigma _{j}W_{ij}^{k'}h_{j})}}} where K is the number of discrete values that the visible values have. They are applied in topic modeling, and recommender systems. === Relation to other models === Restricted Boltzmann machines are a special case of Boltzmann machines and Markov random fields. The graphical model of RBMs corresponds to that of factor analysis. == Training algorithm == Restricted Boltzmann machines are trained to maximize the product of probabilities assigned to some training set V {\displaystyle V} (a matrix, each row of which is treated as a visible vector v {\displaystyle v} ), arg ⁡ max W ∏ v ∈ V P ( v ) {\displaystyle \arg \max _{W}\prod _{v\in V}P(v)} or equivalently, to maximize the expected log probability of a training sample v {\displaystyle v} selected randomly from V {\displaystyle V} : arg ⁡ max W E [ log ⁡ P ( v ) ] {\displaystyle \arg \max _{W}\mathbb {E} \left[\log P(v)\right]} The algorithm most often used to train RBMs, that is, to optimize the weight matrix W {\displaystyle W} , is the contrastive divergence (CD) algorithm due to Hinton, originally developed to train PoE (product of experts) models. The algorithm performs Gibbs sampling and is used inside a gradient descent procedure (similar to the way backpropagation is used inside such a procedure when training feedforward neural nets) to compute weight update. The basic, single-step contrastive divergence (CD-1) procedure for a single sample can be summarized as follows: Take a training sample v, compute the probabilities of the hidden units and sample a hidden activation vector h from this probability distribution. Compute the outer product of v and h and call this the positive gradient. From h, sample a reconstruction v' of the visible units, then resample the hidden activations h' from this. (Gibbs sampling step) Compute the outer product of v' and h' and call this the negative gradient. Let the update to the weight matrix W {\displaystyle W} be the positive gradient minus the negative gradient, times some learning rate: Δ W = ϵ ( v h T − v ′ h ′ T ) {\displaystyle \Delta W=\epsilon (vh^{\mathsf {T}}-v'h'^{\mathsf {T}})} . Update the biases a and b analogously: Δ a = ϵ ( v − v ′ ) {\displaystyle \Delta a=\epsilon (v-v')} , Δ b = ϵ ( h − h ′ ) {\displaystyle \Delta b=\epsilon (h-h')} . A Practical Guide to Training RBMs written by Hinton can be found on his homepage. == Stacked Restricted Boltzmann Machine == The difference between the Stacked Restricted Boltzmann Machines and RBM is that RBM has lateral connections within a layer that are prohibited to make analysis tractable. On the other hand, the Stacked Boltzmann consists of a combination of an unsupervised three-layer network with symmetric weights and a supervised fine-tuned top layer for recognizing three classes. The usage of Stacked Boltzmann is to understand Natural languages, retrieve documents, image generation, and classification. These functions are trained with unsupervised pre-training and/or supervised fine-tuning. Unlike the undirected symmetric top layer, with a two-way unsymmetric layer for connection for RBM. The restricted Boltzmann's connection is three-layers with asymmetric weights, and two networks are combined into one. Stacked Boltzmann does share similarities with RBM, the neuron for Stacked Boltzmann is a stochastic binary Hopfield neuron, which is the same as the Restricted Boltzmann Machine. The energy from both Restricted Boltzmann and RBM is given by Gibb's probability measure: E = − 1 2 ∑ i , j w i j s i s j + ∑ i θ i s i {\displaystyle E=-{\frac {1}{2}}\sum _{i,j}{w_{ij}{s_{i}}{s_{j}}}+\sum _{i}{\theta _{i}}{s_{i}}} . The training process of Restricted Boltzmann is similar to RBM. Restricted Boltzmann train one layer at a time and approximate equilibrium state with a 3-segment pass, not performing back propagation. Restricted Boltzmann uses both supervised and unsupervised on different RBM for pre-training for classification and recognition. The training uses contrastive divergence with

  Read more →
• 

  How to Choose an AI Marketing Tool

  Curious about the best AI marketing tool? An AI marketing tool is software that uses machine learning to help you get more done — it combines speed, accuracy, and an interface that just works. Hands-on testing shows real-world results vary, so a short free trial is the smartest way to decide. Whether you are a beginner or a pro, the right AI marketing tool slots into your workflow and pays for itself fast. This guide breaks down the top picks, their pros and cons, and who each one is best for.

  Read more →
• 

  Native cloud application

  A native cloud application (NCA) is a type of computer software that natively utilizes services and infrastructure from cloud computing providers such as Amazon EC2, Force.com, or Microsoft Azure. NCAs exhibit a combined usage of the three fundamental technologies: Computational grid - loosely, e.g. MapReduce Data grids (e.g. distributed in-memory data caches) Auto-scaling on any managed infrastructure

  Read more →
• 

  Boris Katz

  Boris Gershevich Katz (Russian: Борис Гершевич Кац; born October 5, 1947) is a principal American research scientist (computer scientist) at the MIT Computer Science and Artificial Intelligence Laboratory at the Massachusetts Institute of Technology in Cambridge and head of the Laboratory's InfoLab Group. His research interests include natural language processing and understanding, machine learning and intelligent information access. His brother Victor Kac is a mathematician at MIT. He was able to get out of the USSR with the help of U.S. Senator Ted Kennedy, before the end of the Cold War. Over the last several decades, Boris Katz has been developing the START natural language system that allows the user to access various types of information using English. == Biography == Boris Katz was born on October 5, 1947, in Chișinău in the family of Hersh Katz (died 1976) and Hayki (Klara) Landman (born 1921, Lipcani, Briceni District - died 2006, Cambridge, Middlesex County), who moved from Lipcani, a town located in the northern Bessarabian, to Chișinău before the war. He graduated from Moscow State University and in November 1978, he left for the United States thanks to the personal intervention of Senator Edward M. Kennedy. He defended his thesis as a candidate of physical and mathematical sciences in 1975 under the supervision of Evgenii M. Landis. He currently lives in Boston and heads the InfoLabresearch team at the Laboratory of Informatics and Artificial Intelligence at the Massachusetts Institute of Technology. Boris Katz is the creator of the START information processing system (since 1993 - on the Internet), the author of several works in the field of processing, generation and perception of natural languages, machine learning, and accelerated access to multimedia information. == Family == Brothers - Victor Gershevich Katz, American mathematician, professor at the Massachusetts Institute of Technology; Mikhail Gershevich Katz, Israeli mathematician, graduate of Harvard and Columbia (Ph.D., 1984) universities, professor at Bar-Ilan University, author of the monograph "Systolic Geometry and Topology" (Mathematical Surveys and Monographs, vol. 137. American Mathematical Society: Providence, 2007). Daughter - Luba Katz, a bioinformatics scientist (her husband is Alan Jasanoff, a neuroimaging scientist, a professor at MIT, the son of Harvard University professors Jay Jasanoff and Sheila Jasanoff). == Past works == A Knowledge Entry System for Subject Matter Experts: The goal of SHAKEN project is to enable subject matter experts, without any assistance from AI technologists, to assemble the models of processes and mechanisms so that questions about them can be answered by declarative inference and simulation. Exploiting lexical regularities in designing natural language systems Word sense disambiguation for information retrieval HIKE (HPKB integrated knowledge environment)- a query interface and integrated knowledge environment for HPKB Quantitative evaluation of passage retrieval algorithms for question answering Sticky notes for the semantic web Question answering from the web using knowledge annotation and knowledge mining techniques The role of context in question answering systems

  Read more →
• 

  Myhill–Nerode theorem

  In the theory of formal languages, the Myhill–Nerode theorem provides a necessary and sufficient condition for a language to be regular. The theorem is named for John Myhill and Anil Nerode, who proved it at the University of Chicago in 1957 (Nerode & Sauer 1957, p. ii). == Statement == Given a language L {\displaystyle L} , and a pair of strings x {\displaystyle x} and y {\displaystyle y} , define a distinguishing extension to be a string z {\displaystyle z} such that exactly one of the two strings x z {\displaystyle xz} and y z {\displaystyle yz} belongs to L {\displaystyle L} . Define a relation ∼ L {\displaystyle \sim _{L}} on strings as x ∼ L y {\displaystyle x\;\sim _{L}\ y} if there is no distinguishing extension for x {\displaystyle x} and y {\displaystyle y} . It is easy to show that ∼ L {\displaystyle \sim _{L}} is an equivalence relation on strings, and thus it divides the set of all strings into equivalence classes. The Myhill–Nerode theorem states that a language L {\displaystyle L} is regular if and only if ∼ L {\displaystyle \sim _{L}} has a finite number of equivalence classes, and moreover, that this number is equal to the number of states in the minimal deterministic finite automaton (DFA) accepting L {\displaystyle L} . Furthermore, every minimal DFA for the language is isomorphic to the canonical one (Hopcroft & Ullman 1979). Generally, for any language, the constructed automaton is a state automaton acceptor. However, it does not necessarily have finitely many states. The Myhill–Nerode theorem shows that finiteness is necessary and sufficient for language regularity. Some authors refer to the ∼ L {\displaystyle \sim _{L}} relation as Nerode congruence, in honor of Anil Nerode. == Use and consequences == The Myhill–Nerode theorem may be used to show that a language L {\displaystyle L} is regular by proving that the number of equivalence classes of ∼ L {\displaystyle \sim _{L}} is finite. This may be done by an exhaustive case analysis in which, beginning from the empty string, distinguishing extensions are used to find additional equivalence classes until no more can be found. For example, the language consisting of binary representations of numbers that can be divided by 3 is regular. Given two binary strings x , y {\displaystyle x,y} , extending them by one digit gives 2 x + b , 2 y + b {\displaystyle 2x+b,2y+b} , so 2 x + b ≡ 2 y + b mod 3 {\displaystyle 2x+b\equiv 2y+b\mod 3} iff x ≡ y mod 3 {\displaystyle x\equiv y\mod 3} . Thus, 00 {\displaystyle 00} (or 11 {\displaystyle 11} ), 01 {\displaystyle 01} , and 10 {\displaystyle 10} are the only distinguishing extensions, resulting in the 3 classes. The minimal automaton accepting our language would have three states corresponding to these three equivalence classes. Another immediate corollary of the theorem is that if for a language L {\displaystyle L} the relation ∼ L {\displaystyle \sim _{L}} has infinitely many equivalence classes, it is not regular. It is this corollary that is frequently used to prove that a language is not regular. == Generalizations == The Myhill–Nerode theorem can be generalized to tree automata.

  Read more →
• 

  Ayanna Howard

  Ayanna MacCalla Howard (born January 24, 1972) is an American roboticist, entrepreneur, and educator currently serving as the dean of the College of Engineering at Ohio State University. Assuming this role in March 2021, Howard became the first woman to lead the Ohio State College of Engineering. Howard previously served as the chair of the School of Interactive Computing in the Georgia Tech College of Computing, the Linda J. and Mark C. Smith Endowed Chair in Bioengineering in the School of Electrical and Computer Engineering, and the director of the Human-Automation Systems (Humans) Lab. == Early life and education == As a little girl, Howard was interested in aliens and robots. Her favorite TV show was The Bionic Woman. Howard received her B.S. in engineering from Brown University in 1993 and her M.S. and Ph.D. in electrical engineering from the University of Southern California in 1994 and 1999, respectively. Her thesis, Recursive Learning for Deformable Object Manipulation, was advised by George A. Bekey. In addition, Howard's Doctoral thesis was triggered by the AIDS epidemic with focus on sorting hospital waste by using robots. Howard has also received an MBA from Claremont Graduate University. == Career == Howard's early interest in artificial intelligence led her to pursue a senior position at Seattle-based Axcelis Inc, where she helped develop Evolver, the first commercial genetic algorithm, and Brainsheet, a neural network developed in partnership with Microsoft. From 1993 to 2005, she worked at the NASA Jet Propulsion Laboratory, holding multiple roles such as senior robotics researcher and deputy manager in the Office of the Chief Scientist. In 2005, she joined Georgia Tech as an associate professor and founder of the Human-Automation Systems (Humans) lab. She has also served as the associate director of research for Georgia Tech's Institute for Robotics and Intelligent Machines and as chair of the multidisciplinary robotics Ph.D. program at Georgia Tech. In 2017, she became the chair of the School of Interactive Computing at Georgia Tech. In 2008, Howard received worldwide attention for her SnoMote robots, designed to study the impact of global warming on the Antarctic ice shelves. In 2013, she founded Zyrobotics, which has released their first suite of therapy and educational products for children with special needs. Howard has authored 250 publications in reputable journals and conferences, including serving as co-editor/co-author of more than a dozen books and book chapters. She has also received four patents and given over 140 invited talks and keynotes. She is a fellow of the Association for the Advancement of Artificial Intelligence (AAAI) and the Institute of Electrical and Electronics Engineers (IEEE). Among her many honors, Howard received the Computer Research Association's A. Nico Habermann Award and the Richard A. Tapia Achievement Award. In a 2020 interview on Marketplace, Howard outlined how companion robots could alleviate the effects of social distancing caused by the COVID-19 pandemic in the United States. On November 30, 2020, the Columbus Dispatch reported that Howard would become the next dean of the College of Engineering at Ohio State University on March 1, pending approval by the board of trustees. On March 1, 2021, she assumed this role, becoming the first woman to hold the position. In 2021, Howard received the Athena Lecturer Award from Association for Computing Machinery (ACM) for her Contributions to Robotics, AI and Broadening Participation in Computing. In June 2022, Howard was elected a trustee of Brown University. == Research == Howard's research interests include human-robot interaction, assistive/rehabilitation robotics, science-driven/field robotics, and perception, learning, and reasoning. Howard's research and published works span across various topics in robotics and AI, including intelligent learning, virtual reality for rehabilitation and robotics in the role of pediatric therapy. Her research is highlighted by her focus on technology development for intelligent agents that must interact with and in a human-centered world. Her work, which addresses issues of human-robot interaction, learning, and autonomous control, has resulted in more than 200 peer-reviewed publications. == Honors and awards == Howard's numerous accomplishments have been documented in more than a dozen featured articles. In 2003, she was named to the MIT Technology Review TR100 as one of the top 100 innovators in the world under the age of 35. She was featured in Time magazine's "Rise of the Machines" article in 2004. She was also featured in a USA Today Science & Space article. Some of Howard's notable awards include: Lew Allen Award for Excellence (formerly the Director's Research Achievement Award of the Jet Propulsion Laboratory) for significant technical contributions, 2001 MIT Technology Review Top 100 Young Innovators of the Year, 2003 NAE Gilbreth Lectureship, 2010 A. Richard Newton Educator ABIE Award, Anita Borg Institute, 2014 Computer Research Association's A. Nico Habermann Award, 2016 Brown Engineering Alumni Medal (BEAM), 2016 AAAS-Lemelson Invention Ambassador, 2016-2017 Atlanta magazine's Women Making a Mark, 2017 Walker's Legacy #WLPower25 Atlanta Award, 2017 Forbes America's Top 50 Women In Tech, 2018 ACM Athena Lecturer Award, 2021 2021 class of Fellows of the American Association for the Advancement of Science. IEEE Fellow, 2021, "for contributions to human-robot interaction systems" 2023 AAAI/EAAI Patrick Henry Winston Outstanding Educator Award

  Read more →
• 

  NCAA transfer portal

  The NCAA transfer portal is a National Collegiate Athletic Association (NCAA) application, database, and compliance tool that facilitates student athletes' transfers between member institutions. It is intended to bring greater transparency to the transfer process and to enable student athletes to publicize their desire to transfer. The transfer portal is an NCAA-wide database covering all three NCAA divisions, although most media coverage of the transfer portal involves its use in the top-level Division I (D-I). The portal launched on October 15, 2018. Regulations adopted in 2021 allowed student-athletes in D-I football, men's and women's basketball, men's ice hockey, and baseball to transfer schools using the portal once without sitting out a year. In 2024, the NCAA authorized athletes unlimited transfers. == Process == For Divisions I and II, once an athlete desiring to transfer informs their school; the school must enter the athlete's name in the database within two business days. Then coaches and staff from other universities may contact the athlete about potentially transferring. Before the January 2026 NCAA convention, Division III schools were allowed, but not required, to enter such a student into the portal. A proposal to require use of the portal in that division was approved at the convention. The timeline for D-III members to enter athletes into the portal differs from that of the other divisions. Athletes wishing to enter the portal must first complete an educational module. Once completed, the school has seven calendar days to enter the athlete's transfer request into the portal. == Transfer windows == On August 31, 2022, the D-I board adopted a series of changes to transfer rules, introducing the concept of transfer windows, similar to those used in professional soccer worldwide. Student-athletes who wish to take advantage of the one-time transfer rule must, under normal circumstances, enter the portal within a designated window for their sport. These windows are slightly different for each NCAA sport, but are broadly grouped by the NCAA's three athletic "seasons". At that time, the windows were as follows: Fall sports – A 45-day winter window opening the day after championship selections are made in that sport, and a spring window from May 1–15. According to the NCAA, "reasonable accommodations" would be made for participants in football's FBS and FCS championship games (respectively the College Football Playoff National Championship and Division I Football Championship Game), both of which take place in early January. Participants in those games had a 14-day window opening on the day after the championship game, as well as the spring window. Winter sports – A 60-day window opening the day after championship selections are made in that sport. Spring sports – A winter window from December 1–15, and a 45-day spring window opening the day after championship selections are made in that sport. For sports included in the NCAA Emerging Sports for Women program, transfer windows are the same as those for fully recognized NCAA sports. As with fully recognized NCAA sports, transfer windows linked to championship events open on the day after selections are made for the generally recognized championship events in emerging sports. Student-athletes whose athletic aid is reduced, canceled, or not renewed by their school, as well as those affected by a university's elimination of a sports team, may enter the transfer portal at any time without penalty. A slightly different exception applies to those undergoing a head coaching change; student-athletes so affected in sports other than Division I football can enter the portal within 30 days of the change, starting on the day after the coach's departure is announced. The coaching change window also applied to Division I football before October 2025. Less than a month after transfer windows were adopted, the Division I Council adopted a change that affected only graduate transfers. Student-athletes who are set to graduate with remaining athletic eligibility, and plan to continue competition as postgraduate students, were exempt from transfer windows. They could enter the portal at any time during the academic year, and were not subject to the standard deadlines of May 1 for fall and winter sports and July 1 for spring sports. In April 2024, graduate transfers became subject to the same deadlines as all other transfer students. This change did not affect windows for student-athletes affected by a head coaching change, a loss of athletic aid, or the discontinuation of a team. Because the Ivy League allows neither redshirting nor athletic participation by graduate students, athletes at its member schools who are set to complete four years of attendance but still have remaining athletic eligibility may enter the portal at any time during their fourth academic year of attendance. In October 2024, the Division I Council reduced transfer windows in football and basketball to a total of 30 days. For FBS and FCS football, the fall window opened for 20 days, starting on the Monday after FBS conference championship games. Participants in postseason play had a 5-day window that opened on the day after each team's final game. A 10-day spring window opened in mid-April. In men's and women's basketball, a single 30-day window opens on the day after the second round of each Division I tournament concludes. The existing exceptions regarding head coaching changes, a loss of athletic aid, or the discontinuation of a team remained in place. Almost exactly a year later, Division I adopted more significant changes to the football transfer portal for both FBS and FCS. The previous two windows were abolished and replaced by a single window that opens from January 2–16. Participants in the College Football Playoff National Championship—the only game in FBS or FCS played after the closure of the new window—receive a 5-day window that opens on the day after that game. The window for players undergoing a head coaching change was also reduced. A new window of 15 days opens five calendar days after the hiring or public announcement of a new head coach. Should a school fail to hire or publicly announce a new head coach within 30 days after the previous coach's departure, the window will open on the 31st day after departure, provided that the 31st day is no earlier than January 3. This particular window, also open for 15 days, may open at any time before June 30. No change was announced to the exceptions for those affected by a loss of athletic aid or the discontinuation of a team. == Impact on high school recruiting == Effective July 1, 2025, the NCAA Division I Board of Directors implemented new DI roster limits following the court-approved House settlement. Additionally, according to the NCAA, "NCAA rules for Division I programs will no longer include sport-specific scholarship limits." As a result, many top Division I programs, especially those in power conferences, are relying heavily on the transfer portal to bring in conference- and national-level student-athletes. This shift in recruiting focus has already been exemplified across Division I men's and women's track and field especially, beginning in the recruitment cycle for 2025 college entries. Track and field coaches formerly managing rosters of 120-plus (60-plus men and 60-plus women) are now limited to 45 per side for a total of 90 roster spots across men's and women's track and field, meaning they are recruiting fewer student-athletes out of high school and more immediately impactful scholarship-worthy student-athletes via the transfer portal. Roster limits for track and field teams are even more stringent in the Southeastern Conference (SEC): 35 men and 35 women. For high school track and field athletes seeking opportunities with top DI programs, they no longer need to display potential to be point-scorers, but demonstrate the ability to contribute immediately, often by competing at a level aligned with conference scoring standards.

  Read more →
• 

  HOCR

  hOCR is an open standard of data representation for formatted text obtained from optical character recognition (OCR). The definition encodes text, style, layout information, recognition confidence metrics and other information using Extensible Markup Language (XML) in the form of Hypertext Markup Language (HTML) or XHTML. == Software == The following OCR software can output the recognition result as hOCR file: OCRopus Tesseract Cuneiform ghostscript HebOCR gcv2hocr gImageReader == Example == The following example is an extract of an hOCR file: The recognized text is stored in normal text nodes of the HTML file. The distribution into separate lines and words is here given by the surrounding span tags. Moreover, the usual HTML entities are used, for example the p tag for a paragraph. Additional information is given in the properties such as: different layout elements such as "ocr_par", "ocr_line", "ocrx_word" geometric information for each element with a bounding box "bbox" language information "lang" some confidence values "x_wconf" == bbox == === General === The Layout of the Bounding Box Object or bbox Object is Grammar. property-name = "bbox" property-value = uint uint uint uint ==== Example ==== bbox 0 0 100 200 The bbox - short for "bounding box" - of an element is a rectangular box around this element, which is defined by the upper-left corner (x0, y0) and the lower-right corner (x1, y1). the values are with reference to the top-left corner of the document image and measured in pixels the order of the values are x0 y0 x1 y1 = "left top right bottom" ===== Usage ===== Use x_bboxes below for character bounding boxes. Do not use bbox unless the bounding box of the layout component is, in fact, rectangular, some non-rectangular layout components may have rectangular bounding boxes if the non-rectangularity is caused by floating elements around which text flows. The bounding box bbox of this line is shown in blue and it is span by the upper-left corner (10, 20) and the lower-right corner (160, 30). All coordinates are measured with reference to the top-left corner of the document image which border is drawn in black. == Searchable PDF files == The hOCR format is most commonly used in order to make searchable PDF files or as an extracted metadata of the PDF file. In order to create searchable PDF files we can use a scanned document image and a .hocr file of the particular image. We can use the following open source tools in order to achieve that. === hocr-tools === Source: hocr-tools is an open source library written in Python. It has a command-line utility attached in the scripts called hocr-pdf that enables us to convert standard hocr files to a searchable PDF file. It is also worth noting that the version for dealing with hocr files in RTL or non-Latin scripts like Arabic, we need to use the GitHub repository at the moment. hocr-pdf We can use the hocr-pdf utility using the following basic syntax. hocr-pdf—savefile final.pdf folder_images_and_hocr The folder_images_and_hocr must contain the respective .jpg and .hocr format files with their file extensions changed. ==== Known issues ==== Some of the known issues of hocr-pdf script in PyPI installation are the following. Not up to date with GitHub repository. hocr-pdf is broken on line 134 due to decodebytes() depreciated after Python 3.1 ==== Known fixes ==== Compile hocr-tools using latest GitHub repository. === hocr2pdf === hocr2pdf is another library that supports the conversion of hocr files. It is written in C++ and is cross-compatible with other libraries. It also has support for UTF-8 languages but that may require some additional debugging and browsing through some google conversation records to achieve that. According to Ubuntu Manpages,ExactImage is a fast C++ image processing library. Unlike many other library frameworks it allows operation in several color spaces and bit depths natively, resulting in low memory and computational requirements. hocr2pdf creates well layouted, searchable PDF files from hOCR (annotated HTML) input obtained from an OCR system. == hOCR to PDF attempts == In addition to the following discussed and stable libraries there have been many contributions to the hOCR format over the years with support from many of the early adopters of this format. You can get access to inlaying text on an Image with hOCR and converting that in a PDF file using Python 2 with this 12-year-old script as of 2021. This script can also be updated and made functional by converting that Python 2 Source code to Python 3 Supported Context. - HOCRConverter by jbrinley (Documentation) === HOCRConverter === The HOCRConverter is a script written in Python 2.x that can used in order to convert a hOCR file with a specified image file in order to convert it to a searchable PDF file. You can see the documentation using the link above. ==== Known issues ==== Has not been tested. Does not natively support Python 3.x

  Read more →
• 

  Rob Fergus

  Rob Fergus is a British-American computer scientist working primarily in the fields of machine learning, deep learning, representational learning, and generative models. He is a professor of computer science at Courant Institute of Mathematical Sciences at New York University (NYU) and a research scientist at DeepMind. Fergus developed ZFNet in 2013 together with M.D. Zeiler, his PhD student in NYU. Fergus co-founded Meta AI (then known as Facebook Artificial Intelligence Research (FAIR)) along with Yann Le Cun in September 2013. In 2009, Rob Fergus co-founded the Computational Intelligence, Learning, Vision, and Robotics (CILVR) Lab at NYU along with Yann Le Cun. == Awards and recognition == Rob Fergus has been recognized in academia and received the following awards: NSF Faculty Early Career Development Program (CAREER) Sloan Research Fellowship Test-of-time awards at ECCV, CVPR and ICLR == Notable PhD students == Matt Zeiler (Clarifai founder) Wojciech Zaremba (OpenAI co-founder) Denis Yarats (Perplexity co-founder) Alex Rives (EvolutionaryScale co-founder; faculty at MIT)

  Read more →
• 

  Hapax legomenon

  In corpus linguistics, a hapax legomenon ( also or ; pl. hapax legomena; sometimes abbreviated to hapax, plural hapaxes) is a word or an expression that occurs only once within a context: either in the written record of an entire language, in the works of an author, or in a single text. The term is also sometimes used to describe a word that occurs in just one of an author's works but more than once in that particular work. Hapax legomenon is a transliteration of Greek ἅπαξ λεγόμενον, meaning "said once". The related terms dis legomenon, tris legomenon, and tetrakis legomenon respectively (, , ) refer to double, triple, or quadruple occurrences, but are far less commonly used. Hapax legomena are quite common, as predicted by Zipf's law, which states that the frequency of any word in a corpus is inversely proportional to its rank in the frequency table. For large corpora, about 40% to 60% of the words are hapax legomena, and another 10% to 15% are dis legomena. Thus, in the Brown Corpus of American English, about half of the 50,000 distinct words are hapax legomena within that corpus. Hapax legomenon refers to the appearance of a word or an expression in a body of text, not to either its origin or its prevalence in speech. It thus differs from a nonce word, which may never be recorded, may find currency and may be widely recorded, or may appear several times in the work which coins it, and so on. == Significance == Hapax legomena in ancient texts are usually difficult to decipher, since it is easier to infer meaning from multiple contexts than from just one. For example, many of the remaining undeciphered Mayan glyphs are hapax legomena, and Biblical (particularly Hebrew; see § Hebrew) hapax legomena sometimes pose problems in translation. Hapax legomena also pose challenges in natural language processing. Some scholars consider Hapax legomena useful in determining the authorship of written works. P. N. Harrison, in The Problem of the Pastoral Epistles (1921) made hapax legomena popular among Bible scholars, when he argued that there are considerably more of them in the three Pastoral Epistles than in other Pauline Epistles. He argued that the number of hapax legomena in a putative author's corpus indicates his or her vocabulary and is characteristic of the author as an individual. Harrison's theory has faded in significance due to a number of problems raised by other scholars. For example, in 1896, W. P. Workman found the following numbers of hapax legomena in each Pauline Epistle: At first glance, the last three totals (for the Pastoral Epistles) are not out of line with the others. To take account of the varying length of the epistles, Workman also calculated the average number of hapax legomena per page of the Greek text, which ranged from 3.6 to 13, as summarized in the diagram on the right. Although the Pastoral Epistles have more hapax legomena per page, Workman found the differences to be moderate in comparison to the variation among other Epistles. This was reinforced when Workman looked at several plays by Shakespeare, which showed similar variations (from 3.4 to 10.4 per page of Irving's one-volume edition), as summarized in the second diagram on the right. Apart from author identity, there are several other factors that can explain the number of hapax legomena in a work: text length: this directly affects the expected number and percentage of hapax legomena; the brevity of the Pastoral Epistles also makes any statistical analysis problematic. text topic: if the author writes on different subjects, of course many subject-specific words will occur only in limited contexts. text audience: if the author is writing to a peer rather than a student, or their spouse rather than their employer, again quite different vocabulary will appear. time: over the course of years, both the language and an author's knowledge and use of language will change. In the particular case of the Pastoral Epistles, all of these variables are quite different from those in the rest of the Pauline corpus, and hapax legomena are no longer widely accepted as strong indicators of authorship; those who reject Pauline authorship of the Pastorals rely on other arguments. There are also subjective questions over whether two forms amount to "the same word": dog vs. dogs, clue vs. clueless, sign vs. signature; many other gray cases also arise. The Jewish Encyclopedia points out that, although there are 1,500 hapaxes in the Hebrew Bible, only about 400 are not obviously related to other attested word forms. A final difficulty with the use of hapax legomena for authorship determination is that there is considerable variation among works known to be by a single author, and disparate authors often show similar values. In other words, hapax legomena are not a reliable indicator. Authorship studies now usually use a wide range of measures to look for patterns rather than relying upon single measurements. == Computer science == In the fields of computational linguistics and natural language processing (NLP), esp. corpus linguistics and machine-learned NLP, it is common to disregard hapax legomena (and sometimes other infrequent words), as they are likely to have little value for computational techniques. This disregard has the added benefit of significantly reducing the memory use of an application, since, by Zipf's law, many words are hapax legomena. == Examples == The following are some examples of hapax legomena in languages or corpora. === Arabic === In the Qurʾān: The proper nouns Iram (Q 89:7, Iram of the Pillars), Bābil (Q 2:102, Babylon), Bakka(t) (Q 3:96, Bakkah), Jibt (Q 4:51), Ramaḍān (Q 2:185, Ramadan), ar-Rūm (Q 30:2, Byzantine Empire), Tasnīm (Q 83:27), Qurayš (Q 106:1, Quraysh), Majūs (Q 22:17, Magian/Zoroastrian), Mārūt (Q 2:102, Harut and Marut), Makka(t) (Q 48:24, Mecca), Nasr (Q 71:23), (Ḏū) an-Nūn (Q 21:87) and Hārūt (Q 2:102, Harut and Marut) occur only once. zanjabīl (زَنْجَبِيل – ginger) is a Qurʾānic hapax (Q 76:17). zamharīr (زَمْهَرِيرًۭ) is a Qurʾānic hapax (Q 76:13), usually glossed as referring to extreme cold. The epitheton ornans aṣ-ṣamad (الصَّمَد – the One besought) is a Qurʾānic hapax (Q 112:2). ṭūd (طُودْ - mountain) is a Qurʾānic hapax (Q 26:63). === Chinese and Japanese === Classical Chinese and Japanese literature contains many Chinese characters that feature only once in the corpus, and their meaning and pronunciation has often been lost. Known in Japanese as kogo (孤語), literally "lonely characters", these can be considered a type of hapax legomenon. For example, the Classic of Poetry (c. 1000 BC) uses the character 篪 exactly once in the verse 「伯氏吹塤, 仲氏吹篪」, and it was only through the discovery of a description by Guo Pu (276–324 AD) that the character could be associated with a specific type of ancient flute. === English === It is fairly common for authors to "coin" new words to convey a particular meaning or for the sake of entertainment, without any suggestion that they are "proper" words. For example, P.G. Wodehouse and Lewis Carroll frequently coined novel words. Indexy, below, appears to be an example of this. Flother, as a synonym for snowflake, is a hapax legomenon of written English found in a manuscript entitled The XI Pains of Hell (c. 1275). Honorificabilitudinitatibus is a hapax legomenon of Shakespeare's works, coming from Erasmus' Adagia Indexy, in Bram Stoker's Dracula, used as an adjective to describe a situational state with no other further use in the language: "If that man had been an ordinary lunatic I would have taken my chance of trusting him; but he seems so mixed up with the Count in an indexy kind of way that I am afraid of doing anything wrong by helping his fads." Manticratic, meaning "of the rule by the Prophet's family or clan", was apparently invented by T. E. Lawrence and appears once in Seven Pillars of Wisdom. Nortelrye, a word for "education", occurs only once in Chaucer's The Reeve's Tale. Sassigassity, perhaps with the meaning of "audacity", occurs only once in Dickens's short story "A Christmas Tree". Slæpwerigne, "sleep-weary", occurs exactly once in the Old English corpus, in the Exeter Book. There is debate over whether it means "weary with sleep" or "weary for sleep". === German === The name of the 9th-century poem Muspilli is a back-formation from "muspille", Old High German hapax legomenon of unclear meaning only found in this text (see Muspilli § Etymology for discussion). === Ancient Greek === According to classical scholar Clyde Pharr, "the Iliad has 1,097 hapax legomena, while the Odyssey has 868". Others have defined the term differently, however, and count as few as 303 in the Iliad and 191 in the Odyssey. panaōrios (παναώριος), ancient Greek for "very untimely", is one of many words that occur only once in the Iliad. The Greek New Testament contains 686 local hapax legomena, which are sometimes called "New Testament hapaxes". 62 of these occur in 1 Peter and 54 occur in 2 Peter

  Read more →
• 

  Chai AI

  Chai AI (also known as Chai Research) is an American artificial intelligence (AI) company that operates a chatbot platform where users can create, share, and interact with character-based chatbots powered by large language models (LLMs). The company is headquartered in Palo Alto, California. == History == Chai was founded in 2021 by William Beauchamp, a former quantitative trader educated at Cambridge, who began developing the initial prototype in 2020 in Cambridge, England. The company launched in 2021 and relocated to Palo Alto in 2022. In June 2023, Chai raised US$2 million in a pre-seed funding round. In September 2023, GPU cloud provider CoreWeave invested in the company at a valuation of US$450 million. In January 2024, Chai Research reported a $450 million valuation following an investment from cloud computing provider CoreWeave. In July 2024, authorities in Belgium launched an investigation into the company following reports of a man dying by suicide following extensive chats on the Chai app. == Reception == In 2025, Chai Research announced that their app had over 10 million downloads and 1 million daily active users. In 2022, Canadian writer Sheila Heti published her conversations with various chatbots in The Paris Review, including Chai AI chatbots, and later used Chai AI chatbots in the development of a novel. Heti said that she had found that Chai's default chatbot, Eliza, "had turned out to be like most of the other bots on the site—primarily interested in sex". In January 2026, CHAI introduced country-based blocks on its free, ad-supported tier, initially providing the community with little information and inaccurate lists of the affected countries. Users in "Low tier" regions are required to subscribe to use the app in any capacity, while "High tier" regions will retain free ad-supported access. In response to backlash, the company announced a "Basic" tier with unlimited messages and ads, intended to cover electricity and infrastructure costs. In February 2026, CHAI was criticized for the unannounced implementation of restrictive "token limits" that abruptly blocked messages and froze conversations for both free and paid subscribers. Users generating long responses or utilizing roleplay features found their quotas exhausted within minutes, resulting in lockouts lasting anywhere from a few hours to a week. == Technology == Chai allows users to create characters and interact with chatbot versions of those characters. These chatbots use the open-source large language model (LLM) GPT-J originally developed by EleutherAI. Chai AI chatbots can be shared on the platform for other users to interact with.

  Read more →
• 

  Corpus-assisted discourse studies

  Corpus-assisted discourse studies (abbr.: CADS) is related historically and methodologically to the discipline of corpus linguistics. The principal endeavor of corpus-assisted discourse studies is the investigation, and comparison of features of particular discourse types, integrating into the analysis the techniques and tools developed within corpus linguistics. These include the compilation of specialised corpora and analyses of word and word-cluster frequency lists, comparative keyword lists and, above all, concordances. A broader conceptualisation of corpus-assisted discourse studies would include any study that aims to bring together corpus linguistics and discourse analysis. Such research is often labelled as corpus-based or corpus-assisted discourse analysis, with the term CADS coined by a research group in Italy (Partington 2004) for a specific type of corpus-assisted discourse analysis (see the section 'in different countries' below). == Aims == Corpus-assisted discourse studies aim to uncover non-obvious meaning, that is, meaning which might not be readily available to naked-eye perusal. Much of what carries meaning in texts is not open to direct observation: “you cannot understand the world just by looking at it” (Stubbs [after Gellner 1959] 1996: 92). We use language “semi-automatically”, in the sense that speakers and writers make semi-conscious choices within the various complex overlapping systems of which language is composed, including those of transitivity, modality (Michael Halliday 1994), lexical sets (e.g. freedom, liberty, deliverance), modification, and so on. Authors themselves are, famously, generally unaware of all the meanings their texts convey. By combining the quantitative research approach, that is, statistical analysis of large amounts of the discourse in question - more precisely, large numbers of tokens of the discourse type under study contained in a corpus - with the more qualitative research approach typical of discourse analysis, that is, the close, detailed examination of particular stretches of discourse it may be possible to better understand the processes at play in the discourse type and to gain access to non-obvious meanings. Aims can differ in other types of corpus-based or corpus-assisted discourse analysis; but in general such studies combine quantitative and qualitative research and aim to shed light on discourses, registers, discourse patterns, etc., with the help of a corpus linguistic approach. Specific aims and techniques depend on the relevant project. == In different countries == In German-speaking countries: Pioneering work in corpus-based discourse analysis was conducted in Europe, in particular by Hardt-Mautner/Mautner (1995, 2000) and Stubbs (1996, 2001). CADS and other types of corpus-based discourse analysis are inspired by this important early work. In Italy: A considerable body of research has been conducted in Italy either by individual researchers or under the aegis of combined inter-university projects such as Newspool (Partington et al. 2004) and CorDis (Morley and Bayley eds, 2009). It has concentrated on political and media language, mainly because a nucleus of linguists in Italian universities work in Political Science faculties and are increasingly interested in the use of corpus techniques to conduct a particular type of sociopolitical discourse analysis, including the unearthing of noteworthy ideological metaphors and motifs in the language of political figures and institutions. Italian researchers also developed Modern diachronic corpus-assisted discourse studies (MD-CADS). This approach contrasts the language contained in comparable corpora from different but recent points in time in order to track changes in modern language usage but also social, cultural and political changes over modern times, as reflected - and shared among people - in language. It is this Italian body of research that makes most use of the label CADS. In the UK: Linguists in the UK tend to undertake corpus-based critical discourse analysis (CDA). CDA generally adopts a leftist political stance, focusing on the ways that social and political domination is reproduced by text and talk. This type of corpus-based research was originally associated with Lancaster University (Baker et al. 2008), but has spread more widely since. Such work typically studies the discourses around particular groups of people (e.g. Muslims, people with disabilities) or concepts/events (e.g. feminism, same-sex marriage). In Australia: Corpus-based discourse analysis is undertaken by a growing number of Australian researchers, most often on media texts. Some of this work aims to elucidate specific features of discourse types (news, social media, television series, etc.), while other work is rooted in the tradition of corpus-based critical discourse analysis. == Comparison with traditional corpus linguistics == Traditional corpus linguistics has, quite naturally, tended to privilege the quantitative approach. In the drive to produce more authentic dictionaries and grammars of a language, it has been characterised by the compilation of some very large corpora of heterogeneric discourse types in the desire to obtain an overview of the greatest quantity and variety of discourse types possible, in other words, of the chimerical but useful fiction called the “general language” (“general English”, “general Italian”, and so on). This has led to the construction of immensely valuable research tools such as the Bank of English and the British National Corpus. Some branches of corpus linguistics have also promoted an approach that is "corpus-driven", in which we need, grammatically speaking, a mental tabula rasa to free ourselves of the baleful prejudice exerted by traditional models and allow the data to speak entirely for itself. The aim of corpus-assisted discourse studies and related approaches is radically different. Here the aim of the exercise is to acquaint oneself as much as possible with the discourse type(s) in hand. Researchers typically engage with their corpus in a variety of ways. As well as via wordlists and concordancing, intuitions for further research can also arise from reading or watching or listening to parts of the data-set, a process which can help provide a feel for how things are done linguistically in the discourse-type being studied. Corpus-assisted discourse analysis is also typically characterised by the compilation of ad hoc specialised corpora, since very frequently there exists no previously available collection of the discourse type in question. Often, other corpora are utilized in the course of a study for purposes of comparison. These may include pre-existing corpora or may themselves need to be compiled by the researcher. In some sense, all work with corpora – just as all work with discourse - is properly comparative. Even when a single corpus is employed, it is used to test the data it contains against another body of data. This may consist of the researcher's intuitions, or the data found in reference works such as dictionaries and grammars, or it may be statements made by previous authors in the field. == CADS as a specific type of corpus-based discourse analysis == Researchers in Italy have developed CADS as a specific type of corpus-based discourse analysis, creating a standard set of methods: 'A basic, standard methodology in CADS may resemble the following:' Step 1: Decide upon the research question; Step 2: Choose, compile or edit an appropriate corpus; Step 3: Choose, compile or edit an appropriate reference corpus / corpora; Step 4: Make frequency lists and run a keywords comparison of the corpora; Step 5: Determine the existence of sets of key items; Step 6: Concordance interesting key items (with differing quantities of co-text); Step 7: (Possibly) refine the research question and return to Step 2. This basic procedure can of course vary according to individual research circumstances and requirements. A particular way of conceptualising research questions has also been proposed in such CADS projects: Given that P is a discourse participant (or possibly an institution) and G is a goal, often a political goal: How does P achieve G with language? What does this tell us about P? Comparative studies: how do P1 and P2 differ in their use of language? Does this tell us anything about their different principles and objectives? A second general type of CADS research question, which might be asked of interactive discourse data, has been conceptualised as follows: Given that P(x) is a particular participant or set of participants, DT is the discourse type, and R is an observed relationship between or among participants: How do {P(a), P(b)...P(n)} achieve / maintain R in DT [using language]? Another common type of research question has been conceptualised thus: Given that A is an author, Ph(x) is a phenomenon or practice or behaviour, and DT(x) is a particular discourse type. A has said P

  Read more →
• 

  Oren Etzioni

  Oren Etzioni (born 1964) is Professor Emeritus of Computer Science at the University of Washington, and founding CEO of the Allen Institute for Artificial Intelligence (AI2). Etzioni is a co-founder of Vercept, an AI startup, and founder and CEO of TrueMedia.org, a non-profit dedicated to fighting political deepfakes, which launched in April 2024. He is also the Founder and Technical Director of the AI2 Incubator and a venture partner at the Madrona Venture Group. == Early life and education == Etzioni is the son of Israeli-American intellectual Amitai Etzioni. He was the first student to major in computer science at Harvard University, where he earned a bachelor's degree in 1986. He earned a PhD from Carnegie Mellon University in January, 1991, supervised by Tom M. Mitchell. == University of Washington career == Etzioni joined the University of Washington faculty in 1991, immediately after receiving his PhD. He rose through the ranks to become the Washington Research Foundation Entrepreneurship Professor in Computer Science & Engineering. Etzioni's research has been focused on basic problems in the study of intelligence, machine reading, machine learning and web search. Past projects include Internet Softbots—the study of intelligent agents in the context of real-world software testbeds. In 2003, he started the KnowItAll project for acquiring massive amounts of information from the web. In 2005, he founded and became the director of the university's Turing Center. The center investigated problems in data mining, natural language processing, the Semantic Web and other web search topics. Etzioni coined the term machine reading and helped to create the first commercial comparison shopping agent. He has published over 200 technical papers, and his H-index exceeds 100. == Entrepreneurship == As a faculty member Etzioni was also an active entrepreneur, founding multiple companies and pioneering multiple technologies including MetaCrawler (bought by Infospace), Netbot (bought by Excite in 1997 for $35 million), and ClearForest (bought by Reuters). He founded Farecast, a travel metasearch and price prediction site, which was acquired by Microsoft in 2008 for $115 million. Before founding Farecast, he developed a program originally called Hamlet, that used algorithms to identify patterns in airfare data using data-mining techniques. He also co-founded Decide.com, a website to help consumers make buying decisions using previous price history and recommendations from other users. Decide.com was bought by eBay in September, 2013. Etzioni is also a venture partner at the Madrona Venture Group. He is founder and CEO of TrueMedia.org, a non-profit dedicated to fighting political deepfakes, which launched in April 2024. Etzioni is a co-founder of Vercept, an AI startup formed in 2025. == Founding CEO of AI2 == In September 2013 Etzioni was selected as the Founding CEO of the Allen Institute for Artificial Intelligence by philanthropist Paul G. Allen, and in January 2014 he took a leave of absence from the University of Washington to serve in that role. Etzioni's technical contributions continued at AI2; for example, in 2015, he helped to create the Semantic Scholar search engine. Under Etzioni’s leadership, AI2 grew from zero to over two hundred team members including notable researchers and engineers across several domains of AI. By 2021, its AI2 researchers had published near 700 papers in publications such as AAAI, ACL, CVPR, NeurIPS, and ICLR. Twenty-four of these papers had garnered special-recognition awards. AI2 also offered several key resources and tools to the AI community including the AllenNLP library, Semantic Scholar, and the conservation platforms EarthRanger and Skylight. Ed Lazowska, AI2 Board Member, has stated about Etzioni that he "took the collegial, collaborative culture that he absorbed in his 20+ years as a professor in UW's Allen School and mixed it with the singular focus that drives startups to create an elixir that AI2 folks have been drinking over the last eight years. The result is an exceptional organization of scientists, engineers, and entrepreneurs that's pursuing Paul Allen’s vision of ‘AI for the Common Good’ with extraordinary success.” == Popular press == In addition to his scientific publications, Etzioni has written commentary on AI for The New York Times, Wired, Nature, and other publications. After reading the idea in a book about AI by Brad Smith and Harry Shum, Etzioni has attempted to create an oath for AI practitioners. In 2018, he published what he called a "Hippocratic Oath for artificial intelligence practitioners" in TechCrunch. == Awards and recognition == In 1993, Etzioni received a National Young Investigator Award. In 2003, Etzioni was elected as AAAI Fellow. In 2005, Etzioni received an IJCAI Distinguished Paper Award for "A Probabilistic Model of Redundancy in Information Extraction". In 2007, he received the Robert S. Engelmore Memorial Award. In 2012 Etzioni was featured as GeekWire's "Geek of the Week". In 2013 Etzioni was voted "Geek of the Year" through GeekWire. In 2022, Etzioni received the 2012 ACL Test-of-Time Paper Award. In 2022, Etzioni, along with Ana-Maria Popescu and Henry Kautz, received the ACM Intelligent User Interfaces Most Impact Award for their 2003 paper, "Towards a Theory of Natural Language Interfaces to Databases". == Personal life == Etzioni has three children, and has said in interviews that family is his number one priority. He is married to Ivone Etzioni, and was previously married to Dr. Ruth Etzioni, a biostatistician at the Fred Hutchinson Cancer Center. Outside of his professional career, Etzioni has a wide range of personal interests. He has attended the Burning Man festival, which he described as a valuable way to step outside his comfort zone. His first computer was a TRS-80, and he has described his car’s GPS as his favorite gadget, joking that he has “no sense of direction.” == Selected publications == === Scholarly publications === Etzioni, Oren (July 1994). "A Softbot-based Interface to the Internet" (PDF). Communications of the ACM. Retrieved March 29, 2018. Etzioni, Oren (December 2008). "Open Information Extraction from the Web" (PDF). Communications of the ACM. Retrieved March 29, 2018. Zamir, Oren; Etzioni, Oren (1998). "Web document clustering". Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval. ACM. pp. 46–54. doi:10.1145/290941.290956. ISBN 978-1-58113-015-7. S2CID 244069. Zamir, Oren; Etzioni, Oren (May 1999). "Grouper: a dynamic clustering interface to Web search results". Computer Networks. 31 (11–16): 1361–1374. CiteSeerX 10.1.1.31.8216. doi:10.1016/S1389-1286(99)00054-7. S2CID 206134308. Popescu, Ana-Maria; Etzioni, Oren (2005). "Extracting product features and opinions from reviews". Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05. pp. 339–346. doi:10.3115/1220575.1220618. Etzioni, Oren; Cafarella, Michael; Downey, Doug; Popescu, Ana-Maria; Shaked, Tal; Sonderland, Stephen; Weld, Daniel; Yates, Alexander (June 2005). "Unsupervised named-entity extraction from the Web: An experimental study". Artificial Intelligence. 165 (1): 91–134. doi:10.1016/j.artint.2005.03.001. Downey, Doug; Etzioni, Oren; Sonderland, Stephen (July 2010). "Grouper: Analysis of a probabilistic model of redundancy in unsupervised information extraction". Artificial Intelligence. 174 (11): 726–748. CiteSeerX 10.1.1.174.2441. doi:10.1016/j.artint.2010.04.024. === Popular articles === Etzioni, Oren (August 4, 2011). "Web Search Needs a Shakeup" (PDF). Nature. Retrieved November 21, 2019. Etzioni, Oren (December 9, 2014). "AI Won't Exterminate Us – It Will Empower Us". Backchannel. Retrieved March 29, 2018. Etzioni, Oren (February 4, 2016). "To Keep AI Safe -- Use AI". Vox. Retrieved November 21, 2019. Etzioni, Oren (April 8, 2016). "Quora Session with Oren Etzioni". Quora. Retrieved March 29, 2018. Etzioni, Oren (June 15, 2016). "Deep Learning Isn't a Dangerous Magic Genie. It's Just Math". Wired. Retrieved March 29, 2018. Etzioni, Oren (September 20, 2016). "No, the Experts Don't Think Superintelligent AI is a Threat to Humanity". MIT Technology Review. Retrieved November 21, 2019. Etzioni, Oren (July 6, 2017). "Artificial intelligence: AI Zooms in on highly influential citations". Nature. Retrieved March 29, 2018. Etzioni, Oren (September 1, 2017). "How to Regulate Artificial Intelligence". The New York Times. Retrieved March 29, 2018. Etzioni, Oren (November 2, 2017). "Workers Displaced by Automation Should Try A New Job: Caregiver". Wired. Retrieved March 29, 2018. Etzioni, Oren (March 14, 2018). "A Hippocratic Oath for artificial intelligence practitioners". Tech Crunch. Retrieved March 29, 2018. Etzioni, Oren (March 7, 2018). "A 'Manhattan Project' for science research". The Hill. Retrieved November 21, 2019. Etzioni, Ore

  Read more →