The distributional learning theory or learning of probability distribution is a framework in computational learning theory. It has been proposed from Michael Kearns, Yishay Mansour, Dana Ron, Ronitt Rubinfeld, Robert Schapire and Linda Sellie in 1994 and it was inspired from the PAC-framework introduced by Leslie Valiant. In this framework the input is a number of samples drawn from a distribution that belongs to a specific class of distributions. The goal is to find an efficient algorithm that, based on these samples, determines with high probability the distribution from which the samples have been drawn. Because of its generality, this framework has been used in a large variety of different fields like machine learning, approximation algorithms, applied probability and statistics. This article explains the basic definitions, tools and results in this framework from the theory of computation point of view. == Definitions == Let X {\displaystyle \textstyle X} be the support of the distributions of interest. As in the original work of Kearns et al. if X {\displaystyle \textstyle X} is finite it can be assumed without loss of generality that X = { 0 , 1 } n {\displaystyle \textstyle X=\{0,1\}^{n}} where n {\displaystyle \textstyle n} is the number of bits that have to be used in order to represent any y ∈ X {\displaystyle \textstyle y\in X} . We focus in probability distributions over X {\displaystyle \textstyle X} . There are two possible representations of a probability distribution D {\displaystyle \textstyle D} over X {\displaystyle \textstyle X} . probability distribution function (or evaluator) an evaluator E D {\displaystyle \textstyle E_{D}} for D {\displaystyle \textstyle D} takes as input any y ∈ X {\displaystyle \textstyle y\in X} and outputs a real number E D [ y ] {\displaystyle \textstyle E_{D}[y]} which denotes the probability that of y {\displaystyle \textstyle y} according to D {\displaystyle \textstyle D} , i.e. E D [ y ] = Pr [ Y = y ] {\displaystyle \textstyle E_{D}[y]=\Pr[Y=y]} if Y ∼ D {\displaystyle \textstyle Y\sim D} . generator a generator G D {\displaystyle \textstyle G_{D}} for D {\displaystyle \textstyle D} takes as input a string of truly random bits y {\displaystyle \textstyle y} and outputs G D [ y ] ∈ X {\displaystyle \textstyle G_{D}[y]\in X} according to the distribution D {\displaystyle \textstyle D} . Generator can be interpreted as a routine that simulates sampling from the distribution D {\displaystyle \textstyle D} given a sequence of fair coin tosses. A distribution D {\displaystyle \textstyle D} is called to have a polynomial generator (respectively evaluator) if its generator (respectively evaluator) exists and can be computed in polynomial time. Let C X {\displaystyle \textstyle C_{X}} a class of distribution over X, that is C X {\displaystyle \textstyle C_{X}} is a set such that every D ∈ C X {\displaystyle \textstyle D\in C_{X}} is a probability distribution with support X {\displaystyle \textstyle X} . The C X {\displaystyle \textstyle C_{X}} can also be written as C {\displaystyle \textstyle C} for simplicity. In order to evaluate learnability, it is necessary to have a way to measure how well an approximated distribution D ′ {\displaystyle \textstyle D'} fits the sampled distribution D {\displaystyle \textstyle D} . There are several ways to measure the divergence between two distributions. Three common possibilities are Kullback–Leibler divergence Total variation distance of probability measures Kolmogorov distance Total variation and Kolmogorov distance are true metrics, while KL divergence is not (it lacks symmetry). These measures are ordered by convergence strength: closeness in KL divergence implies closeness in total variation (via Pinsker's inequality), which in turn implies closeness in Kolmogorov distance. Therefore, a learnability result proven under KL divergence automatically holds under the weaker measures, but not vice versa. Since certain measures may be more appropriate in specific applications, we will use d ( D , D ′ ) {\displaystyle \textstyle d(D,D')} to denote a selected divergence between the distribution D {\displaystyle \textstyle D} and the distribution D ′ {\displaystyle \textstyle D'} . The basic input that we use in order to learn a distribution is a number of samples drawn by this distribution. For the computational point of view the assumption is that such a sample is given in a constant amount of time. So it's like having access to an oracle G E N ( D ) {\displaystyle \textstyle GEN(D)} that returns a sample from the distribution D {\displaystyle \textstyle D} . Sometimes the interest is, apart from measuring the time complexity, to measure the number of samples that have to be used in order to learn a specific distribution D {\displaystyle \textstyle D} in class of distributions C {\displaystyle \textstyle C} . This quantity is called sample complexity of the learning algorithm. In order for the problem of distribution learning to be more clear consider the problem of supervised learning as defined in. In this framework of statistical learning theory a training set S = { ( x 1 , y 1 ) , … , ( x n , y n ) } {\displaystyle \textstyle S=\{(x_{1},y_{1}),\dots ,(x_{n},y_{n})\}} and the goal is to find a target function f : X → Y {\displaystyle \textstyle f:X\rightarrow Y} that minimizes some loss function, e.g. the square loss function. More formally f = arg min g ∫ V ( y , g ( x ) ) d ρ ( x , y ) {\displaystyle f=\arg \min _{g}\int V(y,g(x))d\rho (x,y)} , where V ( ⋅ , ⋅ ) {\displaystyle V(\cdot ,\cdot )} is the loss function, e.g. V ( y , z ) = ( y − z ) 2 {\displaystyle V(y,z)=(y-z)^{2}} and ρ ( x , y ) {\displaystyle \rho (x,y)} the probability distribution according to which the elements of the training set are sampled. If the conditional probability distribution ρ x ( y ) {\displaystyle \rho _{x}(y)} is known then the target function has the closed form f ( x ) = ∫ y y d ρ x ( y ) {\displaystyle f(x)=\int _{y}yd\rho _{x}(y)} . So the set S {\displaystyle S} is a set of samples from the probability distribution ρ ( x , y ) {\displaystyle \rho (x,y)} . Now the goal of distributional learning theory if to find ρ {\displaystyle \rho } given S {\displaystyle S} which can be used to find the target function f {\displaystyle f} . Definition of learnability A class of distributions C {\displaystyle \textstyle C} is called efficiently learnable if for every ϵ > 0 {\displaystyle \textstyle \epsilon >0} and 0 < δ ≤ 1 {\displaystyle \textstyle 0<\delta \leq 1} given access to G E N ( D ) {\displaystyle \textstyle GEN(D)} for an unknown distribution D ∈ C {\displaystyle \textstyle D\in C} , there exists a polynomial time algorithm A {\displaystyle \textstyle A} , called learning algorithm of C {\displaystyle \textstyle C} , that outputs a generator or an evaluator of a distribution D ′ {\displaystyle \textstyle D'} such that Pr [ d ( D , D ′ ) ≤ ϵ ] ≥ 1 − δ {\displaystyle \Pr[d(D,D')\leq \epsilon ]\geq 1-\delta } If we know that D ′ ∈ C {\displaystyle \textstyle D'\in C} then A {\displaystyle \textstyle A} is called proper learning algorithm, otherwise is called improper learning algorithm. In some settings the class of distributions C {\displaystyle \textstyle C} is a class with well known distributions which can be described by a set of parameters. For instance C {\displaystyle \textstyle C} could be the class of all the Gaussian distributions N ( μ , σ 2 ) {\displaystyle \textstyle N(\mu ,\sigma ^{2})} . In this case the algorithm A {\displaystyle \textstyle A} should be able to estimate the parameters μ , σ {\displaystyle \textstyle \mu ,\sigma } . In this case A {\displaystyle \textstyle A} is called parameter learning algorithm. Obviously the parameter learning for simple distributions is a very well studied field that is called statistical estimation and there is a very long bibliography on different estimators for different kinds of simple known distributions. But distributions learning theory deals with learning class of distributions that have more complicated description. == First results == In their seminal work, Kearns et al. deal with the case where A {\displaystyle \textstyle A} is described in term of a finite polynomial sized circuit and they proved the following for some specific classes of distribution. O R {\displaystyle \textstyle OR} gate distributions for this kind of distributions there is no polynomial-sized evaluator, unless # P ⊆ P / poly {\displaystyle \textstyle \#P\subseteq P/{\text{poly}}} . On the other hand, this class is efficiently learnable with generator. Parity gate distributions this class is efficiently learnable with both generator and evaluator. Mixtures of Hamming Balls this class is efficiently learnable with both generator and evaluator. Probabilistic Finite Automata this class is not efficiently learnable with evaluator under the Noisy Parity Assumption which is an impossibility assumption in the PAC learning fram
Weak supervision
Weak supervision (also known as semi-supervised learning) is a paradigm in machine learning, the relevance and notability of which increased with the advent of large language models due to the large amount of data required to train them. It is characterized by using a combination of a small amount of human-labeled data (exclusively used in more expensive and time-consuming supervised learning paradigm), followed by a large amount of unlabeled data (used exclusively in unsupervised learning paradigm). In other words, the desired output values are provided only for a subset of the training data. The remaining data is unlabeled or imprecisely labeled. Intuitively, it can be seen as an exam and labeled data as sample problems that the teacher solves for the class as an aid in solving another set of problems. In the transductive setting, these unsolved problems act as exam questions. In the inductive setting, they become practice problems of the sort that will make up the exam. == Problem == The acquisition of labeled data for a learning problem often requires a skilled human agent (e.g. to transcribe an audio segment) or a physical experiment (e.g. determining the 3D structure of a protein or determining whether there is oil at a particular location). The cost associated with the labeling process thus may render large, fully labeled training sets infeasible, whereas acquisition of unlabeled data is relatively inexpensive. In such situations, semi-supervised learning can be of great practical value. Semi-supervised learning is also of theoretical interest in machine learning and as a model for human learning. == Technique == More formally, semi-supervised learning assumes a set of l {\displaystyle l} independently identically distributed examples x 1 , … , x l ∈ X {\displaystyle x_{1},\dots ,x_{l}\in X} with corresponding labels y 1 , … , y l ∈ Y {\displaystyle y_{1},\dots ,y_{l}\in Y} and u {\displaystyle u} unlabeled examples x l + 1 , … , x l + u ∈ X {\displaystyle x_{l+1},\dots ,x_{l+u}\in X} are processed. Semi-supervised learning combines this information to surpass the classification performance that can be obtained either by discarding the unlabeled data and doing supervised learning or by discarding the labels and doing unsupervised learning. Semi-supervised learning may refer to either transductive learning or inductive learning. The goal of transductive learning is to infer the correct labels for the given unlabeled data x l + 1 , … , x l + u {\displaystyle x_{l+1},\dots ,x_{l+u}} only. The goal of inductive learning is to infer the correct mapping from X {\displaystyle X} to Y {\displaystyle Y} . It is unnecessary (and, according to Vapnik's principle, imprudent) to perform transductive learning by way of inferring a classification rule over the entire input space; however, in practice, algorithms formally designed for transduction or induction are often used interchangeably. == Assumptions == In order to make any use of unlabeled data, some relationship to the underlying distribution of data must exist. Semi-supervised learning algorithms make use of at least one of the following assumptions: === Continuity / smoothness assumption === Points that are close to each other are more likely to share a label. This is also generally assumed in supervised learning and yields a preference for geometrically simple decision boundaries. In the case of semi-supervised learning, the smoothness assumption additionally yields a preference for decision boundaries in low-density regions, so few points are close to each other but in different classes. === Cluster assumption === The data tend to form discrete clusters, and points in the same cluster are more likely to share a label (although data that shares a label may spread across multiple clusters). This is a special case of the smoothness assumption and gives rise to feature learning with clustering algorithms. === Manifold assumption === The data lie approximately on a manifold of much lower dimension than the input space. In this case learning the manifold using both the labeled and unlabeled data can avoid the curse of dimensionality. Then learning can proceed using distances and densities defined on the manifold. The manifold assumption is practical when high-dimensional data are generated by some process that may be hard to model directly, but which has only a few degrees of freedom. For instance, human voice is controlled by a few vocal folds, and images of various facial expressions are controlled by a few muscles. In these cases, it is better to consider distances and smoothness in the natural space of the generating problem, rather than in the space of all possible acoustic waves or images, respectively. == History == The heuristic approach of self-training (also known as self-learning or self-labeling) is historically the oldest approach to semi-supervised learning, with examples of applications starting in the 1960s. The transductive learning framework was formally introduced by Vladimir Vapnik in the 1970s. Interest in inductive learning using generative models also began in the 1970s. A probably approximately correct learning bound for semi-supervised learning of a Gaussian mixture was demonstrated by Ratsaby and Venkatesh in 1995. == Methods == === Generative models === Generative approaches to statistical learning first seek to estimate p ( x | y ) {\displaystyle p(x|y)} , the distribution of data points belonging to each class. The probability p ( y | x ) {\displaystyle p(y|x)} that a given point x {\displaystyle x} has label y {\displaystyle y} is then proportional to p ( x | y ) p ( y ) {\displaystyle p(x|y)p(y)} by Bayes' rule. Semi-supervised learning with generative models can be viewed either as an extension of supervised learning (classification plus information about p ( x ) {\displaystyle p(x)} ) or as an extension of unsupervised learning (clustering plus some labels). Generative models assume that the distributions take some particular form p ( x | y , θ ) {\displaystyle p(x|y,\theta )} parameterized by the vector θ {\displaystyle \theta } . If these assumptions are incorrect, the unlabeled data may actually decrease the accuracy of the solution relative to what would have been obtained from labeled data alone. However, if the assumptions are correct, then the unlabeled data necessarily improves performance. The unlabeled data are distributed according to a mixture of individual-class distributions. In order to learn the mixture distribution from the unlabeled data, it must be identifiable, that is, different parameters must yield different summed distributions. Gaussian mixture distributions are identifiable and commonly used for generative models. The parameterized joint distribution can be written as p ( x , y | θ ) = p ( y | θ ) p ( x | y , θ ) {\displaystyle p(x,y|\theta )=p(y|\theta )p(x|y,\theta )} by using the chain rule. Each parameter vector θ {\displaystyle \theta } is associated with a decision function f θ ( x ) = argmax y p ( y | x , θ ) {\displaystyle f_{\theta }(x)={\underset {y}{\operatorname {argmax} }}\ p(y|x,\theta )} . The parameter is then chosen based on fit to both the labeled and unlabeled data, weighted by λ {\displaystyle \lambda } : argmax Θ ( log p ( { x i , y i } i = 1 l | θ ) + λ log p ( { x i } i = l + 1 l + u | θ ) ) {\displaystyle {\underset {\Theta }{\operatorname {argmax} }}\left(\log p(\{x_{i},y_{i}\}_{i=1}^{l}|\theta )+\lambda \log p(\{x_{i}\}_{i=l+1}^{l+u}|\theta )\right)} === Low-density separation === Another major class of methods attempts to place boundaries in regions with few data points (labeled or unlabeled). One of the most commonly used algorithms is the transductive support vector machine, or TSVM (which, despite its name, may be used for inductive learning as well). Whereas support vector machines for supervised learning seek a decision boundary with maximal margin over the labeled data, the goal of TSVM is a labeling of the unlabeled data such that the decision boundary has maximal margin over all of the data. In addition to the standard hinge loss ( 1 − y f ( x ) ) + {\displaystyle (1-yf(x))_{+}} for labeled data, a loss function ( 1 − | f ( x ) | ) + {\displaystyle (1-|f(x)|)_{+}} is introduced over the unlabeled data by letting y = sign f ( x ) {\displaystyle y=\operatorname {sign} {f(x)}} . TSVM then selects f ∗ ( x ) = h ∗ ( x ) + b {\displaystyle f^{}(x)=h^{}(x)+b} from a reproducing kernel Hilbert space H {\displaystyle {\mathcal {H}}} by minimizing the regularized empirical risk: f ∗ = argmin f ( ∑ i = 1 l ( 1 − y i f ( x i ) ) + + λ 1 ‖ h ‖ H 2 + λ 2 ∑ i = l + 1 l + u ( 1 − | f ( x i ) | ) + ) {\displaystyle f^{}={\underset {f}{\operatorname {argmin} }}\left(\displaystyle \sum _{i=1}^{l}(1-y_{i}f(x_{i}))_{+}+\lambda _{1}\|h\|_{\mathcal {H}}^{2}+\lambda _{2}\sum _{i=l+1}^{l+u}(1-|f(x_{i})|)_{+}\right)} An exact solution is intractable due to the non-convex term ( 1 − | f ( x ) | ) + {\displayst
DFA minimization
In automata theory (a branch of theoretical computer science), DFA minimization is the task of transforming a given deterministic finite automaton (DFA) into an equivalent DFA that has a minimum number of states. Here, two DFAs are called equivalent if they recognize the same regular language. Several different algorithms accomplishing this task are known and described in standard textbooks on automata theory. == Minimal DFA == For each regular language, there also exists a minimal automaton that accepts it, that is, a DFA with a minimum number of states and this DFA is unique (except that states can be given different names). The minimal DFA ensures minimal computational cost for tasks such as pattern matching. There are three classes of states that can be removed or merged from the original DFA without affecting the language it accepts. Unreachable states are the states that are not reachable from the initial state of the DFA, for any input string. These states can be removed. Dead states are the states from which no final state is reachable. These states can be removed unless the automaton is required to be complete. Nondistinguishable states are those that cannot be distinguished from one another for any input string. These states can be merged. DFA minimization is usually done in three steps: remove dead and unreachable states (this will accelerate the following step), merge nondistinguishable states, optionally, re-create a single dead state ("sink" state) if the resulting DFA is required to be complete. == Unreachable states == The state p {\displaystyle p} of a deterministic finite automaton M = ( Q , Σ , δ , q 0 , F ) {\displaystyle M=(Q,\Sigma ,\delta ,q_{0},F)} is unreachable if no string w {\displaystyle w} in Σ ∗ {\displaystyle \Sigma ^{}} exists for which p = δ ∗ ( q 0 , w ) {\displaystyle p=\delta ^{}(q_{0},w)} . In this definition, Q {\displaystyle Q} is the set of states, Σ {\displaystyle \Sigma } is the set of input symbols, δ {\displaystyle \delta } is the transition function (mapping a state and an input symbol to a set of states), δ ∗ {\displaystyle \delta ^{}} is its extension to strings (also known as extended transition function), q 0 {\displaystyle q_{0}} is the initial state, and F {\displaystyle F} is the set of accepting (also known as final) states. Reachable states can be obtained with the following algorithm: Assuming an efficient implementation of the state sets (e.g. new_states) and operations on them (such as adding a state or checking whether it is present), this algorithm can be implemented with time complexity O ( n + m ) {\displaystyle O(n+m)} , where n {\displaystyle n} is the number of states and m {\displaystyle m} is the number of transitions of the input automaton. Unreachable states can be removed from the DFA without affecting the language that it accepts. == Nondistinguishable states == The following algorithms present various approaches to merging nondistinguishable states. === Hopcroft's algorithm === One algorithm for merging the nondistinguishable states of a DFA, due to Hopcroft (1971), is based on partition refinement, partitioning the DFA states into groups by their behavior. These groups represent equivalence classes of the Nerode congruence, whereby every two states are equivalent if they have the same behavior for every input sequence. That is, for every two states p1 and p2 that belong to the same block of the partition P, and every input word w, the transitions determined by w should always take states p1 and p2 to either states that both accept or states that both reject. It should not be possible for w to take p1 to an accepting state and p2 to a rejecting state or vice versa. The following pseudocode describes the form of the algorithm as given by Xu. Alternative forms have also been presented. The algorithm starts with a partition that is too coarse: every pair of states that are equivalent according to the Nerode congruence belong to the same set in the partition, but pairs that are inequivalent might also belong to the same set. It gradually refines the partition into a larger number of smaller sets, at each step splitting sets of states into pairs of subsets that are necessarily inequivalent. The initial partition is a separation of the states into two subsets of states that clearly do not have the same behavior as each other: the accepting states and the rejecting states. The algorithm then repeatedly chooses a set A from the current partition and an input symbol c, and splits each of the sets of the partition into two (possibly empty) subsets: the subset of states that lead to A on input symbol c, and the subset of states that do not lead to A. Since A is already known to have different behavior than the other sets of the partition, the subsets that lead to A also have different behavior than the subsets that do not lead to A. When no more splits of this type can be found, the algorithm terminates. Lemma. Given a fixed character c and an equivalence class Y that splits into equivalence classes B and C, only one of B or C is necessary to refine the whole partition. Example: Suppose we have an equivalence class Y that splits into equivalence classes B and C. Suppose we also have classes D, E, and F; D and E have states with transitions into B on character c, while F has transitions into C on character c. By the Lemma, we can choose either B or C as the distinguisher, let's say B. Then the states of D and E are split by their transitions into B. But F, which doesn't point into B, simply doesn't split during the current iteration of the algorithm; it will be refined by other distinguisher(s). Observation. All of B or C is necessary to split referring classes like D, E, and F correctly—subsets won't do. The purpose of the outermost if statement (if Y is in W) is to patch up W, the set of distinguishers. We see in the previous statement in the algorithm that Y has just been split. If Y is in W, it has just become obsolete as a means to split classes in future iterations. So Y must be replaced by both splits because of the Observation above. If Y is not in W, however, only one of the two splits, not both, needs to be added to W because of the Lemma above. Choosing the smaller of the two splits guarantees that the new addition to W is no more than half the size of Y; this is the core of the Hopcroft algorithm: how it gets its speed, as explained in the next paragraph. The worst case running time of this algorithm is O(ns log n), where n is the number of states and s is the size of the alphabet. This bound follows from the fact that, for each of the ns transitions of the automaton, the sets drawn from Q that contain the target state of the transition have sizes that decrease relative to each other by a factor of two or more, so each transition participates in O(log n) of the splitting steps in the algorithm. The partition refinement data structure allows each splitting step to be performed in time proportional to the number of transitions that participate in it. This remains the most efficient algorithm known for solving the problem, and for certain distributions of inputs its average-case complexity is even better, O(n log log n). Once Hopcroft's algorithm has been used to group the states of the input DFA into equivalence classes, the minimum DFA can be constructed by forming one state for each equivalence class. If S is a set of states in P, s is a state in S, and c is an input character, then the transition in the minimum DFA from the state for S, on input c, goes to the set containing the state that the input automaton would go to from state s on input c. The initial state of the minimum DFA is the one containing the initial state of the input DFA, and the accepting states of the minimum DFA are the ones whose members are accepting states of the input DFA. === Moore's algorithm === Moore's algorithm for DFA minimization is due to Edward F. Moore (1956). Like Hopcroft's algorithm, it maintains a partition that starts off separating the accepting from the rejecting states, and repeatedly refines the partition until no more refinements can be made. At each step, it replaces the current partition with the coarsest common refinement of s + 1 partitions, one of which is the current one and the rest of which are the preimages of the current partition under the transition functions for each of the input symbols. The algorithm terminates when this replacement does not change the current partition. Its worst-case time complexity is O(n2s): each step of the algorithm may be performed in time O(ns) using a variant of radix sort to reorder the states so that states in the same set of the new partition are consecutive in the ordering, and there are at most n steps since each one but the last increases the number of sets in the partition. The instances of the DFA minimization problem that cause the worst-case behavior are the same as for Hopcroft's algorithm. The number of steps th
Best AI Paragraph Rewriters in 2026
In search of the best AI paragraph rewriter? An AI paragraph rewriter is software that uses machine learning to help you get more done — it turns a rough idea into a polished result in seconds. When choosing one, weigh output quality, pricing, export formats, and how well it fits the tools you already use. Whether you are a beginner or a pro, the right AI paragraph rewriter slots into your workflow and pays for itself fast. We tested the leading options and ranked them by quality, value, and ease of use.
Rayid Ghani
Rayid Ghani (born 1977) is a Distinguished Career Professor in the Machine Learning Department (in the School of Computer Science) and the Heinz College of Information Systems and Public Policy at Carnegie Mellon University. Previously, he was the director of the Center for Data Science and Public Policy, research associate professor in the department of computer science, and a senior fellow at the Harris School of Public Policy at the University of Chicago. He was also the co-founder of Edgeflip, an analytics startup that grew out of the Obama 2012 Campaign, focused on social media products for non-profits, advocacy groups, and charities. In September 2019, it was announced that he will be leaving the University of Chicago and joining Carnegie Mellon University's School of Computer Science and Heinz College of Information Systems and Public Policy. Prior to that, Rayid was the Chief Scientist of the Obama 2012 Election Campaign and focused on using data science, machine learning, and technology to improve fundraising, volunteer mobilization, voter registration, persuasion, and turnout. Ghani started and runs the Eric & Wendy Schmidt Data Science for Social Good Summer Fellowship. He's also the co-founder of Coleridge Initiative, a nonprofit organization working with governments to ensure that data and evidence is used more effectively for policymaking. == Education and career == Ghani completed his schooling at the Karachi Grammar School, in Karachi, Pakistan. Ghani completed his graduate studies in the machine learning department at Carnegie Mellon University with Tom M. Mitchell on machine learning and text classification and received his undergraduate degrees in computer science and mathematics from University of the South. Before his role at the University of Chicago, he was the chief scientist of the Obama 2012 Campaign. Before that, he was a senior research scientist and director of analytics research at Accenture Labs, where he led a technology research team focused on applied R&D in analytics, machine learning, and data mining for large-scale and emerging business problems. == Policy efforts == Ghani has been actively working with government agencies and non-profits on designing AI and Machine Learning Systems to help tackle societal problems in public health, criminal justice, social services, education, economic development, and workforce development He has also testified in front of the US Senate in 2023 and the US House of Representatives in 2020, on AI Governance and Regulation. == Research contributions == Ghani's research focuses on developing and applying machine learning, data science, and artificial intelligence methods to large-scale social problems in areas such as education, healthcare, economic development, criminal justice, energy, transportation, and public safety. His work has previously focused on text analytics, fundraising, volunteer, and voter mobilization using analytics, social media, and machine learning., and data mining. Rayid's research contributions have been in the areas of text mining, co-training, active learning, consumer behavior modeling, and fraud detection. His research focus has been on 1) dealing with bias and fairness issues in machine learning and AI, 2) designing Human-AI collaborative systems that support people in making decisions, and 3) evaluating AI systems to focus on the entire workflow and outcomes He has given keynote speeches on Analytics and the Presidential Elections (for example at Predictive Analytics World, Digital Leaders Forum, Carnegie Mellon University, and CeBIT Australia), on Business Applications of Data Mining, and Data Science for Social Good. == Selected publications == Big Data and Social Science: A Practical Guide to Methods and Tools. Editors: Ian Foster, Rayid Ghani, Ron Jarmin, Frauke Kreuter, Julia Lane. CRC Press 2016. Empirical observation of negligible fairness–accuracy trade-offs in machine learning for public policy. Kit Rodolfa, Hemank Lamba, Rayid Ghani. Nature Machine Intelligence 2021. Explainable machine learning for public policy: Use cases, gaps, and research directions. Kasun Amarasinghe, Kit T. Rodolfa, Hemank Lamba, Rayid Ghani. Data and Policy 2023. Data Mining for Business Applications. Editors: Carlos Soares, Rayid Ghani. Book. IOS Press 2010. Mining the Web to Add Semantics to Retail Data Mining. R. Ghani. Invited Paper. Web Mining: From Web to Semantic Web. Springer Lecture Notes in Artificial Intelligence, Vol. 3209. Berendt, B.; Hotho, A.; Mladenic, D.; van Someren, M.; Spiliopoulou, M.; Stumme, G. (Eds.) 2004
Automated Mathematician
The Automated Mathematician (AM) is one of the earliest successful discovery systems. It was created by Douglas Lenat in Lisp, and in 1977 led to Lenat being awarded the IJCAI Computers and Thought Award. AM worked by generating and modifying short Lisp programs which were then interpreted as defining various mathematical concepts; for example, a program that tested equality between the length of two lists was considered to represent the concept of numerical equality, while a program that produced a list whose length was the product of the lengths of two other lists was interpreted as representing the concept of multiplication. The system had elaborate heuristics for choosing which programs to extend and modify, based on the experiences of working mathematicians in solving mathematical problems. == Controversy == Lenat claimed that the system was composed of hundreds of data structures called "concepts", together with hundreds of "heuristic rules" and a simple flow of control: "AM repeatedly selects the top task from the agenda and tries to carry it out. This is the whole control structure!" Yet the heuristic rules were not always represented as separate data structures; some had to be intertwined with the control flow logic. Some rules had preconditions that depended on the history, or otherwise could not be represented in the framework of the explicit rules. What's more, the published versions of the rules often involve vague terms that are not defined further, such as "If two expressions are structurally similar, ..." (Rule 218) or "... replace the value obtained by some other (very similar) value..." (Rule 129). Another source of information is the user, via Rule 2: "If the user has recently referred to X, then boost the priority of any tasks involving X." Thus, it appears quite possible that much of the real discovery work is buried in unexplained procedures. Lenat claimed that the system had rediscovered both Goldbach's conjecture and the fundamental theorem of arithmetic. Later critics accused Lenat of over-interpreting the output of AM. In his paper Why AM and Eurisko appear to work, Lenat conceded that any system that generated enough short Lisp programs would generate ones that could be interpreted by an external observer as representing equally sophisticated mathematical concepts. However, he argued that this property was in itself interesting—and that a promising direction for further research would be to look for other languages in which short random strings were likely to be useful. == Successor == This intuition was the basis of AM's successor Eurisko, which attempted to generalize the search for mathematical concepts to the search for useful heuristics.
Jiaya Jia
Jiaya Jia (Chinese: 贾佳亚) is a Chair Professor of the Department of Computer Science and Engineering at The Hong Kong University of Science and Technology (HKUST). He is an IEEE Fellow, the associate editor-in-chief of one of IEEE’s flagship and premier journals- Transactions on Pattern Analysis and Machine Intelligence (TPAMI), as well as on the editorial board of International Journal of Computer Vision (IJCV). == Early life and education == Jiaya Jia joined CUHK in 2004 as an assistant professor, and was promoted to full professor in 2015. He obtained his PhD degree in computer science jointly from Hong Kong University of Science and Technology and Microsoft Research in 2004. From March 2003 to August 2004, he was a visiting scholar at Microsoft. He conducted collaborative research at Adobe Research in 2007. == Career == Jiaya Jia is a distinguished scientist in the fields of computer vision and artificial intelligence. His research team at HKUST, DV Lab, is one of the largest vision AI research teams in the world and has been making significant contribution to advanced development of computer vision algorithms and technologies with focuses on image/video understanding, detection and segmentation, multi-modal AI, computational imaging, practical optimization, and advanced learning for visual content since 2000. Jiaya Jia has published 200+ top papers and was cited 80,000+ times on Google Scholar with H-Index 110+. 40+ PhDs and fellows from this group are now active in academia and industry, and have become prominent AI tech leaders as professors, directors in major research labs, and founders of several successful startups. Jiaya Jia assumes the position of associate editor-in-chief of IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) since 2021. He is also on the editorial board of International Journal of Computer Vision (IJCV). Jiaya Jia has served as the area chair of ICCV, CVPR, AAAI, ECCV, and several other premium international AI conferences for years. He was on program committees of major conferences in graphics and computational imaging, including ICCP, SIGGRAPH, and SIGGRAPH Asia. == Research == The research areas of Jiaya Jia are computer vision, large X models, and deep learning. Jiaya Jia has made outstanding contributions to computer vision technology, algorithms and engineering, and is among the world's leading experts in the field. His research partners include numerous renowned multinational technology companies, such as Microsoft, Qualcomm, Adobe, Intel, NVIDIA, Amazon, and Lenovo. Jia has cultivated a number of outstanding talents with Master's and PhDs who continue to engage in scientific research and development in computer vision. Many technologies in image analysis and processing developed by Jiaya Jia are still leading in the field worldwide. Wherein, his achievements in image deblurring, filtering, image sparse processing, multi-band image signal fusion and enhancement, large range motion estimation, texture and structure-based layering, etc. have been published in the industry's most influential conferences and publications, and implemented in the real-world applications. These achievements have demonstrated outstanding performance in established systems, and most of which are open source so as to enable wider applications across industries such as aviation, medical imaging, safety management, robotic design, meteorological analysis and many more. == Selected publications == In his over 20 years of research experience, Jiaya Jia has published 200+ top papers that have been cited more than 80,000 times. According to HKUST Website in August 2024, Jiaya Jia has accumulatively published over 200 scientific papers in books, journals and conferences, such as IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), International Journal of Computer Vision (IJCV) "Computer Vision and Pattern Recognition (CVPR)", and "International Conference on Computer Vision (ICCV)". Representative papers include: Jiaya Jia: Mathematical Models and Practical Solvers for Uniform Motion Deblurring (in Motion Deblurring: Algorithms and Systems), Cambridge University Press, ISBN 9781107044364, 2014; Jiaya Jia: “Matte Extraction” Book: Computer Vision - A Reference Guide, Springer, ISBN 9780387307718 Editor-in-chief: Ikeuchi, Katsushi; Jiaya Jia, Chi-Keung Tang：Image Stitching Using Structure Deformation，IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 30, No. 4, 2008; Jiaya Jia, Jian Sun, Chi-Keung Tang, Heung-Yeung Shum：Drag-and-Drop Pasting，ACM Transactions on Graphics (also in SIGGRAPH 2006), Vol. 25, No. 3, 2006. Xiaojuan Qi, Zheng zhe Liu, Renjie Liao, Philip HS Torr, Raquel Urtasun, Jiaya Jia：GeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation，IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). Accepted. == Selected honors and awards == ACM Fellow. 1st Place of WAD Drivable Area Segmentation Challenge 2018; 1st Place of LSUN'17 Instance and Semantic Segmentation Challenges; 1st Place of COCO Instance Segmentation Challenge 2017; 2nd Place in COCO Detection Challenge 2017; 1st Place of ImageNet Scene Parsing Challenge 2016 with the paper PSPNet presented in CVPR 2017.