Artificial intelligence (AI) has a range of uses in government. It can be used to further public policy objectives (in areas such as emergency services, health and welfare), as well as assist the public to interact with the government (through the use of virtual assistants, for example). According to the Harvard Business Review, "Applications of artificial intelligence to the public sector are broad and growing, with early experiments taking place around the world." Hila Mehr from the Ash Center for Democratic Governance and Innovation at Harvard University notes that AI in government is not new, with postal services using machine methods in the late 1990s to recognise handwriting on envelopes to automatically route letters. The use of AI in government comes with significant benefits, including efficiencies resulting in cost savings (for instance by reducing the number of front office staff) and reducing the opportunities for corruption. However, it also carries risks (described below). == Uses of AI in government == The potential uses of AI in government are wide and varied, with Deloitte considering that "Cognitive technologies could eventually revolutionize every facet of government operations". Mehr suggests that six types of government problems are appropriate for AI applications: Resource allocation—such as where administrative support is required to complete tasks more quickly. Large datasets—where these are too large for employees to work efficiently and multiple datasets could be combined to provide greater insights. Expert shortage—including where basic questions could be answered and niche issues can be learned. Predictable scenario—historical data makes the situation predictable. Procedural tasks refer to repetitive tasks in which the answers to inputs or outputs are binary. Diverse data—where data takes various forms (such as visual and linguistic) and needs to be summarized regularly. Mehr states that "While applications of AI in government work have not kept pace with the rapid expansion of AI in the private sector, the potential use cases in the public sector mirror common applications in the private sector." Potential and actual uses of AI in government can be divided into three broad categories: those that contribute to public policy objectives, those that assist public interactions with the government, and other uses. === Contributing to public policy objectives === There are a range of examples of where AI can contribute to public policy objectives. These include: Receiving benefits at job loss, retirement, bereavement and child birth almost immediately, in an automated way (thus without requiring any actions from citizens at all) Social insurance service provision Classifying emergency calls based on their urgency (like the system used by the Cincinnati Fire Department in the United States) Detecting and preventing the spread of diseases Assisting public servants in making welfare payments and immigration decisions Adjudicating bail hearings Triaging health care cases Monitoring social media for public feedback on policies Monitoring social media to identify emergency situations Identifying fraudulent benefits claims Predicting a crime and recommending optimal police presence Predicting traffic congestion and car accidents Anticipating road maintenance requirements Identifying breaches of health regulations Providing personalised education to students Marking exam papers Assisting with defence and national security (see Artificial intelligence § Military and Applications of artificial intelligence § Other fields in which AI methods are implemented respectively) Artificial Intelligence in China has been used to drive both political and economic markets. In 2019, Shanghai’s government rolled out 100 billion yuan to assist in funding enterprises that used AI to introduce 22 new policy agendas. Shanghai invested in these enterprises to attract top international talent in order to set up the Shanghai Municipal Big Data Center. City Brain AI is an urban management platform made by Alibaba. China uses City Brain AI to maintain a significant share of capital investment through public and state owned enterprises. The synergy between public and private sectors are more than capital-driven with City Brain AI. The blend of both public and private shareholding is only made out to be through the role of provincial and sub-provincial governments. Both hold control over the direction that City Brain AI makes both socially and economically. === Assisting public interactions with government === AI can be used to assist members of the public to interact with government and access government services, for example by: Answering questions using virtual assistants or chatbots (see below) Directing requests to the appropriate area within government Filling out forms Assisting with searching documents (e.g. IP Australia's trade mark search) Scheduling appointments Various governments, including those of Australia and Estonia, have implemented virtual assistants to aid citizens in navigating services, with applications ranging from tax inquiries to life-event registrations. === Gerrymandering === Gerrymandering is a method of influencing political process by drawing map boundaries in favor of incumbent parties. Academic researchers Wendy Tam Cho and Bruce Cain have proposed partially automating the map-drawing process with an AI system to reduce partisan gerrymandering. Even with this AI system, the process may still be manipulated to favor partisan interests, so the researchers emphasized the importance of transparency and human involvement. === Other uses === Other uses of AI in government include: Translation Language interpretation pioneered by the European Commission's Directorate General for Interpretation and Florika Fink-Hooijer. Drafting documents == Potential benefits == AI offers potential efficiencies and cost savings for the government. For example, Deloitte has estimated that automation could save US Government employees between 96.7 million to 1.2 billion hours a year, resulting in potential savings of between $3.3 billion to $41.1 billion a year. The Harvard Business Review has stated that while this may lead a government to reduce employee numbers, "Governments could instead choose to invest in the quality of its services. They can re-employ workers' time towards more rewarding work that requires lateral thinking, empathy, and creativity—all things at which humans continue to outperform even the most sophisticated AI program." == Risks == Risks associated with the use of AI in government include AI becoming susceptible to bias, a lack of transparency in how an AI application may make decisions, and the accountability for any such decisions. For example, a 2026 lawsuit alleged that the U.S. Department of Government Efficiency used ChatGPT to flag and cancel federal humanities grants, including projects on Jewish history and Israeli culture, over some objections from NEH officials, illustrating how automated decision-making could affect funding outcomes.
Paprika (app)
Paprika is an app and website that helps users organize recipes, produce meal plans, and create grocery lists. The app is available for Android, iOS, macOS, and Windows devices. == Overview == The app allows users to import recipes from various sources, including websites and other apps. The app also allows users to automatically generate meal plans, which are also customizable, in order to achieve specific objectives such as weight loss, muscle gain, adherence to various dietary preferences, or personal taste. The app is also capable of generating grocery lists based on the daily or weekly meal plans chosen by the user. All the recipes, menus, and grocery lists of each user are accessible from smartphones, tablets, and computers. The app is part of a broader category of mobile apps focused on meal planning, recipe management, and shopping list automation, which have grown in popularity with the expansion of smartphone usage and digital cooking tools. == History == Paprika Recipe Manager for iPad version 1.0 was initially released in September 2010 by Hindsight LLC. Paprika 2.0 was released for iPhone and iPad in November 2013, and Paprika 3.0 was released for iOS and macOS in November 2017. == Reception == Paprika has been featured in technology and lifestyle publications as a recipe management and meal planning application. Coverage has noted features such as importing recipes from websites, ingredient scaling, and cross-platform synchronization. The app has also appeared in lists of cooking and meal planning tools published by outlets including The Verge and The Kitchn.
Krohn–Rhodes theory
In mathematics and computer science, the Krohn–Rhodes theory (or algebraic automata theory) is an approach to the study of finite semigroups and automata that seeks to decompose them in terms of elementary components. These components correspond to finite aperiodic semigroups and finite simple groups that are combined in a feedback-free manner (called a "wreath product" or "cascade"). Krohn and Rhodes found a general decomposition for finite automata. The authors discovered and proved an unexpected major result in finite semigroup theory, revealing a deep connection between finite automata and semigroups. Decidability of Krohn-Rhodes complexity long motivated much work in semigroup theory. In June 2024, Stuart Margolis, John Rhodes, and Anne Schilling announced a proof that the complexity is decidable. == Definitions and description of the Krohn–Rhodes theorem == Let T {\displaystyle T} be a semigroup. A semigroup S {\displaystyle S} that is a homomorphic image of a subsemigroup of T {\displaystyle T} is said to be a divisor of T {\displaystyle T} . The Krohn–Rhodes theorem for finite semigroups states that every finite semigroup S {\displaystyle S} is a divisor of a finite alternating wreath product of finite simple groups, each a divisor of S {\displaystyle S} , and finite aperiodic semigroups (which contain no nontrivial subgroups). In the automata formulation, the Krohn–Rhodes theorem for finite automata states that given a finite automaton A {\displaystyle A} with states Q {\displaystyle Q} and input alphabet I {\displaystyle I} , output alphabet U {\displaystyle U} , then one can expand the states to Q ′ {\displaystyle Q'} such that the new automaton A ′ {\displaystyle A'} embeds into a cascade of "simple", irreducible automata: In particular, A {\displaystyle A} is emulated by a feed-forward cascade of (1) automata whose transformation semigroups are finite simple groups and (2) automata that are banks of flip-flops running in parallel. The new automaton A ′ {\displaystyle A'} has the same input and output symbols as A {\displaystyle A} . Here, both the states and inputs of the cascaded automata have a very special hierarchical coordinate form. Moreover, each simple group (prime) or non-group irreducible semigroup (subsemigroup of the flip-flop monoid) that divides the transformation semigroup of A {\displaystyle A} must divide the transformation semigroup of some component of the cascade, and only the primes that must occur as divisors of the components are those that divide A {\displaystyle A} 's transformation semigroup. == Group complexity == The Krohn–Rhodes complexity (also called group complexity or just complexity) of a finite semigroup S is the least number of groups in a wreath product of finite groups and finite aperiodic semigroups of which S is a divisor. All finite aperiodic semigroups have complexity 0, while non-trivial finite groups have complexity 1. In fact, there are semigroups of every non-negative integer complexity. For example, for any n greater than 1, the multiplicative semigroup of all (n+1) × (n+1) upper-triangular matrices over any fixed finite field has complexity n (Kambites, 2007). A major open problem in finite semigroup theory is the decidability of complexity: is there an algorithm that will compute the Krohn–Rhodes complexity of a finite semigroup, given its multiplication table? Upper bounds and ever more precise lower bounds on complexity have been obtained (see, e.g. Rhodes & Steinberg, 2009). Rhodes has conjectured that the problem is decidable. In June 2024, Stuart Margolis, John Rhodes, and Anne Schilling announced a proof in the affirmative of the conjecture, though as of 2025 the result has yet to be confirmed. == History and applications == At a conference in 1962, Kenneth Krohn and John Rhodes announced a method for decomposing a (deterministic) finite automaton into "simple" components that are themselves finite automata. This joint work, which has implications for philosophy, comprised both Krohn's doctoral thesis at Harvard University and Rhodes' doctoral thesis at MIT. Simpler proofs, and generalizations of the theorem to infinite structures, have been published since then (see Chapter 4 of Rhodes and Steinberg's 2009 book The q-Theory of Finite Semigroups for an overview). In the 1965 paper by Krohn and Rhodes, the proof of the theorem on the decomposition of finite automata (or, equivalently sequential machines) made extensive use of the algebraic semigroup structure. Later proofs contained major simplifications using finite wreath products of finite transformation semigroups. The theorem generalizes the Jordan–Hölder decomposition for finite groups (in which the primes are the finite simple groups), to all finite transformation semigroups (for which the primes are again the finite simple groups plus all subsemigroups of the "flip-flop" (see above)). Both the group and more general finite automata decomposition require expanding the state-set of the general, but allow for the same number of input symbols. In the general case, these are embedded in a larger structure with a hierarchical "coordinate system". One must be careful in understanding the notion of "prime" as Krohn and Rhodes explicitly refer to their theorem as a "prime decomposition theorem" for automata. The components in the decomposition, however, are not prime automata (with prime defined in a naïve way); rather, the notion of prime is more sophisticated and algebraic: the semigroups and groups associated to the constituent automata of the decomposition are prime (or irreducible) in a strict and natural algebraic sense with respect to the wreath product (Eilenberg, 1976). Also, unlike earlier decomposition theorems, the Krohn–Rhodes decompositions usually require expansion of the state-set, so that the expanded automaton covers (emulates) the one being decomposed. These facts have made the theorem difficult to understand and challenging to apply in a practical way—until recently, when computational implementations became available (Egri-Nagy & Nehaniv 2005, 2008). H.P. Zeiger (1967) proved an important variant called the holonomy decomposition (Eilenberg 1976). The holonomy method appears to be relatively efficient and has been implemented computationally by A. Egri-Nagy (Egri-Nagy & Nehaniv 2005). Meyer and Thompson (1969) give a version of Krohn–Rhodes decomposition for finite automata that is equivalent to the decomposition previously developed by Hartmanis and Stearns, but for useful decompositions, the notion of expanding the state-set of the original automaton is essential (for the non-permutation automata case). Many proofs and constructions now exist of Krohn–Rhodes decompositions (e.g., [Krohn, Rhodes & Tilson 1968], [Ésik 2000], [Diekert et al. 2012]), with the holonomy method the most popular and efficient in general (although not in all cases). [Zimmermann 2010] gives an elementary proof of the theorem. Owing to the close relation between monoids and categories, a version of the Krohn–Rhodes theorem is applicable to category theory. This observation and a proof of an analogous result were offered by Wells (1980). The Krohn–Rhodes theorem for semigroups/monoids is an analogue of the Jordan–Hölder theorem for finite groups (for semigroups/monoids rather than groups). As such, the theorem is a deep and important result in semigroup/monoid theory. The theorem was also surprising to many mathematicians and computer scientists since it had previously been widely believed that the semigroup/monoid axioms were too weak to admit a structure theorem of any strength, and prior work (Hartmanis & Stearns) was only able to show much more rigid and less general decomposition results for finite automata. Work by Egri-Nagy and Nehaniv (2005, 2008–) continues to further automate the holonomy version of the Krohn–Rhodes decomposition extended with the related decomposition for finite groups (so-called Frobenius–Lagrange coordinates) using the computer algebra system GAP. Applications outside of the semigroup and monoid theories are now computationally feasible. They include computations in biology and biochemical systems (e.g. Egri-Nagy & Nehaniv 2008), artificial intelligence, finite-state physics, psychology, and game theory (see, for example, Rhodes 2009).
Vera Demberg
Vera Demberg (born 1981) is a German computational linguist and professor of computer science and computational linguistics at Saarland University. Her research interests include cognitive models of human language comprehension, natural language generation, experimental psycholinguistics, multimodal language processing in a dual-task setting, and experimental and computational discourse research and pragmatics. == Career and research == Vera Demberg studied computational linguistics at the Institute for Machine Language Processing at the University of Stuttgart from 2001 to 2006. She then completed a Master's degree in Artificial Intelligence at the University of Edinburgh from 2004 to 2005. She received her Ph.D. from the Department of Computer Science there from 2006 to 2010. Her dissertation paper, titled “Broad-Coverage Model of Prediction in Human Sentence Processing”, was awarded the Cognitive Science Society's “Glushko Dissertation Prize in Cognitive Science” in 2011. In her work, she designed a model of human sentence processing that can be used to predict difficulties in processing at the syntactic level. From 2010 to 2016, Vera Demberg led an independent research group on cognitive models of human language processing and their application to speech dialog systems in the Cluster of Excellence “Multimodal Computing and Interaction” at the University of Saarland. In 2016, she was appointed there to a professorship in computer science and computational linguistics. Demberg's professorship is in the Department of Computer Science (Faculty of Mathematics and Computer Science). She is also a co-opted professor in the Department of Linguistics and Language Technology (Faculty of Philosophy). Since 2020, she has led the ERC Starting Grant “Individualized Interaction in Discourse”. The project conducts research on how to make linguistic interaction with computer systems more natural. She has authored and co-authored numerous papers on the study of computational linguistics and natural language processing. According to Google Scholar, Vera Demberg has an H-index of 30. == Publications == Vera Demberg has authored more than 200 papers; please refer to her scholar page at https://scholar.google.com/citations?user=l2CFSAMAAAAJ == Awards == 2011: Cognitive Science Society Glushko Dissertation Prize in Cognitive Science 2020: ERC Starting Grant “Individualized Interaction in Discourse” 2024: Member of the Academy of Sciences and Literature
Sasha Luccioni
Alexandra Sasha Luccioni (née Vorobyova; born 1990) is a computer scientist specializing in the intersection of artificial intelligence (AI) and climate change. Her work focuses on quantifying the environmental impact of AI technologies and promoting sustainable practices in machine learning development. == Early life and education == Alexandra Sasha Vorobyova was born in the Ukrainian Soviet Socialist Republic in 1990. When she was four years old, her family relocated to Ontario, Canada. Her interest in science is influenced by her family's history; her mother, grandmother, and great-grandmother all pursued careers in scientific fields. Luccioni earned a B.A. in language science from University of Paris III: Sorbonne Nouvelle in 2010. Subsequently, she completed a M.S. in cognitive science, with a minor in natural language processing, at École normale supérieure in Paris in 2012. Luccioni obtained her PhD in cognitive computing from Université du Québec à Montréal (UQAM) in 2018. == Career == Luccioni began her professional career at Nuance Communications in 2017, where she focused on natural language processing (NLP) and machine learning (ML) techniques to enhance conversational agents. She then joined Morgan Stanley’s AI/ML Center of Excellence in 2018, working on explainable artificial intelligence (AI) and decision-making systems. In 2019, she became a postdoctoral researcher at Université de Montréal and Mila, collaborating with computer scientist Yoshua Bengio on a project titled This Climate Does Not Exist. This initiative used generative adversarial networks to visualize the effects of climate change. During this time, she also contributed to integrating fairness and accountability into machine learning education at Mila. Luccioni briefly worked with the United Nations Global Pulse in 2021, developing tools to monitor COVID-19 misinformation. Later that year, she joined Hugging Face as a research scientist. Her role includes quantifying the carbon footprint of AI systems, co-chairing the carbon working group in the Big Science project, and advancing responsible machine learning practices. She helped create "CodeCarbon," an open-source software tool that estimates the carbon emissions produced during the training and operation of machine learning models. In addition to her research, she has developed tools to measure the environmental impact of AI models, communicated findings through media engagements, and presented at international conferences, including a TED Talk. In 2024, she was listed on BBC 100 Women and Time 100 AI.
News analytics
In trading strategy, news analysis refers to the measurement of the various qualitative and quantitative attributes of textual (unstructured data) news stories. Some of these attributes are: sentiment, relevance, and novelty. Expressing news stories as numbers and metadata permits the manipulation of everyday information in a mathematical and statistical way. This data is often used in financial markets as part of a trading strategy or by businesses to judge market sentiment and make better business decisions. News analytics are usually derived through automated text analysis and applied to digital texts using elements from natural language processing and machine learning such as latent semantic analysis, support vector machines, "bag of words" among other techniques. == Applications and strategies == The application of sophisticated linguistic analysis to news and social media has grown from an area of research to mature product solutions since 2007. News analytics and news sentiment calculations are now routinely used by both buy-side and sell-side in alpha generation, trading execution, risk management, and market surveillance and compliance. There is however a good deal of variation in the quality, effectiveness and completeness of currently available solutions. A large number of companies use news analysis to help them make better business decisions. Academic researchers have become interested in news analysis especially with regards to predicting stock price movements, volatility and traded volume. Provided a set of values such as sentiment and relevance as well as the frequency of news arrivals, it is possible to construct news sentiment scores for multiple asset classes such as equities, Forex, fixed income, and commodities. Sentiment scores can be constructed at various horizons to meet the different needs and objectives of high and low frequency trading strategies, whilst characteristics such as direction and volatility of asset returns as well as the traded volume may be addressed more directly via the construction of tailor-made sentiment scores. Scores are generally constructed as a range of values. For instance, values may range between 0 and 100, where values above and below 50 convey positive and negative sentiment, respectively. === Absolute return strategies === The objective of absolute return strategies is absolute (positive) returns regardless of the direction of the financial market. To meet this objective, such strategies typically involve opportunistic long and short positions in selected instruments with zero or limited market exposure. In statistical terms, absolute return strategies should have very low correlation with the market return. Typically, hedge funds tend to employ absolute return strategies. Below, a few examples show how news analysis can be applied in the absolute return strategy space with the purpose to identify alpha opportunities applying a market neutral strategy or based on volatility trading. Example 1 Scenario: The gap between the news sentiment scores for direction, S {\displaystyle S} , of Company X {\displaystyle X} and Market Y {\displaystyle Y} has moved beyond + 20 {\displaystyle +20} . That is, S X − S Y {\displaystyle S_{X}-S_{Y}} ≥ 20 {\displaystyle 20} . Action: Buy the stock on Company X {\displaystyle X} and short the future on Market Y {\displaystyle Y} . Exit Strategy: When the gap in the news sentiment scores for direction of Company X {\displaystyle X} and Market Y {\displaystyle Y} has disappeared, S X − S Y {\displaystyle S_{X}-S_{Y}} = 0 {\displaystyle 0} , sell the stock on Company X {\displaystyle X} and go long the future on Market Y {\displaystyle Y} to close the positions. Example 2 Scenario: The news sentiment score for volatility of Company X {\displaystyle X} goes above 70 {\displaystyle 70} out of 100 {\displaystyle 100} indicating an expected volatility above the option implied volatility. Action: Buy a short-dated straddle (the purchase of both a put and a call) on the stock of Company X {\displaystyle X} . Exit Strategy: Keep the straddle on Company X {\displaystyle X} until expiry or until a certain profit target has been reached. === Relative return strategies === The objective of relative return strategies is to either replicate (passive management) or outperform (active management) a theoretical passive reference portfolio or benchmark. To meet these objectives such strategies typically involve long positions in selected instruments. In statistical terms, relative return strategies often have high correlation with the market return. Typically, mutual funds tend to employ relative return strategies. Below, a few examples show how news analysis can be applied in the relative return strategy space with the purpose to outperform the market applying a stock picking strategy and by making tactical tilts to ones asset allocation model. Example 1 Scenario: The news sentiment score for direction of Company X {\displaystyle X} goes above 70 {\displaystyle 70} out of 100 {\displaystyle 100} . Action: Buy the stock on Company X {\displaystyle X} . Exit Strategy: When the news sentiment score for direction of Company X {\displaystyle X} falls below 60 {\displaystyle 60} , sell the stock on Company X {\displaystyle X} to close the position. Example 2 Scenario: The news sentiment score for direction of Sector Z {\displaystyle Z} goes above 70 {\displaystyle 70} out of 100 {\displaystyle 100} . Action: Include Sector Z {\displaystyle Z} as a tactical bet in the asset allocation model. Exit Strategy: When the news sentiment score for direction of Sector Z {\displaystyle Z} falls below 60 {\displaystyle 60} , remove the tactical bet for Sector Z {\displaystyle Z} from the asset allocation model. === Financial risk management === The objective of financial risk management is to create economic value in a firm or to maintain a certain risk profile of an investment portfolio by using financial instruments to manage risk exposures, particularly credit risk and market risk. Other types include Foreign exchange, Shape, Volatility, Sector, Liquidity, Inflation risks, etc. Below, a few examples show how news analysis can be applied in the financial risk management space with the purpose to either arrive at better risk estimates in terms of Value at Risk (VaR) or to manage the risk of a portfolio to meet ones portfolio mandate. Example 1 Scenario: The bank operates a VaR model to manage the overall market risk of its portfolio. Action: Estimate the portfolio covariance matrix taking into account the development of the news sentiment score for volume. Implement the relevant hedges to bring the VaR of the bank in line with the desired levels. Example 2 Scenario: A portfolio manager operates his portfolio towards a certain desired risk profile. Action: Estimate the portfolio covariance matrix taking into account the development of the news sentiment score for volume. Scale the portfolio exposure according to the targeted risk profile. === Computer algorithms using news analytics === Within 0.33 seconds, computer algorithms using news analytics can notify subscribers which company the news is about, if the news article sentiment is positive or negative, if the news is ranked as high or low relative importance … relative relevance. the stock price reaction and the increase in trade volume is concentrated in the first 5 seconds after an news article is released. === Algorithmic order execution === The objective of algorithmic order execution, which is part of the concept of algorithmic trading, is to reduce trading costs by optimizing on the timing of a given order. It is widely used by hedge funds, pension funds, mutual funds, and other institutional traders to divide up large trades into several smaller trades to manage market impact, opportunity cost, and risk more effectively. The example below shows how news analysis can be applied in the algorithmic order execution space with the purpose to arrive at more efficient algorithmic trading systems. Example 1 Scenario: A large order needs to be placed in the market for the stock on Company X {\displaystyle X} . Action: Scale the daily volume distribution for Company X {\displaystyle X} applied in the algorithmic trading system, thus taking into account the news sentiment score for volume. This is followed by the creation of the desired trading distribution forcing greater market participation during the periods of the day when volume is expected to be heaviest. == Effects == Being able to express news stories as numbers permits the manipulation of everyday information in a statistical way that allows computers not only to make decisions once made only by humans, but to do so more efficiently. Since market participants are always looking for an edge, the speed of computer connections and the delivery of news analysis, measured in milliseconds, have become essential.
IBM alignment models
The IBM alignment models are a sequence of increasingly complex models used in statistical machine translation to train a translation model and an alignment model, starting with lexical translation probabilities and moving to reordering and word duplication. They underpinned the majority of statistical machine translation systems for almost twenty years starting in the early 1990s, until neural machine translation began to dominate. These models offer principled probabilistic formulation and (mostly) tractable inference. The IBM alignment models were published in parts in 1988 and 1990, and the entire series is published in 1993. Every author of the 1993 paper subsequently went to the hedge fund Renaissance Technologies. The original work on statistical machine translation at IBM proposed five models, and a model 6 was proposed later. The sequence of the six models can be summarized as: Model 1: lexical translation Model 2: additional absolute alignment model Model 3: extra fertility model Model 4: added relative alignment model Model 5: fixed deficiency problem. Model 6: Model 4 combined with a HMM alignment model in a log linear way == Mathematical setup == The IBM alignment models translation as a conditional probability model. For each source-language ("foreign") sentence f {\displaystyle f} , we generate both a target-language ("English") sentence e {\displaystyle e} and an alignment a {\displaystyle a} . The problem then is to find a good statistical model for p ( e , a | f ) {\displaystyle p(e,a|f)} , the probability that we would generate English language sentence e {\displaystyle e} and an alignment a {\displaystyle a} given a foreign sentence f {\displaystyle f} . The meaning of an alignment grows increasingly complicated as the model version number grew. See Model 1 for the most simple and understandable version. == Model 1 == === Word alignment === Given any foreign-English sentence pair ( e , f ) {\displaystyle (e,f)} , an alignment for the sentence pair is a function of type { 1 , . , . . . , l e } → { 0 , 1 , . , . . . , l f } {\displaystyle \{1,.,...,l_{e}\}\to \{0,1,.,...,l_{f}\}} . That is, we assume that the English word at location i {\displaystyle i} is "explained" by the foreign word at location a ( i ) {\displaystyle a(i)} . For example, consider the following pair of sentences It will surely rain tomorrow -- 明日 は きっと 雨 だWe can align some English words to corresponding Japanese words, but not everyone:it -> ? will -> ? surely -> きっと rain -> 雨 tomorrow -> 明日This in general happens due to the different grammar and conventions of speech in different languages. English sentences require a subject, and when there is no subject available, it uses a dummy pronoun it. Japanese verbs do not have different forms for future and present tense, and the future tense is implied by the noun 明日 (tomorrow). Conversely, the topic-marker は and the grammar word だ (roughly "to be") do not correspond to any word in the English sentence. So, we can write the alignment as 1-> 0; 2 -> 0; 3 -> 3; 4 -> 4; 5 -> 1where 0 means that there is no corresponding alignment. Thus, we see that the alignment function is in general a function of type { 1 , . , . . . , l e } → { 0 , 1 , . , . . . , l f } {\displaystyle \{1,.,...,l_{e}\}\to \{0,1,.,...,l_{f}\}} . Future models will allow one English world to be aligned with multiple foreign words. === Statistical model === Given the above definition of alignment, we can define the statistical model used by Model 1: Start with a "dictionary". Its entries are of form t ( e i | f j ) {\displaystyle t(e_{i}|f_{j})} , which can be interpreted as saying "the foreign word f j {\displaystyle f_{j}} is translated to the English word e i {\displaystyle e_{i}} with probability t ( e i | f j ) {\displaystyle t(e_{i}|f_{j})} ". After being given a foreign sentence f {\displaystyle f} with length l f {\displaystyle l_{f}} , we first generate an English sentence length l e {\displaystyle l_{e}} uniformly in a range U n i f o r m [ 1 , 2 , . . . , N ] {\displaystyle Uniform[1,2,...,N]} . In particular, it does not depend on f {\displaystyle f} or l f {\displaystyle l_{f}} . Then, we generate an alignment uniformly in the set of all possible alignment functions { 1 , . , . . . , l e } → { 0 , 1 , . , . . . , l f } {\displaystyle \{1,.,...,l_{e}\}\to \{0,1,.,...,l_{f}\}} . Finally, for each English word e 1 , e 2 , . . . e l e {\displaystyle e_{1},e_{2},...e_{l_{e}}} , generate each one independently of every other English word. For the word e i {\displaystyle e_{i}} , generate it according to t ( e i | f a ( i ) ) {\displaystyle t(e_{i}|f_{a(i)})} . Together, we have the probability p ( e , a | f ) = 1 / N ( 1 + l f ) l e ∏ i = 1 l e t ( e i | f a ( i ) ) {\displaystyle p(e,a|f)={\frac {1/N}{(1+l_{f})^{l_{e}}}}\prod _{i=1}^{l_{e}}t(e_{i}|f_{a(i)})} IBM Model 1 uses very simplistic assumptions on the statistical model, in order to allow the following algorithm to have closed-form solution. === Learning from a corpus === If a dictionary is not provided at the start, but we have a corpus of English-foreign language pairs { ( e ( k ) , f ( k ) ) } k {\displaystyle \{(e^{(k)},f^{(k)})\}_{k}} (without alignment information), then the model can be cast into the following form: fixed parameters: the foreign sentences { f ( k ) } k {\displaystyle \{f^{(k)}\}_{k}} . learnable parameters: the entries of the dictionary t ( e i | f j ) {\displaystyle t(e_{i}|f_{j})} . observable variables: the English sentences { e ( k ) } k {\displaystyle \{e^{(k)}\}_{k}} . latent variables: the alignments { a ( k ) } k {\displaystyle \{a^{(k)}\}_{k}} In this form, this is exactly the kind of problem solved by expectation–maximization algorithm. Due to the simplistic assumptions, the algorithm has a closed-form, efficiently computable solution, which is the solution to the following equations: { max t ′ ∑ k ∑ i ∑ a ( k ) t ( a ( k ) | e ( k ) , f ( k ) ) ln t ( e i ( k ) | f a ( k ) ( i ) ( k ) ) ∑ x t ′ ( e x | f y ) = 1 ∀ y {\displaystyle {\begin{cases}\max _{t'}\sum _{k}\sum _{i}\sum _{a^{(k)}}t(a^{(k)}|e^{(k)},f^{(k)})\ln t(e_{i}^{(k)}|f_{a^{(k)}(i)}^{(k)})\\\sum _{x}t'(e_{x}|f_{y})=1\quad \forall y\end{cases}}} This can be solved by Lagrangian multipliers, then simplified. For a detailed derivation of the algorithm, see chapter 4 and. In short, the EM algorithm goes as follows:INPUT. a corpus of English-foreign sentence pairs { ( e ( k ) , f ( k ) ) } k {\displaystyle \{(e^{(k)},f^{(k)})\}_{k}} INITIALIZE. matrix of translations probabilities t ( e x | f y ) {\displaystyle t(e_{x}|f_{y})} .This could either be uniform or random. It is only required that every entry is positive, and for each y {\displaystyle y} , the probability sums to one: ∑ x t ( e x | f y ) = 1 {\displaystyle \sum _{x}t(e_{x}|f_{y})=1} . LOOP. until t ( e x | f y ) {\displaystyle t(e_{x}|f_{y})} converges: t ( e x | f y ) ← t ( e x | f y ) λ y ∑ k , i , j δ ( e x , e i ( k ) ) δ ( f y , f j ( k ) ) ∑ j ′ t ( e i ( k ) | f j ′ ( k ) ) {\displaystyle t(e_{x}|f_{y})\leftarrow {\frac {t(e_{x}|f_{y})}{\lambda _{y}}}\sum _{k,i,j}{\frac {\delta (e_{x},e_{i}^{(k)})\delta (f_{y},f_{j}^{(k)})}{\sum _{j'}t(e_{i}^{(k)}|f_{j'}^{(k)})}}} where each λ y {\displaystyle \lambda _{y}} is a normalization constant that makes sure each ∑ x t ( e x | f y ) = 1 {\displaystyle \sum _{x}t(e_{x}|f_{y})=1} .RETURN. t ( e x | f y ) {\displaystyle t(e_{x}|f_{y})} .In the above formula, δ {\displaystyle \delta } is the Dirac delta function -- it equals 1 if the two entries are equal, and 0 otherwise. The index notation is as follows: k {\displaystyle k} ranges over English-foreign sentence pairs in corpus; i {\displaystyle i} ranges over words in English sentences; j {\displaystyle j} ranges over words in foreign language sentences; x {\displaystyle x} ranges over the entire vocabulary of English words in the corpus; y {\displaystyle y} ranges over the entire vocabulary of foreign words in the corpus. === Limitations === There are several limitations to the IBM model 1. No fluency: Given any sentence pair ( e , f ) {\displaystyle (e,f)} , any permutation of the English sentence is equally likely: p ( e | f ) = p ( e ′ | f ) {\displaystyle p(e|f)=p(e'|f)} for any permutation of the English sentence e {\displaystyle e} into e ′ {\displaystyle e'} . No length preference: The probability of each length of translation is equal: ∑ e has length l p ( e | f ) = 1 N {\displaystyle \sum _{e{\text{ has length }}l}p(e|f)={\frac {1}{N}}} for any l ∈ { 1 , 2 , . . . , N } {\displaystyle l\in \{1,2,...,N\}} . Does not explicitly model fertility: some foreign words tend to produce a fixed number of English words. For example, for German-to-English translation, ja is usually omitted, and zum is usually translated to one of to the, for the, to a, for a. == Model 2 == Model 2 allows alignment to be conditional on sentence lengths. That is, we have a probability distribution p a ( j | i , l e , l f ) {\displaystyle