In machine learning, attention is a method that determines the importance of each component in a sequence relative to the other components in that sequence. In natural language processing, importance is represented by "soft" weights assigned to each word in a sentence. More generally, attention encodes vectors called token embeddings across a fixed-width sequence that can range from tens to millions of tokens in size. Unlike "hard" weights, which are computed during the backwards training pass, "soft" weights exist only in the forward pass and therefore change with every step of the input. Earlier designs implemented the attention mechanism in a serial recurrent neural network (RNN) language translation system, but a more recent design, namely the transformer, removed the slower sequential RNN and relied more heavily on the faster parallel attention scheme. Inspired by ideas about attention in humans, the attention mechanism was developed to address the weaknesses of using information from the hidden layers of recurrent neural networks. Recurrent neural networks favor information contained in words at the end of a sentence and thus deemed more recent, thereby tending to attenuate the significance and associated predictive weight assigned to information earlier in the sentence. Attention allows a token equal access to any part of a sentence directly, rather than only through the previous state. == History == Additional surveys of the attention mechanism in deep learning are provided by Niu et al. and Soydaner. The major breakthrough came with self-attention, where each element in the input sequence attends to all others, enabling the model to capture global dependencies. This idea was central to the Transformer architecture, which replaced recurrence with attention mechanisms. As a result, Transformers became the foundation for models like BERT, T5 and generative pre-trained transformers (GPT). == Overview == The modern era of machine attention was revitalized by grafting an attention mechanism (Fig 1. orange) to an Encoder-Decoder. Figure 2 shows the internal step-by-step operation of the attention block (A) in Fig 1. === Interpreting attention weights === In translating between languages, alignment is the process of matching words from the source sentence to words of the translated sentence. Networks that perform verbatim translation without regard to word order would show the highest scores along the (dominant) diagonal of the matrix. The off-diagonal dominance shows that the attention mechanism is more nuanced. Consider an example of translating I love you to French. On the first pass through the decoder, 94% of the attention weight is on the first English word I, so the network offers the word je. On the second pass of the decoder, 88% of the attention weight is on the third English word you, so it offers t'. On the last pass, 95% of the attention weight is on the second English word love, so it offers aime. In the I love you example, the second word love is aligned with the third word aime. Stacking soft row vectors together for je, t', and aime yields an alignment matrix: Sometimes, alignment can be multiple-to-multiple. For example, the English phrase look it up corresponds to cherchez-le. Thus, "soft" attention weights work better than "hard" attention weights (setting one attention weight to 1, and the others to 0), as we would like the model to make a context vector consisting of a weighted sum of the hidden vectors, rather than "the best one", as there may not be a best hidden vector. == Variants == Many variants of attention implement soft weights, such as fast weight programmers, or fast weight controllers (1992). A "slow" neural network outputs the "fast" weights of another neural network through outer products. The slow network learns by gradient descent. It was later renamed as "linearized self-attention". Bahdanau-style attention, also referred to as additive attention, Luong-style attention, which is known as multiplicative attention, Early attention mechanisms similar to modern self-attention were proposed using recurrent neural networks. However, the highly parallelizable self-attention was introduced in 2017 and successfully used in the Transformer model, positional attention and factorized positional attention. For convolutional neural networks, attention mechanisms can be distinguished by the dimension on which they operate, namely: spatial attention, channel attention, or combinations. These variants recombine the encoder-side inputs to redistribute those effects to each target output. Often, a correlation-style matrix of dot products provides the re-weighting coefficients. In the figures below, W is the matrix of context attention weights, similar to the formula in Overview section above. == Optimizations == === Flash attention === The size of the attention matrix is proportional to the square of the number of input tokens. Therefore, when the input is long, calculating the attention matrix requires a lot of GPU memory. Flash attention is an implementation that reduces the memory needs and increases efficiency without sacrificing accuracy. It achieves this by partitioning the attention computation into smaller blocks that fit into the GPU's faster on-chip memory, reducing the need to store large intermediate matrices and thus lowering memory usage while increasing computational efficiency. === FlexAttention === FlexAttention is an attention kernel developed by Meta that allows users to modify attention scores prior to softmax and dynamically chooses the optimal attention algorithm. == Applications == Attention is widely used in natural language processing, computer vision, and speech recognition. In NLP, it improves context understanding in tasks like question answering and summarization. In vision, visual attention helps models focus on relevant image regions, enhancing object detection and image captioning. === Attention maps as explanations for vision transformers === From the original paper on vision transformers (ViT), visualizing attention scores as a heat map (called saliency maps or attention maps) has become an important and routine way to inspect the decision making process of ViT models. One can compute the attention maps with respect to any attention head at any layer, while the deeper layers tend to show more semantically meaningful visualization. Attention rollout is a recursive algorithm to combine attention scores across all layers, by computing the dot product of successive attention maps. Because vision transformers are typically trained in a self-supervised manner, attention maps are generally not class-sensitive. When a classification head is attached to the ViT backbone, class-discriminative attention maps (CDAM) combines attention maps and gradients with respect to the class [CLS] token. Some class-sensitive interpretability methods originally developed for convolutional neural networks can be also applied to ViT, such as GradCAM, which back-propagates the gradients to the outputs of the final attention layer. Using attention as basis of explanation for the transformers in language and vision is not without debate. While some pioneering papers analyzed and framed attention scores as explanations, higher attention scores do not always correlate with greater impact on model performances. == Mathematical representation == === Standard scaled dot-product attention === For matrices: Q ∈ R m × d k , K ∈ R n × d k {\displaystyle Q\in \mathbb {R} ^{m\times d_{k}},K\in \mathbb {R} ^{n\times d_{k}}} and V ∈ R n × d v {\displaystyle V\in \mathbb {R} ^{n\times d_{v}}} , the scaled dot-product, or QKV attention, is defined as: Attention ( Q , K , V ) = softmax ( Q K T d k ) V ∈ R m × d v {\displaystyle {\text{Attention}}(Q,K,V)={\text{softmax}}\left({\frac {QK^{T}}{\sqrt {d_{k}}}}\right)V\in \mathbb {R} ^{m\times d_{v}}} where T {\displaystyle {}^{T}} denotes transpose and the softmax function is applied independently to every row of its argument. The matrix Q {\displaystyle Q} contains m {\displaystyle m} queries, while matrices K , V {\displaystyle K,V} jointly contain an unordered set of n {\displaystyle n} key-value pairs. Value vectors in matrix V {\displaystyle V} are weighted using the weights resulting from the softmax operation, so that the rows of the m {\displaystyle m} -by- d v {\displaystyle d_{v}} output matrix are confined to the convex hull of the points in R d v {\displaystyle \mathbb {R} ^{d_{v}}} given by the rows of V {\displaystyle V} . To understand the permutation invariance and permutation equivariance properties of QKV attention, let A ∈ R m × m {\displaystyle A\in \mathbb {R} ^{m\times m}} and B ∈ R n × n {\displaystyle B\in \mathbb {R} ^{n\times n}} be permutation matrices; and D ∈ R m × n {\displaystyle D\in \mathbb {R} ^{m\times n}} an arbitrary matrix. The softmax function is permutation equivariant in the sense that: softmax ( A D B ) = A softmax ( D ) B {\displays
Automated Lip Reading
Automated Lip Reading (ALR) is a software technology developed by speech recognition expert Frank Hubner. A video image of a person talking can be analysed by the software. The shapes made by the lips can be examined and then turned into sounds. The sounds are compared to a dictionary to create matches to the words being spoken. The technology was used successfully to analyse silent home movie footage of Adolf Hitler taken by Eva Braun at their Bavarian retreat Berghof. The video, with words, was included in a documentary titled "Hitler's Private World", Revealed Studios, 2006 Source: New Technology catches Hitler off guard
Type-2 fuzzy sets and systems
Type-2 fuzzy sets and systems generalize standard type-1 fuzzy sets and systems so that more uncertainty can be handled. From the beginning of fuzzy sets, criticism was made about the fact that the membership function of a type-1 fuzzy set has no uncertainty associated with it, something that seems to contradict the word fuzzy, since that word has the connotation of much uncertainty. So, what does one do when there is uncertainty about the value of the membership function? The answer to this question was provided in 1975 by the inventor of fuzzy sets, Lotfi A. Zadeh, when he proposed more sophisticated kinds of fuzzy sets, the first of which he called a "type-2 fuzzy set". A type-2 fuzzy set lets us incorporate uncertainty about the membership function into fuzzy set theory, and is a way to address the above criticism of type-1 fuzzy sets head-on. And, if there is no uncertainty, then a type-2 fuzzy set reduces to a type-1 fuzzy set, which is analogous to probability reducing to determinism when unpredictability vanishes. Type1 fuzzy systems are working with a fixed membership function, while in type-2 fuzzy systems the membership function is fluctuating. A fuzzy set determines how input values are converted into fuzzy variables. == Overview == In order to symbolically distinguish between a type-1 fuzzy set and a type-2 fuzzy set, a tilde symbol is put over the symbol for the fuzzy set; so, A denotes a type-1 fuzzy set, whereas à denotes the comparable type-2 fuzzy set. When the latter is done, the resulting type-2 fuzzy set is called a "general type-2 fuzzy set" (to distinguish it from the special interval type-2 fuzzy set). Zadeh didn't stop with type-2 fuzzy sets, because in that 1976 paper he also generalized all of this to type-n fuzzy sets. The present article focuses only on type-2 fuzzy sets because they are the next step in the logical progression from type-1 to type-n fuzzy sets, where n = 1, 2, ... . Although some researchers are beginning to explore higher than type-2 fuzzy sets, as of early 2009, this work is in its infancy. The membership function of a general type-2 fuzzy set, Ã, is three-dimensional (Fig. 1), where the third dimension is the value of the membership function at each point on its two-dimensional domain that is called its "footprint of uncertainty"(FOU). For an interval type-2 fuzzy set that third-dimension value is the same (e.g., 1) everywhere, which means that no new information is contained in the third dimension of an interval type-2 fuzzy set. So, for such a set, the third dimension is ignored, and only the FOU is used to describe it. It is for this reason that an interval type-2 fuzzy set is sometimes called a first-order uncertainty fuzzy set model, whereas a general type-2 fuzzy set (with its useful third-dimension) is sometimes referred to as a second-order uncertainty fuzzy set model. The FOU represents the blurring of a type-1 membership function, and is completely described by its two bounding functions (Fig. 2), a lower membership function (LMF) and an upper membership function (UMF), both of which are type-1 fuzzy sets! Consequently, it is possible to use type-1 fuzzy set mathematics to characterize and work with interval type-2 fuzzy sets. This means that engineers and scientists who already know type-1 fuzzy sets will not have to invest a lot of time learning about general type-2 fuzzy set mathematics in order to understand and use interval type-2 fuzzy sets. Work on type-2 fuzzy sets languished during the 1980s and early-to-mid 1990s, although a small number of articles were published about them. People were still trying to figure out what to do with type-1 fuzzy sets, so even though Zadeh proposed type-2 fuzzy sets in 1976, the time was not right for researchers to drop what they were doing with type-1 fuzzy sets to focus on type-2 fuzzy sets. This changed in the latter part of the 1990s as a result of Jerry Mendel and his student's works on type-2 fuzzy sets and systems. Since then, more researchers around the world are writing articles about type-2 fuzzy sets and systems. == Interval type-2 fuzzy sets == Interval type-2 fuzzy sets have received the most attention because the mathematics that is needed for such sets—primarily Interval arithmetic—is much simpler than the mathematics that is needed for general type-2 fuzzy sets. The literature about interval type-2 fuzzy sets is large, whereas the literature about general type-2 fuzzy sets is much smaller. Both kinds of fuzzy sets are being actively researched by an ever-growing number of researchers around the world and have resulted in successful employment in a variety of domains such as robot control. Formally, the following have already been worked out for interval type-2 fuzzy sets: Fuzzy set operations: union, intersection and complement Centroid (a very widely used operation by practitioners of such sets, and also an important uncertainty measure for them) Other uncertainty measures [fuzziness, cardinality, variance and skewness and uncertainty bounds Similarity Subsethood Embedded fuzzy sets Fuzzy set ranking Fuzzy rule ranking and selection Type-reduction methods Firing intervals for an interval type-2 fuzzy logic system Fuzzy weighted average Linguistic weighted average Synthesizing an FOU from data that are collected from a group of subject == Interval type-2 fuzzy logic systems == Type-2 fuzzy sets are finding very wide applicability in rule-based fuzzy logic systems (FLSs) because they let uncertainties be modeled by them whereas such uncertainties cannot be modeled by type-1 fuzzy sets. A block diagram of a type-2 FLS is depicted in Fig. 3. This kind of FLS is used in fuzzy logic control, fuzzy logic signal processing, rule-based classification, etc., and is sometimes referred to as a function approximation application of fuzzy sets, because the FLS is designed to minimize an error function. The following discussions, about the four components in Fig. 3 rule-based FLS, are given for an interval type-2 FLS, because to-date they are the most popular kind of type-2 FLS; however, most of the discussions are also applicable for a general type-2 FLS. Rules, that are either provided by subject experts or are extracted from numerical data, are expressed as a collection of IF-THEN statements, e.g., IF temperature is moderate and pressure is high, then rotate the valve a bit to the right. Fuzzy sets are associated with the terms that appear in the antecedents (IF-part) or consequents (THEN-part) of rules, and with the inputs to and the outputs of the FLS. Membership functions are used to describe these fuzzy sets, and in a type-1 FLS they are all type-1 fuzzy sets, whereas in an interval type-2 FLS at least one membership function is an interval type-2 fuzzy set. An interval type-2 FLS lets any one or all of the following kinds of uncertainties be quantified: Words that are used in antecedents and consequents of rules—because words can mean different things to different people. Uncertain consequents—because when rules are obtained from a group of experts, consequents will often be different for the same rule, i.e. the experts will not necessarily be in agreement. Membership function parameters—because when those parameters are optimized using uncertain (noisy) training data, the parameters become uncertain. Noisy measurements—because very often it is such measurements that activate the FLS. In Fig. 3, measured (crisp) inputs are first transformed into fuzzy sets in the Fuzzifier block because it is fuzzy sets and not numbers that activate the rules which are described in terms of fuzzy sets and not numbers. Three kinds of fuzzifiers are possible in an interval type-2 FLS. When measurements are: Perfect, they are modeled as a crisp set; Noisy, but the noise is stationary, they are modeled as a type-1 fuzzy set; and, Noisy, but the noise is non-stationary, they are modeled as an interval type-2 fuzzy set (this latter kind of fuzzification cannot be done in a type-1 FLS). In Fig. 3, after measurements are fuzzified, the resulting input fuzzy sets are mapped into fuzzy output sets by the Inference block. This is accomplished by first quantifying each rule using fuzzy set theory, and by then using the mathematics of fuzzy sets to establish the output of each rule, with the help of an inference mechanism. If there are M rules then the fuzzy input sets to the Inference block will activate only a subset of those rules, where the subset contains at least one rule and usually way fewer than M rules. The inference is done one rule at a time. So, at the output of the Inference block, there will be one or more fired-rule fuzzy output sets. In most engineering applications of an FLS, a number (and not a fuzzy set) is needed as its final output, e.g., the consequent of the rule given above is "Rotate the valve a bit to the right." No automatic valve will know what this means because "a bit to the right" is a linguistic expression, and a valv
SCIgen
SCIgen is a paper generator that uses context-free grammar to randomly generate nonsense in the form of computer science research papers. Its original data source was a collection of computer science papers downloaded from CiteSeer. All elements of the papers are formed, including graphs, diagrams, and citations. Created by scientists at the Massachusetts Institute of Technology, its stated aim is "to maximize amusement, rather than coherence." Originally created in 2005 to expose the lack of scrutiny of submissions to conferences, the generator subsequently became used, primarily by Chinese academics, to create large numbers of fraudulent conference submissions, leading to the retraction of 122 SCIgen generated papers and the creation of detection software to combat its use. == Sample output == Opening abstract of Rooter: A Methodology for the Typical Unification of Access Points and Redundancy: Many physicists would agree that, had it not been for congestion control, the evaluation of web browsers might never have occurred. In fact, few hackers worldwide would disagree with the essential unification of voice-over-IP and public/private key pair. In order to solve this riddle, we confirm that SMPs can be made stochastic, cacheable, and interposable. == Prominent results == In 2005, a paper generated by SCIgen, Rooter: A Methodology for the Typical Unification of Access Points and Redundancy, was accepted as a non-reviewed paper to the 2005 World Multiconference on Systemics, Cybernetics and Informatics (WMSCI) and the authors were invited to speak. The authors of SCIgen described their hoax on their website, and it soon received great publicity when picked up by Slashdot. WMSCI withdrew their invitation, but the SCIgen team went anyway, renting space in the hotel separately from the conference and delivering a series of randomly generated talks on their own "track". The organizer of these WMSCI conferences is Professor Nagib Callaos. From 2000 until 2005, the WMSCI was also sponsored by the Institute of Electrical and Electronics Engineers. The IEEE stopped granting sponsorship to Callaos from 2006 to 2008. Submitting the paper was a deliberate attempt to embarrass WMSCI, which the authors claim accepts low-quality papers and sends unsolicited requests for submissions in bulk to academics. As the SCIgen website states: One useful purpose for such a program is to auto-generate submissions to conferences that you suspect might have very low submission standards. A prime example, which you may recognize from spam in your inbox, is SCI/IIIS and its dozens of co-located conferences (check out the very broad conference description on the WMSCI 2005 website). Computing writer Stan Kelly-Bootle noted in ACM Queue that many sentences in the "Rooter" paper were individually plausible, which he regarded as posing a problem for automated detection of hoax articles. He suggested that even human readers might be taken in by the effective use of jargon ("The pun on root/router is par for MIT-graduate humor, and at least one occurrence of methodology is mandatory") and attribute the paper's apparent incoherence to their own limited knowledge. His conclusion was that "a reliable gibberish filter requires a careful holistic review by several peer domain experts". === Schlangemann === The pseudonym "Herbert Schlangemann" was used to publish fake scientific articles in international conferences that claimed to practice peer review. The name is taken from the Swedish short film Der Schlangemann. In 2008, in response to a series of Call-for-Paper e-mails, SCIgen was used to generate a false scientific paper titled Towards the Simulation of E-Commerce, using "Herbert Schlangemann" as the author. The article was accepted at the 2008 International Conference on Computer Science and Software Engineering (CSSE 2008), co-sponsored by the IEEE, to be held in Wuhan, China, and the author was invited to be a session chair on grounds of his fictional Curriculum Vitae. The official review comment: "This paper presents cooperative technology and classical Communication. In conclusion, the result shows that though the much-touted amphibious algorithm for the refinement of randomized algorithms is impossible, the well-known client-server algorithm for the analysis of voice-over-IP by Kumar and Raman runs in _(n) time. The authors can clearly identify important features of visualization of DHTs and analyze them insightfully. It is recommended that the authors should develop ideas more cogently, organizes them more logically, and connects them with clear transitions." The paper was available for a short time in the IEEE Xplore Database, but was then removed. The entire story is described in the official "Herbert Schlangemann" blog, and it also received attention in Slashdot and the German-language technology-news site Heise Online. In 2009, the same incident happened and Herbert Schlangemann's latest fake paper PlusPug: A Methodology for the Improvement of Local-Area Networks was accepted for oral presentation at the 2009 International Conference on e-Business and Information System Security (EBISS 2009), also co-sponsored by IEEE, to be held again in Wuhan, China. In all cases, the published papers were withdrawn from the conferences' proceedings, and the conference organizing committee as well as the names of the keynote speakers were removed from their websites. === List of works with notable acceptance === ==== In conferences ==== Rob Thomas: Rooter: A Methodology for the Typical Unification of Access Points and Redundancy, 2005 for WMSCI (see above) Mathias Uslar's paper was accepted to the IPSI-BG conference. Professor Genco Gulan published a paper in the 3rd International Symposium of Interactive Media Design. A 2013 scientometrics paper demonstrated that at least 85 SCIgen papers have been published by IEEE and Springer. Over 120 SCIgen papers were removed according to this research. ==== In journals ==== Students at Iran's Sharif University of Technology published a paper in Elsevier's Journal of Applied Mathematics and Computation. The students wrote under the surname "MosallahNejad", which translates literally from Persian language (in spite of not being a traditional Persian name) as "from an Armed Breed". The paper was subsequently removed when the publishers were informed that it was a joke paper. Mikhail Gelfand published a translation of the "Rooter" article in the Russian-language Journal of Scientific Publications of Aspirants and Doctorants in August 2008. Gelfand was protesting against the journal, which was apparently not peer-reviewed and was being used by Russian PhD candidates to publish in an "accredited" scientific journal, charging them 4,000 Rubles to do so. The accreditation was revoked two weeks later. (See Dissernet for related information.) Springer Science+Business Media and IEEE were also the subject of similar pranks. === Spoofing Google Scholar and h-index calculators === Refereeing performed on behalf of the Institute of Electrical and Electronics Engineers has also been subject to criticism after fake papers were discovered in conference publications, most notably by Labbé and a researcher using the pseudonym of Schlangemann. Cyril Labbé from Grenoble University demonstrated the vulnerability of h-index calculations based on Google Scholar output by feeding it a large set of SCIgen-generated documents that were citing each other, effectively an academic link farm, in a 2010 paper. Using this method the author managed to rank "Ike Antkare" ahead of Albert Einstein for instance. === 2013 retractions === In 2013, over 122 published conference papers created by SCIgen were retracted by Springer and the IEEE. Unlike previous submissions that were intended to be pranks, this submission were largely made by Chinese academics, who were using SCIgen papers to boost their publication record. === SciDetect === In 2015, SciDetect was released by Springer. This software, developed by Cyril Labbé, is designed to automatically detect papers generated by SCIgen. === 2021 report === In 2021, a study was published on 243 SCIgen papers that had been published in the academic literature. They found that SCIgen papers made up 75 per million papers (< 0.01%) in information science, and that only a small fraction of the detected papers had been dealt with.
The MANIAC
The MANIAC is a 2023 novel by Chilean author Benjamín Labatut, written in English. It is a fictionalised biography of polymath John von Neumann, whom Labatut calls "the smartest human being of the 20th century". The book focuses on von Neumann, but is also about physicist Paul Ehrenfest, the history of artificial intelligence, and Lee Sedol's Go match against AlphaGo. The book received mostly positive reviews from critics. == Background == John von Neumann was a Jewish Hungarian-born polymath who was a prodigy from an early childhood. Von Neumann worked in multiple fields of science, theoretical (mathematical foundations of quantum mechanics, game theory, cellular automata) and applied (nuclear weapons research during the Manhattan Project in World War II, computer architecture later named after him, and many other subjects). Labatut calls him "the smartest human being of the 20th century". The title of the book is derived from an early computer based on von Neumann architecture, built after the war at Los Alamos laboratory, called MANIAC I. Benjamín Labatut is a Chilean author known for his 2020 book When We Cease to Understand the World, a collection of fictionalised stories about famous scientists that received positive reviews and was translated into multiple languages from Spanish. The MANIAC is Labatut's first book written in English. In an interview, Labatut said he prefers to write in English: English is my preferred form of thought. ... English is the language I do most if not all my reading it. And it is a far better language than Spanish, in so many ways. Writing "clean" prose in Spanish is almost impossible, because so many of its sounds clash. Borges said that he found English "a far finer language than Spanish" because it's both Germanic and Latin; because of its wonderful vocabulary ("Regal is not exactly the same thing as saying kingly," he explained); because of its physicality; and because you can do almost anything with verbs and prepositions. Labatut was inspired to write The MANIAC by George Dyson's book Turing's Cathedral. == Synopsis == The book has three chapters. The first chapter, "Paul or the Discovery of the Irrational", written in the third person, is about physicist Paul Ehrenfest. The chapter opens with Ehrenfest shooting dead his son Vassily, who suffered from Down syndrome, and then himself. It then recounts Ehrenfest's life story, describing his relationships with his wife Tatyana, his mistress Nelly Meyjes, and his eminent physicist colleagues. It chronicles his descent into despair and depression over his marriage's disintegration, the advent of quantum mechanics, and the direction Europe was heading in with the Nazi Party's rise to power in Germany, looping back to the initial scene of the chapter. The second chapter, "John or the Mad Dreams of Reason", is about John von Neumann, and is written as a series of interviews of his family members, wives, friends, and colleagues, each in a distinctive voice. It is divided into three parts. Part I, "The Limits of Logic", is about his early life, as told by von Neumann's childhood friend Eugene Wigner, mother Margrit Kann, brother Nicholas von Neumann, first wife Mariette Kövesi, and scientists Theodore von Karman, George Polya, and Gábor Szegő. It climaxes with von Neumann's participation in David Hilbert's program to create a logical basis for mathematics based on a consistent set of axioms, a quest ultimately scuppered by Kurt Gödel. Part II, "The Delicate Balance of Terror", discusses von Neumann's role in the Manhattan Project (as told by Richard Feynman); his development of game theory and the doctrine of mutual assured destruction (MAD) (as told by Oskar Morgenstern); and his creation of the MANIAC I computer and the von Neumann architecture (as told by Julian Bigelow). In Part III, "Ghosts in the Machine", Sydney Brenner discusses von Neumann's contributions to biology, his theoretical work on self-replicating and self-repairing machines, and his vision of Von Neumann probes exploring the universe. Nils Aall Barricelli talks about his ideas of digital life and his disagreements with von Neumann. Von Neumann's wife Klára Dán, daughter Marina, and Wigner talk about his final years, personal life, and death. The third chapter, "Lee or The Delusions of Artificial Intelligence", is about Lee Sedol's Go match against AlphaGo. The narrative reverts to the third person. The chapter also tells the story of Demis Hassabis, a chess prodigy in childhood who decided to work on artificial intelligence and founded DeepMind, the company behind AlphaGo. The way is pointed to the future, as artificial intelligence's growing capabilities outpace the human mind. The book ends with Lee Sedol's retirement from Go, and new version of DeepMind's program, AlphaZero, that did not train on human games but nevertheless became the strongest player in Go, chess, and Shogi. == Reception == The book received mostly positive reviews. In his review for The New York Times Tom McCarthy noted the ambiguity of genre: "At its best, as in the stunning opening sequence reconstructing the murder-suicide of the physicist Paul Ehrenfest and his disabled son, or in the final section's gripping account of a computer defeating the world's best human Go player, you just throw up your hands and think, Who cares what discourse label we assign this stuff? It's great." Becca Rothfeld of the Washington Post praised the book, writing that it is "Labatut's latest virtuosic effort, at once a historical novel and a philosophical foray": "The MANIAC is a work of dark, eerie and singular beauty." She noted that the book "can also be difficult to read" because of its unusual narrative structure: "The book is narrated by a cluttered polyphony of characters, among them both of von Neumann's wives and a number of his teachers and colleagues. ... Like von Neumann, The MANIAC strives to adopt the impartial standpoint of the universe." Killian Fox of The Guardian sees the book as "darkly fascinating novel", and notes Labatut's "impressive dexterity, unpicking complex ideas in long, elegant sentences that propel us forward at speed (this is his first book written in English). Even in the more feverish passages, when yet another great mind succumbs to madness, haunted by the spectres they've helped unleash on the world, he feels in full control of his material." Sam Byers of The Guardian praises the book and the author's style: "The opening chapter of Benjamín Labatut's second novel is such a perfect distillation of his technique that it could serve as a manifesto." and "Readers ... will recognise the sense of breathlessness his best writing can evoke. Seemingly loosened from the laws of physics they describe, his sentences range freely through time and space, connecting not only characters and events, but the delicate tissue of intellectual history, often with a lightness of touch that belies their underlying complexity." He writes on the narrative structure: "Through a cascade of staccato chapters, an ensemble of narrators offer their piecemeal insights." Byers adds that "a brilliant novel is not quite what we end up with" and sees the problem in the "diffusion": "Labatut simply spreads himself too thin. Too many years in too few pages; too many voices with far too little to distinguish them. Initially intriguing, the bite-size monologues quickly come to feel inadequate." Some reviewers did not see the book as a biography. In an essay for the Cleveland Review of Books, Ben Cosman juxtaposes the book with Christopher Nolan's biopic Oppenheimer, and writes that it "follows the development of artificial intelligence—first as an idea at the beginning of the twentieth century, and then as a practicality at the beginning of the twenty-first—through the lives of three men who faced it." He also compared the book's structure to "witness testimony". Another reviewer called the book "perfect for anyone thirsting for more nuclear anxiety after watching Oppenheimer". Garrett Biggs of the Chicago Review of Books writes of the book's style: "Labatut writes about scientists the way Roberto Bolaño writes about poets. They are near mythical figures, captured at the corner of the novel's eye. They become historical in the most fraught sense of the term: subject to rumor and speculation and, eventually, the novel's form inflates their personas into something so large they can only be understood as narrative, never known in any objective capacity." Biggs criticises the last chapter: "the story of artificial intelligence has yet to be written. And so when Labatut's narration editorializes about artificial intelligence as 'a future that inspires hope and horror,' The MANIAC disassembles as a novel and starts to sound like a stale thinkpiece. AlphaGo might represent the first glimmer of a true artificial intelligence, as Labatut suggests. It also could one day be considered nothing more than a souped-up cousin to IBM's DeepBlue.
Language Server Protocol
The Language Server Protocol (LSP) is an open, JSON-RPC-based protocol for use between source-code editors or integrated development environments (IDEs) and servers that provide "language intelligence tools": programming language-specific features like code completion, syntax highlighting and marking of warnings and errors, as well as refactoring routines. The goal of the protocol is to allow programming language support to be implemented and distributed independently of any given editor or IDE. In the early 2020s, LSP quickly became a "norm" for language intelligence tools providers. == History == LSP was originally developed for Microsoft Visual Studio Code and is now an open standard. On June 27, 2016, Microsoft announced a collaboration with Red Hat and Codenvy to standardize the protocol's specification. Its specification is hosted and developed on GitHub. == Background == Modern IDEs provide programmers with sophisticated features like code completion, refactoring, navigating to a symbol's definition, syntax highlighting, and error and warning markers. For example, in a text-based programming language, a programmer might want to rename a method read. The programmer could either manually edit the respective source code files and change the appropriate occurrences of the old method name into the new name, or instead use an IDE's refactoring capabilities to make all the necessary changes automatically. To be able to support this style of refactoring, an IDE needs a sophisticated understanding of the programming language that the program's source is written in. A programming tool without such an understanding—for example, one that performs a naive search-and-replace instead—could introduce errors. When renaming a read method, for example, the tool should not replace the partial match in a variable that might be called readyState, nor should it replace the portion of a code comment containing the word "already". Neither should renaming a local variable read, for example, end up altering identically-named variables in other scopes. Conventional compilers or interpreters for a specific programming language are typically unable to provide these language services, because they are written with the goal of either transforming the source code into object code or immediately executing the code. Additionally, language services must be able to handle source code that is not well-formed, e.g. because the programmer is in the middle of editing and has not yet finished typing a statement, procedure, or other construct. Additionally, small changes to a source code file which are done during typing usually change the semantics of the program. In order to provide instant feedback to the user, the editing tool must be able to very quickly evaluate the syntactical and semantical consequences of a specific modification. Compilers and interpreters therefore provide a poor candidate for producing the information needed for an editing tool to consume. Prior to the design and implementation of the Language Server Protocol for the development of Visual Studio Code, most language services were generally tied to a given IDE or other editor. In the absence of the Language Server Protocol, language services are typically implemented by using a tool-specific extension API. Providing the same language service to another editing tool requires effort to adapt the existing code so that the service may target the second editor's extension interfaces. The Language Server Protocol allows for decoupling language services from the editor so that the services may be contained within a general-purpose language server. Any editor can inherit sophisticated support for many different languages by making use of existing language servers. Similarly, a programmer involved with the development of a new programming language can make services for that language available to existing editing tools. Making use of language servers via the Language Server Protocol thus also reduces the burden on vendors of editing tools, because vendors do not need to develop language services of their own for the languages the vendor intends to support, as long as the language servers have already been implemented. The Language Server Protocol also enables the distribution and development of servers contributed by an interested third party, such as end users, without additional involvement by either the vendor of the compiler for the programming language in use or the vendor of the editor to which the language support is being added. LSP is not restricted to programming languages. It can be used for any kind of text-based language, like specifications or domain-specific languages (DSL). == Technical overview == When a user edits one or more source code files using a language server protocol-enabled tool, the tool acts as a client that consumes the language services provided by a language server. The tool may be a text editor or IDE and the language services could be refactoring, code completion, etc. The client informs the server about what the user is doing, e.g., opening a file or inserting a character at a specific text position. The client can also request the server to perform a language service, e.g. to format a specified range in the text document. The server answers a client's request with an appropriate response. For example, the formatting request is answered either by a response that transfers the formatted text to the client or by an error response containing details about the error. The Language Server Protocol defines the messages to be exchanged between client and language server. They are JSON-RPC preceded by headers similar to HTTP. Messages may originate from the server or client. The protocol does not make any provisions about how requests, responses and notifications are transferred between client and server. For example, client and server could be components within the same process exchanging JSON strings via method calls. They could also be different processes on the same or on different machines communicating via network sockets. == Registry == There are lists of LSP-compatible implementations, maintained by the community-driven Langserver.org or Microsoft.
AI Action Summit 2025
The Artificial Intelligence (AI) Action Summit (French: Sommet pour l'action sur l'intelligence artificielle or Sommet pour l'action sur l'IA, SAIA) was held at the Grand Palais in Paris, France, from 10 to 11 February 2025. The summit was co-chaired by French President Emmanuel Macron and Indian Prime Minister Narendra Modi. The 2025 AI Action Summit followed the 2023 AI Safety Summit hosted at Bletchley Park in the UK, and the 2024 AI Seoul Summit in South Korea. This series of AI summits continued with the AI Impact Summit in Delhi, which was hosted by India in February 2026. Whereas the 2023 AI Safety Summit was attended by representatives from 29 governments and executives from only a handful of AI companies, over 1,000 participants from more than 100 countries attended the 2025 Paris AI Summit, representing government leaders, international organisations, the academic and research community, the private sector, and civil society. == Background == The First International AI Safety Report was published on 29 January 2025. Commissioned after the Bletchley Park AI Safety Summit, the report focused on the risks and threats posed by general-purpose AI, and was slated for discussion at the Paris summit as part of the "Trust in AI" pillar. Whereas the first summit was focused on the catastrophic risks of AI and their mitigation, the Paris meeting was recast as an "AI Action Summit" emphasising innovation, practical implementation, and potential economic opportunities of AI, while also exploring a broader range of risks including its environmental impact and disruptions to the labour market. In the weeks leading up to the Paris summit, government leaders had also started to rally around "national champions" in AI, partly in response to Chinese AI startup DeepSeek, which had released a new model rivalling OpenAI o1. On Sunday 9 February, French President Emmanuel Macron posted a compilation of AI-generated deepfake video clips of himself on Instagram to help publicise the start of the 2025 AI Action Summit the following day. While acknowledging the humour of the deepfakes, the real Macron states in the video that using artificial intelligence, "we can do some very big things: change healthcare, energy, life in our society". == Proceedings == === Day 1 === In her opening address, French special envoy Anne Bouverot discussed the environmental impact of AI, acknowledging the technology's "current trajectory is unsustainable". General secretary Christy Hoffman of the UNI Global Union said that "AI-driven productivity gains risk turning the technology into yet another engine of inequality, further straining our democracies". Chinese Vice Premier Zhang Guoqing made a speech expressing China's willingness "to work with other countries to promote development, safeguard security, and share achievements in the field of artificial intelligence". Google CEO Sundar Pichai said in his speech that while the rise of AI brings many risks, "The biggest risk is missing out". He discussed Google's long track record of AI research and said that the company is investing further into "deep research" agents that can autonomously search the Internet and compile a full analysis for users. A new coalition, the Robust Open Online Safety Tools (ROOST) initiative, debuted at the summit. Supported by Google, Discord, OpenAI, and Roblox, and incubated at the Institute of Global Politics at Columbia University, the organisation is developing free, open-source tools to detect and report child sexual abuse material (CSAM). In his speech closing the first day, President Emmanuel Macron emphasized that France has the capability to deliver the power required by AI companies, thanks to its production of nuclear energy. While declaring that Europe was "back in the race" for AI, Macron said that the region was "too slow" for investors, and called on the EU to "simplify regulation" and "resynchronize with the rest of the world". === Day 2 === On 11 February 2025, the French government announced its $400 million endowment of Current AI, a new foundation to support the creation of AI "public goods" including high-quality datasets and open-source tools and infrastructure. Launched by President Macron, Current AI is backed by nine governments – Finland, France, Germany, Chile, India, Kenya, Morocco, Nigeria, Slovenia, and Switzerland – plus various philanthropic organisations such as the Omidyar Group and the McGovern Foundation, and private companies such as Google and Salesforce. Another initiative launched at the summit was the Coalition for Sustainable AI. Led by France, the UN Environment Programme (UNEP), and the International Telecommunication Union (ITU), the coalition has the support of 11 countries, five international organisations, and 37 tech companies including EDF, IBM, Nvidia, and SAP. The Summit of Heads of State and Government took place with a plenary session in the Grand Palais. Prime Minister Narendra Modi of India stressed the need to "democratise technology" and "[ensure] access to all, especially in the Global South". Vice President JD Vance of the United States used his speech to warn against "excessive regulation of the AI" which "could kill a transformative sector just as it's taking off". Vance also warned other leaders against cooperating with "authoritarian regimes" on AI, a comment widely interpreted as a reference to China. == Investments == At the summit, the European Union made several announcements related to planned investments supporting AI development. President Ursula von der Leyen of the European Commission launched InvestAI, a €200 billion initiative, including €20 billion to build four AI gigafactories to train highly complex, very large models. In addition, a coalition of more than 60 European companies launched the EU AI Champions Initiative. Led by venture capital firm General Catalyst, the coalition plans to invest €150 billion in AI-related businesses and infrastructure in Europe over five years. President Emmanuel Macron announced that private investors had pledged to invest nearly €110 billion in the AI sector in France. Financing of between €30 and €50 billion is expected from the United Arab Emirates to build a very large data centre campus, with another €20 billion from the Canadian investment firm Brookfield Corporation. French startup Mistral AI and Helsing, a German-British company, announced their partnership in developing vision-language-action models helping soldiers use AI on the battlefield. == Reactions == The Financial Times editorial board noted that the Paris summit "highlighted a shift in the dynamics towards geopolitical competition", which it characterised as "a new AI arms race" between the US and China, with Europe "trying to carve out its role". Fortune.com AI editor Jeremy Kahn described the 2025 Paris Summit as an "AI festival, complete with glitzy corporate side events and even a late night dance party", contrasting it with the "decidedly sober" mood of the inaugural AI Safety Summit at Bletchley Park. Many experts of the AI Safety Community expressed disappointment that the Paris Summit did not do enough to address AI risks, with Anthropic CEO Dario Amodei calling it a "missed opportunity". Others voicing similar concerns included David Leslie of the Alan Turing Institute and Max Tegmark of the Future of Life Institute. Reporting from Paris, technology columnist Kevin Roose of The New York Times wrote, "The biggest surprise of the Paris summit, for me, has been that policymakers can't seem to grasp how soon powerful AI systems could arrive, or how disruptive they could be." == Statement on inclusive and sustainable AI == At the summit, 58 countries, including France, China, and India, signed a joint declaration, the Statement on Inclusive and Sustainable Artificial Intelligence for People and the Planet. The statement outlines general principles such as accessibility and overcoming the digital divide; developing AI that is open, transparent, ethical, safe, and trustworthy; avoiding market concentration of AI development to encourage innovation; positive outcomes for labour markets; making AI sustainable; and promoting international cooperation and governance. The US and UK refused to sign the declaration on inclusive and sustainable AI. The UK government said in a brief statement that the international agreement did not go far enough in defining global governance of AI and addressing concerns about its impact on national security. === Signatories === The list of signatory countries to the statement for inclusive and sustainable AI in alphabetical order: Additional signatories included the following international bodies and research institutes: ALAI (Latin American Association on Internet) African Union (AU) Commission BEUC The European Consumer Organisation Center for Democracy and Technology Council of Europe European Commission (and the 27 member states) Hugging Face INRIA Institute of Advanced Study OEC