David Krueger is an American machine learning professor and advocate for the reduction of risks related to artificial intelligence. Krueger is an assistant professor in Robust, Reasoning, and Responsible AI at the University of Montreal and a Core Academic Member at Mila. == Early life and education == Krueger obtained a B.A. in mathematics from Reed College, and completed his MSc and Ph.D. in Computer Science at the University of Montreal. He trained in deep learning under Yoshua Bengio, Roland Memisevic, and Aaron Courville from 2013 to 2021. Krueger was also an intern on Google DeepMind's AI Safety team in 2018. == Career == Krueger researches deep learning, AI alignment, and AI safety. His work is focused on reducing the risk of human extinction resulting from out-of-control AI systems. Krueger was an assistant professor at the University of Cambridge from 2021 to 2024, before taking a faculty position at the University of Montreal in 2024. In 2023, he was a founding research director at the UK AI Security Institute. That same year, Krueger initiated the Statement on AI Risk, which argues that AI could cause human extinction and was signed by Anthropic's Dario Amodei, OpenAI's Sam Altman, AI expert Geoffrey Hinton, and other leaders. In April 2026, Krueger discussed the risks of advanced AI at a Capitol Hill event hosted by Senator Bernie Sanders. === Evitable === In 2025, Krueger founded Evitable, a nonprofit organization that advocates for an AI moratorium. == Views == Krueger argues that AI will lead to a "gradual disempowerment" of workers, likening AI chips to nuclear bombs. He also says the military use of AI "poses an existential risk to humanity."
Open-source software security
Open-source software security is the measure of assurance or guarantee in the freedom from danger and risk inherent to an open-source software system. == Implementation debate == === Benefits === Proprietary software forces the user to accept the level of security that the software vendor is willing to deliver and to accept the rate that patches and updates are released. It is assumed that any compiler that is used creates code that can be trusted, but it has been demonstrated by Ken Thompson that a compiler can be subverted using a compiler backdoor to create faulty executables that are unwittingly produced by a well-intentioned developer. With access to the source code for the compiler, the developer has at least the ability to discover if there is any mal-intention. Kerckhoffs' principle is based on the idea that an enemy can steal a secure military system and not be able to compromise the information. His ideas were the basis for many modern security practices, and followed that security through obscurity is a bad practice. === Drawbacks === Simply making source code available does not guarantee review. An example of this occurring is when Marcus Ranum, an expert on security system design and implementation, released his first public firewall toolkit. At one time, there were over 2,000 sites using his toolkit, but only 10 people gave him any feedback or patches. Having a large amount of eyes reviewing code can "lull a user into a false sense of security". Having many users look at source code does not guarantee that security flaws will be found and fixed. == Metrics and models == There are a variety of models and metrics to measure the security of a system. These are a few methods that can be used to measure the security of software systems. === Number of days between vulnerabilities === It is argued that a system is most vulnerable after a potential vulnerability is discovered, but before a patch is created. By measuring the number of days between the vulnerability and when the vulnerability is fixed, a basis can be determined on the security of the system. There are a few caveats to such an approach: not every vulnerability is equally bad, and fixing a lot of bugs quickly might not be better than only finding a few and taking a little bit longer to fix them, taking into account the operating system, or the effectiveness of the fix. === Poisson process === The Poisson process can be used to measure the rates at which different people find security flaws between open and closed source software. The process can be broken down by the number of volunteers Nv and paid reviewers Np. The rates at which volunteers find a flaw is measured by λv and the rate that paid reviewers find a flaw is measured by λp. The expected time that a volunteer group is expected to find a flaw is 1/(Nv λv) and the expected time that a paid group is expected to find a flaw is 1/(Np λp). === Morningstar model === By comparing a large variety of open source and closed source projects a star system could be used to analyze the security of the project similar to how Morningstar, Inc. rates mutual funds. With a large enough data set, statistics could be used to measure the overall effectiveness of one group over the other. An example of such as system is as follows: 1 Star: Many security vulnerabilities. 2 Stars: Reliability issues. 3 Stars: Follows best security practices. 4 Stars: Documented secure development process. 5 Stars: Passed independent security review. === Coverity scan === Coverity in collaboration with Stanford University has established a new baseline for open-source quality and security. The development is being completed through a contract with the Department of Homeland Security. They are utilizing innovations in automated defect detection to identify critical types of bugs found in software. The level of quality and security is measured in rungs. Rungs do not have a definitive meaning, and can change as Coverity releases new tools. Rungs are based on the progress of fixing issues found by the Coverity Analysis results and the degree of collaboration with Coverity. They start with Rung 0 and currently go up to Rung 2. Rung 0 The project has been analyzed by Coverity's Scan infrastructure, but no representatives from the open-source software have come forward for the results. Rung 1 At rung 1, there is collaboration between Coverity and the development team. The software is analyzed with a subset of the scanning features to prevent the development team from being overwhelmed. Rung 2 There are 11 projects that have been analyzed and upgraded to the status of Rung 2 by reaching zero defects in the first year of the scan. These projects include: AMANDA, ntp, OpenPAM, OpenVPN, Overdose, Perl, PHP, Postfix, Python, Samba, and Tcl.
Computational law
Computational law is the branch of legal informatics concerned with the automation of legal reasoning. What distinguishes Computational Law systems from other instances of legal technology is their autonomy, i.e. the ability to answer legal questions without additional input from human legal experts. While there are many possible applications of Computational Law, the primary focus of work in the field today is compliance management, i.e. the development and deployment of computer systems capable of assessing, facilitating, or enforcing compliance with rules and regulations. Some systems of this sort already exist. TurboTax is a good example. And the potential is particularly significant now due to recent technological advances – including the prevalence of the Internet in human interaction and the proliferation of embedded computer systems (such as smart phones, self-driving cars, and robots). There are also applications that do not involve governmental laws. The regulations can just as well be the terms of contracts (e.g. delivery schedules, insurance covenants, real estate transactions, financial agreements). They can be the policies of corporations (e.g. constraints on travel, expenditure reporting, pricing rules). They can even be the rules of games (embodied in computer game playing systems). == History == Speculation about potential benefits to legal practice through applying methods from computational science and AI research to automate parts of the law date back at least to the middle 1940s. Further, AI and law and computational law do not seem easily separable, as perhaps most of AI research focusing on the law and its automation appears to utilize computational methods. The forms that speculation took are multiple and not all related in ways to readily show closeness to one another. This history will sketch them as they were, attempting to show relationships where they can be found to have existed. By 1949, a minor academic field aiming to incorporate electronic and computational methods to legal problems had been founded by American legal scholars, called jurimetrics. Though broadly said to be concerned with the application of the "methods of science" to the law, these methods were actually of a quite specifically defined scope. Jurimetrics was to be "concerned with such matters as the quantitative analysis of judicial behavior, the application of communication and information theory to legal expression, the use of mathematical logic in law, the retrieval of legal data by electronic and mechanical means, and the formulation of a calculus of legal predictability". These interests led in 1959 to the founding a journal, Modern Uses of Logic in Law, as a forum wherein articles would be published about the applications of techniques such as mathematical logic, engineering, statistics, etc. to the legal study and development. In 1966, this Journal was renamed as Jurimetrics. Today, however, the journal and meaning of jurimetrics seems to have broadened far beyond what would fit under the areas of applications of computers and computational methods to law. Today the journal not only publishes articles on such practices as found in computational law, but has broadened jurimetrical concerns to mean also things like the use of social science in law or the "policy implications [of] and legislative and administrative control of science". Independently in 1958, at the Conference for the Mechanization of Thought held at the National Physical Laboratory in Teddington, Middlesex, UK, the French jurist Lucien Mehl presented a paper both on the benefits of using computational methods for law and on the potential means to use such methods to automate law for a discussion that included AI luminaries like Marvin Minsky. Mehl believed that the law could by automated by two basic distinct, though not wholly separable, types of machine. These were the "documentary or information machine", which would provide the legal researcher quick access to relevant case precedents and legal scholarship, and the "consultation machine", which would be "capable of answering any question put to it over a vast field of law". The latter type of machine would be able to basically do much of a lawyer's job by simply giving the "exact answer to a [legal] problem put to it". By 1970, Mehl's first type of machine, one that would be able to retrieve information, had been accomplished but there seems to have been little consideration of further fruitful intersections between AI and legal research. There were, however, still hopes that computers could model the lawyer's thought processes through computational methods and then apply that capacity to solve legal problems, thus automating and improving legal services via increased efficiency as well as shedding light on the nature of legal reasoning. By the late 1970s, computer science and the affordability of computer technology had progressed enough that the retrieval of "legal data by electronic and mechanical means" had been achieved by machines fitting Mehl's first type and were in common use in American law firms. During this time, research focused on improving the goals of the early 1970s occurred, with programs like Taxman being worked on in order to both bring useful computer technology into the law as practical aids and to help specify the exact nature of legal concepts. Nonetheless, progress on the second type of machine, one that would more fully automate the law, remained relatively inert. Research into machines that could answer questions in the way that Mehl's consultation machine would picked up somewhat in the late 1970s and 1980s. A 1979 convention in Swansea, Wales marked the first international effort solely to focus upon applying artificial intelligence research to legal problems in order to "consider how computers can be used to discover and apply the legal norms embedded within the written sources of the law". Considerable progress on the development of the second type of machine was made in the following decade, with the development of a variety of expert systems. According to Thorne McCarty, "these systems all have the following characteristics: They do backward chaining inference from a specified goal; they ask questions to elicit information from the user; and they produce a suggested answer along with a trace of the supporting legal rules." According to Prakken and Sartor the representation of the British Nationality Act as a logic program, which introduced this approach, was "hugely influential for the development of computational representations of legislation, showing how logic programming enables intuitively appealing representations that can be directly deployed to generate automatic inferences". In 2021, this work received the Inaugural CodeX Prize as "one of the first and best-known works in computational law, and one of the most widely cited papers in the field." In a 1988 review of Anne Gardner's book An Artificial Intelligence Approach to Legal Reasoning (1987), the Harvard academic legal scholar and computer scientist Edwina Rissland wrote that "She plays, in part, the role of pioneer; artificial intelligence ("AI") techniques have not yet been widely applied to perform legal tasks. Therefore, Gardner, and this review, first describe and define the field, then demonstrate a working model in the domain of contract offer and acceptance." Eight years after the Swansea conference had passed, and still AI and law researchers merely trying to delineate the field could be described by their own kind as "pioneer[s]". In the 1990s and early 2000s more progress occurred. Computational research generated insights for law. The First International Conference on AI and the Law occurred in 1987, but it is in the 1990s and 2000s that the biannual conference began to build up steam and to delve more deeply into the issues involved with work intersecting computational methods, AI, and law. Classes began to be taught to undergraduates on the uses of computational methods to automating, understanding, and obeying the law. Further, by 2005, a team largely composed of Stanford computer scientists from the Stanford Logic group had devoted themselves to studying the uses of computational techniques to the law. Computational methods in fact advanced enough that members of the legal profession began in the 2000s to both analyze, predict and worry about the potential future of computational law and a new academic field of computational legal studies seems to be now well established. As insight into what such scholars see in the law's future due in part to computational law, here is quote from a recent conference about the "New Normal" for the legal profession: "Over the last 5 years, in the fallout of the Great Recession, the legal profession has entered the era of the New Normal. Notably, a series of forces related to technological change, globalization, and the pressure to do more with less (in both corpo
Algorithmic accountability
Algorithmic accountability refers to the allocation of responsibility for the consequences of real-world actions influenced by algorithms used in decision-making processes. Ideally, algorithms should be designed to eliminate bias from their decision-making outcomes. This means they ought to evaluate only relevant characteristics of the input data, avoiding distinctions based on attributes that are generally inappropriate in social contexts, such as an individual's ethnicity in legal judgments. However, adherence to this principle is not always guaranteed, and there are instances where individuals may be adversely affected by algorithmic decisions. Responsibility for any harm resulting from a machine's decision may lie with the algorithm itself or with the individuals who designed it, particularly if the decision resulted from bias or flawed data analysis inherent in the algorithm's design. == Algorithm usage == Algorithms are widely utilized across various sectors of society that incorporate computational techniques in their control systems. These applications span numerous industries, including but not limited to medical, transportation, and payment services. In these contexts, algorithms perform functions such as: Approving or denying credit card applications; Approving or denying immigrant visas; Determining which taxpayers will be audited on their income taxes; Managing systems that control self-driving cars on a highway; Scoring individuals as potential criminals for use in legal proceedings; Search engines that match and rank database and internet search results; Recommendation systems that filter which news, entertainment, or purchase items are featured in a feed; Market-making algorithms that match sellers and buyers, such as in transportation (ride-hailing) or financial platforms. However, the implementation of these algorithms can be complex and opaque. Generally, algorithms function as "black boxes," meaning that the specific processes an input undergoes during execution are often not transparent, with users typically only seeing the resulting output. This lack of transparency raises concerns about potential biases within the algorithms, as the parameters influencing decision-making may not be well understood. The outputs generated can lead to perceptions of bias, especially if individuals in similar circumstances receive different results. According to Nicholas Diakopoulos: But these algorithms can make mistakes. They have biases. Yet they sit in opaque black boxes, their inner workings, their inner “thoughts” hidden behind layers of complexity. We need to get inside that black box, to understand how they may be exerting power on us, and to understand where they might be making unjust mistakes == Wisconsin Supreme Court case == Algorithms are prevalent across various fields and significantly influence decisions that affect the population at large. Their underlying structures and parameters often remain unknown to those impacted by their outcomes. A notable case illustrating this issue is a recent ruling by the Wisconsin Supreme Court concerning "risk assessment" algorithms used in criminal justice. The court determined that scores generated by such algorithms, which analyze multiple parameters from individuals, should not be used as a determining factor for arresting an accused individual. Furthermore, the court mandated that all reports submitted to judges must include information regarding the accuracy of the algorithm used to compute these scores. This ruling is regarded as a noteworthy development in how society should manage software that makes consequential decisions, highlighting the importance of reliability, particularly in complex settings like the legal system. The use of algorithms in these contexts necessitates a high degree of impartiality in processing input data. However, experts note that there is still considerable work to be done to ensure the accuracy of algorithmic results. Questions about the transparency of data processing continue to arise, which raises issues regarding the appropriateness of the algorithms and the intentions of their designers. == Controversies == A notable instance of potential algorithmic bias is highlighted in an article by The Washington Post regarding the ride-hailing service Uber. An analysis of collected data revealed that estimated waiting times for users varied based on the neighborhoods in which they resided. Key factors influencing these discrepancies included the predominant ethnicity and average income of the area. Specifically, neighborhoods with a majority white population and higher economic status tended to have shorter waiting times, while those with more diverse ethnic compositions and lower average incomes experienced longer waits. It’s important to clarify that this observation reflects a correlation identified in the data, rather than a definitive cause-and-effect relationship. No value judgments are made regarding the behavior of the Uber app in these cases. In TechCrunch website, Hemant Taneja wrote: Concern about “black box” algorithms that govern our lives has been spreading. New York University’s Information Law Institute hosted a conference on algorithmic accountability, noting: “Scholars, stakeholders, and policymakers question the adequacy of existing mechanisms governing algorithmic decision-making and grapple with new challenges presented by the rise of algorithmic power in terms of transparency, fairness, and equal treatment.” Yale Law School’s Information Society Project is studying this, too. “Algorithmic modeling may be biased or limited, and the uses of algorithms are still opaque in many critical sectors,” the group concluded. == Possible solutions == Discussions among experts have sought viable solutions to understand the operations of algorithms, often referred to as "black boxes." It is generally proposed that companies responsible for developing and implementing these algorithms should ensure their reliability by disclosing the internal processes of their systems. Hemant Taneja, writing for TechCrunch, emphasizes that major technology companies, such as Google, Amazon, and Uber, must actively incorporate algorithmic accountability into their operations. He suggests that these companies should transparently monitor their own systems to avoid stringent regulatory measures. One potential approach is the introduction of regulations in the tech sector to enforce oversight of algorithmic processes. However, such regulations could significantly impact software developers and the industry as a whole. It may be more beneficial for companies to voluntarily disclose the details of their algorithms and decision-making parameters, which could enhance the trustworthiness of their solutions. Another avenue discussed is the possibility of self-regulation by the companies that create these algorithms, allowing them to take proactive steps in ensuring accountability and transparency in their operations. In TechCrunch website, Hemant Taneja wrote: There’s another benefit — perhaps a huge one — to software-defined regulation. It will also show us a path to a more efficient government. The world’s legal logic and regulations can be coded into software and smart sensors can offer real-time monitoring of everything from air and water quality, traffic flows and queues at the DMV. Regulators define the rules, technologist create the software to implement them and then AI and ML help refine iterations of policies going forward. This should lead to much more efficient, effective governments at the local, national and global levels.
Mark I Perceptron
The Mark I Perceptron was a pioneering supervised image classification learning system developed by Frank Rosenblatt in 1958. It was the first implementation of an artificial intelligence (AI) machine. It differs from the Perceptron which is a software architecture proposed in 1943 by Warren McCulloch and Walter Pitts, which was also employed in Mark I, and enhancements of which have continued to be an integral part of cutting edge AI technologies like the Transformer. == Architecture == The Mark I Perceptron was organized into three layers: A set of sensory units which receive optical input A set of association units, each of which fire based on input from multiple sensory units A set of response units, which fire based on input from multiple association units The connection between sensory units and association units were random. The working of association units was very similar to the response units. Different versions of the Mark I used different numbers of units in each of the layers. == Capabilities == In his 1957 proposal for funding for development of the "Cornell Photoperceptron", Rosenblatt claimed:"Devices of this sort are expected ultimately to be capable of concept formation, language translation, collation of military intelligence, and the solution of problems through inductive logic."With the first version of the Mark I Perceptron as early as 1958, Rosenblatt demonstrated a simple binary classification experiment, namely distinguishing between sheets of paper marked on the right versus those marked on the left side. One of the later experiments distinguished a square from a circle printed on paper. The shapes were perfect and their sizes fixed; the only variation was in their position and orientation. The Mark I Perceptron achieved 99.8% accuracy on a test dataset with 500 neurons in a single layer. The size of the training dataset was 10,000 example images. It took 3 seconds for the training pipeline to go through a single image. Higher accuracy was observed with thick outline figures compared to solid figures, likely because outline figures reduced overfitting. Another experiment distinguished between a square and a diamond for which 100% accuracy was achieved with only 60 training images, with a Perceptron having 1,000 neurons in a single layer. The time taken to process each training input for this larger perceptron was 15 seconds. The only variation was in position of the image, since rotation would have been ambiguous. In that same experiment, it could distinguish between the letters X and E with 100% accuracy when trained with only 20 images (10 images of each letter). Variations in the images included both position and rotation by up to 30 degrees. When variation in rotation was increased to any angle (both in training and test datasets), the accuracy reduced to 90% with 60 training images (30 images of each letter). For distinguishing between the letters E and F, a more challenging problem due to their similarity, the same 1,000 neuron perceptron achieved an accuracy of more than 80% with 60 training images. Variation was only in the position of the image, with no rotation.
Frankenstein complex
The Frankenstein complex is a term coined by Isaac Asimov in his robot series, referring to the fear of mechanical men. == History == Some of Asimov's science fiction short stories and novels predict that this suspicion will become strongest and most widespread in respect of "mechanical men" that most-closely resemble human beings (see android), but it is also present on a lower level against robots that are plainly electromechanical automatons. The "Frankenstein complex" is similar in many respects to Masahiro Mori's uncanny valley hypothesis. The name, "Frankenstein complex", is derived from the name of Victor Frankenstein in the 1818 novel Frankenstein; or, The Modern Prometheus by Mary Shelley. In Shelley's story, Frankenstein created an intelligent, somewhat superhuman being, but he finds that his creation is horrifying to behold and abandons it. This ultimately leads to Victor's death at the conclusion of a vendetta between himself and his creation. In much of his fiction, Asimov depicts the general attitude of the public towards robots as negative, with ordinary people fearing that robots will either replace them or dominate them, although dominance would not be allowed under the specifications of the Three Laws of Robotics, the first of which is: "A robot may not harm a human being or, through inaction, allow a human being to come to harm." However, Asimov's fictitious earthly public is not fully persuaded by this, and remains largely suspicious and fearful of robots. I, Robot's short story "Little Lost Robot" is about this "fear of robots". In Asimov's robot novels, the Frankenstein complex is a major problem for roboticists and robot manufacturers. They do all they can to reassure the public that robots are harmless, even though this sometimes involves hiding the truth because they think that the public would misunderstand it. The fear by the public and the response of the manufacturers is an example of the theme of paternalism, the dread of paternalism, and the conflicts that arise from it in Asimov's fiction. The same theme occurs in many later works of fiction featuring robots, although it is rarely referred to as such.
Eclipse Phase
Eclipse Phase is a science fiction horror role-playing game with transhumanist themes. It was originally published by Catalyst Game Labs, and is now published by the game's creators, Posthuman Studios, and is released under a Creative Commons license. == Setting == Eclipse Phase is a science fiction horror role-playing game with transhumanist, post-apocalyptic, and conspiracy themes. The game is set after a World War III project to create artificial intelligence known as TITANs has gone rogue, resulting in the deaths of over 90% of the inhabitants of Earth. Earth is subsequently abandoned, and existing colonies throughout the Solar System are expanded to accommodate the refugees. The setting explores a spectrum of socioeconomic systems in each of these colonies: A capitalist / republican system exists in the Inner System (Mars, the Moon, and Mercury), under the Planetary Consortium, a corporate body which allows the election of representatives but whose shareholders are nominally most powerful. An Extropian/Propertarian system is established in the Asteroid Belt. The Extropians are split into two subfactions, an anarcho-capitalist group, more closely related to the Hypercapitalists, and a mutualist group, related closely to the Anarchists. A military oligarchy rules the moons around Jupiter. An alliance of Scandinavia-style social democracy and Collectivist anarchism are dominant in the Outer System. From there, the setting explores various scientific advances, extrapolated far into the future. Nanotechnology, terraforming, Zero-G living, upgrading animal sapience, and reputation systems are all used as plot points and background. With all of this, the game encourages players to confront existential threats like aliens, weapons of mass destruction, Exsurgent Virus outbreaks, and political unrest. == Mechanics == Eclipse Phase uses a simple roll-under percentile die system for task resolution. Unlike most percentile systems, a roll of 00 does not count as a 100. In addition, any roll of a double (11, 22, 33 etc.) is a critical. If the double is under the target number it is a critical success, while being over the target number constitutes a critical failure. For damage resolution (whether physical damage caused by injury or mental stress caused by traumatic events), players roll a designated number of ten-sided dice and add the values together, along with any modifiers. == Books == === Publications === Eclipse Phase (Core Rulebook) (2009) ISBN 978-0-9845835-0-8 GM Screen (2010) Sunward, Boyle, Rob; Knevitt, James (2010). Sunward : the inner system, a location sourcebook for Eclipse Phase. UK: Cubicle 7. ISBN 978-0984583522. Gatecrashing Boyle, Rob; Graham, Jack; Rosenberg, Aaron (2011). Gatecrashing. UK: Cubicle 7. ISBN 978-0984583539. Panopticon Volume 1: Habitats, Surveillance, Uplifts (2011) (2011) Rimward (2012) Transhuman: The Eclipse Phase Player’s Guide (2013) Firewall (2015) X-Risks (2016) Eclipse Phase (Core Rulebook, Second Edition) (2019) === Nano Ops === Nano Op: Grinder Nano Op: All That Glitters Nano Op: Better on the Inside Nano Op: Binge Nano Op: Body Count == Creative Commons License == The Eclipse Phase roleplaying game was released under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 license, and newer printings have updated to the Creative Commons Attribution-Noncommercial-Share Alike 4.0 license; the text found on the Eclipse Phase website is licensed under the Creative Commons Attribution-Noncommercial-Share Alike 4.0 License. As stated on their website, the publishers encourage players and gamemasters to recreate, alter, and "remix" the material for non-commercial purposes as long as Posthuman Studios is attributed, and any derivatives are licensed under the same Creative Commons Attribution-Noncommercial-Share Alike 4.0 License. Further, copying and sharing the game's electronic versions non-commercially is legal. == Reception == In 2010, it won the 36th Annual Origins award for Best Roleplaying Game of 2009. It also won three 2010 ENnie awards: Gold for Best Writing, Silver for Best Cover Art, and Silver for Product of the Year.