Concurrency control

Concurrency control

In information technology and computer science, especially in the fields of computer programming, operating systems, multiprocessors, and databases, concurrency control ensures that correct results for concurrent operations are generated, while getting those results as quickly as possible. Computer systems, both software and hardware, consist of modules, or components. Each component is designed to operate correctly, i.e., to obey or to meet certain consistency rules. When components that operate concurrently interact by messaging or by sharing accessed data (in memory or storage), a certain component's consistency may be violated by another component. The general area of concurrency control provides rules, methods, design methodologies, and theories to maintain the consistency of components operating concurrently while interacting, and thus the consistency and correctness of the whole system. Introducing concurrency control into a system means applying operation constraints which typically result in some performance reduction. Operation consistency and correctness should be achieved with as good as possible efficiency, without reducing performance below reasonable levels. Concurrency control can require significant additional complexity and overhead in a concurrent algorithm compared to the simpler sequential algorithm. For example, a failure in concurrency control can result in data corruption from torn read or write operations. == Concurrency control in databases == Comments: This section is applicable to all transactional systems, i.e., to all systems that use database transactions (atomic transactions; e.g., transactional objects in Systems management and in networks of smartphones which typically implement private, dedicated database systems), not only general-purpose database management systems (DBMSs). DBMSs need to deal also with concurrency control issues not typical just to database transactions but rather to operating systems in general. These issues (e.g., see Concurrency control in operating systems below) are out of the scope of this section. Concurrency control in Database management systems (DBMS; e.g., Bernstein et al. 1987, Weikum and Vossen 2001), other transactional objects, and related distributed applications (e.g., Grid computing and Cloud computing) ensures that database transactions are performed concurrently without violating the data integrity of the respective databases. Thus concurrency control is an essential element for correctness in any system where two database transactions or more, executed with time overlap, can access the same data, e.g., virtually in any general-purpose database system. Consequently, a vast body of related research has been accumulated since database systems emerged in the early 1970s. A well established concurrency control theory for database systems is outlined in the references mentioned above: serializability theory, which allows to effectively design and analyze concurrency control methods and mechanisms. An alternative theory for concurrency control of atomic transactions over abstract data types is presented in (Lynch et al. 1993), and not utilized below. This theory is more refined, complex, with a wider scope, and has been less utilized in the Database literature than the classical theory above. Each theory has its pros and cons, emphasis and insight. To some extent they are complementary, and their merging may be useful. To ensure correctness, a DBMS usually guarantees that only serializable transaction schedules are generated, unless serializability is intentionally relaxed to increase performance, but only in cases where application correctness is not harmed. For maintaining correctness in cases of failed (aborted) transactions (which can always happen for many reasons) schedules also need to have the recoverability (from abort) property. A DBMS also guarantees that no effect of committed transactions is lost, and no effect of aborted (rolled back) transactions remains in the related database. Overall transaction characterization is usually summarized by the ACID rules below. As databases have become distributed, or needed to cooperate in distributed environments (e.g., Federated databases in the early 1990, and Cloud computing currently), the effective distribution of concurrency control mechanisms has received special attention. === Database transaction and the ACID rules === The concept of a database transaction (or atomic transaction) has evolved in order to enable both a well understood database system behavior in a faulty environment where crashes can happen any time, and recovery from a crash to a well understood database state. A database transaction is a unit of work, typically encapsulating a number of operations over a database (e.g., reading a database object, writing, acquiring lock, etc.), an abstraction supported in database and also other systems. Each transaction has well defined boundaries in terms of which program/code executions are included in that transaction (determined by the transaction's programmer via special transaction commands). Every database transaction obeys the following rules (by support in the database system; i.e., a database system is designed to guarantee them for the transactions it runs): Atomicity - Either the effects of all or none of its operations remain ("all or nothing" semantics) when a transaction is completed (committed or aborted respectively). In other words, to the outside world a committed transaction appears (by its effects on the database) to be indivisible (atomic), and an aborted transaction does not affect the database at all. Either all the operations are done or none of them are. Consistency - Every transaction must leave the database in a consistent (correct) state, i.e., maintain the predetermined integrity rules of the database (constraints upon and among the database's objects). A transaction must transform a database from one consistent state to another consistent state (however, it is the responsibility of the transaction's programmer to make sure that the transaction itself is correct, i.e., performs correctly what it intends to perform (from the application's point of view) while the predefined integrity rules are enforced by the DBMS). Thus since a database can be normally changed only by transactions, all the database's states are consistent. Isolation - Transactions cannot interfere with each other (as an end result of their executions). Moreover, usually (depending on concurrency control method) the effects of an incomplete transaction are not even visible to another transaction. Providing isolation is the main goal of concurrency control. Durability - Effects of successful (committed) transactions must persist through crashes (typically by recording the transaction's effects and its commit event in a non-volatile memory). The concept of atomic transaction has been extended during the years to what has become Business transactions which actually implement types of Workflow and are not atomic. However also such enhanced transactions typically utilize atomic transactions as components. === Why is concurrency control needed? === If transactions are executed serially, i.e., sequentially with no overlap in time, no transaction concurrency exists. However, if concurrent transactions with interleaving operations are allowed in an uncontrolled manner, some unexpected, undesirable results may occur, such as: The lost update problem: A second transaction writes a second value of a data-item (datum) on top of a first value written by a first concurrent transaction, and the first value is lost to other transactions running concurrently which need, by their precedence, to read the first value. The transactions that have read the wrong value end with incorrect results. The dirty read problem: Transactions read a value written by a transaction that has been later aborted. This value disappears from the database upon abort, and should not have been read by any transaction ("dirty read"). The reading transactions end with incorrect results. The incorrect summary problem: While one transaction takes a summary over the values of all the instances of a repeated data-item, a second transaction updates some instances of that data-item. The resulting summary does not reflect a correct result for any (usually needed for correctness) precedence order between the two transactions (if one is executed before the other), but rather some random result, depending on the timing of the updates, and whether certain update results have been included in the summary or not. Most high-performance transactional systems need to run transactions concurrently to meet their performance requirements. Thus, without concurrency control such systems can neither provide correct results nor maintain their databases consistently. === Concurrency control mechanisms === ==== Categories ==== The main categories of concurrency control mechanis

Two-phase locking

In databases and transaction processing, two-phase locking (2PL) is a pessimistic concurrency control method that guarantees conflict-serializability. It is also the name of the resulting set of database transaction schedules (histories). The protocol uses locks, applied by a transaction to data, which may block (interpreted as signals to stop) other transactions from accessing the same data during the transaction's life. By the 2PL protocol, locks are applied and removed in two phases: Expanding phase: locks are acquired and no locks are released. Shrinking phase: locks are released and no locks are acquired. Two types of locks are used by the basic protocol: Shared and Exclusive locks. Refinements of the basic protocol may use more lock types. Using locks that block processes, 2PL, S2PL, and SS2PL may be subject to deadlocks that result from the mutual blocking of two or more transactions. == Read and write locks == Locks are used to guarantee serializability. A transaction is holding a lock on an object if that transaction has acquired a lock on that object which has not yet been released. For 2PL, the only used data-access locks are read-locks (shared locks) and write-locks (exclusive locks). Below are the rules for read-locks and write-locks: A transaction is allowed to read an object if and only if it is holding a read-lock or write-lock on that object. A transaction is allowed to write an object if and only if it is holding a write-lock on that object. A schedule (i.e., a set of transactions) is allowed to hold multiple locks on the same object simultaneously if and only if none of those locks are write-locks. If a disallowed lock attempts on being held simultaneously, it will be blocked. == Variants == Note that all conflict serializable schedules are also view serializable (but not vice-versa). === Two-phase locking === According to the two-phase locking protocol, each transaction handles its locks in two distinct, consecutive phases during the transaction's execution: Expanding phase (aka Growing phase): locks are acquired and no locks are released (the number of locks can only increase). Shrinking phase (aka Contracting phase): locks are released and no locks are acquired. The two phase locking rules can be summarized as: each transaction must never acquire a lock after it has released a lock. The serializability property is guaranteed for a schedule with transactions that obey this rule. Typically, without explicit knowledge in a transaction on end of phase 1, the rule is safely determined only when a transaction has completed processing and requested commit. In this case, all the locks can be released at once (phase 2). === Conservative two-phase locking === Conservative two-phase locking (C2PL) differs from 2PL in that transactions obtain all the locks they need before the actual execution begins. This is to ensure that a transaction that already holds some locks will not block waiting for other locks. C2PL prevents deadlocks. In cases of heavy lock contention, C2PL reduces the time locks are held on average, relative to 2PL and Strict 2PL, because transactions that hold locks are never blocked. In light lock contention, C2PL holds more locks than is necessary, because it is difficult to predict which locks will be needed in the future, thus leading to higher overhead. A C2PL transaction will not obtain any locks if it cannot obtain all the locks it needs in its initial request. Furthermore, each transaction needs to declare its read and write set (the data items that will be read/written), which is not always possible. Because of these limitations, C2PL is not used very frequently. === Strict two-phase locking === To comply with the strict two-phase locking (S2PL) protocol, a transaction needs to comply with 2PL, and release its write (exclusive) locks only after the transaction has ended (i.e., either committed or aborted). On the other hand, read (shared) locks are released regularly during the shrinking phase. Unlike 2PL, S2PL provides strictness (a special case of cascade-less recoverability). This protocol is not appropriate in B-trees because it causes Bottleneck (while B-trees always starts searching from the parent root). === Strong strict two-phase locking === or Rigorousness, or Rigorous scheduling, or Rigorous two-phase locking To comply with strong strict two-phase locking (SS2PL), a transaction's read and write locks are released only after that transaction has ended (i.e., either committed or aborted). A transaction obeying SS2PL has only a phase 1 and lacks a phase 2 until the transaction has completed. Every SS2PL schedule is also an S2PL schedule, but not vice versa.

Mustafa Suleyman

Mustafa Suleyman (born in August 1984) is a British artificial intelligence (AI) entrepreneur. He is the CEO of Microsoft AI, and the co-founder and former head of applied AI at DeepMind, an AI company which was acquired by Google. After leaving DeepMind, he co-founded Inflection AI, a machine learning and generative AI company, in 2022. == Early life and education == Suleyman's Syrian father worked as a taxi driver and his English mother was a nurse. He grew up off Caledonian Road, London, where he lived with his parents and his two younger brothers. Suleyman went to Thornhill Primary School, a state school in Islington, followed by Queen Elizabeth's School, Barnet, a boys' grammar school. Around that time, he met his DeepMind co-founder, Demis Hassabis, through his best friend, who was Demis's younger brother. Suleyman shared that he and Hassabis often discussed how they could make a positive impact on the world. Suleyman enrolled to study philosophy and theology at the University of Oxford where he was an undergraduate student at Mansfield College, Oxford, before dropping out at 19. == Career == In August 2001, while still a teenager and a "strong atheist", Suleyman helped Mohammed Mamdani establish a telephone counselling service called the Muslim Youth Helpline. The organization would later become one of the largest mental health support services. Suleyman subsequently worked as a policy officer on human rights for Ken Livingstone, the Mayor of London, before going on to start Reos Partners, a "systemic change" consultancy that uses methods from conflict resolution to navigate social problems. As a negotiator and facilitator, Mustafa worked for a wide range of clients such as the United Nations, the Dutch government, and the World Wide Fund for Nature. === DeepMind and Google === In 2010 Suleyman co-founded DeepMind Technologies, an artificial intelligence (AI) and machine learning company, and became its chief product officer. The company quickly established itself as one of the leaders in the AI sector. In 2014 DeepMind was acquired by Google for a reported £400 million, the company's largest acquisition in Europe at that time. Following the acquisition, Suleyman became head of applied AI at DeepMind, taking on responsibility for integrating the company's technology across a wide range of Google products. In February 2016 Suleyman launched DeepMind Health at the Royal Society of Medicine. DeepMind Health builds clinician-led technology for the National Health Service (NHS) and other partners to improve frontline healthcare services. Under Suleyman, DeepMind also developed research collaborations with healthcare organizations in the United Kingdom, including Moorfields Eye Hospital NHS foundation trust. In 2016, Suleyman led an effort to apply DeepMind's machine learning algorithms to help reduce the energy required to cool Google's data centres. The system evaluated the billions of possible combinations of actions that the data centre operators could take, and came up with recommendations based on the predicted power usage. The system discovered novel methods of cooling, leading to a reduction of up to 40% of the amount of energy used for cooling, and a 15% improvement in the buildings' overall energy efficiency. Since June 2019, Suleyman has served on the board of The Economist Group, which publishes The Economist newspaper. In August 2019, Suleyman was placed on administrative leave following allegations of bullying employees. The company hired an external lawyer to investigate, and shortly thereafter Suleyman left to take a VP role at parent company Google. An email circulated by DeepMind's leadership to staff after the story broke, as well as additional details published by Business Insider, said Suleyman's "management style fell short" of expected standards. In December 2019, Suleyman announced he would be leaving DeepMind to join Google, working in a policy role. === Inflection AI === Suleyman left Google in January 2022 and joined Greylock Partners as a venture partner and in March 2022, Suleyman co-founded Inflection AI, a new AI lab venture with Greylock's Reid Hoffman. The company was founded with the goal of leveraging "AI to help humans 'talk' to computers," recruited former staff from companies such as Google and Meta and raised $225 million in its first funding round. In 2023, Inflection AI launched a chatbot named “Pi” for Personal Intelligence. The bot “remembers” past conversations and seems to get to know its users over time. According to Suleyman, the long-term goal for Pi is to be a digital “Chief of Staff”, with the initial design focused on maintaining conversational dialogue with users, asking questions, and offering emotional support. === Microsoft AI === In March 2024, Microsoft appointed Suleyman as Executive Vice President (EVP) and CEO of its newly created consumer AI unit, Microsoft AI. Several members of Inflection AI's team were also appointed to the division, including co-founder Karen Simonyan. === Awards and honours === Suleyman was appointed a Commander of the British Empire (CBE) in the 2019 New Year Honours. Suleyman was named by Time as one of the 100 most influential people in artificial intelligence in 2023 and in 2024. === Views on AI ethics === Suleyman is prominent in the debate over the ethics of AI and has spoken widely about the need for companies, governments and civil society to join in holding technologists accountable for the impacts of their work. He has advocated redesigning incentives in the technology industry to steer business leaders toward prioritising social responsibility alongside their fiduciary duties. Within DeepMind he set up a research unit called DeepMind Ethics & Society to study the real-world impacts of AI and help technologists put ethics into practice. Suleyman is also a founding co-chair of the Partnership on AI – an organisation that includes representatives from companies such as Amazon, Apple, DeepMind, Meta, Google, IBM, and Microsoft. The organisation studies and formulates best practices for AI technologies, advances the public's understanding of AI, and serves as an open platform for discussion and engagement about AI and how it affects people and society. Its board of directors has equal representation from non-profit and for profit entities. In September 2023, Suleyman, in collaboration with researcher Michael Bhaskar, published The Coming Wave, Technology, Power and the 21st Century's Greatest Dilemma, a book that examines the transformative and potentially perilous impact of advanced technologies, particularly AI and synthetic biology. According to Suleyman, AI notably has the potential to bring "radical abundance", address climate change and empower people with its cheap problem-solving capabilities. But it may also improve its own design and manufacturing processes, leading to a period of dangerously rapid AI progress. And it could enable catastrophic misuse, from bioengineered pathogens to autonomous weapons, making global oversight and containment essential to avoid unintended consequences. It was shortlisted for the 2023 Financial Times Business Book of the Year Award. In June 2024, in an interview with Andrew Ross Sorkin at the Aspen Ideas Festival, Suleyman expressed the view that unless a website explicitly specifies otherwise, for "content that is already on the open web, the social contract of that content since the 90s has been that it is fair use. Anyone can copy it, recreate with it, reproduce with it. That has been freeware, if you like. That's been the understanding." The statement sparked controversy over the use of Internet data for training AI models. == Personal life == A Business Insider profile in 2017 described Suleyman as being liberal.

ChipTest

ChipTest was a 1985 chess playing computer built by Feng-hsiung Hsu, Thomas Anantharaman and Murray Campbell at Carnegie Mellon University. It is the predecessor of Deep Thought which in turn evolved into Deep Blue. == History == ChipTest was based on a special VLSI-technology move generator chip developed by Hsu. ChipTest was controlled by a Sun-3/160 workstation and capable of searching approximately 50,000 moves per second. Hsu and Anantharaman entered ChipTest in the 1986 North American Computer Chess Championship, and it was only partially tested when the tournament began. It lost its first two rounds, but finished with an even score. In August 1987, ChipTest was overhauled and renamed ChipTest-M, M standing for microcode. The new version had eliminated ChipTest's bugs and was ten times faster, searching 500,000 moves per second and running on a Sun-4 workstation. ChipTest-M won the North American Computer Chess Championship in 1987 with a 4–0 sweep. ChipTest was invited to play in the 1987 American Open, but the team did not enter due to an objection by the HiTech team, also from Carnegie Mellon University. HiTech and ChipTest shared some code, and Hitech was already playing in the tournament. The two teams became rivals. Designing and implementing ChipTest revealed many possibilities for improvement, so the designers started on a new machine. Deep Thought 0.01 was created in May 1988 and the version 0.02 in November the same year. This new version had two customized VLSI chess processors and it was able to search 720,000 moves per second. With the "0.02" dropped from its name, Deep Thought won the World Computer Chess Championship with a perfect 5–0 score in 1989.

ITU-WHO Focus Group on Artificial Intelligence for Health

The ITU-WHO Focus Group on Artificial Intelligence for Health (AI for Health) was an inter-agency collaboration from 2018 between the World Health Organization and the ITU, which in 2019 created a benchmarking framework to assess the accuracy of AI in health. The organization convened an international network of experts and stakeholders from fields like research, practice, regulation, ethics, public health, etc, that developed guideline documentation and code. The documents have addressed ethics, assessment/evaluation, handling, and regulation of AI for health solutions, covering specific use cases including AI in ophthalmology, histopathology, dentistry, malaria detection, radiology, symptom checker applications, etc. FG-AI4H has established an ad hoc group concerned with digital technologies for health emergencies, including COVID-19. All documentation is public. The idea for the Focus Group came out of the Health Track of the 2018 AI for Good Global Summit. Administratively, FG-AI4H was created by ITU-T Study Group 16. Under ITU-T's framework, participation in Focus Groups is open to anyone from an ITU Member State. The secretariat is provided by the Telecommunication Standardization Bureau (under Director Chaesub Lee). It was first created at the July 2018 meeting with a lifetime of two years, at the July 2020 meeting, this was extended for another two years, where the focus group also submitted its deliverables to its parent body. It was also presented at the NeurIPS 2020 health workshop. In July 2023 "the work was grandfathered in the Global Initiative on AI for Health (GI-AI4H)". == AI for Health Framework == The outline of the benchmarking framework was published in a 2019 commentary in The Lancet. The output of the Focus Group AI for Health were structured in the AI for Health Framework. Depending on their primary domain being health or ICT, the individual components of the AI for Health Framework were ratified by the corresponding United Nations Specialized Agency, as WHO Guidelines and ITU Recommendations respectively. Standards drawn up by FG-AI4H were titled as: AI4H ethics considerations AI4H regulatory [best practices | considerations] AI4H requirements specification AI software life cycle specification Data specification AI training best practices specification AI4H evaluation considerations AI4H scale-up and adoption AI4H applications and platforms Use cases of the ITU-WHO Focus Group on AI for Health

Belief–desire–intention model

For popular psychology, the belief–desire–intention (BDI) model of human practical reasoning was developed by Michael Bratman as a way of explaining future-directed intention. BDI is fundamentally reliant on folk psychology (the 'theory theory'), which is the notion that our mental models of the world are theories. It was used as a basis for developing the belief–desire–intention software model. == Applications == BDI was part of the inspiration behind the BDI software architecture, which Bratman was also involved in developing. Here, the notion of intention was seen as a way of limiting time spent on deliberating about what to do, by eliminating choices inconsistent with current intentions. BDI has also aroused some interest in psychology. BDI formed the basis for a computational model of childlike reasoning CRIBB.

Inbenta

Inbenta is an AI company that originated in Barcelona, Spain, in 2005. Inbenta is currently headquartered in Allen, Texas, with additional offices in Spain, São Paulo, Brazil, Toulouse, France, and Tokyo, Japan. Inbenta provides natural language processing and semantic search through artificial intelligence. == History == Inbenta raised $12 Million in their Series B funding round to extend the reach of their artificial intelligence for business solutions. In 2023 Inbenta's new chief executive officer Melissa Solis moved Inbenta's headquarters to One Bethany West in Allen, Texas from Foster City, California. == Controversy == On 23 June 2018, Ticketmaster UK identified malicious software on a customer support product hosted by Inbenta Technologies, compromising personal data and payment details for thousands of Ticketmaster customers. Three days later, Inbenta's CEO Issued a message about the incident to convey the full scope of the breach. Also on its FAQ section, Inbenta claimed that "After a careful analysis of all clues and snapshots from our systems, the technical team at Inbenta discovered that the script had been implemented on the payment page. We were unaware of this, and would have advised against doing so had we known, as it presents a point of vulnerability". On November 13, 2020, the Information Commissioner's Office fined Ticketmaster UK Limited £1.25 million for failing to protect customers' payment details. According to the ICO, "It was because of Ticketmaster's business decision to include the [Inbenta] chat bot on its payment page that the chat bot was able to unlawfully process the personal data of customers."