Evaluation of a binary classifier typically assigns a numerical value, or values, to a classifier that represent its accuracy. An example is error rate, which measures how frequently the classifier makes a mistake. There are many metrics that can be used; different fields have different preferences. For example, in medicine sensitivity and specificity are often used, while in computer science precision and recall are preferred. An important distinction is between metrics that are independent of the prevalence or skew (how often each class occurs in the population), and metrics that depend on the prevalence – both types are useful, but they have very different properties. Often, evaluation is used to compare two methods of classification, so that one can be adopted and the other discarded. Such comparisons are more directly achieved by a form of evaluation that results in a single unitary metric rather than a pair of metrics. == Contingency table == Given a data set, a classification (the output of a classifier on that set) gives two numbers: the number of positives and the number of negatives, which add up to the total size of the set. To evaluate a classifier, one compares its output to another reference classification – ideally a perfect classification, but in practice the output of another gold standard test – and cross tabulates the data into a 2×2 contingency table, comparing the two classifications. One then evaluates the classifier relative to the gold standard by computing summary statistics of these 4 numbers. Generally these statistics will be scale invariant (scaling all the numbers by the same factor does not change the output), to make them independent of population size, which is achieved by using ratios of homogeneous functions, most simply homogeneous linear or homogeneous quadratic functions. Say we test some people for the presence of a disease. Some of these people have the disease, and our test correctly says they are positive. They are called true positives (TP). Some have the disease, but the test incorrectly claims they don't. They are called false negatives (FN). Some don't have the disease, and the test says they don't – true negatives (TN). Finally, there might be healthy people who have a positive test result – false positives (FP). These can be arranged into a 2×2 contingency table (confusion matrix), conventionally with the test result on the vertical axis and the actual condition on the horizontal axis. These numbers can then be totaled, yielding both a grand total and marginal totals. Totaling the entire table, the number of true positives, false negatives, true negatives, and false positives add up to 100% of the set. Totaling the columns (adding vertically) the number of true positives and false positives add up to 100% of the test positives, and likewise for negatives. Totaling the rows (adding horizontally), the number of true positives and false negatives add up to 100% of the condition positives (conversely for negatives). The basic marginal ratio statistics are obtained by dividing the 2×2=4 values in the table by the marginal totals (either rows or columns), yielding 2 auxiliary 2×2 tables, for a total of 8 ratios. These ratios come in 4 complementary pairs, each pair summing to 1, and so each of these derived 2×2 tables can be summarized as a pair of 2 numbers, together with their complements. Further statistics can be obtained by taking ratios of these ratios, ratios of ratios, or more complicated functions. The contingency table and the most common derived ratios are summarized below; see sequel for details. Note that the rows correspond to the condition actually being positive or negative (or classified as such by the gold standard), as indicated by the color-coding, and the associated statistics are prevalence-independent, while the columns correspond to the test being positive or negative, and the associated statistics are prevalence-dependent. There are analogous likelihood ratios for prediction values, but these are less commonly used, and not depicted above. == Pairs of metrics == Often accuracy is evaluated with a pair of metrics composed in a standard pattern. === Sensitivity and specificity === The fundamental prevalence-independent statistics are sensitivity and specificity. Sensitivity or True Positive Rate (TPR), also known as recall, is the proportion of people that tested positive and are positive (True Positive, TP) of all the people that actually are positive (Condition Positive, CP = TP + FN). It can be seen as the probability that the test is positive given that the patient is sick. With higher sensitivity, fewer actual cases of disease go undetected (or, in the case of the factory quality control, fewer faulty products go to the market). Specificity (SPC) or True Negative Rate (TNR) is the proportion of people that tested negative and are negative (True Negative, TN) of all the people that actually are negative (Condition Negative, CN = TN + FP). As with sensitivity, it can be looked at as the probability that the test result is negative given that the patient is not sick. With higher specificity, fewer healthy people are labeled as sick (or, in the factory case, fewer good products are discarded). The relationship between sensitivity and specificity, as well as the performance of the classifier, can be visualized and studied using the Receiver Operating Characteristic (ROC) curve. In theory, sensitivity and specificity are independent in the sense that it is possible to achieve 100% in both (such as in the red/blue ball example given above). In more practical, less contrived instances, however, there is usually a trade-off, such that they are inversely proportional to one another to some extent. This is because we rarely measure the actual thing we would like to classify; rather, we generally measure an indicator of the thing we would like to classify, referred to as a surrogate marker. The reason why 100% is achievable in the ball example is because redness and blueness is determined by directly detecting redness and blueness. However, indicators are sometimes compromised, such as when non-indicators mimic indicators or when indicators are time-dependent, only becoming evident after a certain lag time. The following example of a pregnancy test will make use of such an indicator. Modern pregnancy tests do not use the pregnancy itself to determine pregnancy status; rather, human chorionic gonadotropin is used, or hCG, present in the urine of gravid females, as a surrogate marker to indicate that a woman is pregnant. Because hCG can also be produced by a tumor, the specificity of modern pregnancy tests cannot be 100% (because false positives are possible). Also, because hCG is present in the urine in such small concentrations after fertilization and early embryogenesis, the sensitivity of modern pregnancy tests cannot be 100% (because false negatives are possible). === Positive and negative predictive values === In addition to sensitivity and specificity, the performance of a binary classification test can be measured with positive predictive value (PPV), also known as precision, and negative predictive value (NPV). The positive prediction value answers the question "If the test result is positive, how well does that predict an actual presence of disease?". It is calculated as TP/(TP + FP); that is, it is the proportion of true positives out of all positive results. The negative prediction value is the same, but for negatives, naturally. ==== Impact of prevalence on predictive values ==== Prevalence has a significant impact on prediction values. As an example, suppose there is a test for a disease with 99% sensitivity and 99% specificity. If 2000 people are tested and the prevalence (in the sample) is 50%, 1000 of them are sick and 1000 of them are healthy. Thus about 990 true positives and 990 true negatives are likely, with 10 false positives and 10 false negatives. The positive and negative prediction values would be 99%, so there can be high confidence in the result. However, if the prevalence is only 5%, so of the 2000 people only 100 are really sick, then the prediction values change significantly. The likely result is 99 true positives, 1 false negative, 1881 true negatives and 19 false positives. Of the 19+99 people tested positive, only 99 really have the disease – that means, intuitively, that given that a patient's test result is positive, there is only 84% chance that they really have the disease. On the other hand, given that the patient's test result is negative, there is only 1 chance in 1882, or 0.05% probability, that the patient has the disease despite the test result. === Precision and recall === Precision and recall can be interpreted as (estimated) conditional probabilities: Precision is given by P ( C = P | C ^ = P ) {\displaystyle P(C=P|{\hat {C}}=P)} while recall is given by P ( C ^ = P | C = P ) {\displaystyle P({\hat {C}}=P|C=P)} , where C ^ {\
Imaging phantom
An imaging phantom, or simply phantom (less commonly spelled fantom), is a specially designed object that is scanned or imaged in the field of medical imaging to evaluate, analyze, and tune the performance of various imaging devices. A phantom is more readily available and provides more consistent results than the use of a living subject or cadaver, while also avoiding direct risks to living subjects. Phantoms were originally employed in 2D x-ray–based imaging techniques such as radiography or fluoroscopy, but more recently phantoms with desired imaging characteristics have been developed for 3D techniques such as SPECT, MRI, CT, ultrasound, PET, and other imaging modalities. == Design == A phantom used to evaluate an imaging device should respond in a similar manner to how human tissues and organs would act in that specific imaging modality. For instance, phantoms made for 2D radiography may hold various quantities of x-ray contrast agents with similar x-ray absorbing properties (such as the attenuation coefficient) to normal tissue to tune the contrast of the imaging device or modulate the patient's exposure to radiation. In such a case, the radiography phantom would not necessarily need to have similar textures and mechanical properties since these are not relevant in x-ray imaging modalities. However, in the case of ultrasonography, a phantom with similar rheological and ultrasound scattering properties to real tissue would be essential, but x-ray absorbing properties would not be relevant. The term "phantom" describes an object that is designed to resemble human tissue and can be evaluated, analyzed or manipulated to study the performance of a medical device. Phantoms are created using a digital file that is rendered through magnetic resonance imaging (MRI) or computer-aided design (CAD). The digital files allow for quick modifications that are read by the 3D printer. The 3D printer will create the product in successive layers using polymeric materials. There are several types of phantoms including tissue-mimicking, radiological phantoms, dental phantoms, BOMABs (used to calibrate whole-body counters), and more.
Computational theory of mind
In philosophy of mind, the computational theory of mind (CTM), also known as computationalism, is a family of views that hold that the human mind is an information processing system and that cognition and consciousness together are a form of computation. It is closely related to functionalism, a broader theory that defines mental states by what they do rather than what they are made of. == History == Warren McCulloch and Walter Pitts (1943) were the first to suggest that neural activity is computational. They argued that neural computations explain cognition. A version of the theory was put forward by Peter Putnam and Robert W. Fuller in 1964. The theory was proposed in its modern form by Hilary Putnam in 1960 and 1961, aided by his then PhD student, philosopher and cognitive scientist Jerry Fodor, who continued the research as a post-doc in the 1960s, 1970s, and 1980s. It was later criticized by Putnam himself, John Searle, and others. == Classical computational theory of mind == The CTM holds that the human mind is a computational system that is realized (i.e., physically implemented) by neural activity in the brain. The theory can be elaborated in many ways and varies largely based on how the term computation is understood. In classical computational theory of mind (CCTM), computation is modeled in terms of Turing machines which manipulate symbols according to a rule, in combination with the internal state of the machine. A Turing machine is an abstract machine with unlimited time and storage. CCTM does not pretend that the mind looks like a Turing machine, but instead uses Turing machines as a formalism. Alan Turing argued that any symbolic algorithm executed by a human brain can in theory be replicated on a Turing machine. The critical aspect of such a computational model is that it allows to abstract away from particular physical details of the machine that is implementing the computation. For example, the appropriate computation could be implemented either by silicon chips or biological neural networks, so long as there is a series of outputs based on manipulations of inputs and internal states, performed according to a rule. Computational theories of mind are often said to require mental representation because 'input' into a computation comes in the form of symbols or representations of other objects. A computer cannot compute an actual object but must interpret and represent the object in some form and then compute the representation. Unlike CTM, the representational theory of mind shifts the focus to the symbols being manipulated. This approach better accounts for systematicity and productivity. In Fodor's view, the mind is a computational system that processes the language of thought. == Variants == Connectionist computationalism models the mind as a neural network. Steven Pinker and Alan Prince distinguish two types of connectionists: eliminative and implementationist. Eliminative connectionists generally reject classical CTMs and the idea of a structured, symbolic mind, whereas implementationists view neural networks and Turing machines as two potentially complementary levels of analysis. It is indeed possible in theory to implement a neural network in a Turing machine, or a Turing machine in a neural network. Building from the tradition of McCulloch and Pitts, the computational theory of cognition (CTC) states that neural computations explain cognition. The computational theory of mind asserts that not only cognition, but also phenomenal consciousness or qualia, are computational. That is to say, CTM entails CTC. While phenomenal consciousness could fulfill some other functional role, computational theory of cognition leaves open the possibility that some aspects of the mind could be non-computational. CTC, therefore, provides an important explanatory framework for understanding neural networks, while avoiding counter-arguments that center around phenomenal consciousness. == "Computer metaphor" == Computational theory of mind is not the same as the computer metaphor, comparing the mind to a modern-day digital computer. While the computer metaphor draws an analogy between the mind as software and the brain as hardware, CTM is the claim that the mind is literally a computational system. "Computational system" is not intended to mean a modern-day electronic computer. == Pancomputationalism == CTM raises a question that remains a subject of debate: what does it take for a physical system (such as a mind, or an artificial computer) to perform computations? A very straightforward account is based on a simple mapping between abstract mathematical computations and physical systems: a system performs computation C if and only if there is a mapping between a sequence of states individuated by C and a sequence of states individuated by a physical description of the system. Putnam (1988) and Searle (1992) argue that this simple mapping account (SMA) trivializes the empirical import of computational descriptions. As Putnam put it, "everything is a Probabilistic Automaton under some Description". Even rocks, walls, and buckets of water—contrary to appearances—are computing systems. Gualtiero Piccinini identifies different versions of pancomputationalism. Searle wrote:the wall behind my back is right now implementing the WordStar program, because there is some pattern of molecule movements that is isomorphic with the formal structure of WordStar. But if the wall is implementing WordStar, if it is a big enough wall it is implementing any program, including any program implemented in the brain.In response to the trivialization criticism, and to restrict SMA, philosophers of mind have offered different accounts of computational systems. These typically include causal account, semantic account, syntactic account, and mechanistic account. Instead of a semantic restriction, the syntactic account imposes a syntactic restriction. The mechanistic account was first introduced by Gualtiero Piccinini in 2007. == Criticism == A range of arguments have been proposed against physicalist conceptions used in computational theories of mind. An early, though indirect, criticism of the computational theory of mind comes from philosopher John Searle. In his thought experiment known as the Chinese room, Searle attempts to refute the claims that artificially intelligent agents can be said to have intentionality and understanding and that these systems, because they can be said to be minds themselves, are sufficient for the study of the human mind. Searle asks us to imagine that there is a man in a room with no way of communicating with anyone or anything outside of the room except for a piece of paper with symbols written on it that is passed under the door. With the paper, the man is to use a series of provided rule books to return paper containing different symbols. Unknown to the man in the room, these symbols are of a Chinese language, and this process generates a conversation that a Chinese speaker outside of the room can actually understand. Searle contends that the man in the room does not understand the Chinese conversation. This was originally written as a repudiation of the idea that computers work like minds. Objections like Searle's might be called insufficiency objections. They claim that computational theories of mind fail because computation is insufficient to account for some capacity of the mind. Arguments from qualia, such as Frank Jackson's knowledge argument, can be understood as objections to computational theories of mind in this way—though they take aim at physicalist conceptions of the mind in general, and not computational theories specifically. Objections have also been put forth that are directly tailored for computational theories of mind. Jerry Fodor himself argues that the mind is still a very long way from having been explained by the computational theory of mind. The main reason for this shortcoming is that most cognition is abductive and global, hence sensitive to all possibly relevant background beliefs to (dis)confirm a belief. This creates, among other problems, the frame problem for the computational theory, because the relevance of a belief is not one of its local, syntactic properties but context-dependent. Putnam himself (see in particular Representation and Reality and the first part of Renewing Philosophy) became a prominent critic of computationalism for a variety of reasons, including ones related to Searle's Chinese room arguments, questions of world-word reference relations, and thoughts about the mind-body problem. Regarding functionalism in particular, Putnam has claimed along lines similar to, but more general than Searle's arguments, that the question of whether the human mind can implement computational states is not relevant to the question of the nature of mind, because "every ordinary open system realizes every abstract finite automaton." Computationalists have responded by aiming to develop criteri
Shane Legg
Shane Legg (born 1973 or 1974) is a machine learning researcher and entrepreneur. With Demis Hassabis and Mustafa Suleyman, he cofounded DeepMind Technologies (later bought by Google and now called Google DeepMind), and works there as the chief AGI scientist. He is also known for his academic work on artificial general intelligence, including his thesis supervised by Marcus Hutter. == Early life and education == Legg attended Rotorua Lakes High School in Rotorua, on New Zealand's North Island. He completed his undergraduate studies at Waikato University in 1996. Also in 1996, he obtained his MSc degree with a thesis entitled "Solomonoff Induction", with Cristian S. Calude at the University of Auckland. == Research interests == In the early 2000s, Legg re-introduced and popularized with Ben Goertzel the term "artificial general intelligence" (AGI), to describe an AI that can do practically any cognitive task a human can do. At that time, talking about AGI "would put you on the lunatic fringe". Legg is known for his concern of existential risk from AI, highlighted in 2011 in an interview on LessWrong and in 2023 he signed the statement on AI risk of extinction. == Career == Before his PhD and before cofounding DeepMind, Shane Legg worked at "a number of software development positions at private companies", including the "big data firm Adaptive Intelligence" and the startup WebMind founded by Ben Goertzel. === Research === Legg later obtained a PhD at the Dalle Molle Institute for Artificial Intelligence Research (IDSIA), a joint research institute of USI Università della Svizzera italiana and SUPSI. He worked on theoretical models of super intelligent machines (AIXI) with Marcus Hutter, and completed in 2008 his doctoral thesis entitled "Machine Super Intelligence". He then went on to complete a postdoctoral fellowship in finance at USI, and began a further fellowship at University College London's Gatsby Computational Neuroscience Unit. === DeepMind === Demis Hassabis and Shane Legg first met in 2009 at University College London, where Legg was a postdoctoral researcher. In 2010, Legg cofounded the start-up DeepMind Technologies along with Demis Hassabis and Mustafa Suleyman. DeepMind Technologies was bought in 2014 by Google. After the merge with Google Brain in 2023, the company is now known as Google DeepMind. According to a 2017 article, a significant part of his job as the chief scientist was to supervise recruitment, to decide where DeepMind should focus its efforts, and to lead DeepMind's AI safety work. As of July 2023, Legg works at Google DeepMind as the Chief AGI Scientist. == Awards and honors == Legg was awarded the $10,000 prize of the Singularity Institute for Artificial Intelligence for his PhD done in 2008. Legg was appointed Commander of the Order of the British Empire (CBE) in the 2019 Birthday Honours for services to the science and technology sector and to investment.
Social History and Industrial Classification
Social History and Industrial Classification (SHIC) is a classification system used by many British museums for social history and industrial collections. It was first published in 1983. == Purpose == SHIC classifies materials (books, objects, recordings etc.) by their interaction with the people who used them. For example, a carpenter's hammer is classified with other tools of the carpenter, and not with a blacksmith's hammer. In contrast other classification systems, for example the Dewey Decimal Classification, might class all hammers together and close to the classification for other percussive tools. The specialist subject network, Social History Curator's Group (SHCG), obtained funding in 2012 to develop an on-line version, now on their website http://www.shcg.org.uk/ == Scheme == Materials are classified under four major category numbers: Community life Domestic and family life Personal life Working life Further classification within a category is by the use of further numbers after the decimal point. It is permissible to assign more than one classification in cases where the object had more than one use.
Bitstrips
Bitstrips, Inc. was a Canadian media and technology company based in Toronto, founded in 2007 by Jacob Blackstock, David Kennedy, Shahan Panth, Dorian Baldwin, and Jesse Brown. The company created and offered a web application, Bitstrips.com, which allowed users to create comic strips using personalized avatars, and preset templates and poses. Brown and Blackstock explained that the service was meant to enable self-expression without the need to have artistic skills. Bitstrips was first presented in 2008 at South by Southwest in Austin, Texas, and the service later piloted and launched a version designed for use as educational software. The service achieved increasing prominence following the launch of versions for Facebook and mobile platforms. In 2014, Bitstrips launched a spin-off app known as Bitmoji, which allows users to create personalized stickers for use in instant messaging. In July 2016, Snapchat Inc. announced that it had acquired the company; the Bitstrips comic service was shut down, but Bitmoji remains operational, and has subsequently been given greater prominence within Snapchat's overall platform. == History == Bitstrips was co-developed by Toronto-based comic artist Jacob Blackstock and his high school friend, journalist Jesse Brown. The service was originally envisioned as a means to allow anyone to create their own comic strip without needing artistic skills. Brown explained that "it's so difficult and time-consuming to tell a story in comic book form, drawing the same characters again and again in these tiny little panels, and just the amount of craftsmanship required. And even if you can do it well, which I never could, it takes years to make a story." Brown stated that the service would be "groundwork for a whole new way to communicate", and went as far as describing the service as being a "YouTube for comics". Blackstock explained that the concept of Bitstrips was influenced by his own use of comics as a form of socialization; a student, Blackstock and his friends drew comics featuring each other and shared them during classes. He felt that Bitstrips was a "medium for self-expression", stating that "It's not just about you making the comics, but since you and your friends star in these comics, it's like you're the medium. The visual nature of comics just speaks so much louder than text." The service was publicly unveiled at South by Southwest in 2008. In 2009, the service introduced a version oriented towards the educational market, Bitstrips for Schools, which was initially piloted at a number of schools in Ontario. The service was praised by educators for being engaging to students, especially within language classes. Brown noted that students were using the service to create comics outside of class as well, stating that it was "so gratifying and shocking what people do with your tool to make their own stories in ways that you never would have anticipated. Some of them are just brilliant." In December 2012, Bitstrips launched a version for Facebook; by July 2013, Bitstrips had 10 million unique users on Facebook, having created over 50 million comics. In October 2013, Bitstrips launched a mobile app; in two months, Bitstrips became a top-downloaded app in 40 countries, and over 30 million avatars had been created with it. In November 2013, Bitstrips secured a round of funding from Horizons Ventures and Li Ka-shing. In October 2014, Bitstrips launched Bitmoji, a spin-off app that allows users to create stickers featuring Bitstrips characters in various templates. In July 2016, following unconfirmed reports earlier in the year, Snapchat Inc. announced that it had acquired Bitstrips. The company's staff continue to operate out of Toronto, but the original Bitstrips comic service was shut down in favour of focusing exclusively on Bitmoji, leaving many Bitstrips users to call for a reboot of the comic service.
UMBEL
UMBEL (Upper Mapping and Binding Exchange Layer) is a logically organized knowledge graph of 34,000 concepts and entity types that can be used in information science for relating information from disparate sources to one another. It was retired at the end of 2019. UMBEL was first released in July 2008. Version 1.00 was released in February 2011. Its current release is version 1.50. The grounding of this information occurs by common reference to the permanent URIs for the UMBEL concepts; the connections within the UMBEL upper ontology enable concepts from sources at different levels of abstraction or specificity to be logically related. Since UMBEL is an open-source extract of the OpenCyc knowledge base, it can also take advantage of the reasoning capabilities within Cyc. UMBEL has two means to promote the semantic interoperability of information:. It is: An ontology of about 35,000 reference concepts, designed to provide common mapping points for relating different ontologies or schema to one another, and A vocabulary for aiding that ontology mapping, including expressions of likelihood relationships distinct from exact identity or equivalence. This vocabulary is also designed for interoperable domain ontologies. UMBEL is written in the Semantic Web languages of SKOS and OWL 2. It is a class structure used in Linked Data, along with OpenCyc, YAGO, and the DBpedia ontology. Besides data integration, UMBEL has been used to aid concept search, concept definitions, query ranking, ontology integration, and ontology consistency checking. It has also been used to build large ontologies and for online question answering systems. Including OpenCyc, UMBEL has about 65,000 formal mappings to DBpedia, PROTON, GeoNames, and schema.org, and provides linkages to more than 2 million Wikipedia pages (English version). All of its reference concepts and mappings are organized under a hierarchy of 31 different "super types", which are mostly disjoint from one another. Each of these "super types" has its own typology of entity classes to provide flexible tie-ins for external content. 90% of UMBEL is contained in these entity classes.