AI Chatbot Image Generator

AI Chatbot Image Generator — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Application permissions

  Permissions are a means of controlling and regulating access to specific system- and device-level functions by software. Typically, types of permissions cover functions that may have privacy implications, such as the ability to access a device's hardware features (including the camera and microphone), and personal data (such as storage devices, contacts lists, and the user's present geographical location). Permissions are typically declared in an application's manifest, and certain permissions must be specifically granted at runtime by the user—who may revoke the permission at any time. Permission systems are common on mobile operating systems, where permissions needed by specific apps must be disclosed via the platform's app store. == Mobile devices == On mobile operating systems for smartphones and tablets, typical types of permissions regulate: Access to storage and personal information, such as contacts, calendar appointments, etc. Location tracking. Access to the device's internal camera and/or microphone. Access to biometric sensors, including fingerprint readers and other health sensors.. Internet access. Access to communications interfaces (including their hardware identifiers and signal strength where applicable, and requests to enable them), such as Bluetooth, Wi-Fi, NFC, and others. Making and receiving phone calls. Sending and reading text messages The ability to perform in-app purchases. The ability to "overlay" themselves within other apps. Installing, deleting and otherwise managing applications. Authentication tokens (e.g., OAuth tokens) from web services stored in system storage for sharing between apps. Prior to Android 6.0 "Marshmallow", permissions were automatically granted to apps at runtime, and they were presented upon installation in Google Play Store. Since Marshmallow, certain permissions now require the app to request permission at runtime by the user. These permissions may also be revoked at any time via Android's settings menu. Usage of permissions on Android are sometimes abused by app developers to gather personal information and deliver advertising; in particular, apps for using a phone's camera flash as a flashlight (which have grown largely redundant due to the integration of such functionality at the system level on later versions of Android) have been known to require a large array of unnecessary permissions beyond what is actually needed for the stated functionality. iOS imposes a similar requirement for permissions to be granted at runtime, with particular controls offered for enabling of Bluetooth, Wi-Fi, and location tracking. == WebPermissions == WebPermissions is a permission system for web browsers. When a web application needs some data behind permission, it must request it first. When it does it, a user sees a window asking him to make a choice. The choice is remembered, but can be cleared lately. Currently the following resources are controlled: geolocation desktop notifications service workers sensors audio capturing devices, like sound cards, and their model names and characteristics video capturing devices, like cameras, and their identifiers and characteristics == Analysis == The permission-based access control model assigns access privileges for certain data objects to application. This is a derivative of the discretionary access control model. The access permissions are usually granted in the context of a specific user on a specific device. Permissions are granted permanently with few automatic restrictions. In some cases permissions are implemented in 'all-or-nothing' approach: a user either has to grant all the required permissions to access the application or the user can not access the application. There is still a lack of transparency when the permission is used by a program or application to access the data protected by the permission access control mechanism. Even if a user can revoke a permission, the app can blackmail a user by refusing to operate, for example by just crashing or asking user to grant the permission again in order to access the application. The permission mechanism has been widely criticized by researchers for several reasons, including; Intransparency of personal data extraction and surveillance, including the creation of a false sense of security; End-user fatigue of micro-managing access permissions leading to a fatalistic acceptance of surveillance and intransparency; Massive data extraction and personal surveillance carried out once the permissions are granted. Some apps, such as XPrivacy and Mockdroid spoof data in order to act as a measure for privacy. Further transparency methods include longitudinal behavioural profiling and multiple-source privacy analysis of app data access.

  Read more →
• 

  Black in AI

  Black in AI, formally called the Black in AI Workshop, is a technology research organization and affinity group, founded by computer scientists Timnit Gebru and Rediet Abebe in 2017. It started as a conference workshop, later pivoting into an organization. Black in AI increases the presence and inclusion of Black people in the field of artificial intelligence (AI) by creating space for sharing ideas, fostering collaborations, mentorship, and advocacy. == History == Black in AI was created in 2017 to address issues of lack of diversity in AI workshops, and was started as its own workshop within the Conference on Neural Information Processing Systems (NeurIPS) conference. Because of algorithmic bias, ethical issues, and underrepresentation of Black people in AI roles; there has been an ongoing need for unity within the AI community to have focus on these issues. Black in AI has strived to continue the progress of improving the presence of people of color in the field of artificial intelligence. In 2018 and 2019, the Black in AI workshop had many immigration visa issues to Canada, which spurred the conference to be planned for 2020 in Addis Ababa, Ethiopia. On December 7, 2020, Black in AI held its fourth annual workshop and first virtual workshop (due to the COVID-19 pandemic). In 2021, Black in AI, alongside the groups Queer in AI and Widening NLP, released a public statement refusing funding from Google in an act of protest of Google's treatment of Timnit Gebru, Margaret Mitchell, and April Christina Curley in the events that occurred in December 2020. == Founders == Rediet Abebe is an Ethiopian computer scientist who specializes in algorithms and artificial intelligence. She is a Computer Science Assistant Professor at the University of California, Berkeley. She was previously a Junior Fellow at Harvard's Society of Fellows. She was the first Black woman to receive a Ph.D. in computer science at Cornell University. She "designs and analyzes algorithms, discrete optimizations, network-based, [and] computational strategies to increase access to opportunity for historically disadvantaged populations," according to her web bio. Timnit Gebru was born in Ethiopia and moved to the United States at the age of fifteen. She got her B.S. and M.S. in electrical engineering from Stanford University, as well as a PhD from the Stanford Artificial Intelligence Laboratory, where she studied computer vision under Fei-Fei Li. She formerly worked as a postdoctoral researcher at Microsoft Research in the Fairness Accountability Transparency, and Ethics (FATE) division. She's also worked with Apple, where she assisted in the development of signal-processing algorithms for the original iPad. == Grants == Black in AI received grants and support from private foundations like MacArthur Foundation and Rockefeller Foundation. The organization received $10,000 in 2018 for its annual workshop and $150,000 in 2019 for its long-term organizational planning. In 2020, during the pandemic, the organization received a grant of $300,000 by MacArthur Foundation in order to provide broad organizational support. In 2022, Rockefeller Foundation announced $300,000 to fight prejudice in artificial intelligence (AI) across the globe and incorporate equity into this rapidly expanding field. == Programs == "Black in AI works in academics, advocacy, entrepreneurship, financial support, and summer research programs." The Black in AI Academic Program is a resource for Black junior researchers applying to graduate schools, navigating graduate school, and transitioning into the postgraduate employment market. They provide online education sessions, offer scholarships to cover application fees, pair participants with peer and senior mentors, and distribute crowdsourced papers that simplify the application process. They also undertake research projects to investigate and highlight the difficulties that Black young researchers face, as well as push for structural reforms to eliminate these barriers and build equitable research settings. Moses Namara is a Facebook Research Fellow at Clemson University and a PhD candidate in Human-Centered Computing (HCC). He is the mentor for the new Black in AI Academic Program. During the graduate school admissions season in 2021, Black in AI served more than 200 potential graduate program candidates in some capacity. Furthermore, the organization's study identified greater problems encountered by Black graduate school candidates, such as the high cost of graduate school admissions examinations (GREs), which are known to be biased against those from low-income backgrounds. Black in AI's attempts to encourage institutions to eliminate the obstacles were supported by the findings. Black in AI is also developing a program to help and connect Black tech startups with investors. Black in AI also mentors early-career Black AI academics and is forming relationships with Historically Black Colleges and Universities to extend its academic program. In 2021, Black in AI launched two summer research programs, one for undergraduate internships and another for unconstrained research mentorship, including one aimed explicitly at empowering Black women's AI research projects. == Conferences and workshops == At NeurIPS 2017, the first Black in AI event took place in December 8, 2017 in Long Beach, California. The goal was to bring together experts in the area to share ideas and debate efforts aimed at increasing the participation of Black people in artificial intelligence, both for diversity and to avoid data bias. Black AI researchers had the opportunity to share their work at the workshop's oral and poster sessions. The second workshop was hosted in Montréal, Canada, on December 7, 2018. According to AI experts, visa issues stymie efforts to make their area more inclusive, making technology that discriminates or disadvantages individuals who aren't white or Western less likely. Hundreds of participants who were supposed to attend or present work at the Black in AI session on Friday were unable to fly to Canada; many of the participants were from African countries. The third workshop was held in NeurIPS 2019, one of the premier machine learning conferences Vancouver, Canada. The workshop was able to give travel scholarships and visa support to hundreds of academics who would not have been able to attend NeurIPS without the help of sponsors. For instance, Ramon Vilarino of the University of Sao Paulo, who presented a poster at the conference on his study of geographical and racial prejudice in credit scoring in Brazil, would not have been able to attend NeurIPS without the help of Black in AI. Twenty-four academics from Africa and South America were denied visas to attend this session during the conference, according to Victor Silva, the workshop organizer. He noted that, less than a month before the conference, 40 applicants from both continents had been given visas but that more than 70 applications were still waiting. For the second year in a row, visa restrictions have stopped several African scholars from attending the 2018 meeting in Montreal. The AAAI announced the first Black in AI lunch, which was held in conjunction with AAAI-19. The lunch was hosted on Tuesday, January 29, 2019. This event was intended to promote networking, discussion of various AI career options, and the exchange of ideas in order to boost the number of Black researchers in the area. The fourth Black in AI workshop, which was held in conjunction with NeurIPS 2020, took place the week of December 7, 2020. The workshop was scheduled to take place in Vancouver, British Columbia. Due to the pandemic, the session was held for the first time in a virtual format. Victor Silva, an AI4Society student, served as the event's chair. The fifth annual Black in AI workshop was also held virtually in 2021. Oral presentations, guest keynote speakers, a combined poster session with other affinity groups, sponsored sessions, and startup showcases was all featured. The goal of the session was to raise the visibility of black scholars at NeurIPS.

  Read more →
• 

  Ginger Software

  Ginger Software is an American and Israeli start-up specialized in natural language processing and AI. The main products are tools aiming to improve written communications, develop English speaking skills and boost productivity. The company was founded in 2008 by Yael Karov and Avner Zangvil. Ginger Software uses the context of complete sentences to suggest corrections. In December 2011, Ginger Software was one of nine projects approved by the Board of Governors of the Israel-U.S. Binational Industrial Research and Development Foundation for a funding of $8.1 million. The company also raised $3 million from private Israeli and US investors in 2009. In May, 2014 Intel acquired one of Ginger's business units and the rights to use the company's patented technology. == Founders == Before founding Ginger Software, Yael Karov had worked with Rosetta Genomics as its Chief Technology Officer and Vice President of Research and Development from 2003 to 2006, and with ClickSoftware Technologies as a Director of Research and Development from 1990 to 1994. Karov also founded Agentics, a company specializing in free-text classification of e-commerce product information based on natural language processing, in 1996. Avner Zangvil is the co-founder of Ginger Software. Zangvil co-founded Menta Software in 1996 with his brother Arnon Zangvil to develop a product that transforms any Windows-based application into a Web-enabled application usable from any remote computer running a Web browser. Menta was acquired by GraphOn Corporation in 2001. == Technology == Ginger Software uses patented software algorithms in the field of natural language processing. The company claims that the algorithm allows it to correct the written sentences with relatively high accuracy (eliminating up to 95 percent of writing errors), compared to standard spell checkers. Its unique algorithm allows the software to understand the context of the sentence rather than correcting based solely on a word. According to its founder, Karov, the software operates on the logic of sentence context in addition to the memory of a database of words. The company is at the heart of a growing revolution in the world of assistive technology. Ginger claims that the benefits of the software have been leveraged by native English and non-native speakers alike, and have also found value in niche markets like dyslexia management. They further claim that ESL users derive great benefit from the use of the software, as it lets them write error-free English text. Its use also extends to native English speaking business professionals and students who use it as a 'safety net' for their email edits, as well as international students writing in English. More recently, the company has focused on implementing its technology in mobile devices as an integral component of its mobile keyboard products. == Products == Ginger Software products include Ginger Page, a cross-platform writing enhancement app, and Ginger Keyboard which is available for Android devices. Ginger Writer can be used as an online service or installed on your PC or Mac. It supports MS-Word, MS-Outlook, MS-PowerPoint, Microsoft Edge, Chrome, and functions as a writing enhancement app for Android and iOS mobile devices. Its main feature is English grammar and spelling checker that runs seamlessly with the different user interfaces. It also has an advanced paraphrasing tool, contextual synonyms and definitions, translation and a text-to-speech function that enables users to hear sentences before and after correction. Ginger Keyboard for Android replaces the stock keyboard and functions as a productivity boosting keyboard app. Featuring a full set of advanced keyboard features like Stream (swipe-like) typing, adaptive word prediction, a wide variety of customizable themes and emoji, Ginger Keyboard is the only 3rd party keyboard to offer proofreading and other writing tools via one tap access to Ginger Page. == Target segment == Ginger Software started off targeting people with dyslexia. The algorithm underlying the software studies a vast pool of proper sentences in English and builds a model of proper language. The software does not analyze the text at the level of the word, but of the whole sentence. Dyslexics can have trouble choosing the right word – hence the attention to the sentence as a whole. From 2010, Ginger Software included a new target segment in its marketing outreach – users of English as a second language (ESL). Its contextual-based writing correction tool could benefit those who are not proficient in the English language. == Business model == The main business model for consumers is freemium. The free version offers contextual-based grammar and spelling checker with some limitations. Its premium features include unlimited access to Grammar Checker, the grammar and spelling checker, and Sentence Rephraser the rephrasing tool. Ginger Keyboard is free to download and use, although it does offer in-app purchases like themes and theme packs. It also disables your original spell checker. Ginger also provides a powerful Rest API which can correct full documents in one call.

  Read more →
• 

  Conversational AI Platforms: Free vs Paid (2026)

  Comparing the best conversational AI platform? An conversational AI platform is software that uses machine learning to help you get more done — it lowers the barrier so anyone can produce professional output. Privacy matters too: check whether your data trains the model and whether a no-log or enterprise tier is available. Whether you are a beginner or a pro, the right conversational AI platform slots into your workflow and pays for itself fast. Below we compare features, pricing, and real output so you can choose with confidence.

  Read more →
• 

  Lesk algorithm

  The Lesk algorithm is a classical algorithm for word sense disambiguation introduced by Michael E. Lesk in 1986. It operates on the premise that words within a given context are likely to share a common meaning. This algorithm compares the dictionary definitions of an ambiguous word with the words in its surrounding context to determine the most appropriate sense. Variations, such as the Simplified Lesk algorithm, have demonstrated improved precision and efficiency. However, the Lesk algorithm has faced criticism for its sensitivity to definition wording and its reliance on brief glosses. Researchers have sought to enhance its accuracy by incorporating additional resources like thesauruses and syntactic models. == Overview == The Lesk algorithm is based on the assumption that words in a given "neighborhood" (section of text) will tend to share a common topic. A simplified version of the Lesk algorithm is to compare the dictionary definition of an ambiguous word with the terms contained in its neighborhood. Versions have been adapted to use WordNet. An implementation might look like this: for every sense of the word being disambiguated one should count the number of words that are in both the neighborhood of that word and in the dictionary definition of that sense the sense that is to be chosen is the sense that has the largest number of this count. A frequently used example illustrating this algorithm is for the context "pine cone". The following dictionary definitions are used: PINE 1. kinds of evergreen tree with needle-shaped leaves 2. waste away through sorrow or illness CONE 1. solid body which narrows to a point 2. something of this shape whether solid or hollow 3. fruit of certain evergreen trees As can be seen, the best intersection is Pine #1 ⋂ Cone #3 = 2. == Simplified Lesk algorithm == In Simplified Lesk algorithm, the correct meaning of each word in a given context is determined individually by locating the sense that overlaps the most between its dictionary definition and the given context. Rather than simultaneously determining the meanings of all words in a given context, this approach tackles each word individually, independent of the meaning of the other words occurring in the same context. "A comparative evaluation performed by Vasilescu et al. (2004) has shown that the simplified Lesk algorithm can significantly outperform the original definition of the algorithm, both in terms of precision and efficiency. By evaluating the disambiguation algorithms on the Senseval-2 English all words data, they measure a 58% precision using the simplified Lesk algorithm compared to the only 42% under the original algorithm. Note: Vasilescu et al. implementation considers a back-off strategy for words not covered by the algorithm, consisting of the most frequent sense defined in WordNet. This means that words for which all their possible meanings lead to zero overlap with current context or with other word definitions are by default assigned sense number one in WordNet." Simplified LESK Algorithm with smart default word sense (Vasilescu et al., 2004) The COMPUTEOVERLAP function returns the number of words in common between two sets, ignoring function words or other words on a stop list. The original Lesk algorithm defines the context in a more complex way. == Criticisms == Unfortunately, Lesk’s approach is very sensitive to the exact wording of definitions, so the absence of a certain word can radically change the results. Further, the algorithm determines overlaps only among the glosses of the senses being considered. This is a significant limitation in that dictionary glosses tend to be fairly short and do not provide sufficient vocabulary to relate fine-grained sense distinctions. A lot of work has appeared offering different modifications of this algorithm. These works use other resources for analysis (thesauruses, synonyms dictionaries or morphological and syntactic models): for instance, it may use such information as synonyms, different derivatives, or words from definitions of words from definitions. == Lesk variants == Original Lesk (Lesk, 1986) Adapted/Extended Lesk (Banerjee and Pederson, 2002/2003): In the adaptive lesk algorithm, a word vector is created corresponds to every content word in the wordnet gloss. Concatenating glosses of related concepts in WordNet can be used to augment this vector. The vector contains the co-occurrence counts of words co-occurring with w in a large corpus. Adding all the word vectors for all the content words in its gloss creates the Gloss vector g for a concept. Relatedness is determined by comparing the gloss vector using the Cosine similarity measure. There are a lot of studies concerning Lesk and its extensions: Wilks and Stevenson, 1998, 1999; Mahesh et al., 1997; Cowie et al., 1992; Yarowsky, 1992; Pook and Catlett, 1988; Kilgarriff and Rosensweig, 2000; Kwong, 2001; Nastase and Szpakowicz, 2001; Gelbukh and Sidorov, 2004.

  Read more →
• 

  Phraselator

  The Phraselator is a weatherproof handheld language translation device developed by Applied Data Systems and VoxTec, a former division of the military contractor Marine Acoustics, located in Annapolis, Maryland, USA. It was designed to serve as a handheld computer device that translates English into one of 40 different languages. == The device == The Phraselator is a small speech translation PDA-sized device designed to aid in interpretation. The device does not produce synthesized speech like that utilized by Stephen Hawking; instead, it plays pre-recorded foreign language MP3 files. Users can select the phrase they wish to convey from an English list on the screen or speak into the device. It then uses speech recognition technology called DynaSpeak, developed by SRI International, to play the proper sound file. The accuracy of the speech recognition software is over 70 percent according to software developer Jack Buchanan. The device can also record replies for translation later. Pre-recorded phrases are stored on Secure Digital flash memory cards. A 128 MB card can hold up to 12,000 phrases in four or five languages. Users can download phrase modules from the official website, which contained over 300,000 phrases as of March 2005. Users can also construct their own custom phrase modules. Earlier devices were known to have run on an SA-1110 Strong Arm 206 MHz CPU with 32MB SDRAM and 32MB onboard Flash RAM. A newer model, the P2, was released in 2004 and developed according to feedback from U.S. soldiers. It translates one way from English to approximately 60 other languages. It has a directional microphone, a larger library of phrases and a longer battery life. The 2004 release was created by and utilizes a computer board manufactured by InHand Electronics, Inc. In the future, the device will be able to display pictures so users can ask questions such as "Have you seen this person?" Developer Ace Sarich notes that the device is inferior to human interpreter. Conclusions derived from a Nepal field test conducted by U.S. and Nepal based NGO Himalayan Aid in 2004 seemed to confirm Sarich's comparisons: The very concept of using a machine as a communication point between individuals seemed to actually encourage a more limited form of interaction between tester and respondent. Usually, when limited language skills are present between parties, the genuine struggle and desire to communicate acts as a display of good will – we openly display our weakness in this regard – and the result is a more relaxed and human encounter. This was not necessarily present with the Phraselator as all parties abandoned learning about each other and instead focused on learning how to work with the device. As a tool for bridging any cultural differences or communicating effectively at any length, the Phraselator would not be recommended. This device, at least in the form tested, would best be used in large-scale operations where there is no time for language training and there is a need to communicate fixed ideas, quickly, over the greatest distance by employing large amounts of unskilled users. Large humanitarian or natural disasters in remote areas of third-world countries might be an effective example. == Origin == The original idea for the device came from Lee Morin, a Navy doctor in Operation Desert Storm. To communicate with patients, he played Arabic audio files from his laptop. He informed Ace Sarich, the vice president of VoxTec, about the idea. VoxTec won a DARPA Small Business Innovation Research grant in early 2001 to develop a military-grade handheld phrase translator. During its development, the Phraselator was tested and evaluated by scientists from the Army Research Laboratory. The device was first field tested in Afghanistan in 2001. By 2002, about 500 Phraselators were built for soldiers around the world with another 250 ordered by the U.S. Special Forces. The device cost $2000 to develop and could convert spoken English into one of 200,000 recorded commands and questions in 30 languages. However, the device could only translate one-way. At the time, the only existing two-way voice translator that could convert speech back and forth between languages was the Audio Voice Translation Guide System, or TONGUES, which was developed by Carnegie Mellon University for Lockheed Martin. As part of a DARPA program known as the Spoken Language Communication and Translation System for Tactical Use, SRI International has further developed two-way translation software for use in Iraq called IraqComm in 2006 which contains a vocabulary of 40,000 English words and 50,000 words in Iraqi Arabic. == Notable users == The handheld translator was recently used by U.S. troops while providing relief to tsunami victims in early 2005. About 500 prototypes of the device were provided to U.S. military forces in Operation Enduring Freedom. Units loaded with Haitian dialects have been provided to U.S. troops in Haiti. Army military police have used it in Kandahar to communicate with POWs. In late 2004, the U.S. Navy began to augment some ships with a version of the device attached to large speakers in order to broadcast clear voice instructions up to 400 yards (370 m) away. Corrections officers and law enforcement in Oneida County, New York, have tested the device. Hospital emergency rooms and health departments have also evaluated it. Several Native American tribes such as the Choctaw Nation, the Ponca, and the Comanche Nation have also used the device to preserve their dying languages. Various law enforcement agencies, such as the Los Angeles Police Department, also use the phraselator in their patrol cars. == Awards == In March 2004, DARPA director Dr. Tony Tether presented the Small Business Innovative Research Award to the VoxTec division of Marine Acoustics at DARPATech 2004 in Anaheim, CA. The device was recently listed as one of "Ten Emerging Technologies That Will Change Your World" in MIT's Technology Review. == Pop culture == Software developer Jack Buchanan believes that building a device similar to the fictional universal translator seen in Star Trek would be harder than building the Enterprise. The device was mentioned in a list of "Top 10 Star Trek Tech" on Space.com.

  Read more →
• 

  AI Pair Programmers Reviews: What Actually Works in 2026

  Curious about the best AI pair programmer? An AI pair programmer is software that uses machine learning to help you get more done — it combines speed, accuracy, and an interface that just works. Hands-on testing shows real-world results vary, so a short free trial is the smartest way to decide. Whether you are a beginner or a pro, the right AI pair programmer slots into your workflow and pays for itself fast. Read on for hands-on impressions, pricing tiers, and the standout features that matter.

  Read more →
• 

  How to Choose an AI Website Builder

  Shopping for the best AI website builder? An AI website builder is software that uses machine learning to help you get more done — it keeps getting smarter as the underlying models improve. Pricing, accuracy, and the size of the model behind the tool are the three factors that most affect daily usefulness. Whether you are a beginner or a pro, the right AI website builder slots into your workflow and pays for itself fast. We tested the leading options and ranked them by quality, value, and ease of use.

  Read more →
• 

  Argumentation framework

  In artificial intelligence and related fields, an argumentation framework is a way to deal with contentious information and draw conclusions from it using formalized arguments. In an abstract argumentation framework, entry-level information is a set of abstract arguments that, for instance, represent data or a proposition. Conflicts between arguments are represented by a binary relation on the set of arguments. In concrete terms, an argumentation framework is represented with a directed graph such that the nodes are the arguments, and the arrows represent the attack relation. There exist some extensions of the Dung's framework, like the logic-based argumentation frameworks or the value-based argumentation frameworks. == Abstract argumentation frameworks == === Formal framework === Abstract argumentation frameworks, also called argumentation frameworks à la Dung, are defined formally as a pair: A set of abstract elements called arguments, denoted A {\displaystyle A} A binary relation on A {\displaystyle A} , called attack relation, denoted R {\displaystyle R} For instance, the argumentation system S = ⟨ A , R ⟩ {\displaystyle S=\langle A,R\rangle } with A = { a , b , c , d } {\displaystyle A=\{a,b,c,d\}} and R = { ( a , b ) , ( b , c ) , ( d , c ) } {\displaystyle R=\{(a,b),(b,c),(d,c)\}} contains four arguments ( a , b , c {\displaystyle a,b,c} and d {\displaystyle d} ) and three attacks ( a {\displaystyle a} attacks b {\displaystyle b} , b {\displaystyle b} attacks c {\displaystyle c} and d {\displaystyle d} attacks c {\displaystyle c} ). Dung defines some notions : an argument a ∈ A {\displaystyle a\in A} is acceptable with respect to E ⊆ A {\displaystyle E\subseteq A} if and only if E {\displaystyle E} defends a {\displaystyle a} , that is ∀ b ∈ A {\displaystyle \forall b\in A} such that ( b , a ) ∈ R , ∃ c ∈ E {\displaystyle (b,a)\in R,\exists c\in E} such that ( c , b ) ∈ R {\displaystyle (c,b)\in R} , a set of arguments E {\displaystyle E} is conflict-free if there is no attack between its arguments, formally : ∀ a , b ∈ E , ( a , b ) ∉ R {\displaystyle \forall a,b\in E,(a,b)\not \in R} , a set of arguments E {\displaystyle E} is admissible if and only if it is conflict-free and all its arguments are acceptable with respect to E {\displaystyle E} . === Different semantics of acceptance === ==== Extensions ==== To decide if an argument can be accepted or not, or if several arguments can be accepted together, Dung defines several semantics of acceptance that allows, given an argumentation system, sets of arguments (called extensions) to be computed. For instance, given S = ⟨ A , R ⟩ {\displaystyle S=\langle A,R\rangle } , E {\displaystyle E} is a complete extension of S {\displaystyle S} only if it is an admissible set and every acceptable argument with respect to E {\displaystyle E} belongs to E {\displaystyle E} , E {\displaystyle E} is a preferred extension of S {\displaystyle S} only if it is a maximal element (with respect to the set-theoretical inclusion) among the admissible sets with respect to S {\displaystyle S} , E {\displaystyle E} is a stable extension of S {\displaystyle S} only if it is a conflict-free set that attacks every argument that does not belong in E {\displaystyle E} (formally, ∀ a ∈ A ∖ E , ∃ b ∈ E {\displaystyle \forall a\in A\backslash E,\exists b\in E} such that ( b , a ) ∈ R {\displaystyle (b,a)\in R} , E {\displaystyle E} is the (unique) grounded extension of S {\displaystyle S} only if it is the smallest element (with respect to set inclusion) among the complete extensions of S {\displaystyle S} . There exists some inclusions between the sets of extensions built with these semantics : Every stable extension is preferred, Every preferred extension is complete, The grounded extension is complete, If the system is well-founded (there exists no infinite sequence a 0 , a 1 , … , a n , … {\displaystyle a_{0},a_{1},\dots ,a_{n},\dots } such that ∀ i > 0 , ( a i + 1 , a i ) ∈ R {\displaystyle \forall i>0,(a_{i+1},a_{i})\in R} ), all these semantics coincide—only one extension is grounded, stable, preferred, and complete. Some other semantics have been defined. One introduce the notation E x t σ ( S ) {\displaystyle Ext_{\sigma }(S)} to note the set of σ {\displaystyle \sigma } -extensions of the system S {\displaystyle S} . In the case of the system S {\displaystyle S} in the figure above, E x t σ ( S ) = { { a , d } } {\displaystyle Ext_{\sigma }(S)=\{\{a,d\}\}} for every Dung's semantic—the system is well-founded. That explains why the semantics coincide, and the accepted arguments are: a {\displaystyle a} and d {\displaystyle d} . ==== Labellings ==== Labellings are a more expressive way than extensions to express the acceptance of the arguments. Concretely, a labelling is a mapping that associates every argument with a label in (the argument is accepted), out (the argument is rejected), or undec (the argument is undefined—not accepted or refused). One can also note a labelling as a set of pairs ( a r g u m e n t , l a b e l ) {\displaystyle ({\mathit {argument}},{\mathit {label}})} . Such a mapping does not make sense without additional constraint. The notion of reinstatement labelling guarantees the sense of the mapping. L {\displaystyle L} is a reinstatement labelling on the system S = ⟨ A , R ⟩ {\displaystyle S=\langle A,R\rangle } if and only if : ∀ a ∈ A , L ( a ) = i n {\displaystyle \forall a\in A,L(a)={\mathit {in}}} if and only if ∀ b ∈ A {\displaystyle \forall b\in A} such that ( b , a ) ∈ R , L ( b ) = o u t {\displaystyle (b,a)\in R,L(b)={\mathit {out}}} ∀ a ∈ A , L ( a ) = o u t {\displaystyle \forall a\in A,L(a)={\mathit {out}}} if and only if ∃ b ∈ A {\displaystyle \exists b\in A} such that ( b , a ) ∈ R {\displaystyle (b,a)\in R} and L ( b ) = i n {\displaystyle L(b)={\mathit {in}}} ∀ a ∈ A , L ( a ) = u n d e c {\displaystyle \forall a\in A,L(a)={\mathit {undec}}} if and only if L ( a ) ≠ i n {\displaystyle L(a)\neq {\mathit {in}}} and L ( a ) ≠ o u t {\displaystyle L(a)\neq {\mathit {out}}} One can convert every extension into a reinstatement labelling: the arguments of the extension are in, those attacked by an argument of the extension are out, and the others are undec. Conversely, one can build an extension from a reinstatement labelling just by keeping the arguments in. Indeed, Caminada proved that the reinstatement labellings and the complete extensions can be mapped in a bijective way. Moreover, the other Datung's semantics can be associated to some particular sets of reinstatement labellings. Reinstatement labellings distinguish arguments not accepted because they are attacked by accepted arguments from undefined arguments—that is, those that are not defended cannot defend themselves. An argument is undec if it is attacked by at least another undec. If it is attacked only by arguments out, it must be in, and if it is attacked some argument in, then it is out. The unique reinstatement labelling that corresponds to the system S {\displaystyle S} above is L = { ( a , i n ) , ( b , o u t ) , ( c , o u t ) , ( d , i n ) } {\displaystyle L=\{(a,{\mathit {in}}),(b,{\mathit {out}}),(c,{\mathit {out}}),(d,{\mathit {in}})\}} . === Inference from an argumentation system === In the general case when several extensions are computed for a given semantic σ {\displaystyle \sigma } , the agent that reasons from the system can use several mechanisms to infer information: Credulous inference: the agent accepts an argument if it belongs to at least one of the σ {\displaystyle \sigma } -extensions—in which case, the agent risks accepting some arguments that are not acceptable together ( a {\displaystyle a} attacks b {\displaystyle b} , and a {\displaystyle a} and b {\displaystyle b} each belongs to an extension) Skeptical inference: the agent accepts an argument only if it belongs to every σ {\displaystyle \sigma } -extension. In this case, the agent risks deducing too little information (if the intersection of the extensions is empty or has a very small cardinal). For these two methods to infer information, one can identify the set of accepted arguments, respectively C r σ ( S ) {\displaystyle Cr_{\sigma }(S)} the set of the arguments credulously accepted under the semantic σ {\displaystyle \sigma } , and S c σ ( S ) {\displaystyle Sc_{\sigma }(S)} the set of arguments accepted skeptically under the semantic σ {\displaystyle \sigma } (the σ {\displaystyle \sigma } can be missed if there is no possible ambiguity about the semantic). Of course, when there is only one extension (for instance, when the system is well-founded), this problem is very simple: the agent accepts arguments of the unique extension and rejects others. The same reasoning can be done with labellings that correspond to the chosen semantic : an argument can be accepted if it is in for each labelling and refused if it is out for each labelling, the others being in an undecided state (the status of the arguments can remind the

  Read more →
• 

  METEO System

  The METEO System is a machine translation system specifically designed for the translation of the weather forecasts issued daily by Environment Canada. The system was used from 1981 to 30 September 2001 by Environment Canada to translate forecasts issued in French in the province of Quebec into English and those issued in English in other Canadian provinces into French. Since then, a competitor program has replaced METEO System after an open governmental bid. The system was developed by John Chandioux and was often mentioned as one of the few success stories in the field of machine translation. == History == The METEO System was in operational use at Environment Canada from 1982 to 2001. It stems from a prototype developed in 1975–76 by the TAUM Group, known as TAUM-METEO. The initial motivation to develop that prototype was that a junior translator came to TAUM to ask for help in translating weather bulletins at Environment Canada. Since all official communications emanating from the Canadian government must be available in French and English, because of the Official Languages Act of 1969, and weather bulletins represent a large amount of translation in real time, junior translators had to spend several months producing first draft translations, which were then revised by seniors. That was a difficult and tedious job, because of the specificities of the English and French sublanguages used, and not very rewarding, as the lifetime of a bulletin is only 4 hours. TAUM proposed to build a prototype MT system, and Environment Canada agreed to fund the project. A prototype was ready after a few months, with basic integration in the workflow of translation (source and target bulletins travelled over telex lines at the time and MT happened on a mainframe computer). The first version of the system (METEO 1) went into operation on a Control Data CDC 7600 supercomputer in March 1977. Chandioux then left the TAUM group to manage its operation and improve it, while the TAUM group embarked on a different project (TAUM-aviation, 1977–81). Benoit Thouin made improvements to the initial prototype over the subsequent year, and turned it into an operational system. After three years, METEO 1 had demonstrated the feasibility of microcomputer-based machine translation to the satisfaction of the Canadian government's Translation Bureau of Public Works and Government Services Canada. METEO 1 was formally adopted in 1981, replacing the junior translators in the workflow. Because of the need for high-quality translation, the revision step, done by senior translators, was maintained. The quality, measured as the percentage of edit operations (inserting or deleting a word counts as 1, replacing as 2) on the MT results, reached 85% in 1985. Until that time, the MT part was still implemented as a sequence of Q-systems. The Q-systems formalism is a rule-based SLLP (Specialized Language for Linguistic Programming) invented by Alain Colmerauer in 1967 as he was a postdoc coopérant at the TAUM group. He later invented the Prolog language in 1972 after returning to France and becoming a university professor in Marseille-Luminy. As the engine of the Q-systems is highly non-deterministic, and the manipulated data structures are in some ways too simple, without any types such as string or number, Chandioux encountered limitations in his efforts to raise translation quality and lower computation time to the point he could run it on microcomputers. In 1981, Chandioux created a new SLLP, or metalanguage for linguistic applications, based on the same basic algorithmic ideas as the Q-systems, but more deterministic, and offering typed labels on tree nodes. Following the advice of Bernard Vauquois and Colmerauer, he created GramR, and developed it for microcomputers. In 1982, he could start developing in GramR a new system for translating the weather bulletins on a high-end Cromemco microcomputer. METEO 2 went into operation in 1983. The software then ran in 48Kb of central memory with a 5Mb hard disk for paging. METEO 2 was the first MT application to run on a microcomputer. In 1985, the system had nothing left of the initial prototype, and was officially renamed METEO. It translated about 20 million words per year from English into French, and 10 million words from French into English, with a quality of 97%. Typically, it took 4 minutes for a bulletin in English to be sent from Winnipeg and come back in French after MT and human revision. In 1996, Chandioux developed a special version of his system (METEO 96) which was used to translate the weather forecasts (different kinds of bulletins) issued by the US National Weather Service during the 1996 Summer Olympics in Atlanta. The last known version of the system, METEO 5, dates from 1997 and ran on an IBM PC network under Windows NT. It translated 10 pages per second, but was able to fit into a 1.44Mb floppy disk.

  Read more →
• 

  Ancient text corpora

  Ancient text corpora are the entire collection of texts from the period of ancient history, defined in this article as the period from the beginning of writing up to 300 AD. These corpora are important for the study of literature, history, linguistics, and other fields, and are a fundamental component of the world's cultural heritage. Chinese, Latin, and Greek are examples of ancient languages with significant text corpora, although much of these corpora are known to us via transmission (frequently via medieval manuscript copies) rather than in their original form. These texts – both transmitted and original – provide valuable insights into the history and culture of different regions of the world, and have been studied for centuries by scholars and researchers. Other ancient texts – particularly stone inscriptions and papyrus scrolls – have been published following archaeological research, notably the cuneiform corpus of c.10 million words and the c.5 million words in ancient Egyptian. Through advances in technology and digitization, ancient text corpora are more accessible than ever before. Tools such as the Perseus Digital Library and the Digital Corpus of Sanskrit have made it easier for researchers to access and analyze these texts. == Quantifying the corpora == Two types of ancient texts are known to modern scholars – those that have only survived in younger manuscripts, but whose great age is undisputed (this applies to the bulk of the Chinese, Brahmi, Greek, Latin, Hebrew and Avestan tradition), and those known from original inscriptions, papyri and other manuscripts. Counting of the words in each corpus presents significant methodological challenges – in principle, every single occurrence of a word in the text is counted separately, but in the case of parallel transmission of literary texts, only a single transmission is taken into account. Just as the Book of the Dead and the coffin texts are only included once in the number given for the Egyptian, the Greek and Latin literary works should only be counted according to one manuscript. If, on the other hand, tombs, royal inscriptions or economic documents of certain ancient languages often show a more or less identical form, this is not evaluated as a purely "parallel tradition". Attached prepositions are counted as separate words, except in the case of the definite article in Hebrew, Aramaic and Greek since it has no equivalent in most languages, so its frequency would significantly affect the comparability of numbers. === Languages with known size estimates === === South Asian === Sanskrit (Vedic Sanskrit and Classical Sanskrit) Indus script (3,800 items, c.20,000 characters) Brahmi script Old Tamil Early Indian epigraphy and Indian epic poetry Kharosthi Pali literature List of historic Indian texts === Mesoamerican === Olmec hieroglyphs Maya script === East Asian === Old Chinese Chinese classics The pre-Qin corpus: a collection of ancient Chinese texts written before the Qin dynasty (221 BCE). The corpus includes texts from Confucianism, Taoism, Legalism, and other schools of thought. The pre-Han corpus: a collection of ancient Chinese texts written before the Han dynasty (202 BCE). The corpus includes texts from Confucianism, Taoism, Legalism, and other schools of thought. See the Chinese Text Project Chinese bronze inscriptions, Oracle bone script, Seal script, Clerical script === Central Iranian languages === Prior to 300 AD, the Central Iranian languages are mainly in the form of Sassanid stone inscriptions in the two closely related idioms Middle Persian (Pahlavi scripts and Inscriptional Parthian), there are 5000 for the corpus of Middle Persian (mostly 3rd, but also 4th/5th centuries) and for the corpus of Parthian (3rd century) 3000 words. To what extent some of the Manichaean Middle Persian literary texts may date back to the 3rd century is difficult to estimate; Mani is said to have personally written the Shabuhragan totaling about 5000 words. In any case, if we combine Middle Persian and Parthian, we come to over 10,000 words. === Proto-Sinaitic === Proto-Sinaitic script has no more than about 400 letters (number of words is unknown since the script has not been fully interpreted). To a similar extent, there are probably approximately contemporaneous Proto-Canaanite inscriptions (ibid.). === Anatolian === Luwian cuneiform, approx. 3000 words the Palaic language few hundred words. Hieroglyphic Luwian the Lycian alphabet (the best attested Anatolian successor language written in alphabetic script) with about 5000 words The Lydian alphabet 109 inscriptions comprising about 1500 words The Phrygian alphabet the in-tomb inscriptions from the 2nd and 3rd centuries AD (approx. 1000 words) and in the so-called "old Phrygian" inscriptions less than 300 words The Carian alphabets whose texts, mainly from Egypt, contain around 600 words. === Old Italic === the Umbrian language attested essentially by the sacrificial instructions of the Iguvinian Tables with 5000 words the Oscan language (ibid.) with 2000 words the Messapic language with probably a good 1000 words (the estimate is difficult because most texts in this hardly understandable language do not use word separators) the Venetic language a few hundred words the Faliscan language a few hundred words Cisalpine Celtic inscriptions amount to approximately 2000 words, to which are added a number of glosses by classical authors === Iberia === Iberian scripts, more rarely written in Greek or Latin script, approx. 2500 words Celtiberian script, which refers to Celtic language testimonies in Iberian, but also in Latin script from Spain (approx. 1000 words) Southwest Paleohispanic script, 78 inscriptions, a few hundred words Lusitanian language, three monuments in Latin script, approx. 60 words === Germanic Northern Europe === Runic inscriptions dated before the 4th century amount to about 30 pieces, which contain no more than 50 words in total === Africa === Geʽez script: comparatively few inscriptions with a total of around 1,000 words before 300 AD. Following Christianization in the 4th century, more extensive texts are known. Libyco-Berber alphabet: over 1,000 inscriptions from the Maghreb, which are dated to Roman times. Most texts do not use a word separator; Peust estimates that the total number of words could be around 5,000 Meroitic script (Ancient Nubian): about 900 texts are known, which Peust estimates may contain approximately 10,000 words, albeit with uncertainty from the fact that the word separator is not used consistently in the Meroitic script. === Aegean === The Cretan Linear A inscriptions that have not yet been deciphered are available in about 2500 texts, which contain a total of around 20,000 characters. The total number of words can hardly be determined; Peust tentatively put it in the same order of magnitude as in Meroitic. In addition to the Linear A texts, there are also inscriptions Cretan hieroglyphs of a few hundred characters and texts written in the Greek alphabet, but not in Greek, with a few dozen words Cypriot syllabary in the first millennium BC, in which mostly Greek texts were recorded. The relevant texts comprise around 100 to 200 words. === Micro corpora === There are a significant number of ancient micro-corpus languages. Estimating the total number of attested ancient languages may be as difficult as estimating their corpus size. For example, Greek and Latin sources hand down an enormous amount of foreign-language glosses, the seriousness of which is not always certain. == Preservation and curation == Historic preservation and maintaining ancient text corpora presents several challenges, including issues with preservation, translation, and digitization. Many ancient texts have been lost over time, and those that survive may be damaged or fragmented. Translating ancient languages and scripts requires specialized expertise, and digitizing texts can be time-consuming and resource-intensive. == Corpus linguistics == The field of corpus linguistics studies language as expressed in text corpora. This includes the analysis of word frequency, collocations, grammar, and semantics. Ancient text corpora provide a valuable resource for corpus linguistics research, enabling scholars to explore the evolution of language and culture over time.

  Read more →
• 

  Maximum-entropy Markov model

  In statistics, a maximum-entropy Markov model (MEMM), or conditional Markov model (CMM), is a graphical model for sequence labeling that combines features of hidden Markov models (HMMs) and maximum entropy (MaxEnt) models. An MEMM is a discriminative model that extends a standard maximum entropy classifier by assuming that the unknown values to be learnt are connected in a Markov chain rather than being conditionally independent of each other. MEMMs find applications in natural language processing, specifically in part-of-speech tagging and information extraction. == Model == Suppose we have a sequence of observations O 1 , … , O n {\displaystyle O_{1},\dots ,O_{n}} that we seek to tag with the labels S 1 , … , S n {\displaystyle S_{1},\dots ,S_{n}} that maximize the conditional probability P ( S 1 , … , S n ∣ O 1 , … , O n ) {\displaystyle P(S_{1},\dots ,S_{n}\mid O_{1},\dots ,O_{n})} . In a MEMM, this probability is factored into Markov transition probabilities, where the probability of transitioning to a particular label depends only on the observation at that position and the previous position's label: P ( S 1 , … , S n ∣ O 1 , … , O n ) = ∏ t = 1 n P ( S t ∣ S t − 1 , O t ) . {\displaystyle P(S_{1},\dots ,S_{n}\mid O_{1},\dots ,O_{n})=\prod _{t=1}^{n}P(S_{t}\mid S_{t-1},O_{t}).} Each of these transition probabilities comes from the same general distribution P ( s ∣ s ′ , o ) {\displaystyle P(s\mid s',o)} . For each possible label value of the previous label s ′ {\displaystyle s'} , the probability of a certain label s {\displaystyle s} is modeled in the same way as a maximum entropy classifier: P ( s ∣ s ′ , o ) = P s ′ ( s ∣ o ) = 1 Z ( o , s ′ ) exp ⁡ ( ∑ a λ a f a ( o , s ) ) . {\displaystyle P(s\mid s',o)=P_{s'}(s\mid o)={\frac {1}{Z(o,s')}}\exp \left(\sum _{a}\lambda _{a}f_{a}(o,s)\right).} Here, the f a ( o , s ) {\displaystyle f_{a}(o,s)} are real-valued or categorical feature-functions, and Z ( o , s ′ ) {\displaystyle Z(o,s')} is a normalization term ensuring that the distribution sums to one. This form for the distribution corresponds to the maximum entropy probability distribution satisfying the constraint that the empirical expectation for the feature is equal to the expectation given the model: E e ⁡ [ f a ( o , s ) ] = E p ⁡ [ f a ( o , s ) ] for all a . {\displaystyle \operatorname {E} _{e}\left[f_{a}(o,s)\right]=\operatorname {E} _{p}\left[f_{a}(o,s)\right]\quad {\text{ for all }}a.} The parameters λ a {\displaystyle \lambda _{a}} can be estimated using generalized iterative scaling. Furthermore, a variant of the Baum–Welch algorithm, which is used for training HMMs, can be used to estimate parameters when training data has incomplete or missing labels. The optimal state sequence S 1 , … , S n {\displaystyle S_{1},\dots ,S_{n}} can be found using a very similar Viterbi algorithm to the one used for HMMs. The dynamic program uses the forward probability: α t + 1 ( s ) = ∑ s ′ ∈ S α t ( s ′ ) P s ′ ( s ∣ o t + 1 ) . {\displaystyle \alpha _{t+1}(s)=\sum _{s'\in S}\alpha _{t}(s')P_{s'}(s\mid o_{t+1}).} == Strengths and weaknesses == An advantage of MEMMs rather than HMMs for sequence tagging is that they offer increased freedom in choosing features to represent observations. In sequence tagging situations, it is useful to use domain knowledge to design special-purpose features. In the original paper introducing MEMMs, the authors write that "when trying to extract previously unseen company names from a newswire article, the identity of a word alone is not very predictive; however, knowing that the word is capitalized, that is a noun, that it is used in an appositive, and that it appears near the top of the article would all be quite predictive (in conjunction with the context provided by the state-transition structure)." Useful sequence tagging features, such as these, are often non-independent. Maximum entropy models do not assume independence between features, but generative observation models used in HMMs do. Therefore, MEMMs allow the user to specify many correlated, but informative features. Another advantage of MEMMs versus HMMs and conditional random fields (CRFs) is that training can be considerably more efficient. In HMMs and CRFs, one needs to use some version of the forward–backward algorithm as an inner loop in training. However, in MEMMs, estimating the parameters of the maximum-entropy distributions used for the transition probabilities can be done for each transition distribution in isolation. A drawback of MEMMs is that they potentially suffer from the "label bias problem," where states with low-entropy transition distributions "effectively ignore their observations." Conditional random fields were designed to overcome this weakness, which had already been recognised in the context of neural network-based Markov models in the early 1990s. Another source of label bias is that training is always done with respect to known previous tags, so the model struggles at test time when there is uncertainty in the previous tag.

  Read more →
• 

  Kernel density estimation

  In statistics, kernel density estimation (KDE) is the application of kernel smoothing for probability density estimation, i.e., a non-parametric method to estimate the probability density function of a random variable based on kernels as weights. KDE answers a fundamental data smoothing problem where inferences about the population are made based on a finite data sample. In some fields such as signal processing and econometrics it is also termed the Parzen–Rosenblatt window method, after Emanuel Parzen and Murray Rosenblatt, who are usually credited with independently creating it in its current form. One of the famous applications of kernel density estimation is in estimating the class-conditional marginal densities of data when using a naive Bayes classifier, which can improve its prediction accuracy. == Definition == Let x = ( x 1 , x 2 , x 3 , . . . ) {\displaystyle \mathbf {x} =\left(x_{1},x_{2},x_{3},...\right)} be independent and identically distributed samples drawn from some univariate distribution with an unknown density f at any given point x. We are interested in estimating the shape of this function f. Its kernel density estimator is f ^ h ( x ) = 1 n ∑ i = 1 n K h ( x − x i ) = 1 n h ∑ i = 1 n K ( x − x i h ) , {\displaystyle {\hat {f}}_{h}(x)={\frac {1}{n}}\sum _{i=1}^{n}K_{h}(x-x_{i})={\frac {1}{nh}}\sum _{i=1}^{n}K{\left({\frac {x-x_{i}}{h}}\right)},} where K is the kernel — a non-negative function — and h > 0 is a smoothing parameter called the bandwidth or simply width. A kernel with subscript h is called the scaled kernel and defined as Kh(x) = ⁠1/h⁠ K(⁠x/h⁠). Intuitively one wants to choose h as small as the data will allow; however, there is always a trade-off between the bias of the estimator and its variance. The choice of bandwidth is discussed in more detail below. A range of kernel functions are commonly used: uniform, triangular, biweight, triweight, Epanechnikov (parabolic), normal, and others. The Epanechnikov kernel is optimal in a mean square error sense, though the loss of efficiency is small for the kernels listed previously. Due to its convenient mathematical properties, the normal kernel is often used, which means K(x) = ϕ(x), where ϕ is the standard normal density function. The kernel density estimator then becomes f ^ h ( x ) = 1 n ∑ i = 1 n 1 h 2 π exp ⁡ ( − ( x − x i ) 2 2 h 2 ) , {\displaystyle {\hat {f}}_{h}(x)={\frac {1}{n}}\sum _{i=1}^{n}{\frac {1}{h{\sqrt {2\pi }}}}\exp \left({\frac {-(x-x_{i})^{2}}{2h^{2}}}\right),} where h {\displaystyle h} is the standard deviation of the sample x {\displaystyle \mathbf {x} } . The construction of a kernel density estimate finds interpretations in fields outside of density estimation. For example, in thermodynamics, this is equivalent to the amount of heat generated when heat kernels (the fundamental solution to the heat equation) are placed at each data point locations xi. Similar methods are used to construct discrete Laplace operators on point clouds for manifold learning (e.g. diffusion map). == Example == Kernel density estimates are closely related to histograms, but can be endowed with properties such as smoothness or continuity by using a suitable kernel. The diagram below based on these 6 data points illustrates this relationship: For the histogram, first, the horizontal axis is divided into sub-intervals or bins which cover the range of the data: In this case, six bins each of width 2. Whenever a data point falls inside this interval, a box of height 1/12 is placed there. If more than one data point falls inside the same bin, the boxes are stacked on top of each other. For the kernel density estimate, normal kernels with a standard deviation of 1.5 (indicated by the red dashed lines) are placed on each of the data points xi. The kernels are summed to make the kernel density estimate (solid blue curve). The smoothness of the kernel density estimate (compared to the discreteness of the histogram) illustrates how kernel density estimates converge faster to the true underlying density for continuous random variables. == Bandwidth selection == The bandwidth of the kernel is a free parameter which exhibits a strong influence on the resulting estimate. To illustrate its effect, we take a simulated random sample from the standard normal distribution (plotted at the blue spikes in the rug plot on the horizontal axis). The grey curve is the true density (a normal density with mean 0 and variance 1). In comparison, the red curve is undersmoothed since it contains too many spurious data artifacts arising from using a bandwidth h = 0.05, which is too small. The green curve is oversmoothed since using the bandwidth h = 2 obscures much of the underlying structure. The black curve with a bandwidth of h = 0.337 is considered to be optimally smoothed since its density estimate is close to the true density. An extreme situation is encountered in the limit h → 0 {\displaystyle h\to 0} (no smoothing), where the estimate is a sum of n delta functions centered at the coordinates of analyzed samples. In the other extreme limit h → ∞ {\displaystyle h\to \infty } the estimate retains the shape of the used kernel, centered on the mean of the samples (completely smooth). The most common optimality criterion used to select this parameter is the expected L2 risk function, also termed the mean integrated squared error: MISE ⁡ ( h ) = E [ ∫ ( f ^ h ( x ) − f ( x ) ) 2 d x ] {\displaystyle \operatorname {MISE} (h)=\operatorname {E} \!\left[\int \!{\left({\hat {f}}\!_{h}(x)-f(x)\right)}^{2}dx\right]} Under weak assumptions on f and K, (f is the, generally unknown, real density function), MISE ⁡ ( h ) = AMISE ⁡ ( h ) + o ( ( n h ) − 1 + h 4 ) {\displaystyle \operatorname {MISE} (h)=\operatorname {AMISE} (h)+{\mathcal {o}}{\left((nh)^{-1}+h^{4}\right)}} where o is the little o notation, and n the sample size (as above). The AMISE is the asymptotic MISE, i. e. the two leading terms, AMISE ⁡ ( h ) = R ( K ) n h + 1 4 m 2 ( K ) 2 h 4 R ( f ″ ) {\displaystyle \operatorname {AMISE} (h)={\frac {R(K)}{nh}}+{\frac {1}{4}}m_{2}(K)^{2}h^{4}R(f'')} where R ( g ) = ∫ g ( x ) 2 d x {\textstyle R(g)=\int g(x)^{2}\,dx} for a function g, m 2 ( K ) = ∫ x 2 K ( x ) d x {\textstyle m_{2}(K)=\int x^{2}K(x)\,dx} and f ″ {\displaystyle f''} is the second derivative of f {\displaystyle f} and K {\displaystyle K} is the kernel. The minimum of this AMISE is the solution to this differential equation ∂ ∂ h AMISE ⁡ ( h ) = − R ( K ) n h 2 + m 2 ( K ) 2 h 3 R ( f ″ ) = 0 {\displaystyle {\frac {\partial }{\partial h}}\operatorname {AMISE} (h)=-{\frac {R(K)}{nh^{2}}}+m_{2}(K)^{2}h^{3}R(f'')=0} or h AMISE = R ( K ) 1 / 5 m 2 ( K ) 2 / 5 R ( f ″ ) 1 / 5 n − 1 / 5 = C n − 1 / 5 {\displaystyle h_{\operatorname {AMISE} }={\frac {R(K)^{1/5}}{m_{2}(K)^{2/5}R(f'')^{1/5}}}n^{-1/5}=Cn^{-1/5}} Neither the AMISE nor the hAMISE formulas can be used directly since they involve the unknown density function f {\displaystyle f} or its second derivative f ″ {\displaystyle f''} . To overcome that difficulty, a variety of automatic, data-based methods have been developed to select the bandwidth. Several review studies have been undertaken to compare their efficacies, with the general consensus that the plug-in selectors and cross validation selectors are the most useful over a wide range of data sets. Substituting any bandwidth h which has the same asymptotic order n−1/5 as hAMISE into the AMISE gives that AMISE(h) = O(n−4/5), where O is the big O notation. It can be shown that, under weak assumptions, there cannot exist a non-parametric estimator that converges at a faster rate than the kernel estimator. Note that the n−4/5 rate is slower than the typical n−1 convergence rate of parametric methods. If the bandwidth is not held fixed, but is varied depending upon the location of either the estimate (balloon estimator) or the samples (pointwise estimator), this produces a particularly powerful method termed adaptive or variable bandwidth kernel density estimation. Bandwidth selection for kernel density estimation of heavy-tailed distributions is relatively difficult. === A rule-of-thumb bandwidth estimator === If Gaussian basis functions are used to approximate univariate data, and the underlying density being estimated is Gaussian, the optimal choice for h (that is, the bandwidth that minimises the mean integrated squared error) is: h = ( 4 σ ^ 5 3 n ) 1 / 5 ≈ 1.06 σ ^ n − 1 / 5 , {\displaystyle h={\left({\frac {4{\hat {\sigma }}^{5}}{3n}}\right)}^{1/5}\approx 1.06\,{\hat {\sigma }}\,n^{-1/5},} An h {\displaystyle h} value is considered more robust when it improves the fit for long-tailed and skewed distributions or for bimodal mixture distributions. This is often done empirically by replacing the standard deviation σ ^ {\displaystyle {\hat {\sigma }}} by the parameter A {\displaystyle A} below: A = min ( σ ^ , I Q R 1.34 ) {\displaystyle A=\min \left({\hat {\sigma }},{\frac {\mathrm {IQR} }{1.34}}\right)} where IQR is the

  Read more →
• 

  Hebbian theory

  Hebbian theory is a neuropsychological theory claiming that an increase in synaptic efficacy arises from a presynaptic cell's repeated and persistent stimulation of a postsynaptic cell. It is an attempt to explain synaptic plasticity, the adaptation of neurons during the learning process. Hebbian theory was introduced by Donald Hebb in his 1949 book The Organization of Behavior. The theory is also called Hebb's rule, Hebb's law, Hebb's postulate, and cell assembly theory. Hebb states it as follows: Let us assume that the persistence or repetition of a reverberatory activity (or "trace") tends to induce lasting cellular changes that add to its stability. ... When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased. The theory is often summarized as "Neurons that fire together, wire together." However, Hebb emphasized that cell A needs to "take part in firing" cell B, and such causality can occur only if cell A fires just before, not at the same time as, cell B. This aspect of causation in Hebb's work foreshadowed what is now known about spike-timing-dependent plasticity, which requires temporal precedence. Hebbian theory attempts to explain associative or Hebbian learning, in which simultaneous activation of cells leads to pronounced increases in synaptic strength between those cells. It also provides a biological basis for errorless learning methods for education and memory rehabilitation. In the study of neural networks in cognitive function, it is often regarded as the neuronal basis of unsupervised learning. == Engrams, cell assembly theory, and learning == Hebbian theory provides an explanation for how neurons might connect to become engrams, which may be stored in overlapping cell assemblies, or groups of neurons that encode specific information. Initially created as a way to explain recurrent activity in specific groups of cortical neurons, Hebb's theories on the form and function of cell assemblies can be understood from the following: The general idea is an old one, that any two cells or systems of cells that are repeatedly active at the same time will tend to become 'associated' so that activity in one facilitates activity in the other. Hebb also wrote: When one cell repeatedly assists in firing another, the axon of the first cell develops synaptic knobs (or enlarges them if they already exist) in contact with the soma of the second cell. D. Alan Allport posits additional ideas regarding cell assembly theory and its role in forming engrams using the concept of auto-association, or the brain's ability to retrieve information based on a partial cue, described as follows: If the inputs to a system cause the same pattern of activity to occur repeatedly, the set of active elements constituting that pattern will become increasingly strongly inter-associated. That is, each element will tend to turn on every other element and (with negative weights) to turn off the elements that do not form part of the pattern. To put it another way, the pattern as a whole will become 'auto-associated'. We may call a learned (auto-associated) pattern an engram. Research conducted in the laboratory of Nobel laureate Eric Kandel has provided evidence supporting the role of Hebbian learning mechanisms at synapses in the marine gastropod Aplysia californica. Because synapses in the peripheral nervous system of marine invertebrates are much easier to control in experiments, Kandel's research found that Hebbian long-term potentiation along with activity-dependent presynaptic facilitation are both necessary for synaptic plasticity and classical conditioning in Aplysia californica. While research on invertebrates has established fundamental mechanisms of learning and memory, much of the work on long-lasting synaptic changes between vertebrate neurons involves the use of non-physiological experimental stimulation of brain cells. However, some of the physiologically relevant synapse modification mechanisms that have been studied in vertebrate brains do seem to be examples of Hebbian processes. One such review indicates that long-lasting changes in synaptic strengths can be induced by physiologically relevant synaptic activity using both Hebbian and non-Hebbian mechanisms. == Principles == In artificial neurons and artificial neural networks, Hebb's principle can be described as a method of determining how to alter the weights between model neurons. The weight between two neurons increases if the two neurons activate simultaneously, and reduces if they activate separately. Nodes that tend to be either both positive or both negative at the same time have strong positive weights, while those that tend to be opposite have strong negative weights. The following is a formulaic description of Hebbian learning (many other descriptions are possible): w i j = x i x j , {\displaystyle \,w_{ij}=x_{i}x_{j},} where w i j {\displaystyle w_{ij}} is the weight of the connection from neuron j {\displaystyle j} to neuron i {\displaystyle i} , and x i {\displaystyle x_{i}} is the input for neuron i {\displaystyle i} . This is an example of pattern learning, where weights are updated after every training example. In a Hopfield network, connections w i j {\displaystyle w_{ij}} are set to zero if i = j {\displaystyle i=j} (no reflexive connections allowed). With binary neurons (activations either 0 or 1), connections would be set to 1 if the connected neurons have the same activation for a pattern. When several training patterns are used, the expression becomes an average of the individuals: w i j = 1 p ∑ k = 1 p x i k x j k , {\displaystyle w_{ij}={\frac {1}{p}}\sum _{k=1}^{p}x_{i}^{k}x_{j}^{k},} where w i j {\displaystyle w_{ij}} is the weight of the connection from neuron j {\displaystyle j} to neuron i {\displaystyle i} , p {\displaystyle p} is the number of training patterns and x i k {\displaystyle x_{i}^{k}} the k {\displaystyle k} -th input for neuron i {\displaystyle i} . This is learning by epoch, with weights updated after all the training examples are presented and is last term applicable to both discrete and continuous training sets. Again, in a Hopfield network, connections w i j {\displaystyle w_{ij}} are set to zero if i = j {\displaystyle i=j} (no reflexive connections). A variation of Hebbian learning that takes into account phenomena such as blocking and other neural learning phenomena is the mathematical model of Harry Klopf. Klopf's model assumes that parts of a system with simple adaptive mechanisms can underlie more complex systems with more advanced adaptive behavior, such as neural networks. == Relationship to unsupervised learning, stability, and generalization == Because of the simple nature of Hebbian learning, based only on the coincidence of pre- and post-synaptic activity, it may not be intuitively clear why this form of plasticity leads to meaningful learning. However, it can be shown that Hebbian plasticity does pick up the statistical properties of the input in a way that can be categorized as unsupervised learning. This can be mathematically shown in a simplified example. Let us work under the simplifying assumption of a single rate-based neuron of rate y ( t ) {\displaystyle y(t)} , whose inputs have rates x 1 ( t ) . . . x N ( t ) {\displaystyle x_{1}(t)...x_{N}(t)} . The response of the neuron y ( t ) {\displaystyle y(t)} is usually described as a linear combination of its input, ∑ i w i x i {\displaystyle \sum _{i}w_{i}x_{i}} , followed by a response function f {\displaystyle f} : y = f ( ∑ i = 1 N w i x i ) . {\displaystyle y=f\left(\sum _{i=1}^{N}w_{i}x_{i}\right).} As defined in the previous sections, Hebbian plasticity describes the evolution in time of the synaptic weight w {\displaystyle w} : d w i d t = η x i y . {\displaystyle {\frac {dw_{i}}{dt}}=\eta x_{i}y.} Assuming, for simplicity, an identity response function f ( a ) = a {\displaystyle f(a)=a} , we can write d w i d t = η x i ∑ j = 1 N w j x j {\displaystyle {\frac {dw_{i}}{dt}}=\eta x_{i}\sum _{j=1}^{N}w_{j}x_{j}} or in matrix form: d w d t = η x x T w . {\displaystyle {\frac {d\mathbf {w} }{dt}}=\eta \mathbf {x} \mathbf {x} ^{T}\mathbf {w} .} As in the previous chapter, if training by epoch is done an average ⟨ … ⟩ {\displaystyle \langle \dots \rangle } over discrete or continuous (time) training set of x {\displaystyle \mathbf {x} } can be done: d w d t = ⟨ η x x T w ⟩ = η ⟨ x x T ⟩ w = η C w . {\displaystyle {\frac {d\mathbf {w} }{dt}}=\langle \eta \mathbf {x} \mathbf {x} ^{T}\mathbf {w} \rangle =\eta \langle \mathbf {x} \mathbf {x} ^{T}\rangle \mathbf {w} =\eta C\mathbf {w} .} where C = ⟨ x x T ⟩ {\displaystyle C=\langle \,\mathbf {x} \mathbf {x} ^{T}\rangle } is the correlation matrix of the input under the additional assumption that ⟨ x ⟩ = 0 {\displaystyle \langle \mathbf

  Read more →
• 

  Karl Steinbuch

  Karl W. Steinbuch (June 15, 1917 in Stuttgart-Bad Cannstatt – June 4, 2005 in Ettlingen) was a German computer scientist, cyberneticist, and electrical engineer. He was an early and influential researcher in German computer science, and was the developer of the Lernmatrix, an early implementation of artificial neural networks. From the late 1960s onwards the focus of his activity shifted from scientific research to right-wing political activism supporting the Neue Rechte. == Biography == Steinbuch joined the National Socialist German Students' League (NSDStB) and the Nazi Party. Steinbuch studied at the University of Stuttgart and in 1944 he received his PhD in physics. In 1948 he joined Standard Elektrik Lorenz (SEL, part of the ITT group) in Stuttgart, as a computer design engineer and later as a director of research and development, where he filed more than 70 patents. Steinbuch completed the first European fully transistorized computer, the ER 56 marketed by SEL. In 1958 he became professor and director of the Institute of Technology for information processing (ITIV) of the University of Karlsruhe, where he retired in 1980. In 1967 he began publishing books, in which he tried to influence German education policy. Together with books from colleagues like Jean Ziegler from Switzerland, Eric J. Hobsbawm from the UK, and John Naisbitt his books predicted what he regarded as the coming education disaster of the emerging civic lobby society. In 1957, together with Helmut Gröttrup, Steinbuch coined the term Informatik, the German word for computer science, which gave informatics, and the term kybernetische Anthropologie. == Awards and recognition == Wilhelm-Boelsche award - medal in Gold German non-fiction book award Gold medal award of the XXI. International Congresses on Aerospace Medicine Konrad Adenauer award of science Jakob Fugger award medal Medal of merit of the state of Baden-Wuerttemberg member, German Academy of Sciences Leopoldina member, International Academy of Science, Munich. grants from a state government grants program, named "Karl-Steinbuch-Stipendium" Steinbuch Centre for Computing at the Karlsruhe Institute of Technology named after him == Books == Steinbuch wrote several books and articles, including: 1957 Informatik: Automatische Informationsverarbeitung ("Informatics: automatic information processing"). 1963 Learning matrices and their applications (together with U. A. W. Piske) 1965 A critical comparison of two kinds of adaptive classification networks (together with Bernard Widrow) 1966 (1969): Die informierte Gesellschaft. Geschichte und Zukunft der Nachrichtentechnik (The informed society. History and Future of telecommunications) 1989: Die desinformierte Gesellschaft (The disinformed society) 1968: Falsch programmiert. Über das Versagen unserer Gesellschaft in der Gegenwart und vor der Zukunft und was eigentlich geschehen müßte. (as a bestseller listet in: Der Spiegel) (Programmed falsely. About our society's failure in the present and with respect to the future and what should be done.) 1969: Programm 2000. (as a bestseller listet in: Der Spiegel) 1971: Automat und Mensch. Auf dem Weg zu einer kybernetischen Anthropologie (Machine and Man. On the way to a cybernetic anthropology; 4th revised edition) 1971: Mensch Technik Zukunft. Probleme von Morgen (German non-fiction book award) (Man Technology Future. Problems of Tomorrow) 1973: Kurskorrektur (Correcting the Course) 1978: Maßlos informiert. Die Enteignung des Denkens (Excessively informed. The Deprivation of Thinking) 1984: Unsere manipulierte Demokratie. Müssen wir mit der linken Lüge leben? (Our Thought-controlled Democracy. Do we have to live with the leftist lie?)

  Read more →