AI Art News

AI Art News — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

Computational heuristic intelligence

Computational heuristic intelligence (CHI) refers to specialized programming techniques in computational intelligence (also called artificial intelligence, or AI). These techniques have the express goal of avoiding complexity issues, also called NP-hard problems, by using human-like techniques. They are best summarized as the use of exemplar-based methods (heuristics), rather than rule-based methods (algorithms). Hence the term is distinct from the more conventional computational algorithmic intelligence, or symbolic AI. An example of a CHI technique is the encoding specificity principle of Tulving and Thompson. In general, CHI principles are problem solving techniques used by people, rather than programmed into machines. It is by drawing attention to this key distinction that the use of this term is justified in a field already replete with confusing neologisms. Note that the legal systems of all modern human societies employ both heuristics (generalisations of cases) from individual trial records as well as legislated statutes (rules) as regulatory guides. Another recent approach to the avoidance of complexity issues is to employ feedback control rather than feedforward modeling as a problem-solving paradigm. This approach has been called computational cybernetics, because (a) the term 'computational' is associated with conventional computer programming techniques which represent a strategic, compiled, or feedforward model of the problem, and (b) the term 'cybernetic' is associated with conventional system operation techniques which represent a tactical, interpreted, or feedback model of the problem. Of course, real programs and real problems both contain both feedforward and feedback components. A real example which illustrates this point is that of human cognition, which clearly involves both perceptual (bottom-up, feedback, sensor-oriented) and conceptual (top-down, feedforward, motor-oriented) information flows and hierarchies. The AI engineer must choose between mathematical and cybernetic problem solution and machine design paradigms. This is not a coding (program language) issue, but relates to understanding the relationship between the declarative and procedural programming paradigms. The vast majority of STEM professionals never get the opportunity to design or implement pure cybernetic solutions. When pushed, most responders will dismiss the importance of any difference by saying that all code can be reduced to a mathematical model anyway. Unfortunately, not only is this belief false, it fails most spectacularly in many AI scenarios. Mathematical models are not time agnostic, but by their very nature are pre-computed, i.e. feedforward. Dyer [2012] and Feldman [2004] have independently investigated the simplest of all somatic governance paradigms, namely control of a simple jointed limb by a single flexor muscle. They found that it is impossible to determine forces from limb positions- therefore, the problem cannot have a pre-computed (feedforward) mathematical solution. Instead, a top-down command bias signal changes the threshold feedback level in the sensorimotor loop, e.g. the loop formed by the afferent and efferent nerves, thus changing the so-called ‘equilibrium point’ of the flexor muscle/ elbow joint system. An overview of the arrangement reveals that global postures and limb position are commanded in feedforward terms, using global displacements (common coding), with the forces needed being computed locally by feedback loops. This method of sensorimotor unit governance, which is based upon what Anatol Feldman calls the ‘equilibrium Point’ theory, is formally equivalent to a servomechanism such as a car's ‘cruise control’.
Read more →
Ross Quinlan

John Ross Quinlan is a computer science researcher in data mining and decision theory. He has contributed extensively to the development of decision tree algorithms, including inventing the canonical C4.5 and ID3 algorithms. He also contributed to early ILP literature with First Order Inductive Learner (FOIL). He is currently running the company RuleQuest Research which he founded in 1997. == Education == He received his BSc degree in Physics and Computing from the University of Sydney in 1965 and his computer science doctorate at the University of Washington in 1968. He has held positions at the University of New South Wales, University of Sydney, University of Technology Sydney, and RAND Corporation. == Artificial intelligence == Quinlan is a specialist in artificial intelligence, particularly in the aspect involving machine learning and its application to data mining. He is a Founding Fellow of the Association for the Advancement of Artificial Intelligence. === ID3 === Ross Quinlan invented the Iterative Dichotomiser 3 (ID3) algorithm which is used to generate decision trees. ID3 follows the principle of Occam's razor in attempting to create the smallest decision tree possible. === C4.5 === He then expanded upon the principles used in ID3 to create C4.5. C4.5 improved: discrete and continuous attributes, missing attribute values, attributes with differing costs, pruning trees (replacing irrelevant branches with leaf nodes). === C5.0 === C5.0, which Quinlan is commercially selling (single-threaded version is distributed under the terms of the GNU General Public License), is an improvement on C4.5. The advantages are speed (several orders of magnitude faster), memory efficiency, smaller decision trees, boosting (more accuracy), ability to weight different attributes, and winnowing (reducing noise). == Selected works == === Books === 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers. ISBN 1-55860-238-0. === Articles === Quinlan, J. R. (1982) Semi-autonomous acquisition of pattern-based knowledge, In Machine intelligence 10 (eds J. E. Hayes, D. Michie, and Y.-H. Pao). Ellis Norwood,Chichester. Quinlan, J.R. (1985). Decision trees and multi-valued attributes, In J.E. Hayes & D. Michie (Eds.), Machine intelligence 11. Oxford University Press. Quinlan, J. R. (1986). Induction of decision trees. Machine Learning, 1(1):81-106 2008. (with Qiang Yang, Philip S. Yu, Zhou Zhihua, and David Hand et al). Top 10 algorithms in data mining. Knowledge and Information Systems 14.1: 1-37 Quinlan, J. R. (1990). Learning logical definitions from relations. Machine Learning, 5:239-266.
Read more →
Deep Learning Studio

Deep Learning Studio is a software tool that aims to simplify the creation of deep learning models used in artificial intelligence. It is compatible with a number of open-source programming frameworks popularly used in artificial neural networks, including MXNet and Google's TensorFlow. Prior to the release of Deep Learning Studio in January 2017, proficiency in Python, among other programming languages, was essential in developing effective deep learning models. Deep Learning Studio sought to simplify the model creation process through a visual, drag-and-drop interface and the application of pre-trained learning models on available data. Irving, Texas–based Deep Cognition Inc. is the developer behind Deep Learning Studio. In 2017, the software allowed Deep Cognition to become a finalist for Best Innovation in Deep Learning in the Alconics Awards, which are given annually to the best artificial intelligence software. Deep Cognition launched version 2.0 of Deep Learning Studio at NVIDIA's GTC 2018 Conference in San Jose, California. Fremont, California–based computing products supplier Exxact Corp provides desktop computers specifically built to handle Deep Learning Studio workloads. == Features == Source: Deep Learning Studio is available in two versions: Desktop and Cloud, both of which are free software. The Desktop version is available on Windows and Ubuntu. The Cloud version is available in single-user and multi-user configurations. A Deep Cognition account is needed to access the Cloud version. Account registration is free. Deep Learning Studio can import existing Keras models; it also takes a data set as an input. Deep Learning Studio's AutoML feature allows automatic generation of deep learning models. More advanced users may choose to generate their own models using various types of layers and neural networks. Deep Learning Studio also has a library of loss functions and optimizers for use in hyperparameter tuning, a traditionally complicated area in neural network programming. Generated models can be trained using either CPUs or GPUs. Trained models can then be used for predictive analytics.
Read more →
Xuedong Huang

Xuedong David Huang (born October 20, 1962) is a Chinese-American computer scientist and technology executive who has made contributions to spoken language processing and artificial intelligence, including Azure AI Services. He is Zoom's chief technology officer after serving as Microsoft's Technical Fellow and Azure AI Chief Technology Officer for 30 years. Huang is a strong advocate of AI for Accessibility, and AI for Cultural Heritage. == Education == Huang received his PhD from the University of Edinburgh in 1989 (sponsored by the British ORS and Edinburgh University Scholarship), his MS from Tsinghua University in 1984, and BS from Hunan University in 1982. == Career == After receiving his PhD in 1989, Huang joined Carnegie Mellon University and worked with Raj Reddy and Kai-Fu Lee on speech recognition. At CMU, he directed the Sphinx-II speech system research which achieved the best performance in every category of DARPA's 1992 benchmarking. Microsoft Research recruited him to found and lead Microsoft's spoken language initiatives in 1993. His co-authored book Spoken Language Processing and his Historical speech recognition review succinctly summarize several generations of spoken language research. As Microsoft's Mr. Speech for three decades, Huang has been instrumental in creating Microsoft's Speech Application Programming Interface (SAPI), shipping Microsoft Speech Server, and modernizing spoken language and integrative AI services via Azure AI, which not only enables millions of 3rd party customers but also powers up Microsoft's Windows, Office, Teams, and Azure OpenAI Services. Huang helped Microsoft and Azure Cognitive Services achieve multiple industry's first human parity milestones on the following open research tasks: transcribing conversational speech, machine translation, conversational QnA, and computer vision image captioning. Huang has made significant contributions to the software and AI industry through his executive leadership and his scientific publications, owning more than 170 US patents and impacting billions through Azure AI enabled products and services. In 2016, Wired magazine named him one of 25 Geniuses. In 2021, Azure AI was named the winner of InfoWorld's Technology of the Year Award. Huang was awarded the Allen Newell research excellence medal in 1992, and IEEE Speech Processing Best Paper in 1993. He was recognized as an IEEE Fellow by Institute of Electrical and Electronics Engineers in 2000, named ACM Fellow by Association for Computing Machinery in 2017, and a member of Washington State Academy of Sciences. Huang received 2022 Asian American Corporate Leadership Award, and IEEE Amar Bose Industrial Leader Award. In 2023, he was elected a member of the US National Academy of Engineering (NAE), and a member of the American Academy of Arts and Sciences.
Read more →
Molecular graphics

Molecular graphics is the discipline and philosophy of studying molecules and their properties through graphical representation. IUPAC limits the definition to representations on a "graphical display device". Ever since Dalton's atoms and Kekulé's benzene, there has been a rich history of hand-drawn atoms and molecules, and these representations have had an important influence on modern molecular graphics. Colour molecular graphics are often used on chemistry journal covers artistically. == History == Prior to the use of computer graphics in representing molecular structure, Robert Corey and Linus Pauling developed a system for representing atoms or groups of atoms from hard wood on a scale of 1 inch = 1 angstrom connected by a clamping device to maintain the molecular configuration. These early models also established the CPK coloring scheme that is still used today to differentiate the different types of atoms in molecular models (e.g. carbon = black, oxygen = red, nitrogen = blue, etc). This early model was improved upon in 1966 by W.L. Koltun and are now known as Corey-Pauling-Koltun (CPK) models. The earliest efforts to produce models of molecular structure was done by Project MAC using wire-frame models displayed on a cathode ray tube in the mid 1960s. In 1965, Carroll Johnson distributed the Oak Ridge thermal ellipsoid plot (ORTEP) that visualized molecules as a ball-and-stick model with lines representing the bonds between atoms and ellipsoids to represent the probability of thermal motion. Thermal ellipsoid plots quickly became the de facto standard used in the display of X-ray crystallography data, and are still in wide use today. The first practical use of molecular graphics was a simple display of the protein myoglobin using a wireframe representation in 1966 by Cyrus Levinthal and Robert Langridge working at Project MAC. Among the milestones in high-performance molecular graphics was the work of Nelson Max in "realistic" rendering of macromolecules using reflecting spheres. Initially much of the technology concentrated on high-performance 3D graphics. During the 1970s, methods for displaying 3D graphics using cathode ray tubes were developed using continuous tone computer graphics in combination with electro-optic shutter viewing devices. The first devices used an active shutter 3D system, generating different perspective views for the left and right channel to provide the illusion of three-dimensional viewing. Stereoscopic viewing glasses were designed using lead lanthanum zirconate titanate (PLZT) ceramics as electronically controlled shutter elements. Active 3D glasses require batteries and work in concert with the display to actively change the presentation by the lenses to the wearer's eyes. Many modern 3D glasses use a passive, polarized 3D system that enables the wearer to visualize 3D effects based on their own perception. Passive 3D glasses are more common today since they are less expensive. The requirements of macromolecular crystallography also drove molecular graphics because the traditional techniques of physical model-building could not scale. The first two protein structures solved by molecular graphics without the aid of the Richards' Box were built with Stan Swanson's program FIT on the Vector General graphics display in the laboratory of Edgar Meyer at Texas A&M University: First Marge Legg in Al Cotton's lab at A&M solved a second, higher-resolution structure of staph. nuclease (1975) and then Jim Hogle solved the structure of monoclinic lysozyme in 1976. A full year passed before other graphics systems were used to replace the Richards' Box for modelling into density in 3-D. Alwyn Jones' FRODO program (and later "O") were developed to overlay the molecular electron density determined from X-ray crystallography and the hypothetical molecular structure. === Timeline === == Types == === Ball-and-stick models === In the ball-and-stick model, atoms are drawn as small sphered connected by rods representing the chemical bonds between them. === Space-filling models === In the space-filling model, atoms are drawn as solid spheres to suggest the space they occupy, in proportion to their van der Waals radii. Atoms that share a bond overlap with each other. === Surfaces === In some models, the surface of the molecule is approximated and shaded to represent a physical property of the molecule, such as electronic charge density. === Ribbon diagrams === Ribbon diagrams are schematic representations of protein structure and are one of the most common methods of protein depiction used today. The ribbon shows the overall path and organization of the protein backbone in 3D, and serves as a visual framework on which to hang details of the full atomic structure, such as the balls for the oxygen atoms bound to the active site of myoglobin in the adjacent image. Ribbon diagrams are generated by interpolating a smooth curve through the polypeptide backbone. α-helices are shown as coiled ribbons or thick tubes, β-strands as arrows, and non-repetitive coils or loops as lines or thin tubes. The direction of the polypeptide chain is shown locally by the arrows, and may be indicated overall by a colour ramp along the length of the ribbon.
Read more →
Wang-Chiew Tan

Wang-Chiew Tan is a Singaporean computer scientist specializing in data management and natural language processing. Her work in data management includes data provenance (or data lineage) and data integration. She is currently a Research Scientist at Facebook AI, and was previously the Director of Research at Megagon Labs in Mountain View, California. At Megagon Labs, Tan was the lead researcher on a study with the University of Tokyo that concluded that the company of other people is more effective than pets at making people happy. == Education and career == Tan earned her bachelor's degree in computer science (first-class) at the National University of Singapore, and completed her Ph.D. at the University of Pennsylvania. Her 2002 dissertation, Data Annotations, Provenance, and Archiving, was jointly supervised by Peter Buneman and Sanjeev Khanna. Before working at Megagon, she has been a professor of computer science at the University of California, Santa Cruz beginning in 2002, and, from 2010 to 2012, was on leave from Santa Cruz as a researcher at IBM Research - Almaden. == Recognition == Tan was named a Fellow of the Association for Computing Machinery in 2015 "for contributions to data provenance and to the foundations of information integration".
Read more →
Corpus-assisted discourse studies

Corpus-assisted discourse studies (abbr.: CADS) is related historically and methodologically to the discipline of corpus linguistics. The principal endeavor of corpus-assisted discourse studies is the investigation, and comparison of features of particular discourse types, integrating into the analysis the techniques and tools developed within corpus linguistics. These include the compilation of specialised corpora and analyses of word and word-cluster frequency lists, comparative keyword lists and, above all, concordances. A broader conceptualisation of corpus-assisted discourse studies would include any study that aims to bring together corpus linguistics and discourse analysis. Such research is often labelled as corpus-based or corpus-assisted discourse analysis, with the term CADS coined by a research group in Italy (Partington 2004) for a specific type of corpus-assisted discourse analysis (see the section 'in different countries' below). == Aims == Corpus-assisted discourse studies aim to uncover non-obvious meaning, that is, meaning which might not be readily available to naked-eye perusal. Much of what carries meaning in texts is not open to direct observation: “you cannot understand the world just by looking at it” (Stubbs [after Gellner 1959] 1996: 92). We use language “semi-automatically”, in the sense that speakers and writers make semi-conscious choices within the various complex overlapping systems of which language is composed, including those of transitivity, modality (Michael Halliday 1994), lexical sets (e.g. freedom, liberty, deliverance), modification, and so on. Authors themselves are, famously, generally unaware of all the meanings their texts convey. By combining the quantitative research approach, that is, statistical analysis of large amounts of the discourse in question - more precisely, large numbers of tokens of the discourse type under study contained in a corpus - with the more qualitative research approach typical of discourse analysis, that is, the close, detailed examination of particular stretches of discourse it may be possible to better understand the processes at play in the discourse type and to gain access to non-obvious meanings. Aims can differ in other types of corpus-based or corpus-assisted discourse analysis; but in general such studies combine quantitative and qualitative research and aim to shed light on discourses, registers, discourse patterns, etc., with the help of a corpus linguistic approach. Specific aims and techniques depend on the relevant project. == In different countries == In German-speaking countries: Pioneering work in corpus-based discourse analysis was conducted in Europe, in particular by Hardt-Mautner/Mautner (1995, 2000) and Stubbs (1996, 2001). CADS and other types of corpus-based discourse analysis are inspired by this important early work. In Italy: A considerable body of research has been conducted in Italy either by individual researchers or under the aegis of combined inter-university projects such as Newspool (Partington et al. 2004) and CorDis (Morley and Bayley eds, 2009). It has concentrated on political and media language, mainly because a nucleus of linguists in Italian universities work in Political Science faculties and are increasingly interested in the use of corpus techniques to conduct a particular type of sociopolitical discourse analysis, including the unearthing of noteworthy ideological metaphors and motifs in the language of political figures and institutions. Italian researchers also developed Modern diachronic corpus-assisted discourse studies (MD-CADS). This approach contrasts the language contained in comparable corpora from different but recent points in time in order to track changes in modern language usage but also social, cultural and political changes over modern times, as reflected - and shared among people - in language. It is this Italian body of research that makes most use of the label CADS. In the UK: Linguists in the UK tend to undertake corpus-based critical discourse analysis (CDA). CDA generally adopts a leftist political stance, focusing on the ways that social and political domination is reproduced by text and talk. This type of corpus-based research was originally associated with Lancaster University (Baker et al. 2008), but has spread more widely since. Such work typically studies the discourses around particular groups of people (e.g. Muslims, people with disabilities) or concepts/events (e.g. feminism, same-sex marriage). In Australia: Corpus-based discourse analysis is undertaken by a growing number of Australian researchers, most often on media texts. Some of this work aims to elucidate specific features of discourse types (news, social media, television series, etc.), while other work is rooted in the tradition of corpus-based critical discourse analysis. == Comparison with traditional corpus linguistics == Traditional corpus linguistics has, quite naturally, tended to privilege the quantitative approach. In the drive to produce more authentic dictionaries and grammars of a language, it has been characterised by the compilation of some very large corpora of heterogeneric discourse types in the desire to obtain an overview of the greatest quantity and variety of discourse types possible, in other words, of the chimerical but useful fiction called the “general language” (“general English”, “general Italian”, and so on). This has led to the construction of immensely valuable research tools such as the Bank of English and the British National Corpus. Some branches of corpus linguistics have also promoted an approach that is "corpus-driven", in which we need, grammatically speaking, a mental tabula rasa to free ourselves of the baleful prejudice exerted by traditional models and allow the data to speak entirely for itself. The aim of corpus-assisted discourse studies and related approaches is radically different. Here the aim of the exercise is to acquaint oneself as much as possible with the discourse type(s) in hand. Researchers typically engage with their corpus in a variety of ways. As well as via wordlists and concordancing, intuitions for further research can also arise from reading or watching or listening to parts of the data-set, a process which can help provide a feel for how things are done linguistically in the discourse-type being studied. Corpus-assisted discourse analysis is also typically characterised by the compilation of ad hoc specialised corpora, since very frequently there exists no previously available collection of the discourse type in question. Often, other corpora are utilized in the course of a study for purposes of comparison. These may include pre-existing corpora or may themselves need to be compiled by the researcher. In some sense, all work with corpora – just as all work with discourse - is properly comparative. Even when a single corpus is employed, it is used to test the data it contains against another body of data. This may consist of the researcher's intuitions, or the data found in reference works such as dictionaries and grammars, or it may be statements made by previous authors in the field. == CADS as a specific type of corpus-based discourse analysis == Researchers in Italy have developed CADS as a specific type of corpus-based discourse analysis, creating a standard set of methods: 'A basic, standard methodology in CADS may resemble the following:' Step 1: Decide upon the research question; Step 2: Choose, compile or edit an appropriate corpus; Step 3: Choose, compile or edit an appropriate reference corpus / corpora; Step 4: Make frequency lists and run a keywords comparison of the corpora; Step 5: Determine the existence of sets of key items; Step 6: Concordance interesting key items (with differing quantities of co-text); Step 7: (Possibly) refine the research question and return to Step 2. This basic procedure can of course vary according to individual research circumstances and requirements. A particular way of conceptualising research questions has also been proposed in such CADS projects: Given that P is a discourse participant (or possibly an institution) and G is a goal, often a political goal: How does P achieve G with language? What does this tell us about P? Comparative studies: how do P1 and P2 differ in their use of language? Does this tell us anything about their different principles and objectives? A second general type of CADS research question, which might be asked of interactive discourse data, has been conceptualised as follows: Given that P(x) is a particular participant or set of participants, DT is the discourse type, and R is an observed relationship between or among participants: How do {P(a), P(b)...P(n)} achieve / maintain R in DT [using language]? Another common type of research question has been conceptualised thus: Given that A is an author, Ph(x) is a phenomenon or practice or behaviour, and DT(x) is a particular discourse type. A has said P
Read more →
Luca Maria Gambardella

Luca Maria Gambardella (born 4 January 1962) is an Italian computer scientist and author. He is the former director of the Dalle Molle Institute for Artificial Intelligence Research in Lugano, in the Ticino canton of Switzerland. He is currently the prorector of Università della Svizzera italiana, where he directs the Master of Science in Artificial Intelligence degree course. Several of his papers have been extensively cited, with his collaborators including Marco Dorigo, with whom he has published papers on the application of ant colony optimization theory to the traveling salesman problem, and Jürgen Schmidhuber with whom he has published research on deep neural networks.. Beside working in research, Gambardella explores the potentials of AI applied for the generation of art. Some of his artistic installations received significant media coverage. As a novelist, the genres he approached broad from Bildungsroman of his first book "Sei vite" ("Six lives"), to romance of his second book "Il suono dell'alba" ("The sound of sunrise").
Read more →
Learning rate

In machine learning and statistics, the learning rate is a tuning parameter in an optimization algorithm that determines the step size at each iteration while moving toward a minimum of a loss function. Since it influences to what extent newly acquired information overrides old information, it metaphorically represents the speed at which a machine learning model "learns". In the adaptive control literature, the learning rate is commonly referred to as gain. In setting a learning rate, there is a trade-off between the rate of convergence and overshooting. While the descent direction is usually determined from the gradient of the loss function, the learning rate determines how big a step is taken in that direction. Too high a learning rate will make the learning jump over minima, but too low a learning rate will either take too long to converge or get stuck in an undesirable local minimum. In order to achieve faster convergence, prevent oscillations and getting stuck in undesirable local minima the learning rate is often varied during training either in accordance to a learning rate schedule or by using an adaptive learning rate. The learning rate and its adjustments may also differ per parameter, in which case it is a diagonal matrix that can be interpreted as an approximation to the inverse of the Hessian matrix in Newton's method. The learning rate is related to the step length determined by inexact line search in quasi-Newton methods and related optimization algorithms. == Learning rate schedule == Initial rate can be left as system default or can be selected using a range of techniques. A learning rate schedule changes the learning rate during learning and is most often changed between epochs/iterations. This is mainly done with two parameters: decay and momentum. There are many different learning rate schedules but the most common are time-based, step-based and exponential. Decay serves to settle the learning in a nice place and avoid oscillations, a situation that may arise when too high a constant learning rate makes the learning jump back and forth over a minimum, and is controlled by a hyperparameter. Momentum is analogous to a ball rolling down a hill; we want the ball to settle at the lowest point of the hill (corresponding to the lowest error). Momentum both speeds up the learning (increasing the learning rate) when the error cost gradient is heading in the same direction for a long time and also avoids local minima by 'rolling over' small bumps. Momentum is controlled by a hyperparameter analogous to a ball's mass which must be chosen manually—too high and the ball will roll over minima which we wish to find, too low and it will not fulfil its purpose. The formula for factoring in the momentum is more complex than for decay but is most often built in with deep learning libraries such as Keras. Time-based learning schedules alter the learning rate depending on the learning rate of the previous time iteration. Factoring in the decay the mathematical formula for the learning rate is: η n + 1 = η 0 1 + d n {\displaystyle \eta _{n+1}={\frac {\eta _{0}}{1+dn}}} where η {\displaystyle \eta } is the learning rate, η 0 {\displaystyle \eta _{0}} is the original learning rate, d {\displaystyle d} is a decay parameter and n {\displaystyle n} is the iteration step. Step-based learning schedules changes the learning rate according to some predefined steps. The decay application formula is here defined as: η n = η 0 d ⌊ 1 + n r ⌋ {\displaystyle \eta _{n}=\eta _{0}d^{\left\lfloor {\frac {1+n}{r}}\right\rfloor }} where η n {\displaystyle \eta _{n}} is the learning rate at iteration n {\displaystyle n} , η 0 {\displaystyle \eta _{0}} is the initial learning rate, d {\displaystyle d} is how much the learning rate should change at each drop (0.5 corresponds to a halving) and r {\displaystyle r} corresponds to the drop rate, or how often the rate should be dropped (10 corresponds to a drop every 10 iterations). The floor function ( ⌊ … ⌋ {\displaystyle \lfloor \dots \rfloor } ) here drops the value of its input to 0 for all values smaller than 1. Exponential learning schedules are similar to step-based, but instead of steps, a decreasing exponential function is used. The mathematical formula for factoring in the decay is: η n = η 0 e − d n {\displaystyle \eta _{n}=\eta _{0}e^{-dn}} where d {\displaystyle d} is a decay parameter. == Adaptive learning rate == The issue with learning rate schedules is that they all depend on hyperparameters that must be manually chosen for each given learning session and may vary greatly depending on the problem at hand or the model used. To combat this, there are many different types of adaptive gradient descent algorithms such as Adagrad, Adadelta, RMSprop, and Adam which are generally built into deep learning libraries such as Keras.
Read more →
Cognitive Technologies

Cognitive Technologies is a Russian software corporation that develops corporate business applications, AI-based advanced driver assistance systems. Founded in 1993 in Moscow (Russia), the company has offices in Eastern Europe, with R&D Centers in Russia. == History == Cognitive Technologies was founded in 1993 by Olga Uskova and Vladimir Arlazarov. The first employees previously worked in the team that developed the first world computer chess champion "Kaissa". The first programs developed by Cognitive Technologies were optical image and character recognition software – Tiger and CuneiForm. In February 2015 Cognitive Technologies and Kamaz, Russian Dakar Rally-winning truck manufacturer, started working on the self-driving Kamaz truck project. The first field tests took place in June 2015. In 2015 Andrey Chernogorov was appointed CEO of the company. == Products == Cognitive Technologies develops business application software and self-driving vehicle artificial intelligence. The main products are: C-pilot, AI-based ADAS E1 Evfrat – electronic workflow system CognitiveLot – e-purchasing systems == Cooperation with global companies == Under the contract signed between Cognitive Technologies and Hewlett-Packard, all scanners sold in Russia had text recognition software developed by Cognitive Technologies. It was the first contract with HP for an Eastern European company. Afterwards, Cognitive Technologies signed OEM contracts and business agreements with several global IT-companies, including IBM, Canon, Corel, Samsung, Xerox, Brother, Epson, and Olivetti. In 1998 Cognitive Technologies became the first company in Eastern Europe to get the Oracle Complementary Software Provider status. In 2001 Cognitive Technologies sold its Russian language speech corpus to Intel. In 2010 Cognitive Technologies sold its text parsing module to Yandex. The company also signed an agreement with NVIDIA join efforts in the development of intelligent document recognition technologies. == Self-driving car project == The system developed by Cognitive Technologies does not require building smart cities and smart roads equipped with multiple sensors – it works the opposite way, trying to understand the situation on the road like humans do. The system uses a video camera like a driver who uses his eyes, analyzing the information and focusing on the relevant data. For this purpose the system uses a special type of computer vision – foveal computer vision. Only 5–7% of the data gathered by the video cameras and sensors is processed by the system as relevant. The prototype is being tested in Russia on rough roads, on roads without marking, with the goal to prepare the system for work in difficult situations and on bad roads all around the world. == C-Pilot ADAS project == In August 2016 Cognitive Technologies started its own ADAS development project C-Pilot for ground transport control automation. == Self-driving tractors and harvesters project == The experts from Cognitive Technologies claim that the system will track stones, poles, and other obstacles that might be dangerous for the vehicles. This data will enable the engineers to develop an interactive field map, with GPS coordinates for stones and other obstacles. Eventually, this will result in an alteration of the harvester's movement pattern preventing it from running into stones or other objects that may inflict damage. Harvesters will work autonomously on the field, on the territory that is narrowed by radio beacons. == Present international activities == In 2016 Cognitive Technologies has joined the international community OpenPower Foundation, a consortium of open source solutions to developers based on POWER technology from IBM, which includes the world's leading IT map of Google, NVidia, Mellanox, etc. Within the consortium Cognitive Technologies is the initiator of forming of an international working group to develop a single software standard for the self-driving vehicle control. == Awards == In 2016, the leading Russian business newspaper Kommersant, announced that Cognitive Technologies is the TOP-2 Russian software company. TOP-6 Russian software company in 2015 according to Russoft TOP-500 biggest Russian companies according to RBC TOP-2 company of the Russian EDMS market in 2014 according to IDC TOP-20 Russian biggest IT-companies in 2013 according to Cnews Analytics
Read more →
Ernst Dickmanns

Ernst Dieter Dickmanns is a German pioneer of dynamic computer vision and of driverless cars. Dickmanns has been a professor at the University of the Bundeswehr Munich (1975–2001), and visiting professor to Caltech and to MIT, teaching courses on "dynamic vision". == Biography == Dickmanns was born in 1936. He studied aerospace and aeronautics at RWTH Aachen (1956–1961), and control engineering at Princeton University (1964/65); from 1961 to 1975 he was associated with the German Aero-Space Research Establishment (now DLR) Oberpfaffenhofen, working in the fields of flight dynamics and trajectory optimization. In 1971/72 he spent a Post-Doc Research Associateship with the NASA-Marshall Space Flight Center, Huntsville (orbiter re-entry). From 1975 to 2001 he was with UniBw Munich, where he initiated the 'Institut fuer Flugmechanik und Systemdynamik' (IFS), the Institut fuer die 'Technik Autonomer Systeme' (TAS), and the research activities in machine vision for vehicle guidance. == Pioneering work in autonomous driving == In the early 1980s his team equipped a Mercedes-Benz van with cameras and other sensors. The 5-ton van was re-engineered that it was possible to control steering wheel, throttle, and brakes through computer commands based on real-time evaluation of image sequences. Software was written that translated the sensory data into appropriate driving commands. For safety reasons, initial experiments in Bavaria took place on streets without traffic. In 1986 the Robot Car "VaMoRs" managed to drive all by itself and by 1987 was capable of driving itself at speeds up to 96 kilometres per hour (60 mph). One of the greatest challenges in high-speed autonomous driving arises through the rapidly changing visual street scenes. Back then, computers were much slower than they are today (~1% of 1%); therefore, sophisticated computer vision strategies were necessary to react in real time. The team of Dickmanns solved the problem through an innovative approach to dynamic vision. Spatiotemporal models were used right from the beginning, dubbed '4-D approach', which did not need storing previous images but nonetheless was able to yield estimates of all 3-D position and velocity components. Attention control including artificial saccadic movements of the platform carrying the cameras allowed the system to focus its attention on the most relevant details of the visual input. Kalman filters have been extended to perspective imaging and were used to achieve robust autonomous driving even in presence of noise and uncertainty. Feedback of prediction errors allowed bypassing the (ill-conditioned) inversion of perspective projection by least-squares parameter fits. When in 1986/83 the EUREKA-project 'PROgraMme for a European Traffic of Highest Efficiency and Unprecedented Safety' (PROMETHEUS) was initiated by the European car manufacturing industry (funding in the range of several hundred million Euros), the initially planned autonomous lateral guidance by buried cables was dropped and substituted by the much more flexible machine vision approach proposed by Dickmanns, and partially encouraged by his successes. Most of the major car companies participated; so did Dickmanns and his team in cooperation with the Daimler-Benz AG. Substantial progress was made in the following 7 years. In particular, Dickmanns' robot cars learned to drive in traffic under various conditions. An accompanying human driver with a "red button" made sure the robot vehicle could not get out of control and become a danger to the public. Since 1992, driving in public traffic was standard as final step in real-world testing. Several dozen Transputers, a special breed of parallel computers, were used to deal with the (by 1990s standards) enormous computational demands. Two culmination points were achieved in 1994/95, when Dickmanns´ re-engineered autonomous S-Class Mercedes-Benz performed international demonstrations. The first was the final presentation of the PROMETHEUS project in October 1994 on Autoroute 1 near the airport Charles-de-Gaulle in Paris. With guests on board, the twin vehicles of Daimler-Benz (VITA-2) and UniBwM (VaMP) drove more than 1,000 kilometres (620 mi) on the three-lane highway in standard heavy traffic at speeds up to 130 kilometres per hour (81 mph). Driving in free lanes, convoy driving with distance keeping depending on speed, and lane changes left and right with autonomous passing have been demonstrated; the latter required interpreting the road scene also in the rear hemisphere. Two cameras with different focal lengths for each hemisphere have been used in parallel for this purpose. The second culmination point was a 1,758 kilometres (1,092 mi) trip in the fall of 1995 from Munich in Bavaria to Odense in Denmark to a project meeting and back. Both longitudinal and lateral guidance were performed autonomously by vision. On highways, the robot achieved speeds exceeding 175 kilometres per hour (109 mph) (there is no general speed limit on the Autobahn). Publications from Dickmann's research group indicate a mean autonomously driven distance without resets of ~9 kilometres (5.6 mi); the longest autonomously driven stretch reached 158 kilometres (98 mi). More than half of the resets required were achieved autonomously (no human intervention). This is particularly impressive considering that the system used black-and-white video-cameras and did not model situations like road construction sites with yellow lane markings; lane-changes at over 140 kilometres per hour (87 mph), and other traffic with more than 40 kilometres per hour (25 mph) relative speed have been handled. In total, 95% autonomous driving (by distance) was achieved. In the years 1994 to 2004 the elder 5-ton van 'VaMoRs' was used to develop the capabilities needed for driving on networks of minor (also unsealed) roads and for cross-country driving including avoidance of negative obstacles like ditches. Turning off onto crossroads of unknown width and intersection angles required a big effort, but has been achieved with "Expectation-based, Multi-focal, Saccadic vision" (EMS-vision). This vertebrate-type vision uses animation capabilities based on knowledge about subject classes (including the autonomous vehicle itself) and their potential behaviour in certain situations. This rich background is used for control of gaze and attention as well as for locomotion. Beside ground vehicle guidance, also applications of the 4-D approach to dynamic vision for unmanned air vehicles (conventional aircraft and helicopters) have been investigated. Autonomous visual landing approaches and landings have been demonstrated in hardware-in-the-loop simulations with visual/inertial data fusion. Real-world autonomous visual landing approaches till shortly before touchdown have been performed in 1992 with the twin-propeller aircraft Dornier 128 of the University of Brunswick at the airport there. Another success of this machine vision technology was the first ever visually controlled grasping experiment of a free-floating object in weightlessness on board the Space Shuttle Columbia D2-mission in 1993 as part of the 'Rotex'-experiment of DLR.
Read more →
Ali Farhadi

Ali Farhadi is a professor of computer science and currently serves as the CEO of the Allen Institute for Artificial Intelligence (AI2). == Career == Farhadi is an AI professor in the Paul G. Allen School of Computer Science & Engineering at the University of Washington in Seattle. == Recognitions == Among other awards and recognitions, Farhadi was one of the winners of the 2017 Sloan Research Fellowship granted by the Alfred P. Sloan Foundation. == Personal life == Farhadi is married to Hanna Hajishirzi, a computer science professor at the University of Washington and senior director at the Allen Institute who studies natural language processing.
Read more →
Calais (Reuters product)

Calais is a service created by Thomson Reuters that automatically extracts semantic information from web pages in a format that can be used on the semantic web. Calais was launched in January 2008, and is free to use. The technology is now available via the website of Refinitiv, a provider of financial market data and infrastructure founded in 2018, that is a subsidiary of London Stock Exchange Group. The Calais Web service reads unstructured text and returns Resource Description Framework formatted results identifying entities, facts and events within the text. The service appears to be based on technology acquired when Reuters purchased ClearForest in 2007. The technology has also been used to automatically tag blog articles, and organize museum collections. Calais uses natural language processing technologies delivered via a web service interface.
Read more →
Phraselator

The Phraselator is a weatherproof handheld language translation device developed by Applied Data Systems and VoxTec, a former division of the military contractor Marine Acoustics, located in Annapolis, Maryland, USA. It was designed to serve as a handheld computer device that translates English into one of 40 different languages. == The device == The Phraselator is a small speech translation PDA-sized device designed to aid in interpretation. The device does not produce synthesized speech like that utilized by Stephen Hawking; instead, it plays pre-recorded foreign language MP3 files. Users can select the phrase they wish to convey from an English list on the screen or speak into the device. It then uses speech recognition technology called DynaSpeak, developed by SRI International, to play the proper sound file. The accuracy of the speech recognition software is over 70 percent according to software developer Jack Buchanan. The device can also record replies for translation later. Pre-recorded phrases are stored on Secure Digital flash memory cards. A 128 MB card can hold up to 12,000 phrases in four or five languages. Users can download phrase modules from the official website, which contained over 300,000 phrases as of March 2005. Users can also construct their own custom phrase modules. Earlier devices were known to have run on an SA-1110 Strong Arm 206 MHz CPU with 32MB SDRAM and 32MB onboard Flash RAM. A newer model, the P2, was released in 2004 and developed according to feedback from U.S. soldiers. It translates one way from English to approximately 60 other languages. It has a directional microphone, a larger library of phrases and a longer battery life. The 2004 release was created by and utilizes a computer board manufactured by InHand Electronics, Inc. In the future, the device will be able to display pictures so users can ask questions such as "Have you seen this person?" Developer Ace Sarich notes that the device is inferior to human interpreter. Conclusions derived from a Nepal field test conducted by U.S. and Nepal based NGO Himalayan Aid in 2004 seemed to confirm Sarich's comparisons: The very concept of using a machine as a communication point between individuals seemed to actually encourage a more limited form of interaction between tester and respondent. Usually, when limited language skills are present between parties, the genuine struggle and desire to communicate acts as a display of good will – we openly display our weakness in this regard – and the result is a more relaxed and human encounter. This was not necessarily present with the Phraselator as all parties abandoned learning about each other and instead focused on learning how to work with the device. As a tool for bridging any cultural differences or communicating effectively at any length, the Phraselator would not be recommended. This device, at least in the form tested, would best be used in large-scale operations where there is no time for language training and there is a need to communicate fixed ideas, quickly, over the greatest distance by employing large amounts of unskilled users. Large humanitarian or natural disasters in remote areas of third-world countries might be an effective example. == Origin == The original idea for the device came from Lee Morin, a Navy doctor in Operation Desert Storm. To communicate with patients, he played Arabic audio files from his laptop. He informed Ace Sarich, the vice president of VoxTec, about the idea. VoxTec won a DARPA Small Business Innovation Research grant in early 2001 to develop a military-grade handheld phrase translator. During its development, the Phraselator was tested and evaluated by scientists from the Army Research Laboratory. The device was first field tested in Afghanistan in 2001. By 2002, about 500 Phraselators were built for soldiers around the world with another 250 ordered by the U.S. Special Forces. The device cost $2000 to develop and could convert spoken English into one of 200,000 recorded commands and questions in 30 languages. However, the device could only translate one-way. At the time, the only existing two-way voice translator that could convert speech back and forth between languages was the Audio Voice Translation Guide System, or TONGUES, which was developed by Carnegie Mellon University for Lockheed Martin. As part of a DARPA program known as the Spoken Language Communication and Translation System for Tactical Use, SRI International has further developed two-way translation software for use in Iraq called IraqComm in 2006 which contains a vocabulary of 40,000 English words and 50,000 words in Iraqi Arabic. == Notable users == The handheld translator was recently used by U.S. troops while providing relief to tsunami victims in early 2005. About 500 prototypes of the device were provided to U.S. military forces in Operation Enduring Freedom. Units loaded with Haitian dialects have been provided to U.S. troops in Haiti. Army military police have used it in Kandahar to communicate with POWs. In late 2004, the U.S. Navy began to augment some ships with a version of the device attached to large speakers in order to broadcast clear voice instructions up to 400 yards (370 m) away. Corrections officers and law enforcement in Oneida County, New York, have tested the device. Hospital emergency rooms and health departments have also evaluated it. Several Native American tribes such as the Choctaw Nation, the Ponca, and the Comanche Nation have also used the device to preserve their dying languages. Various law enforcement agencies, such as the Los Angeles Police Department, also use the phraselator in their patrol cars. == Awards == In March 2004, DARPA director Dr. Tony Tether presented the Small Business Innovative Research Award to the VoxTec division of Marine Acoustics at DARPATech 2004 in Anaheim, CA. The device was recently listed as one of "Ten Emerging Technologies That Will Change Your World" in MIT's Technology Review. == Pop culture == Software developer Jack Buchanan believes that building a device similar to the fictional universal translator seen in Star Trek would be harder than building the Enterprise. The device was mentioned in a list of "Top 10 Star Trek Tech" on Space.com.
Read more →
Synchronizing word

In computer science, more precisely, in the theory of deterministic finite automata (DFA), a synchronizing word or reset sequence is a word in the input alphabet of the DFA that sends any state of the DFA to one and the same state. That is, if an ensemble of copies of the DFA are each started in different states, and all of the copies process the synchronizing word, they will all end up in the same state. Not every DFA has a synchronizing word; for instance, a DFA with two states, one for words of even length and one for words of odd length, can never be synchronized. == Existence == Given a DFA, the problem of determining if it has a synchronizing word can be solved in polynomial time using a theorem due to Ján Černý. A simple approach considers the power set of states of the DFA, and builds a directed graph where nodes belong to the power set, and a directed edge describes the action of the transition function. A path from the node of all states to a singleton state shows the existence of a synchronizing word. This algorithm is exponential in the number of states. A polynomial algorithm results however, due to a theorem of Černý that exploits the substructure of the problem, and shows that a synchronizing word exists if and only if every pair of states has a synchronizing word. == Length == The problem of estimating the length of synchronizing words has a long history and was posed independently by several authors, but it is commonly known as the Černý conjecture. In 1969, Ján Černý conjectured that (n − 1)2 is the upper bound for the length of the shortest synchronizing word for any n-state complete DFA (a DFA with complete state transition graph). If this is true, it would be tight: in his 1964 paper, Černý exhibited a class of automata (indexed by the number n of states) for which the shortest reset words have this length. The best upper bound known is 0.1654n3, far from the lower bound. For n-state DFAs over a k-letter input alphabet, an algorithm by David Eppstein finds a synchronizing word of length at most 11n3/48 + O(n2), and runs in time complexity O(n3+kn2). This algorithm does not always find the shortest possible synchronizing word for a given automaton; as Eppstein also shows, the problem of finding the shortest synchronizing word is NP-complete. However, for a special class of automata in which all state transitions preserve the cyclic order of the states, he describes a different algorithm with time O(kn2) that always finds the shortest synchronizing word, proves that these automata always have a synchronizing word of length at most (n − 1)2 (the bound given in Černý's conjecture), and exhibits examples of automata with this special form whose shortest synchronizing word has length exactly (n − 1)2. == Road coloring == The road coloring problem is the problem of labeling the edges of a regular directed graph with the symbols of a k-letter input alphabet (where k is the outdegree of each vertex) in order to form a synchronizable DFA. It was conjectured in 1970 by Benjamin Weiss and Roy Adler that any strongly connected and aperiodic regular digraph can be labeled in this way; their conjecture was proven in 2007 by Avraham Trahtman. == Related: transformation semigroups == A transformation semigroup is synchronizing if it contains an element of rank 1, that is, an element whose image is of cardinality 1. A DFA corresponds to a transformation semigroup with a distinguished generator set.
Read more →