Locative media or location-based media (LBM) is a virtual medium of communication functionally bound to a location. The physical implementation of locative media, however, is not bound to the same location to which the content refers. Location-based media delivers multimedia and other content directly to the user of a mobile device dependent upon their location. Location information determined by means such as mobile phone tracking and other emerging real-time locating system technologies like Wi-Fi or RFID can be used to customize media content presented on the device. Locative media are digital media applied to real places and thus triggering real social interactions. While mobile technologies such as the Global Positioning System (GPS), laptop computers and mobile phones enable locative media, they are not the goal for the development of projects in this field. == Description == Media content is managed and organized externally of the device on a standard desktop, laptop, server, or cloud computing system. The device then downloads this formatted content with GPS or other RTLS coordinate-based triggers applied to each media sequence. As the location-aware device enters the selected area, centralized services trigger the assigned media, designed to be of optimal relevance to the user and their surroundings. Use of locative technologies "includes a range of experimental uses of geo-technologies including location-based games, artistic critique of surveillance technologies, experiential mapping, and spatial annotation." Location based media allows for the enhancement of any given environment offering explanation, analysis and detailed commentary on what the user is looking at through a combination of video, audio, images and text. The location-aware device can deliver interpretation of cities, parklands, heritage sites, sporting events or any other environment where location based media is required. The content production and pre-production are integral to the overall experience that is created and must have been performed with ultimate consideration of the location and the users position within that location. The media offers a depth to the environment beyond that which is immediately apparent, allowing revelations about background, history and current topical feeds. == Locative, ubiquitous and pervasive computing == The term 'locative media' was coined by Karlis Kalnins. Locative media is closely related to augmented reality (reality overlaid with virtual reality) and pervasive computing (computers everywhere, as in ubiquitous computing). Whereas augmented reality strives for technical solutions, and pervasive computing is interested in embedded computers, locative media concentrates on social interaction with a place and with technology. Many locative media projects have a social, critical or personal (memory) background. While strictly spoken, any kind of link to additional information set up in space (together with the information that a specific place supplies) would make up location-dependent media, the term locative media is strictly bound to technical projects. Locative media works on locations and yet many of its applications are still location-independent in a technical sense. As in the case of digital media, where the medium itself is not digital but the content is digital, in locative media the medium itself might not be location-oriented, whereas the content is location-oriented. Japanese mobile phone culture embraces location-dependent information and context-awareness. It is projected that in the near future locative media will develop to a significant factor in everyday life. == Enabling technologies == Locative media projects use technology such as Global Positioning System (GPS), laptop computers, the mobile phone, Geographic Information System (GIS), and web map services such as Mapbox, OpenStreetMap, and Google Maps among others. Whereas GPS allows for the accurate detection of a specific location, mobile computers allow interactive media to be linked to this place. The GIS supplies arbitrary information about the geological, strategic or economic situation of a location. Web maps like Google Maps give a visual representation of a specific place. Another important new technology that links digital data to a specific place is radio-frequency identification (RFID), a successor to barcodes like Semacode. Research that contributes to the field of locative media happens in fields such as pervasive computing, context awareness and mobile technology. The technological background of locative media is sometimes referred to as "location-aware computing". == Creative representation == Place is often seen as central to creativity; in fact, "for some—regional artists, citizen journalists and environmental organizations for example—a sense of place is a particularly important aspect of representation, and the starting point of conversations." Locative media can propel such conversations in its function as a "poetic form of data visualization," as its output often traces how people move in, and by proxy, make sense of, urban environments. Given the dynamism and hybridity of cities and the networks which comprise them, locative media extends the internet landscape to physical environments where people forge social relations and actions which can be "mobile, plural, differentiated, adventurous, innovative, but also estranged, alienated, impersonalized." Moreover, in using locative technologies, users can expand how they communicate and assert themselves in their environment and, in doing so, explore this continuum of urban interactions. Furthermore, users can assume a more active role in constructing the environments they are situated in accordingly. In turn, artists have been intrigued with locative media as a means of "user-led mapping, social networking and artistic interventions in which the fabric of the urban environment and the contours of the earth become a 'canvas.'" Such projects demystify how resident behaviors in a given city contribute to the culture and sense of personality that cities are often perceived to take on. Design scholars Anne Galloway and Matthew Ward state that "various online lists of pervasive computing and locative media projects draw out the breadth of current classification schema: everything from mobile games, place-based storytelling, spatial annotation and networked performances to device-specific applications." A prominent use of locative media is in locative art. A sub-category of interactive art or new media art, locative art explores the relationships between the real world and the virtual or between people, places or objects in the real world. == Examples == Notable locative media projects include Bio Mapping by Christian Nold in 2004, locative art projects such as the SpacePlace ZKM/ZKMax bluecasting and participatory urban media access in Munich in 2005 and Britglyph by Alfie Dennen in 2009, and location-based games such as AR Quake by the Wearable Computer Lab at the University of South Australia and Can You See Me Now? in 2001 by Blast Theory in collaboration with the Mixed Reality Lab at the University of Nottingham. In 2005, the Silicon Valley–based collaborators of C5 first exhibited the C5 Landscape Initiative, a suite of four GPS inspired projects that investigate perception of landscape in light of locative media. In William Gibson's 2007 novel Spook Country, locative art is one of the main themes and set pieces in the story. Narrative projects which engage with locative media are sometimes referred to as Location-Aware Fiction, as explored in "Data and Narrative: Location Aware Fiction" a 2003 essay by Kate Armstrong. This location-aware fiction is also known as locative literature, where locative stories and poems can be experienced via digital portals, apps, QR codes and e-books, as well as via analogue forms such as labelling tape, Scrabble tiles, fridge magnets or Post-It notes, and these are forms often used by the writer and artist Matt Blackwood. The Transborder Immigrant Tool by the Electronic Disturbance Theater is a locative media project aimed at providing life saving directions to water for people trying to cross the US / Mexico border. The project attracted global media attention in 2009 and 2010. Articles included a Los Angeles Times cover story focusing on Ricardo Dominguez and an AP story interviewing Micha Cárdenas and Brett Stalbaum. The articles focused on concerns over the legality of the project and the ensuing investigations of the group, which are still underway. The Transborder Immigrant Tool has recently been included in a number of major exhibitions including Here, Not There at the Museum of Contemporary Art San Diego and the 2010 California Biennial at the Orange County Museum of Art. Invisible Threads by Stephanie Rothenberg and Jeff Crouse is a locative media project aimed at creating embodied awareness of sweatshops and just-in-time production t
Abillion
abillion was a mobile application helping users to find vegan and sustainable products. The platform allowed users to review plant-based, cruelty-free and sustainable products, while donating between 0.10 and $1 to nonprofit organisations for each review written. As of May 2023, the company claimed to have donated over $2.8M to various nonprofit organisations including Sea Shepherd and Mercy for Animals. The main objective of the company was to reach the number of one billion people following a vegan diet and lifestyle by 2030. == History == The American entrepreneur Vikas Garg founded the company in Singapore and the app was officially launched in May 2018. The start-up was first named 'abillionveg' and changed its name in 2020 to shorten it to 'abillion'. In 2019, the company raised $3M in its first round of funding (pre-Series A). In 2021, it raised $10M in its Series A funding. In February 2023, the company announced the launch of a community investment round, using the crowdfunding platform Wefunder, which reached a total of $500 000. In May 2023, it celebrated its 5th anniversary and reaching 1M downloads. In March 2026, the company announced that they would be closing down by the end of the month. == Awards == Using data from the reviews published by its users, abillion was awarding the most liked vegan products and brands. In May 2023, the company published a world Top 10 Best Plant Based Burgers, among the winning brands were Beyond Meat, NotCo and Sojasun.
Boosting (machine learning)
In machine learning (ML), boosting is an ensemble learning method that combines a set of less accurate models (called "weak learners") to create a single, highly accurate model (a "strong learner"). Unlike other ensemble methods that build models in parallel (such as bagging), boosting algorithms build models sequentially. Each new model in the sequence is trained to correct the errors made by its predecessors. This iterative process allows the overall model to improve its accuracy, particularly by reducing bias. Boosting is a popular and effective technique used in supervised learning for both classification and regression tasks. The theoretical foundation for boosting came from a question posed by Kearns and Valiant (1988, 1989): "Can a set of weak learners create a single strong learner?" A weak learner is defined as a classifier that performs only slightly better than random guessing, whereas a strong learner is a classifier that is highly correlated with the true classification. Robert Schapire's affirmative answer to this question in a 1990 paper led to the development of practical boosting algorithms. The first such algorithm was developed by Schapire, with Freund and Schapire later developing AdaBoost, which remains a foundational example of boosting. == Algorithms == While boosting is not algorithmically constrained, most boosting algorithms consist of iteratively learning weak classifiers with respect to a distribution and adding them to a final strong classifier. When they are added, they are weighted in a way that is related to the weak learners' accuracy. After a weak learner is added, the data weights are readjusted, known as "re-weighting". Misclassified input data gain a higher weight and examples that are classified correctly lose weight. Thus, future weak learners focus more on the examples that previous weak learners misclassified. There are many boosting algorithms. The original ones, proposed by Robert Schapire (a recursive majority gate formulation), and Yoav Freund (boost by majority), were not adaptive and could not take full advantage of the weak learners. Schapire and Freund then developed AdaBoost, an adaptive boosting algorithm that won the prestigious Gödel Prize. Only algorithms that are provable boosting algorithms in the probably approximately correct learning formulation can accurately be called boosting algorithms. Other algorithms that are similar in spirit to boosting algorithms are sometimes called "leveraging algorithms", although they are also sometimes incorrectly called boosting algorithms. The main variation between many boosting algorithms is their method of weighting training data points and hypotheses. AdaBoost is very popular and the most significant historically as it was the first algorithm that could adapt to the weak learners. It is often the basis of introductory coverage of boosting in university machine learning courses. There are many more recent algorithms such as LPBoost, TotalBoost, BrownBoost, xgboost, MadaBoost, LogitBoost, CatBoost and others. Many boosting algorithms fit into the AnyBoost framework, which shows that boosting performs gradient descent in a function space using a convex cost function. == Object categorization in computer vision == Given images containing various known objects in the world, a classifier can be learned from them to automatically classify the objects in future images. Simple classifiers built based on some image feature of the object tend to be weak in categorization performance. Using boosting methods for object categorization is a way to unify the weak classifiers in a special way to boost the overall ability of categorization. === Problem of object categorization === Object categorization is a typical task of computer vision that involves determining whether or not an image contains some specific category of object. The idea is closely related with recognition, identification, and detection. Appearance based object categorization typically contains feature extraction, learning a classifier, and applying the classifier to new examples. There are many ways to represent a category of objects, e.g. from shape analysis, bag of words models, or local descriptors such as SIFT, etc. Examples of supervised classifiers are Naive Bayes classifiers, support vector machines, mixtures of Gaussians, and neural networks. However, research has shown that object categories and their locations in images can be discovered in an unsupervised manner as well. === Status quo for object categorization === The recognition of object categories in images is a challenging problem in computer vision, especially when the number of categories is large. This is due to high intra class variability and the need for generalization across variations of objects within the same category. Objects within one category may look quite different. Even the same object may appear unalike under different viewpoint, scale, and illumination. Background clutter and partial occlusion add difficulties to recognition as well. Humans are able to recognize thousands of object types, whereas most of the existing object recognition systems are trained to recognize only a few, e.g. human faces, cars, simple objects, etc. Research has been very active on dealing with more categories and enabling incremental additions of new categories, and although the general problem remains unsolved, several multi-category objects detectors (for up to hundreds or thousands of categories) have been developed. One means is by feature sharing and boosting. === Boosting for binary categorization === AdaBoost can be used for face detection as an example of binary categorization. The two categories are faces versus background. The general algorithm is as follows: Form a large set of simple features Initialize weights for training images For T rounds Normalize the weights For available features from the set, train a classifier using a single feature and evaluate the training error Choose the classifier with the lowest error Update the weights of the training images: increase if classified wrongly by this classifier, decrease if correctly Form the final strong classifier as the linear combination of the T classifiers (coefficient larger if training error is small) After boosting, a classifier constructed from 200 features could yield a 95% detection rate under a 10 − 5 {\displaystyle 10^{-5}} false positive rate. Another application of boosting for binary categorization is a system that detects pedestrians using patterns of motion and appearance. This work is the first to combine both motion information and appearance information as features to detect a walking person. It takes a similar approach to the Viola-Jones object detection framework. === Boosting for multi-class categorization === Compared with binary categorization, multi-class categorization looks for common features that can be shared across the categories at the same time. They turn to be more generic edge like features. During learning, the detectors for each category can be trained jointly. Compared with training separately, it generalizes better, needs less training data, and requires fewer features to achieve the same performance. The main flow of the algorithm is similar to the binary case. What is different is that a measure of the joint training error shall be defined in advance. During each iteration the algorithm chooses a classifier of a single feature (features that can be shared by more categories shall be encouraged). This can be done via converting multi-class classification into a binary one (a set of categories versus the rest), or by introducing a penalty error from the categories that do not have the feature of the classifier. In the paper "Sharing visual features for multiclass and multiview object detection", A. Torralba et al. used GentleBoost for boosting and showed that when training data is limited, learning via sharing features does a much better job than no sharing, given same boosting rounds. Also, for a given performance level, the total number of features required (and therefore the run time cost of the classifier) for the feature sharing detectors, is observed to scale approximately logarithmically with the number of class, i.e., slower than linear growth in the non-sharing case. Similar results are shown in the paper "Incremental learning of object detectors using a visual shape alphabet", yet the authors used AdaBoost for boosting. == Convex vs. non-convex boosting algorithms == Boosting algorithms can be based on convex or non-convex optimization algorithms. Convex algorithms, such as AdaBoost and LogitBoost, can be "defeated" by random noise such that they can't learn basic and learnable combinations of weak hypotheses. This limitation was pointed out by Long & Servedio in 2008. However, by 2009, multiple authors demonstrated that boosting algorithms based on non-convex optimization, such as BrownBoost, can learn from nois
Soft independent modelling of class analogies
Soft independent modelling by class analogy (SIMCA) is a statistical method for supervised classification of data. The method requires a training data set consisting of samples (or objects) with a set of attributes and their class membership. The term soft refers to the fact the classifier can identify samples as belonging to multiple classes and not necessarily producing a classification of samples into non-overlapping classes. == Method == In order to build the classification models, the samples belonging to each class need to be analysed using principal component analysis (PCA); only the significant components are retained. For a given class, the resulting model then describes either a line (for one Principal Component or PC), plane (for two PCs) or hyper-plane (for more than two PCs). For each modelled class, the mean orthogonal distance of training data samples from the line, plane, or hyper-plane (calculated as the residual standard deviation) is used to determine a critical distance for classification. This critical distance is based on the F-distribution and is usually calculated using 95% or 99% confidence intervals. New observations are projected into each PC model and the residual distances calculated. An observation is assigned to the model class when its residual distance from the model is below the statistical limit for the class. The observation may be found to belong to multiple classes and a measure of goodness of the model can be found from the number of cases where the observations are classified into multiple classes. The classification efficiency is usually indicated by Receiver operating characteristics. In the original SIMCA method, the ends of the hyper-plane of each class are closed off by setting statistical control limits along the retained principal components axes (i.e., score value between plus and minus 0.5 times score standard deviation). More recent adaptations of the SIMCA method close off the hyper-plane by construction of ellipsoids (e.g. Hotelling's T2 or Mahalanobis distance). With such modified SIMCA methods, classification of an object requires both that its orthogonal distance from the model and its projection within the model (i.e. score value within the region defined by the ellipsoid) are not significant. == Application == SIMCA as a method of classification has gained widespread use especially in applied statistical fields such as chemometrics and spectroscopic data analysis.
Relation network
A relation network (RN) is an artificial neural network component with a structure that can reason about relations among objects. An example category of such relations is spatial relations (above, below, left, right, in front of, behind). RNs can infer relations, they are data efficient, and they operate on a set of objects without regard to the objects' order. == History == In June 2017, DeepMind announced the first relation network. It claimed that the technology had achieved "superhuman" performance on multiple question-answering problem sets. == Design == RNs constrain the functional form of a neural network to capture the common properties of relational reasoning. These properties are explicitly added to the system, rather than established by learning just as the capacity to reason about spatial, translation-invariant properties is explicitly part of convolutional neural networks (CNN). The data to be considered can be presented as a simple list or as a directed graph whose nodes are objects and whose edges are the pairs of objects whose relationships are to be considered. The RN is a composite function: R N ( O ) = f ϕ ( ∑ i , j g θ ( o i , o j , q ) ) , {\displaystyle RN\left(O\right)=f_{\phi }\left(\sum _{i,j}g_{\theta }\left(o_{i},o_{j},q\right)\right),} where the input is a set of "objects" O = { o 1 , o 2 , . . . , o n } , o i ∈ R m {\displaystyle O=\left\lbrace o_{1},o_{2},...,o_{n}\right\rbrace ,o_{i}\in \mathbb {R} ^{m}} is the ith object, and fφ and gθ are functions with parameters φ and θ, respectively and q is the question. fφ and gθ are multilayer perceptrons, while the 2 parameters are learnable synaptic weights. RNs are differentiable. The output of gθ is a "relation"; therefore, the role of gθ is to infer any ways in which two objects are related. Image (128x128 pixel) processing is done with a 4-layer CNN. Outputs from the CNN are treated as the objects for relation analysis, without regard for what those "objects" explicitly represent. Questions were processed with a long short-term memory network.
Azure Maps
Azure Maps is a suite of cloud-based, location-based services provided by Microsoft as part of the company's Azure platform. The platform provides geospatial and location-based services via REST APIs and software development kits (SDKs). The service is typically used to integrate maps or geospatial data into applications. Azure Maps differs from Microsoft's other enterprise mapping service, Bing Maps, in its pricing model, focus on privacy, and its level of integration into the broader Azure cloud ecosystem. == History == Azure Maps was first introduced in public preview mode under the name "Azure Location Based Services" in 2017, primarily as an enterprise solution. The services was intended to add mapping and location-based functionality onto the existing Azure cloud services suite, seen as a critical part of Microsoft's broader Internet-of-Things (IoT) strategy. The preview version included APIs which could be used to develop location aware apps for use cases such as logistics and mobility. In 2018, the software was renamed "Azure Maps," and became generally available to the public, and a number of new functions were added, including route calculation, travel time calculation, and incorporation of real-time traffic data and incident information. Azure Maps was integrated with Azure IoT Central in 2018, which added tracking, monitoring, and geofencing capabilities. A set of mobility APIs on were added in 2019, with applications such as use in public transport apps and shared bicycle fleet management. “Azure Maps Creator,” which converts private facility floor plans into indoor map data, was also introduced in 2019. Some commentators linked these services to Microsoft's broader development of augmented reality products. In 2020, Azure Maps Visual for Power BI was released, integrating location-based features and mapping capabilities into Microsoft's business intelligence software. An elevation API (which was later retired), geolocation services, and an iOS and Android software development kit were introduced in 2021. In 2022, support for historical weather, air quality, and tropical storm data was made generally available and custom styling for indoor maps was also introduced. In 2023, Azure Maps was certified as HIPAA compliant in a move to target healthcare and health insurance companies. == Functionality == === Geocoding === Geocoding is one of the core functionalities of Azure Maps, converting addresses or place names into geographic coordinates. Batch geocoding is used to process large amounts of address data, a function used for route optimization and spatial analysis. === Reverse geocoding === Reverse geocoding derives human-readable information from geographic coordinates like longitude and latitude, used in navigation and by geographic information systems. === Routing === Azure Maps uses map data and routing algorithms to calculate the shortest or fastest routes between locations based on factors like vehicle size and type, traffic conditions, and distance. Routing also supports multi-modal routing, which include multiple modes of transport in a single trip, including cycling, walking, and ferries. This functionality is used for location-based searches and route optimization in applications like fleet management, proximity marketing, and emergency services as well as logistics and delivery, urban planning, ride sharing apps, and outdoor activities. === Map visualization === The platform supports map visualizations that can be modified to reflect real-time data (including from IoT sensors) as well as historical data patterns. Visualizations include heat maps, street maps, satellite imagery and other custom data layers. Maps are rendered using raster or vector tiles which reduce the load of displaying large data sets or complex maps. This can be used in various applications in areas like transportation, smart cities, retail and marketing, public health, and environmental monitoring. For example, it can be used for tracking the spread of diseases or measuring the impact of changing climatic patterns. === Geofencing and spatial analytics === Azure Maps supports polygonal geofencing, which enables the definition of custom geographic boundaries. Geofenced areas can be monitored in real-time for events of interest. For example, an application could send an alert when equipment or persons enter or leave a defined area. Tools for analyzing historical geofencing data are also available via the APIs for optimization purposes. == Industry usage == Azure Maps' geofencing function has seen usage in the construction industry, designating hazardous areas for safety purposes and sending alerts if anyone enters the area. Private facility maps are used by construction companies for monitoring large construction sites to increase productivity and prevent accidents or damage. In emergency management, New Zealand based company Beca has used Azure Maps to provide analysis on the impact of earthquakes to users, including information on the severity and location of an earthquake and the impact on affected properties. Alaska's Department of Transportation uses Azure Maps as part of an information system providing weather-related warnings and analytics to road crews. Airmap, an airspace management platform for drones, uses Azure Maps. Azure Maps has also been used in conjunction with Azure Monitor for risk monitoring by an insurance company. Other companies that use or have used Azure Maps include BMW, Banco Santander, Jvion, MV Transportation, C.H. Robertson, Wise Skulls, Tata Consultancy Services, Providence Health and Services, Gas Brasiliano Distribuidora S.A., Shell plc, Persistent Systems, Phase 2 Dining and Entertainment, Symbio, HID, Globant, and Insight Enterprises. == Partnerships == Azure Maps and TomTom have been partners since 2016, and TomTom provides location data to Azure Maps and can process data from Azure Maps for mapping purposes. In 2021, Azure Maps partnered with AccuWeather to make climatic data available via its APIs, making weather data along all parts of calculated routes available for mobility and logistics purposes. Microsoft has partnered with Esri, the developer of ArcGIS, and there is cross-compatibility between Azure and ArcGIS so that data from Azure Maps can be integrated into ArcGIS and vice versa. Azure Maps partnered with Moovit in 2019, a startup providing software that interfaces with public transport data. Moovit's database on global public transit networks, including information on which stations and facilities are wheelchair accessible, was linked to Azure Maps. This service was noted for its use increasing accessibility to public transport for the visually impaired by means of voice activated route planning assistance. NORAD has used some Azure Maps functions for their NORAD Tracks Santa website during Christmas holidays. == Components == === REST APIs === Various APIs cover the major functionalities across Azure Maps: Data registry API Geolocation API Render API Route API Search API Spatial API Time zone API Traffic API Weather API === SDKs === Azure Maps SDKs uses MapLibre-style specifications and open source MapLibre GL-based libraries as a rendering engine. The Web SDK is used for developing web apps with maps and location-based data and functionality. It includes a map control module as well as modules with drawing tools. It also supports Azure Maps Creator and various spatial data formats. The platform also includes a set of REST SDKs for developers integrating Azure Maps REST APIs into Python, C#, Java or JavaScript applications. Azure Maps also includes Android and iOS SDKs used for developing applications for Android and Apple devices. === Azure Maps Creator === Azure Maps Creator is a tool for generating custom maps for locations like large office complexes, construction sites, or university campuses. These maps can then be integrated into applications and used with other Azure Maps functions for purposes such as wayfinding and maintenance and security in building automation contexts. === Azure Maps Visual for Power BI === Azure Maps is integrated with Microsoft Power BI, a graphical tool for producing data visualizations. Since July 2020, Power BI can be used in conjunction with Azure Maps for developing map-based data visualizations. This functionality entered general availability in May 2023.
Multilayer perceptron
In deep learning, a multilayer perceptron (MLP) is a kind of modern feedforward neural network consisting of fully connected neurons with nonlinear activation functions, organized in layers, notable for being able to distinguish data that is not linearly separable. Modern neural networks are trained using backpropagation and are colloquially referred to as "vanilla" networks. MLPs grew out of an effort to improve on single-layer perceptrons, which could only be applied to linearly separable data. A perceptron traditionally used a Heaviside step function as its nonlinear activation function. However, the backpropagation algorithm requires that modern MLPs use continuous activation functions such as sigmoid or ReLU. Multilayer perceptrons form the basis of deep learning, and are applicable across a vast set of diverse domains. == Timeline == In 1943, Warren McCulloch and Walter Pitts proposed the binary artificial neuron as a logical model of biological neural networks. In 1958, Frank Rosenblatt proposed the multilayered perceptron model, consisting of an input layer, a hidden layer with randomized weights that did not learn, and an output layer with learnable connections. In 1962, Rosenblatt published many variants and experiments on perceptrons in his book Principles of Neurodynamics, including up to 2 trainable layers by "back-propagating errors". However, it was not the backpropagation algorithm, and he did not have a general method for training multiple layers. In 1965, Alexey Grigorevich Ivakhnenko and Valentin Lapa published Group Method of Data Handling. It was one of the first deep learning methods, used to train an eight-layer neural net in 1971. In 1967, Shun'ichi Amari reported the first multilayered neural network trained by stochastic gradient descent, was able to classify non-linearily separable pattern classes. Amari's student Saito conducted the computer experiments, using a five-layered feedforward network with two learning layers. Backpropagation was independently developed multiple times in early 1970s. The earliest published instance was Seppo Linnainmaa's master thesis (1970). Paul Werbos developed it independently in 1971, but had difficulty publishing it until 1982. In 1986, David E. Rumelhart et al. popularized backpropagation. In 2003, interest in backpropagation networks returned due to the successes of deep learning being applied to language modelling by Yoshua Bengio with co-authors. In 2021, a very simple NN architecture combining two deep MLPs with skip connections and layer normalizations was designed and called MLP-Mixer; its realizations featuring 19 to 431 millions of parameters were shown to be comparable to vision transformers of similar size on ImageNet and similar image classification tasks. == Mathematical foundations == === Activation function === If a multilayer perceptron has a linear activation function in all neurons, that is, a linear function that maps the weighted inputs to the output of each neuron, then linear algebra shows that any number of layers can be reduced to a two-layer input-output model. In MLPs some neurons use a nonlinear activation function that was developed to model the frequency of action potentials, or firing, of biological neurons. The two historically common activation functions are both sigmoids, and are described by y ( v i ) = tanh ( v i ) and y ( v i ) = ( 1 + e − v i ) − 1 {\displaystyle y(v_{i})=\tanh(v_{i})~~{\textrm {and}}~~y(v_{i})=(1+e^{-v_{i}})^{-1}} . The first is a hyperbolic tangent that ranges from −1 to 1, while the other is the logistic function, which is similar in shape but ranges from 0 to 1. Here y i {\displaystyle y_{i}} is the output of the i {\displaystyle i} th node (neuron) and v i {\displaystyle v_{i}} is the weighted sum of the input connections. Alternative activation functions have been proposed, including the rectifier and softplus functions. More specialized activation functions include radial basis functions (used in radial basis networks, another class of supervised neural network models). In recent developments of deep learning the rectified linear unit (ReLU) is more frequently used as one of the possible ways to overcome the numerical problems related to the sigmoids. === Layers === The MLP consists of three or more layers (an input and an output layer with one or more hidden layers) of nonlinearly-activating nodes. Since MLPs are fully connected, each node in one layer connects with a certain weight w i j {\displaystyle w_{ij}} to every node in the following layer. === Learning === Learning occurs in the perceptron by changing connection weights after each piece of data is processed, based on the amount of error in the output compared to the expected result. This is an example of supervised learning, and is carried out through backpropagation, a generalization of the least mean squares algorithm in the linear perceptron. We can represent the degree of error in an output node j {\displaystyle j} in the n {\displaystyle n} th data point (training example) by e j ( n ) = d j ( n ) − y j ( n ) {\displaystyle e_{j}(n)=d_{j}(n)-y_{j}(n)} , where d j ( n ) {\displaystyle d_{j}(n)} is the desired target value for n {\displaystyle n} th data point at node j {\displaystyle j} , and y j ( n ) {\displaystyle y_{j}(n)} is the value produced by the perceptron at node j {\displaystyle j} when the n {\displaystyle n} th data point is given as an input. The node weights can then be adjusted based on corrections that minimize the error in the entire output for the n {\displaystyle n} th data point, given by E ( n ) = 1 2 ∑ output node j e j 2 ( n ) {\displaystyle {\mathcal {E}}(n)={\frac {1}{2}}\sum _{{\text{output node }}j}e_{j}^{2}(n)} . Using gradient descent, the change in each weight w i j {\displaystyle w_{ij}} is Δ w j i ( n ) = − η ∂ E ( n ) ∂ v j ( n ) y i ( n ) {\displaystyle \Delta w_{ji}(n)=-\eta {\frac {\partial {\mathcal {E}}(n)}{\partial v_{j}(n)}}y_{i}(n)} where y i ( n ) {\displaystyle y_{i}(n)} is the output of the previous neuron i {\displaystyle i} , and η {\displaystyle \eta } is the learning rate, which is selected to ensure that the weights quickly converge to a response, without oscillations. In the previous expression, ∂ E ( n ) ∂ v j ( n ) {\displaystyle {\frac {\partial {\mathcal {E}}(n)}{\partial v_{j}(n)}}} denotes the partial derivate of the error E ( n ) {\displaystyle {\mathcal {E}}(n)} according to the weighted sum v j ( n ) {\displaystyle v_{j}(n)} of the input connections of neuron i {\displaystyle i} . The derivative to be calculated depends on the induced local field v j {\displaystyle v_{j}} , which itself varies. It is easy to prove that for an output node this derivative can be simplified to − ∂ E ( n ) ∂ v j ( n ) = e j ( n ) ϕ ′ ( v j ( n ) ) {\displaystyle -{\frac {\partial {\mathcal {E}}(n)}{\partial v_{j}(n)}}=e_{j}(n)\phi ^{\prime }(v_{j}(n))} where ϕ ′ {\displaystyle \phi ^{\prime }} is the derivative of the activation function described above, which itself does not vary. The analysis is more difficult for the change in weights to a hidden node, but it can be shown that the relevant derivative is − ∂ E ( n ) ∂ v j ( n ) = ϕ ′ ( v j ( n ) ) ∑ k − ∂ E ( n ) ∂ v k ( n ) w k j ( n ) {\displaystyle -{\frac {\partial {\mathcal {E}}(n)}{\partial v_{j}(n)}}=\phi ^{\prime }(v_{j}(n))\sum _{k}-{\frac {\partial {\mathcal {E}}(n)}{\partial v_{k}(n)}}w_{kj}(n)} . This depends on the change in weights of the k {\displaystyle k} th nodes, which represent the output layer. So to change the hidden layer weights, the output layer weights change according to the derivative of the activation function, and so this algorithm represents a backpropagation of the activation function.