Mathematics of neural networks in machine learning

Mathematics of neural networks in machine learning

An artificial neural network (ANN) or neural network combines biological principles with advanced statistics to solve problems in domains such as pattern recognition and game-play. ANNs adopt the basic model of neuron analogues connected to each other in a variety of ways. == Structure == === Neuron === A neuron with label j {\displaystyle j} receiving an input p j ( t ) {\displaystyle p_{j}(t)} from predecessor neurons consists of the following components: an activation a j ( t ) {\displaystyle a_{j}(t)} , the neuron's state, depending on a discrete time parameter, an optional threshold θ j {\displaystyle \theta _{j}} , which stays fixed unless changed by learning, an activation function f {\displaystyle f} that computes the new activation at a given time t + 1 {\displaystyle t+1} from a j ( t ) {\displaystyle a_{j}(t)} , θ j {\displaystyle \theta _{j}} and the net input p j ( t ) {\displaystyle p_{j}(t)} giving rise to the relation a j ( t + 1 ) = f ( a j ( t ) , p j ( t ) , θ j ) , {\displaystyle a_{j}(t+1)=f(a_{j}(t),p_{j}(t),\theta _{j}),} and an output function f out {\displaystyle f_{\text{out}}} computing the output from the activation o j ( t ) = f out ( a j ( t ) ) . {\displaystyle o_{j}(t)=f_{\text{out}}(a_{j}(t)).} Often the output function is simply the identity function. An input neuron has no predecessor but serves as input interface for the whole network. Similarly an output neuron has no successor and thus serves as output interface of the whole network. === Propagation function === The propagation function computes the input p j ( t ) {\displaystyle p_{j}(t)} to the neuron j {\displaystyle j} from the outputs o i ( t ) {\displaystyle o_{i}(t)} and typically has the form p j ( t ) = ∑ i o i ( t ) w i j . {\displaystyle p_{j}(t)=\sum _{i}o_{i}(t)w_{ij}.} === Bias === A bias term can be added, changing the form to the following: p j ( t ) = ∑ i o i ( t ) w i j + w 0 j , {\displaystyle p_{j}(t)=\sum _{i}o_{i}(t)w_{ij}+w_{0j},} where w 0 j {\displaystyle w_{0j}} is a bias. == Neural networks as functions == Neural network models can be viewed as defining a function that takes an input (observation) and produces an output (decision) f : X → Y {\displaystyle \textstyle f:X\rightarrow Y} or a distribution over X {\displaystyle \textstyle X} or both X {\displaystyle \textstyle X} and Y {\displaystyle \textstyle Y} . Sometimes models are intimately associated with a particular learning rule. A common use of the phrase "ANN model" is really the definition of a class of such functions (where members of the class are obtained by varying parameters, connection weights, or specifics of the architecture such as the number of neurons, number of layers or their connectivity). Mathematically, a neuron's network function f ( x ) {\displaystyle \textstyle f(x)} is defined as a composition of other functions g i ( x ) {\displaystyle \textstyle g_{i}(x)} , that can further be decomposed into other functions. This can be conveniently represented as a network structure, with arrows depicting the dependencies between functions. A widely used type of composition is the nonlinear weighted sum, where f ( x ) = K ( ∑ i w i g i ( x ) ) {\displaystyle \textstyle f(x)=K\left(\sum _{i}w_{i}g_{i}(x)\right)} , where K {\displaystyle \textstyle K} (commonly referred to as the activation function) is some predefined function, such as the hyperbolic tangent, sigmoid function, softmax function, or rectifier function. The important characteristic of the activation function is that it provides a smooth transition as input values change, i.e. a small change in input produces a small change in output. The following refers to a collection of functions g i {\displaystyle \textstyle g_{i}} as a vector g = ( g 1 , g 2 , … , g n ) {\displaystyle \textstyle g=(g_{1},g_{2},\ldots ,g_{n})} . This figure depicts such a decomposition of f {\displaystyle \textstyle f} , with dependencies between variables indicated by arrows. These can be interpreted in two ways. The first view is the functional view: the input x {\displaystyle \textstyle x} is transformed into a 3-dimensional vector h {\displaystyle \textstyle h} , which is then transformed into a 2-dimensional vector g {\displaystyle \textstyle g} , which is finally transformed into f {\displaystyle \textstyle f} . This view is most commonly encountered in the context of optimization. The second view is the probabilistic view: the random variable F = f ( G ) {\displaystyle \textstyle F=f(G)} depends upon the random variable G = g ( H ) {\displaystyle \textstyle G=g(H)} , which depends upon H = h ( X ) {\displaystyle \textstyle H=h(X)} , which depends upon the random variable X {\displaystyle \textstyle X} . This view is most commonly encountered in the context of graphical models. The two views are largely equivalent. In either case, for this particular architecture, the components of individual layers are independent of each other (e.g., the components of g {\displaystyle \textstyle g} are independent of each other given their input h {\displaystyle \textstyle h} ). This naturally enables a degree of parallelism in the implementation. Networks such as the previous one are commonly called feedforward, because their graph is a directed acyclic graph. Networks with cycles are commonly called recurrent. Such networks are commonly depicted in the manner shown at the top of the figure, where f {\displaystyle \textstyle f} is shown as dependent upon itself. However, an implied temporal dependence is not shown. == Backpropagation == Backpropagation training algorithms fall into three categories: steepest descent (with variable learning rate and momentum, resilient backpropagation); quasi-Newton (Broyden–Fletcher–Goldfarb–Shanno, one step secant); Levenberg–Marquardt and conjugate gradient (Fletcher–Reeves update, Polak–Ribiére update, Powell–Beale restart, scaled conjugate gradient). === Algorithm === Let N {\displaystyle N} be a network with e {\displaystyle e} connections, m {\displaystyle m} inputs and n {\displaystyle n} outputs. Below, x 1 , x 2 , … {\displaystyle x_{1},x_{2},\dots } denote vectors in R m {\displaystyle \mathbb {R} ^{m}} , y 1 , y 2 , … {\displaystyle y_{1},y_{2},\dots } vectors in R n {\displaystyle \mathbb {R} ^{n}} , and w 0 , w 1 , w 2 , … {\displaystyle w_{0},w_{1},w_{2},\ldots } vectors in R e {\displaystyle \mathbb {R} ^{e}} . These are called inputs, outputs and weights, respectively. The network corresponds to a function y = f N ( w , x ) {\displaystyle y=f_{N}(w,x)} which, given a weight w {\displaystyle w} , maps an input x {\displaystyle x} to an output y {\displaystyle y} . In supervised learning, a sequence of training examples ( x 1 , y 1 ) , … , ( x p , y p ) {\displaystyle (x_{1},y_{1}),\dots ,(x_{p},y_{p})} produces a sequence of weights w 0 , w 1 , … , w p {\displaystyle w_{0},w_{1},\dots ,w_{p}} starting from some initial weight w 0 {\displaystyle w_{0}} , usually chosen at random. These weights are computed in turn: first compute w i {\displaystyle w_{i}} using only ( x i , y i , w i − 1 ) {\displaystyle (x_{i},y_{i},w_{i-1})} for i = 1 , … , p {\displaystyle i=1,\dots ,p} . The output of the algorithm is then w p {\displaystyle w_{p}} , giving a new function x ↦ f N ( w p , x ) {\displaystyle x\mapsto f_{N}(w_{p},x)} . The computation is the same in each step, hence only the case i = 1 {\displaystyle i=1} is described. w 1 {\displaystyle w_{1}} is calculated from ( x 1 , y 1 , w 0 ) {\displaystyle (x_{1},y_{1},w_{0})} by considering a variable weight w {\displaystyle w} and applying gradient descent to the function w ↦ E ( f N ( w , x 1 ) , y 1 ) {\displaystyle w\mapsto E(f_{N}(w,x_{1}),y_{1})} to find a local minimum, starting at w = w 0 {\displaystyle w=w_{0}} . This makes w 1 {\displaystyle w_{1}} the minimizing weight found by gradient descent. == Learning pseudocode == To implement the algorithm above, explicit formulas are required for the gradient of the function w ↦ E ( f N ( w , x ) , y ) {\displaystyle w\mapsto E(f_{N}(w,x),y)} where the function is E ( y , y ′ ) = | y − y ′ | 2 {\displaystyle E(y,y')=|y-y'|^{2}} . The learning algorithm can be divided into two phases: propagation and weight update. === Propagation === Propagation involves the following steps: Propagation forward through the network to generate the output value(s) Calculation of the cost (error term) Propagation of the output activations back through the network using the training pattern target to generate the deltas (the difference between the targeted and actual output values) of all output and hidden neurons. === Weight update === For each weight: Multiply the weight's output delta and input activation to find the gradient of the weight. Subtract the ratio (percentage) of the weight's gradient from the weight. The learning rate is the ratio (percentage) that influences the speed and quality of learning. The greater the ratio, the faster the neuron trains, but the lower the ratio, the more accurat

ChatGPT

ChatGPT is a generative artificial intelligence chatbot developed by OpenAI. Originally released in November 2022, the product uses large language models—specifically generative pre-trained transformers (GPTs)—to generate text, speech, and images in response to user prompts. ChatGPT accelerated the AI boom, an ongoing period marked by rapid investment and public attention toward the field of artificial intelligence (AI). OpenAI operates the service on a freemium model. Users can interact with ChatGPT through text, audio, and image prompts. ChatGPT was quickly adopted, reaching 100 million monthly active users two months after its release and 900 million weekly active users in February 2026. It has been lauded for its potential to transform numerous professional fields, and has instigated public debate about the nature of creativity and the future of knowledge work. The chatbot has also been criticized for its limitations and potential for unethical use. It can generate plausible-sounding but incorrect or nonsensical answers, known as hallucinations. Biases in its training data have been reflected in its responses. The chatbot can facilitate academic dishonesty, generate misinformation, and create malicious code. The ethics of its development, particularly the use of copyrighted content as training data, have also drawn controversy. == Features == ChatGPT is a chatbot and AI assistant built on large language model (LLM) technology. It is designed to generate human-like text and can carry out a wide variety of tasks. These include, among many others, writing and debugging computer programs, composing music, scripts, fairy tales, and essays, answering questions (sometimes at a level exceeding that of an average human test-taker), and generating business concepts. ChatGPT is frequently used for translation and summarization tasks, and can simulate interactive environments such as a Linux terminal, a multi-user chat room, or simple text-based games such as tic-tac-toe. Users interact with ChatGPT through conversations which consist of text, audio, and image inputs and outputs. The user's inputs to these conversations are referred to as prompts. An optional "Memory" feature allows users to tell ChatGPT to memorize specific information. Another option allows ChatGPT to recall old conversations. GPT-based moderation classifiers are used to reduce the risk of harmful outputs being presented to users. In March 2023, OpenAI added support for plugins for ChatGPT. This includes both plugins made by OpenAI, such as web browsing and code interpretation, and external plugins from developers such as Expedia, OpenTable, and Zapier. From October to December 2024, ChatGPT Search was deployed. It allows ChatGPT to search the web in an attempt to make more accurate and up-to-date responses. It increased OpenAI's direct competition with major search engines. OpenAI allows businesses to tailor how their content appears in the ChatGPT Search results and influence what sources are used. In December 2024, OpenAI launched a new feature allowing users to call ChatGPT with a telephone for up to 15 minutes per month for free. In September 2025, OpenAI added a feature called Pulse, which generates a daily analysis of a user's chats and connected apps such as Gmail and Google Calendar. In October 2025, OpenAI launched ChatGPT Atlas, a browser integrating the ChatGPT assistant directly into web navigation, to compete with existing browsers such as Google Chrome. It has an additional feature called "agentic mode" that allows it to take online actions for the user. === Paid tier === ChatGPT was initially free to the public and remains free in a limited capacity. In February 2023, OpenAI launched a premium service, ChatGPT Plus, that costs US$20 per month. What was offered on the paid plan versus the free tier changed as OpenAI has continued to update ChatGPT, and a Pro tier at $200/mo was introduced in December 2024. The Pro launch coincided with the release of the o1 model. In August 2025, ChatGPT Go was offered in India for ₹399 per month. The plan has higher limits than the free version. === Mobile apps === In May-July 2023, OpenAI began offering ChatGPT iOS and Android apps. ChatGPT can also power Android's assistant. An app for Windows launched on the Microsoft Store on October 15, 2024. === Languages === OpenAI met Icelandic President Guðni Th. Jóhannesson in 2022. In 2023, OpenAI worked with a team of 40 Icelandic volunteers to fine-tune ChatGPT's Icelandic conversation skills as a part of Iceland's attempts to preserve the Icelandic language. ChatGPT (based on GPT-4) was better able to translate Japanese to English when compared to Bing, Bard, and DeepL Translator in 2023. In December 2023, the Albanian government decided to use ChatGPT for the rapid translation of European Union documents and the analysis of required changes needed for Albania's accession to the EU. Several studies have shown that ChatGPT can outperform Google Translate in some mainstream translation tasks. However, as of 2024, no machine translation services match human expert performance. In August 2024, a representative of the Asia Pacific wing of OpenAI made a visit to Taiwan, during which a demonstration of ChatGPT's Chinese abilities was made. ChatGPT's Mandarin Chinese abilities were lauded, but the ability of the AI to produce content in Mandarin Chinese in a Taiwanese accent was found to be "less than ideal" due to differences between mainland Mandarin Chinese and Taiwanese Mandarin. === GPT Store === In November 2023, OpenAI released GPT Builder, a tool allowing users to customize ChatGPT's behavior for a specific use case. The customized systems are referred to as GPTs. In January 2024, OpenAI launched the GPT Store, a marketplace for GPTs. At launch, OpenAI included more than 3 million GPTs created by GPT Builder users in the GPT Store. === ChatGPT Apps === In September 2025, OpenAI added support for Model Context Protocol (MCP) to ChatGPT apps. When enabled in developer mode, this allows for improved third-party access to ChatGPT tools and servers. === Deep Research === In February 2025, OpenAI released Deep Research, a feature that generates reports based on extensive web searches. It was initially based on the reasoning model o3 and took 5 to 30 minutes per report. === Images === In October 2023, OpenAI's image generation model DALL-E 3 was integrated into ChatGPT. The integration used ChatGPT to write prompts for DALL-E guided by conversations with users. In March 2025, OpenAI updated ChatGPT to generate images using GPT Image instead of DALL-E. One of the most significant improvements was in the generation of text within images, which is especially useful for branded content. However, this ability is noticeably worse in non-Latin alphabets. The model can also generate new images based on existing ones provided in the prompt. These images are generated with C2PA metadata, which can be used to verify that they are AI-generated. OpenAI has emplaced additional safeguards to prevent what the company deems to be harmful image generation. === Agents === In 2025, OpenAI added several features to make ChatGPT more agentic (capable of autonomously performing longer tasks). In January, Operator was released. It was capable of autonomously performing tasks through web browser interactions, including filling forms, placing online orders, scheduling appointments, and other browser-based tasks. It was controlling a software environment inside a virtual machine with limited internet connectivity and with safety restrictions. It struggled with complex user interfaces. In May 2025, OpenAI introduced an agent for coding named Codex. It is capable of writing software, answering codebase questions, running tests, and proposing pull requests. It is based on a fine-tuned version of OpenAI o3. It has two versions, one running in a virtual machine in the cloud, and one where the agent runs in the cloud, but performs actions on a local machine connected via API. In July 2025, OpenAI released ChatGPT agent, an AI agent that can perform multi-step tasks. Like Operator, it controls a virtual computer. It also inherits from Deep Research's ability to gather and summarize significant volumes of information. The user can interrupt tasks or provide additional instructions as needed. In September 2025, OpenAI partnered with Stripe, Inc. to release Agentic Commerce Protocol, enabling purchases through ChatGPT. At launch, the feature was limited to purchases on Etsy from US users with a payment method linked to their OpenAI account. OpenAI takes an undisclosed cut from the merchant's payment. === ChatGPT Health === On January 7, 2026, OpenAI introduced a feature called "ChatGPT Health", whereby ChatGPT can discuss the user's health in a way that is separate from other chats. The feature is not available for users in the United Kingdom, Switzerland, or the European Economic Area, and is available on a waitli

Commercial skipping

Commercial skipping is a feature of some digital video recorders that makes it possible to automatically skip commercials in recorded programs. This feature created controversy, with major television networks and movie studios claiming it violates copyright and should be banned. == History == After the video cassette recorder (VCR) became popular in the 1980s, the television industry began studying the impact of users fast forwarding through commercials. Advertising agencies fought the trend by making them more entertaining. For many years, video recorders manufactured for the Japanese market have been able to skip advertisements automatically, which is done by detecting when foreign language audio overdub tracks provided for many programmes go silent, as advertisements were broadcast with a single language only. The first digital video recorder (DVR) with a built-in commercial skipping feature was ReplayTV with its "4000 Series" and "5000 Series" units. In 2002, the main television networks and movie studios sued ReplayTV, claiming that skipping advertisements during replay violates copyright. Later, five owners of ReplayTV represented by Electronic Frontier Foundation and attorneys Ira Rothken and Richard Wiebe countersued, asking the federal judge to uphold consumers' rights to record TV shows and skip commercials, claiming that features like commercial skipping help parents protect their kids from excessive consumerism. ReplayTV ended up filing for bankruptcy in 2003 after fighting a copyright infringement suit over the ReplayTV's ability to skip commercials. === Commercial skipping software === In addition to the DVR devices which existed in the private market since the late 1990s, towards the mid-2000s, due to the significant advances in home computers, Home theater PCs started gaining popularity in the private market and many users began using their Home theater PCs in their living room for entertainment purposes. Following this, many DVR programs were developed, including popular programs such as Windows Media Center, which contained all of the features of the DVR devices in addition to advanced features such as HDTV and the use of Multiple TV Tuner Cards. Some independent developers began developing independent software capable of skipping the commercial segments when playing recorded videos, and permanently removing the commercial segments from recorded video files. By 2014, many DVR programs such as Windows Media Center, SageTV and MythTV had the capability to skip commercials segments in recorded TV broadcasts after installing third-party add-ons such as DVRMSToolbox, Comskip and ShowAnalyzer, which use various advanced techniques to locate the commercial segments in the video files and save their locations to text files. The text files can also be fed into programs such as MEncoder or DVRMSToolboxGUI which can delete the commercial segments from the recorded video files. A few third-party tools such as MCEBuddy automate detection and removal/marking of commercials. One of the weaknesses of commercial skippers is that, operating automatically, they may misidentify program material as a commercial. Some programs like MCEBuddy provide the ability to fine-tune commercial detection for groups of files (e.g. by channel or country) and provide tools to manually fine-tune commercial segments for individual files. In May 2012, the US Dish Network began offering a DVR with what it calls AutoHop. The device would automatically skip commercials when displaying programming that the viewer had previously recorded with the PrimeTime Anytime feature. It does not skip ads on any live programs. US broadcasters were angered at the news, and FOX embarked on legal action. Most, but not all, of Fox's claims were dismissed; ultimately an agreement was reached whereby AutoHop would only become available for Fox stations seven days after a program is transmitted; terms of the settlement were not disclosed. == The future of TV advertisements == The introduction of digital video recorders and services with skipping and fast-forward capabilities enables viewers to avoid viewing interruptive advertisements in recorded programs, either manually or automatically. While advertising separate to television shows can be skipped, advertising in TV shows themselves ("product placement") cannot be skipped. Streaming services such as Hulu show shorter advertisements with a countdown timer and tailored to the viewers interests, asking interactive questions like "Is this ad relevant to you?".

Electronic submission

Electronic submission refers to the submission of a document by electronic means: that is, via e-mail or a web form on the Internet, or on an electronic medium such as a compact disc, a hard disk or a USB flash drive. Traditionally, the term "manuscript" referred to anything that was explicitly "written by hand". However, in popular usage and especially in the context of computers and the internet, the term "manuscript" may even refer to documents (text or otherwise) typed out or prepared on typewriters and computers and can be extended to digital photographs and videos, and online surveys too. In other words, any manuscript prepared and submitted online can be considered to be an electronic submission. == History and early usage == There is no concrete data indicating when and by whom were electronic submissions used for the first time. However, research based universities in several countries have been encouraging the collection of course assignments and projects in the form of electronic submissions for almost a decade now. Several governments and organizations are also switching to electronic submissions for the collection of research papers, grant applications and government application forms. == Types of electronic submissions == Since modern computers can store and process information and data in virtually any format and with the Internet allowing easy transfer of this data, the number of scenarios in which submissions can be collected electronically has increased exponentially in the last few years. Some of these scenarios are described below. In most of these scenarios, submissions were collected on hard paper until the Information Technology revolution occurred. === Academic Submissions === Teachers, professors and teaching assistants often collect course assignments and projects electronically. Electronic submissions are usually collected using a web-based system which more often than not also helps in the management of submissions collected and stored on it. (Explained By Henny L, University of Lethbridge, AB, Canada) === Research Papers === In call-for-paper or academic conferences, prospective presenters are usually asked to submit a short abstract or a full paper on their presentation or research work electronically, which is reviewed before being accepted for the conference. === Proposals for Grants === Several grant-giving organizations like the NSA, W3C, NIA, NIH etc. require grant seekers to submit a proposal which if accepted result in the desired grants. A majority of these proposals are now submitted electronically on systems that also help in the managing and tracking the proposals submitted. === Articles for Publication === Magazines, newspapers and other publishing houses have begun accepting electronic submissions for articles from various sources - both internal (by journalists and writers hired by them) as well as external (by users and popular readers). The submitted articles are stored on a server hosted by the publication house or by a third-party Archived 2019-10-13 at the Wayback Machine vendor and are usually evaluated before being given a green signal. === Contests and Competition Entries === Almost every kind of contest or competition requires participants to submit an entry in a format described by the organizers of the contest. If the contest is an Internet-based one, then the entries or nominations for the contest are collected electronically using e-mail or other electronic means depending on feasibility and the choice of the organizers. === Government Applications === The governments of several countries are turning to electronic submission of applications and forms for various government procedures. Electronic submissions allow easier management of the applications and forms submitted. === Legal documents === Many legal documents may be submitted to the courts electronically. In England and Wales, the Civil Procedure Rules include a suitable "document exchange" as an acceptable "method of service". Case law in employment law cases has established that where a claim is submitted electronically, a prudent legal adviser should "check that it has been received and there must be systems in place for doing that". === Resumés and CVs === It has become commonplace for job-seekers to submit soft copies (electronic versions) of their resumés and CVs to recruiting agencies and online job portals. This is usually done over the Internet using e-mail or a pre-hosted web-based system. == Submission management systems == The art and science of collecting and managing electronic submissions is called Submission Management. Certain software vendors have begun developing submission management systems to assist in the collection, tracking and management of complex submission processes realized electronically. Most of these systems are web based and accessible from any device with a browser and an Internet connection. However, a majority of these systems are application specific and cannot be applied to all submission management scenarios. == Resistance to electronic submissions == Despite the easier management and tracking of electronic submissions compared to their paper-based counterparts, widespread adoption and use of electronic submissions and systems for managing them has been hampered by several facts, which include but are not limited to: Inconvenience while drawing figures, diagrams and equations on a computer Resistance to change and adoption of new technologies Lack of or limited access to the Internet.

Ajax (programming)

The Asynchronous JavaScript and XML, usually referred to as Ajax (or AJAX, ) is a set of web development techniques that uses various web technologies on the client-side to create asynchronous web applications. With Ajax, web applications can send and retrieve data from a server asynchronously (in the background) without interfering with the display and behaviour of the existing page. By decoupling the data interchange layer from the presentation layer, Ajax allows web pages and, by extension, web applications, to change content dynamically without the need to reload the entire page. In practice, modern implementations commonly utilize JSON instead of XML. Ajax is not a technology, but rather a programming pattern. HTML and CSS can be used in combination to mark up and style information. The webpage can be modified by JavaScript to dynamically display (and allow the user to interact with) the new information. The built-in XMLHttpRequest object is used to execute Ajax on webpages, allowing websites to load content onto the screen without refreshing the page. == History == In the early-to-mid 1990s, most Websites were based on complete HTML pages. Each user action required a complete new page to be loaded from the server. This process was inefficient, as reflected by the user experience: all page content disappeared, then the new page appeared. Each time the browser reloaded a page because of a partial change, all the content had to be re-sent, even though only some of the information had changed. This placed additional load on the server and made bandwidth a limiting factor in performance. The foundations of AJAX originate back in 1996 with the introduction of JavaScript 1. Developers quickly discovered that any HTML element which accepted a "src" attribute could be used to fetch remote data. By changing the src of a hidden frame, a developer could fetch remote data, process or display it without a page refresh. The remote data could be a string, JavaScript code, XML or a partial HTML page generated on the server. The same could be done with and tags, but many developers were alarmed at the concept of an executable GIF and preferred to use the hidden