In deep learning, a multilayer perceptron (MLP) is a kind of modern feedforward neural network consisting of fully connected neurons with nonlinear activation functions, organized in layers, notable for being able to distinguish data that is not linearly separable. Modern neural networks are trained using backpropagation and are colloquially referred to as "vanilla" networks. MLPs grew out of an effort to improve on single-layer perceptrons, which could only be applied to linearly separable data. A perceptron traditionally used a Heaviside step function as its nonlinear activation function. However, the backpropagation algorithm requires that modern MLPs use continuous activation functions such as sigmoid or ReLU. Multilayer perceptrons form the basis of deep learning, and are applicable across a vast set of diverse domains. == Timeline == In 1943, Warren McCulloch and Walter Pitts proposed the binary artificial neuron as a logical model of biological neural networks. In 1958, Frank Rosenblatt proposed the multilayered perceptron model, consisting of an input layer, a hidden layer with randomized weights that did not learn, and an output layer with learnable connections. In 1962, Rosenblatt published many variants and experiments on perceptrons in his book Principles of Neurodynamics, including up to 2 trainable layers by "back-propagating errors". However, it was not the backpropagation algorithm, and he did not have a general method for training multiple layers. In 1965, Alexey Grigorevich Ivakhnenko and Valentin Lapa published Group Method of Data Handling. It was one of the first deep learning methods, used to train an eight-layer neural net in 1971. In 1967, Shun'ichi Amari reported the first multilayered neural network trained by stochastic gradient descent, was able to classify non-linearily separable pattern classes. Amari's student Saito conducted the computer experiments, using a five-layered feedforward network with two learning layers. Backpropagation was independently developed multiple times in early 1970s. The earliest published instance was Seppo Linnainmaa's master thesis (1970). Paul Werbos developed it independently in 1971, but had difficulty publishing it until 1982. In 1986, David E. Rumelhart et al. popularized backpropagation. In 2003, interest in backpropagation networks returned due to the successes of deep learning being applied to language modelling by Yoshua Bengio with co-authors. In 2021, a very simple NN architecture combining two deep MLPs with skip connections and layer normalizations was designed and called MLP-Mixer; its realizations featuring 19 to 431 millions of parameters were shown to be comparable to vision transformers of similar size on ImageNet and similar image classification tasks. == Mathematical foundations == === Activation function === If a multilayer perceptron has a linear activation function in all neurons, that is, a linear function that maps the weighted inputs to the output of each neuron, then linear algebra shows that any number of layers can be reduced to a two-layer input-output model. In MLPs some neurons use a nonlinear activation function that was developed to model the frequency of action potentials, or firing, of biological neurons. The two historically common activation functions are both sigmoids, and are described by y ( v i ) = tanh ( v i ) and y ( v i ) = ( 1 + e − v i ) − 1 {\displaystyle y(v_{i})=\tanh(v_{i})~~{\textrm {and}}~~y(v_{i})=(1+e^{-v_{i}})^{-1}} . The first is a hyperbolic tangent that ranges from −1 to 1, while the other is the logistic function, which is similar in shape but ranges from 0 to 1. Here y i {\displaystyle y_{i}} is the output of the i {\displaystyle i} th node (neuron) and v i {\displaystyle v_{i}} is the weighted sum of the input connections. Alternative activation functions have been proposed, including the rectifier and softplus functions. More specialized activation functions include radial basis functions (used in radial basis networks, another class of supervised neural network models). In recent developments of deep learning the rectified linear unit (ReLU) is more frequently used as one of the possible ways to overcome the numerical problems related to the sigmoids. === Layers === The MLP consists of three or more layers (an input and an output layer with one or more hidden layers) of nonlinearly-activating nodes. Since MLPs are fully connected, each node in one layer connects with a certain weight w i j {\displaystyle w_{ij}} to every node in the following layer. === Learning === Learning occurs in the perceptron by changing connection weights after each piece of data is processed, based on the amount of error in the output compared to the expected result. This is an example of supervised learning, and is carried out through backpropagation, a generalization of the least mean squares algorithm in the linear perceptron. We can represent the degree of error in an output node j {\displaystyle j} in the n {\displaystyle n} th data point (training example) by e j ( n ) = d j ( n ) − y j ( n ) {\displaystyle e_{j}(n)=d_{j}(n)-y_{j}(n)} , where d j ( n ) {\displaystyle d_{j}(n)} is the desired target value for n {\displaystyle n} th data point at node j {\displaystyle j} , and y j ( n ) {\displaystyle y_{j}(n)} is the value produced by the perceptron at node j {\displaystyle j} when the n {\displaystyle n} th data point is given as an input. The node weights can then be adjusted based on corrections that minimize the error in the entire output for the n {\displaystyle n} th data point, given by E ( n ) = 1 2 ∑ output node j e j 2 ( n ) {\displaystyle {\mathcal {E}}(n)={\frac {1}{2}}\sum _{{\text{output node }}j}e_{j}^{2}(n)} . Using gradient descent, the change in each weight w i j {\displaystyle w_{ij}} is Δ w j i ( n ) = − η ∂ E ( n ) ∂ v j ( n ) y i ( n ) {\displaystyle \Delta w_{ji}(n)=-\eta {\frac {\partial {\mathcal {E}}(n)}{\partial v_{j}(n)}}y_{i}(n)} where y i ( n ) {\displaystyle y_{i}(n)} is the output of the previous neuron i {\displaystyle i} , and η {\displaystyle \eta } is the learning rate, which is selected to ensure that the weights quickly converge to a response, without oscillations. In the previous expression, ∂ E ( n ) ∂ v j ( n ) {\displaystyle {\frac {\partial {\mathcal {E}}(n)}{\partial v_{j}(n)}}} denotes the partial derivate of the error E ( n ) {\displaystyle {\mathcal {E}}(n)} according to the weighted sum v j ( n ) {\displaystyle v_{j}(n)} of the input connections of neuron i {\displaystyle i} . The derivative to be calculated depends on the induced local field v j {\displaystyle v_{j}} , which itself varies. It is easy to prove that for an output node this derivative can be simplified to − ∂ E ( n ) ∂ v j ( n ) = e j ( n ) ϕ ′ ( v j ( n ) ) {\displaystyle -{\frac {\partial {\mathcal {E}}(n)}{\partial v_{j}(n)}}=e_{j}(n)\phi ^{\prime }(v_{j}(n))} where ϕ ′ {\displaystyle \phi ^{\prime }} is the derivative of the activation function described above, which itself does not vary. The analysis is more difficult for the change in weights to a hidden node, but it can be shown that the relevant derivative is − ∂ E ( n ) ∂ v j ( n ) = ϕ ′ ( v j ( n ) ) ∑ k − ∂ E ( n ) ∂ v k ( n ) w k j ( n ) {\displaystyle -{\frac {\partial {\mathcal {E}}(n)}{\partial v_{j}(n)}}=\phi ^{\prime }(v_{j}(n))\sum _{k}-{\frac {\partial {\mathcal {E}}(n)}{\partial v_{k}(n)}}w_{kj}(n)} . This depends on the change in weights of the k {\displaystyle k} th nodes, which represent the output layer. So to change the hidden layer weights, the output layer weights change according to the derivative of the activation function, and so this algorithm represents a backpropagation of the activation function.
Astrostatistics
Astrostatistics is a discipline which spans astrophysics, statistical analysis and data mining. It is used to process the vast amount of data produced by automated scanning of the cosmos, to characterize complex datasets, and to link astronomical data to astrophysical theory. Many branches of statistics are involved in astronomical analysis including nonparametrics, multivariate regression and multivariate classification, time series analysis, and especially Bayesian inference. The field is closely related to astroinformatics.
Defeasible logic
Defeasible logic is a non-monotonic logic proposed by Donald Nute to formalize defeasible reasoning. In defeasible logic, there are three different types of propositions: strict rules specify that a fact is always a consequence of another; defeasible rules specify that a fact is typically a consequence of another; undercutting defeaters specify exceptions to defeasible rules. A priority ordering over the defeasible rules and the defeaters can be given. During the process of deduction, the strict rules are always applied, while a defeasible rule can be applied only if no defeater of a higher priority specifies that it should not.
Botler AI
Botler AI is a Montreal-based Canadian Artificial Intelligence company that helps users navigate the legal system. Launched in 2017 by Amir Morv and Ritika Dutt, Botler offers a free online tool which provides users who are unaware of their legal rights with information and guidance. Botler is known for its role in unveiling misconduct in the Government of Canada's procurement practices. Botler's findings have prompted numerous investigations, including by the Royal Canadian Mounted Police. == History == Botler's first AI was trained on over 300,000 U.S. and Canadian legal documents to help individuals identify and enforce their legal rights, without fear of judgment. Launched during the height of the #MeToo movement, the tool initially focused on sexual harassment with a goal of creating "a general artificial intelligence that would help the average person with any legal issue." === Department of Justice Canada === In 2020, Botler launched an expanded misconduct detection system in the form of an anonymous chatbot which provided users with an explanation of the law and relevant resources. In March 2021, the Minister of Justice and Attorney General of Canada announced the Government of Canada's support for Botler AI to assist complainants of sexual harassment in the workplace. The initiative, entitled Botler for Citizens and implemented with the support of the Department of Justice Canada, established an Artificial Intelligence-powered hybrid legal service delivery model. == Notable cases == On October 4, 2023, the RCMP confirmed to The Globe and Mail that they "are investigating a file referred from the CBSA (Canada Border Services Agency) that is based on allegations brought to their attention by Botler". In 2019, GCStrategies's managing partner, Kristian Firth, reached out to Botler on behalf of his client, the CBSA, to solicit their misconduct detection chatbot. After interactions with GCStrategies, Dalian Enterprises and Coradix Technology Consulting, the three main contractors involved in developing the controversial ArriveCAN app, Dutt and Morv alerted the CBSA to questionable contracting practices in federal government procurement in September, 2021, and again in November, 2022. In response to Botler's November 2022 report, the CBSA launched an internal review and referred the matter to the RCMP. During testimony before a parliamentary committee, the CBSA's President stated that the CBSA investigation to date has raised some concerns and shows "that there was a pattern of persistent collaboration between certain officials and GCStrategies... to circumvent or ignore certain established processes and roles and responsibilities". The Auditor General of Canada, which extended its study into ArriveCAN following the Botler revelations, found that GCStrategies was directly involved in setting narrow terms for a request for proposal for a $25-million government contract it ultimately won. The firm, which has just two employees, charges the government a commission of between 15 per cent and 30 per cent of each contract's value. The Office of the Procurement Ombudsman of Canada found "numerous examples" where GCStrategies "had simply copied and pasted" the required work experience to meet contracting requirements. To date, more than a dozen probes have been launched into the matter, including by the government, parliamentary committees, independent watchdogs and law-enforcement agencies. On April 17, 2024, GCStrategies' Firth was the first person summoned in over a century to answer questions before Members of Parliament in the House of Commons. During his appearance, Firth testified that the RCMP had raided "my property to obtain electronic goods surrounding the Botler allegations". === Government of Canada Reforms === One day after The Globe reported that the RCMP is investigating allegations of misconduct, the federal government responded by announcing new guidelines from the Treasury Board of Canada aimed at cutting back on the use of private consultants and that outsourcing contracts were under examination. Public Services and Procurement Canada (PSPC) invalidated and replaced all master level user agreements with government client departments in November 2023. The agreements set out the conditions for access to select Professional Services methods of supply which are used for outsourcing. In March 2024, PSPC announced its suspension of the respective security statuses of GCStrategies, Dalian and Coradix, barring them from participating in all federal procurements. Records show that the total value of contracts awarded to the three companies amounts to more than $1 Billion.
Tim Houlne
Tim Houlne is an American business executive, entrepreneur, and author known for his work in outsourcing and homeshoring, remote working, and artificial intelligence (AI) in customer service. He is the founder and CEO of Humach, a company that uses human agents and AI in customer experience solutions. Previously, he was co-founder and CEO of Working Solutions, a virtual contact center company in the United States. == Early life and education == Houlne graduated from Missouri Western State University (MWSU) in 1986 with a bachelor's degree in business administration and from the University of Texas in Dallas with an MBA. In 2024, MWSU and North Central Missouri College renamed the Convergent Technology Alliance Center to the Houlne Center for Convergent Technology. The 20,000 square-foot learning laboratory provides training and applied education experiences in industries such as AI, cybersecurity, manufacturing and construction, and service technologies. == Career == In 1998, Houlne co-founded Working Solutions, a Plano, Texas-based U.S. outsourcing company that provides customer service using remote, home-based agents. As CEO, he oversaw the development of a virtual workforce model that routes service calls to either domestic or offshore agents, according to client needs and service requirements. In 2015, Houlne founded Humach, a customer experience outsourcing provider that uses human service agents with AI-based digital agents. The company derives its name from the combination of services provided by humans and machines. Its clients include Amazon, Carfax and McDonald's. The company acquired InfiniteAI in 2020, and Markets EQ in 2025. In 2013, Houlne was named a finalist for the Ernst & Young Entrepreneur of the Year Award (Southwest Region).He is the co-author of several books focused on the evolution of work, the gig economy, and the influence of AI in customer-facing roles. == Works == The New World of Work: From the Cube to the Cloud (2013) ISBN 0982562276 OCLC 813933360 The New World of Work, Second Edition: The Cube, the Cloud and What's Next (2023) ISBN 9781642258318 OCLC 1389815847 The Intelligent Workforce: How Humans & Machines Will Co-Create a Better Future (2024) ISBN 9798887501604 OCLC 1439598569
2024 National Public Data breach
In August 2024, three class-action lawsuits were filed against National Public Data along with over 14 complaints filed in federal court, claiming that the company permitted hackers to steal sensitive private information covering millions of individuals. The theft was alleged to have occurred in April 2024. One of the lawsuits specifically claims that in April, a hacker going by the moniker "USDoD" posted a notice on the dark web, offering the data for sale at the price of US$3.5 million. The information stolen is alleged to include 2.9 billion records containing full names, current and past addresses, Social Security numbers, dates of birth, and telephone numbers. The stolen data contains records for people in the US, UK, and Canada. National Public Data confirmed on August 16, 2024, there was a breach originating from someone trying to breach their systems since December 2023, with the breach occurring from April 2024 and over the next few months. The company also confirmed that 2.9 billion records were obtained, though they were still working to determine how many people were affected by the breach, and were working with law enforcement to identify the hacker. == Jerico Pictures == Jerico Pictures, Inc., doing business as National Public Data, was a data broker company that performed employee background checks. Their primary service was collecting information from public data sources, including criminal records, addresses, and employment history, and offering that information for sale. On October 2, 2024, Jerico Pictures filed for Chapter 11 bankruptcy as it currently faces over a dozen lawsuits over the breach, and is potentially liable "for credit monitoring for hundreds of millions of potentially impacted individuals." In December 2024, National Public Data shut down, showing a closure notice on its website.
IEEE Transactions on Pattern Analysis and Machine Intelligence
IEEE Transactions on Pattern Analysis and Machine Intelligence (sometimes abbreviated as IEEE PAMI or simply PAMI) is a monthly peer-reviewed scientific journal published by the IEEE Computer Society. == Background == The journal covers research in computer vision and image understanding, pattern analysis and recognition, machine intelligence, machine learning, search techniques, document and handwriting analysis, medical image analysis, video and image sequence analysis, content-based retrieval of image and video, and face and gesture recognition. The editor-in-chief is Kyoung Mu Lee (Seoul National University). According to the Journal Citation Reports, the journal has a 2023 impact factor of 20.8.