The Dartmouth Summer Research Project on Artificial Intelligence was a 1956 summer workshop widely considered to be the founding event of artificial intelligence as a field. The workshop has been referred to as "the Constitutional Convention of AI". The project's four organizers, Claude Shannon, John McCarthy, Nathaniel Rochester and Marvin Minsky, are considered some of the "founding fathers" of AI. However it was not the first conference devoted to what would now be described as the question of artificial intelligence: it postdated meetings such as the 1951 Paris cybernetics conference and the Macy meetings. The project lasted approximately six to eight weeks and consisted largely of brainstorming sessions. Eleven mathematicians and scientists originally planned to attend; not all of them attended, but more than ten others came for short times. == Background == In the early 1950s, there were various names for the field of "thinking machines": cybernetics, automata theory, and complex information processing. The variety of names suggests the variety of conceptual orientations. In 1955, John McCarthy, then a young Assistant Professor of Mathematics at Dartmouth College, decided to organize a group to clarify and develop ideas about thinking machines. He picked the name 'Artificial Intelligence' for the new field. He chose the name partly for its neutrality; avoiding a focus on narrow automata theory, and avoiding cybernetics which was heavily focused on analog feedback, as well as him potentially having to accept the assertive Norbert Wiener as guru or having to argue with him. In early 1955, McCarthy approached the Rockefeller Foundation to request funding for a summer seminar at Dartmouth for about 10 participants. In June, he and Claude Shannon, a founder of information theory then at Bell Labs, met with Robert Morison, Director of Biological and Medical Research to discuss the idea and possible funding, though Morison was unsure whether money would be made available for such a visionary project. On September 2, 1955, the project was formally proposed by McCarthy, Marvin Minsky, Nathaniel Rochester and Claude Shannon. The proposal is credited with introducing the term 'artificial intelligence'. The Proposal states: We propose that a 2-month, 10-man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College in Hanover, New Hampshire. The study is to proceed on the basis of the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer. The proposal goes on to discuss computers, natural language processing, neural networks, theory of computation, abstraction and creativity (these areas within the field of artificial intelligence are considered still relevant to the work of the field). On May 26, 1956, McCarthy notified Robert Morison of the planned 11 attendees: For the full period: 1) Dr. Marvin Minsky 2) Dr. Julian Bigelow 3) Professor D.M. Mackay 4) Mr. Ray Solomonoff 5) Mr. John Holland 6) Dr. John McCarthy For four weeks: 7) Dr. Claude Shannon 8) Mr. Nathaniel Rochester 9) Mr. Oliver Selfridge For the first two weeks: 10) Dr. Allen Newell 11) Professor Herbert Simon He noted, "we will concentrate on a problem of devising a way of programming a calculator to form concepts and to form generalizations. This of course is subject to change when the group gets together." The actual participants came at different times, mostly for much shorter times. Trenchard More replaced Rochester for three weeks and MacKay and Holland did not attend—but the project was set to begin. Around June 18, 1956, the earliest participants (perhaps only Ray Solomonoff, maybe with Tom Etter) arrived at the Dartmouth campus in Hanover, N.H., to join John McCarthy who already had an apartment there. Solomonoff and Minsky stayed at Professors' apartments, but most would stay at the Hanover Inn. == Dates == The Dartmouth Workshop is usually said to have run for six weeks. Ray Solomonoff's notes taken during the workshop, however, indicate that it ran for roughly eight weeks, from about June 18 to August 17. Solomonoff's notes start on June 22; June 28 mentions Minsky, June 30 mentions Hanover, N.H., July 1 mentions Tom Etter. On August 17, Solomonoff gave a final talk. == Participants == Initially, McCarthy lost his list of attendees. Instead, after the workshop, McCarthy sent Solomonoff a preliminary list of participants and visitors plus those interested in the subject. 47 people were listed. Solomonoff, however, made a list of participants in his notes of the summer project: Ray Solomonoff Marvin Minsky John McCarthy Claude Shannon Trenchard More Nat Rochester Oliver Selfridge Julian Bigelow W. Ross Ashby W.S. McCulloch Abraham Robinson Tom Etter John Nash David Sayre Arthur Samuel Kenneth R. Shoulders Shoulders' friend Alex Bernstein Herbert Simon Allen Newell Shannon attended Solomonoff's talk on July 10 and Bigelow gave a talk on August 15. Solomonoff doesn't mention Bernard Widrow, but in 1994 Widrow said that he and an unidentified colleague from the same lab in MIT had attended for one week. In the same interview Widrow recalled that "I think [Wesley] Clark and [Belmont] Farley were there from Lincoln Lab." Trenchard mentions R. Culver and Solomonoff mentions Bill Shutz. Herb Gelernter didn't attend, but was influenced later by what Rochester learned. In an article in IEEE Spectrum, Grace Solomonoff additionally identifies Peter Milner in a photo taken by Nathaniel Rochester in front of Dartmouth Hall. Ray Solomonoff, Marvin Minsky, and John McCarthy were the only three who stayed for the full time. Trenchard took attendance during two weeks of his three-week visit. From three to about eight people would attend the daily sessions. == Event and aftermath == They had the entire top floor of the Dartmouth Math Department to themselves, and most weekdays they would meet at the main math classroom where someone might lead a discussion focusing on his ideas, or more frequently, a general discussion would be held. It was not a directed group research project; discussions covered many topics, but several directions are considered to have been initiated or encouraged by the Workshop: the rise of symbolic methods, systems focused on limited domains (early expert systems), and deductive systems versus inductive systems. One participant, Arthur Samuel, said, "It was very interesting, very stimulating, very exciting". Ray Solomonoff kept notes giving his impression of the talks and the ideas from various discussions. === McCarthy's 1956 AI distribution list === This is the list in the "People Interested in the Artificial Intelligence Problem" document which McCarthy produced in 1956, partly in lieu of a list of attendees at the Dartmouth workshop. According to McCarthy the list was "being sent to the people on the list and a few others", and its purpose was "to let those on it know who is interested in receiving documents on the problem" of artificial intelligence. McCarthy also promised to deliver them a report on the Dartmouth conference, and to send an updated list soon afterwards. It includes people who did not attend the conference and does not include everyone who did attend it.
Internet Security Awareness Training
Internet Security Awareness Training (ISAT) is the training given to members of an organization regarding the protection of various information assets of that organization. ISAT is a subset of general security awareness training (SAT). Even small and medium enterprises are generally recommended to provide such training, but organizations that need to comply with government regulations (e.g., the Gramm–Leach–Bliley Act, the Payment Card Industry Data Security Standard, Health Insurance Portability and Accountability Act, Sarbanes–Oxley Act) normally require formal ISAT for annually for all employees. Often such training is provided in the form of online courses. ISAT, also referred to as Security Education, Training, and Awareness (SETA), organizations train and create awareness of information security management within their environment. It is beneficial to organizations when employees are well trained and feel empowered to take important actions to protect themselves and organizational data. The SETA program target must be based on user roles within organizations and for positions that expose the organizations to increased risk levels, specialized courses must be required. == Coverage == There are general topics to cover for the training, but it is necessary for each organization to have a coverage strategy based on its needs, as this will ensure the training is practical and captures critical topics relevant to the organization. As the threat landscape changes very frequently, organizations should continuously review their training programs to ensure relevance with current trends. Topics covered in ISAT include: Appropriate methods for protecting sensitive information on personal computer systems, including password policy Various computer security concerns, including spam, malware, phishing, social engineering, etc. Consequences of failure to properly protect information, including potential job loss, economic consequences to the firm, damage to individuals whose private records are divulged, and possible civil and criminal law penalties. Being Internet Security Aware means you understand that there are people actively trying to steal data that is stored within your organization's computers. (This often focuses on user names and passwords, so that criminal elements can ultimately get access to bank accounts and other high-value IT assets.) That is why it is important to protect the assets of the organization and stop that from happening. The general scope should include topics such as password security, Email phishing, Social engineering, Mobile device security, Sensitive data security, and Business communications. In contrast, those requiring specialized knowledge are usually required to take technical and in-depth training courses. Suppose an organization determines that it is best to use one of the available training tools on the market, it must ensure it sets objectives that the training can meet, including confirming the training will provide employees with the knowledge to understand risks and the behaviors needed in managing them, actions to take to prevent or detect security incidents, using language easily understandable by the trainees, and ensuring the pricing is reasonable. Organizations are recommended to base ISAT training content on employee roles and their culture; the policy should guide that training for all employees and gave the following as examples of sources of reference materials: National Institute of Standards and Technology (NIST) Special Publication 800-50, Building an Information Technology Security Awareness and Training Program International Standards Organization (ISO) 27002:2013, Information technology—Security techniques—Code of practice for information security controls International Standards Organization (ISO) 27001:2013, Information technology — Security techniques — Information security management systems COBIT 5 Appendix F.2, Detailed Guidance: Services, Infrastructure and Applications Enabler, Security Awareness The training must focus on current threats specific to an organization and the impacts if that materializes as a result of user actions. Including practical examples and ways of dealing with scenarios help users know the appropriate measures to take. It is a good practice to periodically train customers of specific organizations on threats they face from people with malicious intentions. Coverage strategy for SAT should be driven by an organization's policy. It can help truly determine the level of depth of the training and where it should be conducted at a global level or business unit level, or a combination of both. A policy also empowers a responsible party within the organization to run the training. == Importance == Studies show that well-structured security awareness training can significantly reduce the likelihood of cyber incidents caused by human error. According to the Ponemon Institute, organizations that implement regular security training experience up to 70% fewer successful phishing attacks. Additionally, a 2023 Verizon Data Breach Investigations Report found that 74% of breaches involve the human element, highlighting the need for continuous education. Employees are key in whether organizations are breached or not; there must be a policy on creating awareness and training them on emerging threats and actions to take in safeguarding sensitive information and reporting any observed unusual activity within the corporate environment. Research has shown that SAT has helped reduce cyber-attacks within organizations, especially when it comes to phishing, as trainees learned to identify these attack modes and give them the self-assurance to take action appropriately. There is an increase in phishing attacks, and it has become increasingly important for people to understand how to these attacks work, and the actions required to prevent these and SAT has shown a significant impact on the number of successful phishing attacks against organizations. == Compliance Requirements == Various regulations and laws mandate SAT for organizations in specific industries, including the Gramm–Leach–Bliley Act (GLBA) for the financial services, the Federal Information Security Modernization Act of 2014 for federal agencies, and the European Union's General Data Protection Regulation (GDPR). === Federal Information Security Modernization Act === Employees and contractors in federal agencies are required to receive Security Awareness Training annually, and the program needs to address job-related information security risks linked that provide them with the knowledge to lessen security risks. === Health Insurance Portability and Accountability Act === The Health Insurance Portability and Accountability Act has the Security Rule, and Privacy Rule requiring the creation of a security awareness training program and ensuring employees are trained accordingly. === Payment Card Industry Data Security Standard === The Payment Card Industry Security Standards Council, the governing council for stakeholders in the payment industry, formed by American Express, Discover, JCB International, MasterCard, and Visa that developed the DSS as a requirement for the payment industry. Requirement 12.6 requires member organizations to institute a formal security awareness program. There is a published guide for organizations to adhere to when setting up the program. === US States Training Regulations === Some States mandate Security Awareness Training whiles other do not but simply recommend voluntary training. Among states that require the training for its employees include: Colorado (The Colorado Information Security Act, Colorado Revised Statutes 24-37.5-401 et seq.) Connecticut (13 FAM 301.1-1 Cyber Security Awareness Training (PS800)) Florida (Florida Statutes Chapter 282) Georgia (Executive Order GA E.O.182 mandated training within 90 days of issue) Illinois (Cook County) Indiana (IN H 1240) Louisiana (Louisiana Division of Administration, Office of Technology Services p. 52: LA H 633) Maryland (20-07 IT Security Policy) Montana (Mandatory cyber training for executive branch state employees) Nebraska Nevada (agency-by-agency state employee requirement - State Security Standard 123 – IT Security) New Hampshire New Jersey ( NJ A 1654) North Carolina Ohio (IT-15 - Security Awareness and Training) Pennsylvania Texas Utah Vermont Virginia West Virginia (WV Code Section 5A-6-4a) == Training Techniques == Below are some common training techniques, even though some can be blended depending on the operating environment: Interactive video training – This technique allows users to be trained using two-way interactive audio and video instruction. Web-based training – This method allows employees or users to take the training independently and usually has a testing component to determine if learning has taken place. If not, users can be allowed to retake the course and test to ensure there is a complete understanding
Margin classifier
In machine learning (ML), a margin classifier is a type of classification model which is able to give an associated distance from the decision boundary for each data sample. For instance, if a linear classifier is used, the distance (typically Euclidean, though others may be used) of a sample from the separating hyperplane is the margin of that sample. The notion of margins is important in several ML classification algorithms, as it can be used to bound the generalization error of these classifiers. These bounds are frequently shown using the VC dimension. The generalization error bound in boosting algorithms and support vector machines is particularly prominent. == Margin for boosting algorithms == The margin for an iterative boosting algorithm given a dataset with two classes can be defined as follows: the classifier is given a sample pair ( x , y ) {\displaystyle (x,y)} , where x ∈ X {\displaystyle x\in X} is a domain space and y ∈ Y = { − 1 , + 1 } {\displaystyle y\in Y=\{-1,+1\}} is the sample's label. The algorithm then selects a classifier h j ∈ C {\displaystyle h_{j}\in C} at each iteration j {\displaystyle j} where C {\displaystyle C} is a space of possible classifiers that predict real values. This hypothesis is then weighted by α j ∈ R {\displaystyle \alpha _{j}\in R} as selected by the boosting algorithm. At iteration t {\displaystyle t} , the margin of a sample x {\displaystyle x} can thus be defined as y ∑ j t α j h j ( x ) ∑ | α j | . {\displaystyle {\frac {y\sum _{j}^{t}\alpha _{j}h_{j}(x)}{\sum |\alpha _{j}|}}.} By this definition, the margin is positive if the sample is labeled correctly, or negative if the sample is labeled incorrectly. This definition may be modified and is not the only way to define the margin for boosting algorithms. However, there are reasons why this definition may be appealing. == Examples of margin-based algorithms == Many classifiers can give an associated margin for each sample. However, only some classifiers utilize information of the margin while learning from a dataset. Many boosting algorithms rely on the notion of a margin to assign weight to samples. If a convex loss is utilized (as in AdaBoost or LogitBoost, for instance) then a sample with a higher margin will receive less (or equal) weight than a sample with a lower margin. This leads the boosting algorithm to focus weight on low-margin samples. In non-convex algorithms (e.g., BrownBoost), the margin still dictates the weighting of a sample, though the weighting is non-monotone with respect to the margin. == Generalization error bounds == One theoretical motivation behind margin classifiers is that their generalization error may be bound by the algorithm parameters and a margin term. An example of such a bound is for the AdaBoost algorithm. Let S {\displaystyle S} be a set of m {\displaystyle m} data points, sampled independently at random from a distribution D {\displaystyle D} . Assume the VC-dimension of the underlying base classifier is d {\displaystyle d} and m ≥ d ≥ 1 {\displaystyle m\geq d\geq 1} . Then, with probability 1 − δ {\displaystyle 1-\delta } , we have the bound: P D ( y ∑ j t α j h j ( x ) ∑ | α j | ≤ 0 ) ≤ P S ( y ∑ j t α j h j ( x ) ∑ | α j | ≤ θ ) + O ( 1 m d log 2 ( m / d ) / θ 2 + log ( 1 / δ ) ) {\displaystyle P_{D}\left({\frac {y\sum _{j}^{t}\alpha _{j}h_{j}(x)}{\sum |\alpha _{j}|}}\leq 0\right)\leq P_{S}\left({\frac {y\sum _{j}^{t}\alpha _{j}h_{j}(x)}{\sum |\alpha _{j}|}}\leq \theta \right)+O\left({\frac {1}{\sqrt {m}}}{\sqrt {d\log ^{2}(m/d)/\theta ^{2}+\log(1/\delta )}}\right)} for all θ > 0 {\displaystyle \theta >0} .
FastICA
FastICA is an efficient and popular algorithm for independent component analysis invented by Aapo Hyvärinen at Helsinki University of Technology. Like most ICA algorithms, FastICA seeks an orthogonal rotation of prewhitened data, through a fixed-point iteration scheme, that maximizes a measure of non-Gaussianity of the rotated components. Non-gaussianity serves as a proxy for statistical independence, which is a very strong condition and requires infinite data to verify. FastICA can also be alternatively derived as an approximative Newton iteration. == Algorithm == === Prewhitening the data === Let the X := ( x i j ) ∈ R N × M {\displaystyle \mathbf {X} :=(x_{ij})\in \mathbb {R} ^{N\times M}} denote the input data matrix, M {\displaystyle M} the number of columns corresponding with the number of samples of mixed signals and N {\displaystyle N} the number of rows corresponding with the number of independent source signals. The input data matrix X {\displaystyle \mathbf {X} } must be prewhitened, or centered and whitened, before applying the FastICA algorithm to it. Centering the data entails demeaning each component of the input data X {\displaystyle \mathbf {X} } , that is, for each i = 1 , … , N {\displaystyle i=1,\ldots ,N} and j = 1 , … , M {\displaystyle j=1,\ldots ,M} . After centering, each row of X {\displaystyle \mathbf {X} } has an expected value of 0 {\displaystyle 0} . Whitening the data requires a linear transformation L : R N × M → R N × M {\displaystyle \mathbf {L} :\mathbb {R} ^{N\times M}\to \mathbb {R} ^{N\times M}} of the centered data so that the components of L ( X ) {\displaystyle \mathbf {L} (\mathbf {X} )} are uncorrelated and have variance one. More precisely, if X {\displaystyle \mathbf {X} } is a centered data matrix, the covariance of L x := L ( X ) {\displaystyle \mathbf {L} _{\mathbf {x} }:=\mathbf {L} (\mathbf {X} )} is the ( N × N ) {\displaystyle (N\times N)} -dimensional identity matrix, that is, A common method for whitening is by performing an eigenvalue decomposition on the covariance matrix of the centered data X {\displaystyle \mathbf {X} } , E { X X T } = E D E T {\displaystyle E\left\{\mathbf {X} \mathbf {X} ^{T}\right\}=\mathbf {E} \mathbf {D} \mathbf {E} ^{T}} , where E {\displaystyle \mathbf {E} } is the matrix of eigenvectors and D {\displaystyle \mathbf {D} } is the diagonal matrix of eigenvalues. The whitened data matrix is defined thus by === Single component extraction === The iterative algorithm finds the direction for the weight vector w ∈ R N {\displaystyle \mathbf {w} \in \mathbb {R} ^{N}} that maximizes a measure of non-Gaussianity of the projection w T X {\displaystyle \mathbf {w} ^{T}\mathbf {X} } , with X ∈ R N × M {\displaystyle \mathbf {X} \in \mathbb {R} ^{N\times M}} denoting a prewhitened data matrix as described above. Note that w {\displaystyle \mathbf {w} } is a column vector. To measure non-Gaussianity, FastICA relies on a nonquadratic nonlinear function f ( u ) {\displaystyle f(u)} , its first derivative g ( u ) {\displaystyle g(u)} , and its second derivative g ′ ( u ) {\displaystyle g^{\prime }(u)} . Hyvärinen states that the functions are useful for general purposes, while may be highly robust. The steps for extracting the weight vector w {\displaystyle \mathbf {w} } for single component in FastICA are the following: Randomize the initial weight vector w {\displaystyle \mathbf {w} } Let w + ← E { X g ( w T X ) T } − E { g ′ ( w T X ) } w {\displaystyle \mathbf {w} ^{+}\leftarrow E\left\{\mathbf {X} g(\mathbf {w} ^{T}\mathbf {X} )^{T}\right\}-E\left\{g'(\mathbf {w} ^{T}\mathbf {X} )\right\}\mathbf {w} } , where E { . . . } {\displaystyle E\left\{...\right\}} means averaging over all column-vectors of matrix X {\displaystyle \mathbf {X} } Let w ← w + / ‖ w + ‖ {\displaystyle \mathbf {w} \leftarrow \mathbf {w} ^{+}/\|\mathbf {w} ^{+}\|} If not converged, go back to 2 === Multiple component extraction === The single unit iterative algorithm estimates only one weight vector which extracts a single component. Estimating additional components that are mutually "independent" requires repeating the algorithm to obtain linearly independent projection vectors - note that the notion of independence here refers to maximizing non-Gaussianity in the estimated components. Hyvärinen provides several ways of extracting multiple components with the simplest being the following. Here, 1 M {\displaystyle \mathbf {1_{M}} } is a column vector of 1's of dimension M {\displaystyle M} . Algorithm FastICA Input: C {\displaystyle C} Number of desired components Input: X ∈ R N × M {\displaystyle \mathbf {X} \in \mathbb {R} ^{N\times M}} Prewhitened matrix, where each column represents an N {\displaystyle N} -dimensional sample, where C <= N {\displaystyle C<=N} Output: W ∈ R N × C {\displaystyle \mathbf {W} \in \mathbb {R} ^{N\times C}} Un-mixing matrix where each column projects X {\displaystyle \mathbf {X} } onto independent component. Output: S ∈ R C × M {\displaystyle \mathbf {S} \in \mathbb {R} ^{C\times M}} Independent components matrix, with M {\displaystyle M} columns representing a sample with C {\displaystyle C} dimensions. for p in 1 to C: w p ← {\displaystyle \mathbf {w_{p}} \leftarrow } Random vector of length N while w p {\displaystyle \mathbf {w_{p}} } changes w p ← 1 M X g ( w p T X ) T − 1 M g ′ ( w p T X ) 1 M w p {\displaystyle \mathbf {w_{p}} \leftarrow {\frac {1}{M}}\mathbf {X} g(\mathbf {w_{p}} ^{T}\mathbf {X} )^{T}-{\frac {1}{M}}g'(\mathbf {w_{p}} ^{T}\mathbf {X} )\mathbf {1_{M}} \mathbf {w_{p}} } w p ← w p − ∑ j = 1 p − 1 ( w p T w j ) w j {\displaystyle \mathbf {w_{p}} \leftarrow \mathbf {w_{p}} -\sum _{j=1}^{p-1}(\mathbf {w_{p}} ^{T}\mathbf {w_{j}} )\mathbf {w_{j}} } w p ← w p ‖ w p ‖ {\displaystyle \mathbf {w_{p}} \leftarrow {\frac {\mathbf {w_{p}} }{\|\mathbf {w_{p}} \|}}} output W ← [ w 1 , … , w C ] {\displaystyle \mathbf {W} \leftarrow {\begin{bmatrix}\mathbf {w_{1}} ,\dots ,\mathbf {w_{C}} \end{bmatrix}}} output S ← W T X {\displaystyle \mathbf {S} \leftarrow \mathbf {W^{T}} \mathbf {X} }
Genetic programming
Genetic programming (GP) is an evolutionary algorithm, an artificial intelligence technique mimicking natural evolution, which operates on a population of programs. It applies the genetic operators selection according to a predefined fitness measure, mutation and crossover. The crossover operation involves swapping specified parts of selected pairs (parents) to produce new and different offspring that become part of the new generation of programs. Some programs not selected for reproduction are copied from the current generation to the new generation. Mutation involves substitution of some random part of a program with some other random part of a program. Then the selection and other operations are recursively applied to the new generation of programs. Typically, members of each new generation are on average more fit than the members of the previous generation, and the best-of-generation program is often better than the best-of-generation programs from previous generations. Termination of the evolution usually occurs when some individual program reaches a predefined proficiency or fitness level. It may and often does happen that a particular run of the algorithm results in premature convergence to some local maximum that is not a globally optimal or even good solution. Multiple runs (dozens to hundreds) are usually necessary to produce a very good result. It may also be necessary to have a large starting population size and variability of the individuals to avoid pathologies. == History == The first record of the proposal to evolve programs is probably that of Alan Turing in 1950 in "Computing Machinery and Intelligence". There was a gap of 25 years before the publication of John Holland's 'Adaptation in Natural and Artificial Systems' laid out the theoretical and empirical foundations of the science. In 1981, Richard Forsyth demonstrated the successful evolution of small programs, represented as trees, to perform classification of crime scene evidence for the UK Home Office. Although the idea of evolving programs, initially in the computer language Lisp, was current amongst John Holland's students, it was not until they organised the first Genetic Algorithms (GA) conference in Pittsburgh that Nichael Cramer published evolved programs in two specially designed languages, which included the first statement of modern "tree-based" genetic programming (that is, procedural languages organized in tree-based structures and operated on by suitably defined GA-operators). In 1988, John Koza (also a PhD student of John Holland) patented his invention of a GA for program evolution. This was followed by publication in the International Joint Conference on Artificial Intelligence IJCAI-89. Koza followed this with 205 publications on "genetic programming", a term coined by David Goldberg, also a PhD student of John Holland. However, it is the series of 4 books by Koza, starting in 1992 with accompanying videos, that really established GP. Subsequently, there was an enormous expansion of the number of publications with the Genetic Programming Bibliography, surpassing 10,000 entries. In 2010, Koza listed 77 results where genetic programming was human competitive. The departure of GP from the rigid, fixed-length representations typical of early GA models was not entirely without precedent. Early work on variable-length representations laid the groundwork. One notable example is messy genetic algorithms, which introduced irregular, variable-length chromosomes to address building block disruption and positional bias in standard GAs. Another precursor was robot trajectory programming, where genome representations encoded program instructions for robotic movements—structures inherently variable in length. Even earlier, unfixed-length representations were proposed in a doctoral dissertation by Cavicchio, who explored adaptive search using simulated evolution. His work provided foundational ideas for flexible program structures. In 1996, Koza started the annual Genetic Programming conference, which was followed in 1998 by the annual EuroGP conference, and the first book in a GP series edited by Koza. 1998 also saw the first GP textbook. GP continued to flourish, leading to the first specialist GP journal and three years later (2003) the annual Genetic Programming Theory and Practice (GPTP) workshop was established by Rick Riolo. Genetic programming papers continue to be published at a diversity of conferences and associated journals. Today there are nineteen GP books including several for students. === Foundational work in GP === Early work that set the stage for current genetic programming research topics and applications is diverse, and includes software synthesis and repair, predictive modeling, data mining, financial modeling, soft sensors, design, and image processing. Applications in some areas, such as design, often make use of intermediate representations, such as Fred Gruau's cellular encoding. Industrial uptake has been significant in several areas including finance, the chemical industry, bioinformatics and the steel industry. == Methods == === Program representation === GP evolves computer programs, traditionally represented in memory as tree structures. Trees can be easily evaluated in a recursive manner. Every internal node has an operator function and every terminal node has an operand, making mathematical expressions easy to evolve and evaluate. Thus traditionally GP favors the use of programming languages that naturally embody tree structures (for example, Lisp; other functional programming languages are also suitable). Non-tree representations have been suggested and successfully implemented, such as linear genetic programming, which perhaps suits the more traditional imperative languages. The commercial GP software Discipulus uses automatic induction of binary machine code ("AIM") to achieve better performance. μGP uses directed multigraphs to generate programs that fully exploit the syntax of a given assembly language. Multi expression programming uses three-address code for encoding solutions. Other program representations on which significant research and development have been conducted include programs for stack-based virtual machines, and sequences of integers that are mapped to arbitrary programming languages via grammars. Cartesian genetic programming is another form of GP, which uses a graph representation instead of the usual tree based representation to encode computer programs. Most representations have structurally noneffective code (introns). Such non-coding genes may seem to be useless because they have no effect on the performance of any one individual. However, they alter the probabilities of generating different offspring under the variation operators, and thus alter the individual's variational properties. Experiments seem to show faster convergence when using program representations that allow such non-coding genes, compared to program representations that do not have any non-coding genes. Instantiations may have both trees with introns and those without; the latter are called canonical trees. Special canonical crossover operators are introduced that maintain the canonical structure of parents in their children. === Initialisation === The methods for creation of the initial population include: Grow creates the individuals sequentially. Every GP tree is created starting from the root, creating functional nodes with children as well as terminal nodes up to a certain depth. Full is similar to the Grow. The difference is that all brunches in a tree are of same predetermined depth. Ramped half-and-half creates a population consisting of m d − 1 {\displaystyle md-1} parts and a maximum depth of m d {\displaystyle md} for its trees. The first part has a maximum depth of 2, second of 3 and so on up to the m d − 1 {\displaystyle md-1} -th part with maximum depth m d {\displaystyle md} . Half of every part is created by Grow, while the other part is created by Full. === Selection === Selection is a process whereby certain individuals are selected from the current generation that would serve as parents for the next generation. The individuals are selected probabilistically such that the better performing individuals have a higher chance of getting selected. The most commonly used selection method in GP is tournament selection, although other methods such as fitness proportionate selection, lexicase selection, and others have been demonstrated to perform better for many GP problems. Elitism, which involves seeding the next generation with the best individual (or best n individuals) from the current generation, is a technique sometimes employed to avoid regression. === Crossover === In genetic programming two fit individuals are chosen from the population to be parents for one or two children. In tree genetic programming, these parents are represented as inverted lisp like trees, with their root nodes at the top. In subtree cro
Brave Leo
Brave Leo is a large language model-based chatbot developed by Brave Software and included with the Brave browser. == History == In November 2023, the company said versions for iOS and Android would be available "in the coming months". == Features == Since January 2024, Leo has used the open-source Mixtral 8x7B from Mistral AI as its default large language model, in addition to LLaMA 2 from Meta Platforms and Claude from Anthropic, both of which have been used previously. Leo can suggest follow-up questions, and summarize webpages, PDFs, and videos. Leo has a $15 (US) per month premium version that enables more requests and uses larger LLMs. == Privacy == The answers given by Leo are not saved. Brave uses the slogan Love Privacy to emphasize its focus on user privacy and data protection. The phrase has been featured in Brave's official marketing campaigns and has been cited in media coverage of the browser's privacy-first approach. == Controversies == In 2023, PC World reported that Leo evades questions about US elections.
LogitBoost
In machine learning and computational learning theory, LogitBoost is a boosting algorithm formulated by Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The original paper casts the AdaBoost algorithm into a statistical framework. Specifically, if one considers AdaBoost as a generalized additive model and then applies the cost function of logistic regression, one can derive the LogitBoost algorithm. == Minimizing the LogitBoost cost function == LogitBoost can be seen as a convex optimization. Specifically, given that we seek an additive model of the form f = ∑ t α t h t {\displaystyle f=\sum _{t}\alpha _{t}h_{t}} the LogitBoost algorithm minimizes the logistic loss: ∑ i log ( 1 + e − y i f ( x i ) ) {\displaystyle \sum _{i}\log \left(1+e^{-y_{i}f(x_{i})}\right)}