In logic, statistical inference, and supervised learning, transduction or transductive inference is reasoning from observed, specific (training) cases to specific (test) cases. In contrast, induction is reasoning from observed training cases to general rules, which are then applied to the test cases. The distinction is most interesting in cases where the predictions of the transductive model are not achievable by any inductive model. Note that this is caused by transductive inference on different test sets producing mutually inconsistent predictions. Transduction was introduced in a computer science context by Vladimir Vapnik in the 1990s, motivated by his view that transduction is preferable to induction since, according to him, induction requires solving a more general problem (inferring a function) before solving a more specific problem (computing outputs for new cases): "When solving a problem of interest, do not solve a more general problem as an intermediate step. Try to get the answer that you really need but not a more general one.". An example of learning which is not inductive would be in the case of binary classification, where the inputs tend to cluster in two groups. A large set of test inputs may help in finding the clusters, thus providing useful information about the classification labels. The same predictions would not be obtainable from a model which induces a function based only on the training cases. Some people may call this an example of the closely related semi-supervised learning, since Vapnik's motivation is quite different. The most well-known example of a case-bases learning algorithm is the k-nearest neighbor algorithm, which is related to transductive learning algorithms. Another example of an algorithm in this category is the Transductive Support Vector Machine (TSVM). A third possible motivation of transduction arises through the need to approximate. If exact inference is computationally prohibitive, one may at least try to make sure that the approximations are good at the test inputs. In this case, the test inputs could come from an arbitrary distribution (not necessarily related to the distribution of the training inputs), which wouldn't be allowed in semi-supervised learning. An example of an algorithm falling in this category is the Bayesian Committee Machine (BCM). == Historical context == The mode of inference from particulars to particulars, which Vapnik came to call transduction, was already distinguished from the mode of inference from particulars to generalizations in part III of the Cambridge philosopher and logician W.E. Johnson's 1924 textbook, Logic. In Johnson's work, the former mode was called 'eduction' and the latter was called 'induction'. Bruno de Finetti developed a purely subjective form of Bayesianism in which claims about objective chances could be translated into empirically respectable claims about subjective credences with respect to observables through exchangeability properties. An early statement of this view can be found in his 1937 La Prévision: ses Lois Logiques, ses Sources Subjectives and a mature statement in his 1970 Theory of Probability. Within de Finetti's subjective Bayesian framework, all inductive inference is ultimately inference from particulars to particulars. == Example problem == The following example problem contrasts some of the unique properties of transduction against induction. A collection of points is given, such that some of the points are labeled (A, B, or C), but most of the points are unlabeled (?). The goal is to predict appropriate labels for all of the unlabeled points. The inductive approach to solving this problem is to use the labeled points to train a supervised learning algorithm, and then have it predict labels for all of the unlabeled points. With this problem, however, the supervised learning algorithm will only have five labeled points to use as a basis for building a predictive model. It will certainly struggle to build a model that captures the structure of this data. For example, if a nearest-neighbor algorithm is used, then the points near the middle will be labeled "A" or "C", even though it is apparent that they belong to the same cluster as the point labeled "B", compared to semi-supervised learning. Transduction has the advantage of being able to consider all of the points, not just the labeled points, while performing the labeling task. In this case, transductive algorithms would label the unlabeled points according to the clusters to which they naturally belong. The points in the middle, therefore, would most likely be labeled "B", because they are packed very close to that cluster. An advantage of transduction is that it may be able to make better predictions with fewer labeled points, because it uses the natural breaks found in the unlabeled points. One disadvantage of transduction is that it builds no predictive model. If a previously unknown point is added to the set, the entire transductive algorithm would need to be repeated with all of the points in order to predict a label. This can be computationally expensive if the data is made available incrementally in a stream. Further, this might cause the predictions of some of the old points to change (which may be good or bad, depending on the application). A supervised learning algorithm, on the other hand, can label new points instantly, with very little computational cost. == Transduction algorithms == Transduction algorithms can be broadly divided into two categories: those that seek to assign discrete labels to unlabeled points, and those that seek to regress continuous labels for unlabeled points. Algorithms that seek to predict discrete labels tend to be derived by adding partial supervision to a clustering algorithm. Two classes of algorithms can be used: flat clustering and hierarchical clustering. The latter can be further subdivided into two categories: those that cluster by partitioning, and those that cluster by agglomerating. Algorithms that seek to predict continuous labels tend to be derived by adding partial supervision to a manifold learning algorithm. === Partitioning transduction === Partitioning transduction can be thought of as top-down transduction. It is a semi-supervised extension of partition-based clustering. It is typically performed as follows: Consider the set of all points to be one large partition. While any partition P contains two points with conflicting labels: Partition P into smaller partitions. For each partition P: Assign the same label to all of the points in P. Of course, any reasonable partitioning technique could be used with this algorithm. Max flow min cut partitioning schemes are very popular for this purpose. === Agglomerative transduction === Agglomerative transduction can be thought of as bottom-up transduction. It is a semi-supervised extension of agglomerative clustering. It is typically performed as follows: Compute the pair-wise distances, D, between all the points. Sort D in ascending order. Consider each point to be a cluster of size 1. For each pair of points {a,b} in D: If (a is unlabeled) or (b is unlabeled) or (a and b have the same label) Merge the two clusters that contain a and b. Label all points in the merged cluster with the same label. === Continuous Label Transduction === These methods seek to regress continuous labels, often via manifold learning techniques. The idea is to learn a low-dimensional representation of the data and infer values smoothly across the manifold. == Applications and related concepts == Transduction is closely related to: Semi-supervised learning – uses both labeled and unlabeled data but typically induces a model. Case-based reasoning – such as the k-nearest neighbor (k-NN) algorithm, often considered a transductive method. Transductive Support Vector Machines (TSVM) – extend standard SVMs to incorporate unlabeled test data during training. Bayesian Committee Machine (BCM) – an approximation method that makes transductive predictions when exact inference is too costly.
Secure state
A secure state is an information systems security term to describe where entities in a computer system are divided into subjects and objects, and it can be formally proven that each state transition preserves security by moving from one secure state to another secure state. Thereby it can be inductively proven that the system is secure. As defined in the Bell–LaPadula model, the secure state is built on the concept of a state machine with a set of allowable states in a system. The transition from one state to another state is defined by transition functions. A system state is defined to be "secure" if the only permitted access modes of subjects to objects are in accordance with a security policy.
Capsule neural network
A capsule neural network (CapsNet) is a machine learning system that is a type of artificial neural network (ANN) that can be used to better model hierarchical relationships. The approach is an attempt to more closely mimic biological neural organization. The idea is to add structures called "capsules" to a convolutional neural network (CNN), and to reuse output from several of those capsules to form more stable (with respect to various perturbations) representations for higher capsules. The output is a vector consisting of the probability of an observation, and a pose for that observation. This vector is similar to what is done for example when doing classification with localization in CNNs. Among other benefits, capsnets address the "Picasso problem" in image recognition: images that have all the right parts but that are not in the correct spatial relationship (e.g., in a "face", the positions of the mouth and one eye are switched). For image recognition, capsnets exploit the fact that while viewpoint changes have nonlinear effects at the pixel level, they have linear effects at the part/object level. This can be compared to inverting the rendering of an object of multiple parts. == History == In 2000, Geoffrey Hinton et al. described an imaging system that combined segmentation and recognition into a single inference process using parse trees. So-called credibility networks described the joint distribution over the latent variables and over the possible parse trees. That system proved useful on the MNIST handwritten digit database. A dynamic routing mechanism for capsule networks was introduced by Hinton and his team in 2017. The approach was claimed to reduce error rates on MNIST and to reduce training set sizes. Results were claimed to be considerably better than a CNN on highly overlapped digits. In Hinton's original idea one minicolumn would represent and detect one multidimensional entity. == Transformations == An invariant is an object property that does not change as a result of some transformation. For example, the area of a circle does not change if the circle is shifted to the left. Informally, an equivariant is a property that changes predictably under transformation. For example, the center of a circle moves by the same amount as the circle when shifted. A nonequivariant is a property whose value does not change predictably under a transformation. For example, transforming a circle into an ellipse means that its perimeter can no longer be computed as π times the diameter. In computer vision, the class of an object is expected to be an invariant over many transformations. I.e., a cat is still a cat if it is shifted, turned upside down or shrunken in size. However, many other properties are instead equivariant. The volume of a cat changes when it is scaled. Equivariant properties such as a spatial relationship are captured in a pose, data that describes an object's translation, rotation, scale and reflection. Translation is a change in location in one or more dimensions. Rotation is a change in orientation. Scale is a change in size. Reflection is a mirror image. Unsupervised capsnets learn a global linear manifold between an object and its pose as a matrix of weights. In other words, capsnets can identify an object independent of its pose, rather than having to learn to recognize the object while including its spatial relationships as part of the object. In capsnets, the pose can incorporate properties other than spatial relationships, e.g., color (cats can be of various colors). Multiplying the object by the manifold poses the object (for an object, in space). == Pooling == Capsnets reject the pooling layer strategy of conventional CNNs that reduces the amount of detail to be processed at the next higher layer. Pooling allows a degree of translational invariance (it can recognize the same object in a somewhat different location) and allows a larger number of feature types to be represented. Capsnet proponents argue that pooling: violates biological shape perception in that it has no intrinsic coordinate frame; provides invariance (discarding positional information) instead of equivariance (disentangling that information); ignores the linear manifold that underlies many variations among images; routes statically instead of communicating a potential "find" to the feature that can appreciate it; damages nearby feature detectors, by deleting the information they rely upon. == Capsules == A capsule is a set of neurons that individually activate for various properties of a type of object, such as position, size and hue. Formally, a capsule is a set of neurons that collectively produce an activity vector with one element for each neuron to hold that neuron's instantiation value (e.g., hue). Graphics programs use instantiation value to draw an object. Capsnets attempt to derive these from their input. The probability of the entity's presence in a specific input is the vector's length, while the vector's orientation quantifies the capsule's properties. Artificial neurons traditionally output a scalar, real-valued activation that loosely represents the probability of an observation. Capsnets replace scalar-output feature detectors with vector-output capsules and max-pooling with routing-by-agreement. Because capsules are independent, when multiple capsules agree, the probability of correct detection is much higher. A minimal cluster of two capsules considering a six-dimensional entity would agree within 10% by chance only once in a million trials. As the number of dimensions increase, the likelihood of a chance agreement across a larger cluster with higher dimensions decreases exponentially. Capsules in higher layers take outputs from capsules at lower layers, and accept those whose outputs cluster. A cluster causes the higher capsule to output a high probability of observation that an entity is present and also output a high-dimensional (20-50+) pose. Higher-level capsules ignore outliers, concentrating on clusters. This is similar to the Hough transform, the RHT and RANSAC from classic digital image processing. == Routing by agreement == The outputs from one capsule (child) are routed to capsules in the next layer (parent) according to the child's ability to predict the parents' outputs. Over the course of a few iterations, each parents' outputs may converge with the predictions of some children and diverge from those of others, meaning that that parent is present or absent from the scene. For each possible parent, each child computes a prediction vector by multiplying its output by a weight matrix (trained by backpropagation). Next the output of the parent is computed as the scalar product of a prediction with a coefficient representing the probability that this child belongs to that parent. A child whose predictions are relatively close to the resulting output successively increases the coefficient between that parent and child and decreases it for parents that it matches less well. This increases the contribution that that child makes to that parent, thus increasing the scalar product of the capsule's prediction with the parent's output. After a few iterations, the coefficients strongly connect a parent to its most likely children, indicating that the presence of the children imply the presence of the parent in the scene. The more children whose predictions are close to a parent's output, the more quickly the coefficients grow, driving convergence. The pose of the parent (reflected in its output) progressively becomes compatible with that of its children. The coefficients' initial logits are the log prior probabilities that a child belongs to a parent. The priors can be trained discriminatively along with the weights. The priors depend on the location and type of the child and parent capsules, but not on the current input. At each iteration, the coefficients are adjusted via a "routing" softmax so that they continue to sum to 1 (to express the probability that a given capsule is the parent of a given child.) Softmax amplifies larger values and diminishes smaller values beyond their proportion of the total. Similarly, the probability that a feature is present in the input is exaggerated by a nonlinear "squashing" function that reduces values (smaller ones drastically and larger ones such that they are less than 1). This dynamic routing mechanism provides the necessary deprecation of alternatives ("explaining away") that is needed for segmenting overlapped objects. This learned routing of signals has no clear biological equivalent. Some operations can be found in cortical layers, but they do not seem to relate this technique. === Math/code === The pose vector u i {\textstyle \mathbf {u} _{i}} is rotated and translated by a matrix W i j {\textstyle \mathbf {W} _{ij}} into a vector u ^ j | i {\textstyle \mathbf {\hat {u}} _{j|i}} that predicts the output of the parent capsule. u ^ j | i = W i j u i {\displaystyle \mathbf {
Mycin
MYCIN was an early backward chaining expert system that used black box to identify bacteria causing severe infections, such as bacteremia and meningitis, and to recommend antibiotics, with the dosage adjusted for patient's body weight — the name derived from the antibiotics themselves, as many antibiotics have the suffix "-mycin". The Mycin system was also used for the diagnosis of blood clotting diseases. MYCIN was developed over five or six years in the early 1970s at Stanford University. It was written in Lisp as the doctoral dissertation of Edward Shortliffe under the direction of Bruce G. Buchanan, Stanley N. Cohen and others. MYCIN emerged from the Stanford Heuristic Programming Project. MYCIN demonstrated the potential for expert systems in building high-performance medical reasoning programs. MYCIN is often viewed as a pioneer in the field of expert systems, even being referred to as the "grandaddy of them all-the one that launched the field" by Dr. Allen Newell. MYCIN led to the EMYCIN expert system shell ("essential MYCIN") for acquiring knowledge, reasoning with it, and explaining the results, without the specific medical knowledge. It can be described as "EMYCIN = Prolog + uncertainty + caching + questions + explanations + contexts - variables". An introduction is in Chapter 16 of Paradigms of Artificial Intelligence Programming (PAIP). == Method == MYCIN operated using a fairly simple inference engine and a knowledge base of ~600 rules by obtaining individual inferential facts identified by experts and encoding such facts as individual production rules. No other AI program at the time contained as much domain-specific knowledge clearly separated from its inference procedures as MYCIN. It would query the physician running the program via a long series of simple yes/no or textual questions. At the end, it provided a list of possible culprit bacteria ranked from high to low based on the probability of each diagnosis, its confidence in each diagnosis' probability, the reasoning behind each diagnosis (that is, MYCIN would also list the questions and rules which led it to rank a diagnosis a particular way), and its recommended course of drug treatment. MYCIN could additionally respond to queries by physicians related to why it asked the user a certain question, how it arrived at a conclusion, and why it did not consider certain factors. The developers performed studies showing that MYCIN's performance was minimally affected by perturbations in the uncertainty metrics associated with individual rules, suggesting that the power in the system was related more to its knowledge representation and reasoning scheme than to the details of its numerical uncertainty model. Some observers felt that it should have been possible to use classical Bayesian statistics. MYCIN's developers argued that this would require either unrealistic assumptions of probabilistic independence, or require the experts to provide estimates for an unfeasibly large number of conditional probabilities. Subsequent studies later showed that the certainty factor model could indeed be interpreted in a probabilistic sense, and highlighted problems with the implied assumptions of such a model. However the modular structure of the system would prove very successful, leading to the development of graphical models such as Bayesian networks. === Context === A context in MYCIN determines what types of objects can be reasoned about. They are similar to variables in Prolog, or environment variables in operating systems. === Evidence combination === In MYCIN it was possible that two or more rules might draw conclusions about a parameter with different weights of evidence. For example, one rule may conclude that the organism in question is E. Coli with a certainty of 0.8 whilst another concludes that it is E. Coli with a certainty of 0.5 or even −0.8. In the event the certainty is less than zero the evidence is actually against the hypothesis. In order to calculate the certainty factor MYCIN combined these weights using the formula below to yield a single certainty factor: C F ( x , y ) = { X + Y − X Y if X , Y > 0 X + Y + X Y if X , Y < 0 X + Y 1 − min ( | X | , | Y | ) otherwise {\displaystyle CF(x,y)={\begin{cases}X+Y-XY&{\text{if }}X,Y>0\\X+Y+XY&{\text{if }}X,Y<0\\{\frac {X+Y}{1-\min(|X|,|Y|)}}&{\text{otherwise}}\end{cases}}} Where X and Y are the certainty factors. This formula can be applied more than once if more than two rules draw conclusions about the same parameter. It is commutative, so it does not matter in which order the weights were combined. The combination formula was designed to have the following desirable properties: −1 can be interpreted as "false", +1 as "true", and 0 as "uncertain". Combining unknown with anything leaves it unchanged. Combining true with anything (except false) gives true. Similarly for false. Combining true and false is a division-by-zero error. Combining +x and -x gives unknown. Combining two positives (except true) gives a larger positive. Similarly for negatives. Combining a positive and a negative gives something in between. === Examples === The following examples come from Chapter 16 of PAIP, which contains an implementation in Common Lisp of a modified and simplified version of MYCIN for pedagogical purposes. A rule, and an English paraphrase generated by the system: == Results == An evaluation of MYCIN was conducted at the Stanford Medical School. The first phase of the evaluation consisted of 10 test cases of diverse origin, chosen by a physician who was not acquainted with MYCIN's methods or knowledge base. These cases were presented to 7 physicians and 1 senior medical student. 10 prescriptions were compiled for each of the cases, 1 recommended by MYCIN, 1 prescribed by the treating physician at the county hospital, and 8 by the aforementioned individuals. The second phase of the evaluation consisted of eight infectious disease specialists being provided the clinical summary and set of 10 prescriptions for each of the 10 cases and tasked to provide their own recommendations for each case and assess the 10 prescriptions. MYCIN received an acceptability rating of 65%, which was comparable to the 42.5% to 62.5% rating of five faculty members. This study is often cited as showing the potential for disagreement about therapeutic decisions, even among experts, when there is no "gold standard" for correct treatment. == Practical use == MYCIN was never actually used in practice. This wasn't because of any weakness in its performance. Some observers raised ethical and legal issues related to the use of computers in medicine, regarding the responsibility of the physicians in case the system gave wrong diagnosis. However, the greatest problem, and the reason that MYCIN was not used in routine practice, was the state of technologies for system integration, especially at the time it was developed. MYCIN was a stand-alone system that required a user to enter all relevant information about a patient by typing in responses to questions MYCIN posed. MYCIN ran on the DEC KI10 PDP-10, supporting a large time-shared system available over the early Internet (ARPANet), before personal computers were developed. MYCIN's greatest influence was accordingly its demonstration of the power of its representation and reasoning approach. Rule-based systems in many non-medical domains were developed in the years that followed MYCIN's introduction of the approach. In the 1980s, expert system "shells" were introduced (including one based on MYCIN, known as E-MYCIN (followed by Knowledge Engineering Environment - KEE)) and supported the development of expert systems in a wide variety of application areas. A difficulty that rose to prominence during the development of MYCIN and subsequent complex expert systems has been the extraction of the necessary knowledge for the inference engine to use from the human expert in the relevant fields into the rule base (the so-called "knowledge acquisition bottleneck").
Revoscalepy
revoscalepy is a machine learning package in Python created by Microsoft. It is available as part of Machine Learning Services in Microsoft SQL Server 2017 and Machine Learning Server 9.2.0 and later. The package contains functions for creating linear model, logistic regression, random forest, decision tree and boosted decision tree, in addition to some summary functions for inspecting data. Other machine learning algorithms such as neural network are provided in microsoftml, a separate package that is the Python version of MicrosoftML. revoscalepy also contains functions designed to run machine learning algorithms in different compute contexts, including SQL Server, Apache Spark, and Hadoop. In June 2021, Microsoft announced to open source the revoscalepy and RevoScaleR packages, making them freely available under the MIT License.
Maximum inner-product search
Maximum inner-product search (MIPS) is a search problem, with a corresponding class of search algorithms which attempt to maximise the inner product between a query and the data items to be retrieved. MIPS algorithms are used in a wide variety of big data applications, including recommendation algorithms and machine learning. Formally, for a database of vectors x i {\displaystyle x_{i}} defined over a set of labels S {\displaystyle S} in an inner product space with an inner product ⟨ ⋅ , ⋅ ⟩ {\displaystyle \langle \cdot ,\cdot \rangle } defined on it, MIPS search can be defined as the problem of determining a r g m a x i ∈ S ⟨ x i , q ⟩ {\displaystyle {\underset {i\in S}{\operatorname {arg\,max} }}\ \langle x_{i},q\rangle } for a given query q {\displaystyle q} . Although there is an obvious linear-time implementation, it is generally too slow to be used on practical problems. However, efficient algorithms exist to speed up MIPS search. Under the assumption of all vectors in the set having constant norm, MIPS can be viewed as equivalent to a nearest neighbor search (NNS) problem in which maximizing the inner product is equivalent to minimizing the corresponding distance metric in the NNS problem. Like other forms of NNS, MIPS algorithms may be approximate or exact. MIPS search is used as part of DeepMind's RETRO algorithm.
User modeling
User modeling is the subdivision of human–computer interaction which describes the process of building up and modifying a conceptual understanding of the user. The main goal of user modeling is customization and adaptation of systems to the user's specific needs. The system needs to "say the 'right' thing at the 'right' time in the 'right' way". To do so it needs an internal representation of the user. Another common purpose is modeling specific kinds of users, including modeling of their skills and declarative knowledge, for use in automatic software-tests. User-models can thus serve as a cheaper alternative to user testing but should not replace user testing. == Background == A user model is the collection and categorization of personal data associated with a specific user. A user model is a (data) structure that is used to capture certain characteristics about an individual user, and a user profile is the actual representation in a given user model. The process of obtaining the user profile is called user modeling. Therefore, it is the basis for any adaptive changes to the system's behavior. Which data is included in the model depends on the purpose of the application. It can include personal information such as users' names and ages, their interests, their skills and knowledge, their goals and plans, their preferences and their dislikes or data about their behavior and their interactions with the system. There are different design patterns for user models, though often a mixture of them is used. Static user models Static user models are the most basic kinds of user models. Once the main data is gathered they are normally not changed again, they are static. Shifts in users' preferences are not registered and no learning algorithms are used to alter the model. Dynamic user models Dynamic user models allow a more up to date representation of users. Changes in their interests, their learning progress or interactions with the system are noticed and influence the user models. The models can thus be updated and take the current needs and goals of the users into account. Stereotype based user models Stereotype based user models are based on demographic statistics. Based on the gathered information users are classified into common stereotypes. The system then adapts to this stereotype. The application therefore can make assumptions about a user even though there might be no data about that specific area, because demographic studies have shown that other users in this stereotype have the same characteristics. Thus, stereotype based user models mainly rely on statistics and do not take into account that personal attributes might not match the stereotype. However, they allow predictions about a user even if there is rather little information about him or her. Highly adaptive user models Highly adaptive user models try to represent one particular user and therefore allow a very high adaptivity of the system. In contrast to stereotype based user models they do not rely on demographic statistics but aim to find a specific solution for each user. Although users can take great benefit from this high adaptivity, this kind of model needs to gather a lot of information first. == Data gathering == Information about users can be gathered in several ways. There are three main methods: Asking for specific facts while (first) interacting with the system Mostly this kind of data gathering is linked with the registration process. While registering users are asked for specific facts, their likes and dislikes and their needs. Often the given answers can be altered afterwards. Learning users' preferences by observing and interpreting their interactions with the system In this case users are not asked directly for their personal data and preferences, but this information is derived from their behavior while interacting with the system. The ways they choose to accomplish a tasks, the combination of things they takes interest in, these observations allow inferences about a specific user. The application dynamically learns from observing these interactions. Different machine learning algorithms may be used to accomplish this task. A hybrid approach which asks for explicit feedback and alters the user model by adaptive learning This approach is a mixture of the ones above. Users have to answer specific questions and give explicit feedback. Furthermore, their interactions with the system are observed and the derived information are used to automatically adjust the user models. Though the first method is a good way to quickly collect main data it lacks the ability to automatically adapt to shifts in users' interests. It depends on the users' readiness to give information and it is unlikely that they are going to edit their answers once the registration process is finished. Therefore, there is a high likelihood that the user models are not up to date. However, this first method allows the users to have full control over the collected data about them. It is their decision which information they are willing to provide. This possibility is missing in the second method. Adaptive changes in a system that learns users' preferences and needs only by interpreting their behavior might appear a bit opaque to the users, because they cannot fully understand and reconstruct why the system behaves the way it does. Moreover, the system is forced to collect a certain amount of data before it is able to predict the users' needs with the required accuracy. Therefore, it takes a certain learning time before a user can benefit from adaptive changes. However, afterwards these automatically adjusted user models allow a quite accurate adaptivity of the system. The hybrid approach tries to combine the advantages of both methods. Through collecting data by directly asking its users it gathers a first stock of information which can be used for adaptive changes. By learning from the users' interactions it can adjust the user models and reach more accuracy. Yet, the designer of the system has to decide, which of these information should have which amount of influence and what to do with learned data that contradicts some of the information given by a user. == System adaptation == Once a system has gathered information about a user it can evaluate that data by preset analytical algorithm and then start to adapt to the user's needs. These adaptations may concern every aspect of the system's behavior and depend on the system's purpose. Information and functions can be presented according to the user's interests, knowledge or goals by displaying only relevant features, hiding information the user does not need, making proposals what to do next and so on. One has to distinguish between adaptive and adaptable systems. In an adaptable system the user can manually change the system's appearance, behavior or functionality by actively selecting the corresponding options. Afterwards the system will stick to these choices. In an adaptive system a dynamic adaption to the user is automatically performed by the system itself, based on the built user model. Thus, an adaptive system needs ways to interpret information about the user in order to make these adaptations. One way to accomplish this task is implementing rule-based filtering. In this case a set of IF... THEN... rules is established that covers the knowledge base of the system. The IF-conditions can check for specific user-information and if they match the THEN-branch is performed which is responsible for the adaptive changes. Another approach is based on collaborative filtering. In this case information about a user is compared to that of other users of the same systems. Thus, if characteristics of the current user match those of another, the system can make assumptions about the current user by presuming that he or she is likely to have similar characteristics in areas where the model of the current user is lacking data. Based on these assumption the system then can perform adaptive changes. == Usages == Adaptive hypermedia: In an adaptive hypermedia system the displayed content and the offered hyperlinks are chosen on basis of users' specific characteristics, taking their goals, interests, knowledge and abilities into account. Thus, an adaptive hypermedia system aims to reduce the "lost in hyperspace" syndrome by presenting only relevant information. Adaptive educational hypermedia: Being a subdivision of adaptive hypermedia the main focus of adaptive educational hypermedia lies on education, displaying content and hyperlinks corresponding to the user's knowledge on the field of study. Intelligent tutoring system: Unlike adaptive educational hypermedia systems intelligent tutoring systems are stand-alone systems. Their aim is to help students in a specific field of study. To do so, they build up a user model where they store information about abilities, knowledge and needs of the user. The system can now adapt to this user by presenting approp