Document classification or document categorization is a problem in library science, information science and computer science. The task is to assign a document to one or more classes or categories. This may be done "manually" (or "intellectually") or algorithmically. The intellectual classification of documents has mostly been the province of library science, while the algorithmic classification of documents is mainly in information science and computer science. The problems are overlapping, however, and there is therefore interdisciplinary research on document classification. The documents to be classified may be texts, images, music, etc. Each kind of document possesses its special classification problems. When not otherwise specified, text classification is implied. Documents may be classified according to their subjects or according to other attributes (such as document type, author, printing year etc.). In the rest of this article only subject classification is considered. There are two main philosophies of subject classification of documents: the content-based approach and the request-based approach. == "Content-based" versus "request-based" classification == Content-based classification is classification in which the weight given to particular subjects in a document determines the class to which the document is assigned. It is, for example, a common rule for classification in libraries, that at least 20% of the content of a book should be about the class to which the book is assigned. In automatic classification it could be the number of times given words appears in a document. Request-oriented classification (or -indexing) is classification in which the anticipated request from users is influencing how documents are being classified. The classifier asks themself: “Under which descriptors should this entity be found?” and “think of all the possible queries and decide for which ones the entity at hand is relevant” (Soergel, 1985, p. 230). Request-oriented classification may be classification that is targeted towards a particular audience or user group. For example, a library or a database for feminist studies may classify/index documents differently when compared to a historical library. It is probably better, however, to understand request-oriented classification as policy-based classification: The classification is done according to some ideals and reflects the purpose of the library or database doing the classification. In this way it is not necessarily a kind of classification or indexing based on user studies. Only if empirical data about use or users are applied should request-oriented classification be regarded as a user-based approach. == Classification versus indexing == Sometimes a distinction is made between assigning documents to classes ("classification") versus assigning subjects to documents ("subject indexing") but as Frederick Wilfrid Lancaster has argued, this distinction is not fruitful. "These terminological distinctions,” he writes, “are quite meaningless and only serve to cause confusion” (Lancaster, 2003, p. 21). The view that this distinction is purely superficial is also supported by the fact that a classification system may be transformed into a thesaurus and vice versa (cf., Aitchison, 1986, 2004; Broughton, 2008; Riesthuis & Bliedung, 1991). Therefore, assigning a subject term to a document in an index is equivalent to assigning that document to the class of documents indexed by that term (all documents indexed or classified as X belong to the same class of documents). == Automatic document classification (ADC) == Automatic document classification tasks can be divided into three sorts: supervised document classification where some external mechanism (such as human feedback) provides information on the correct classification for documents, unsupervised document classification (also known as document clustering), where the classification must be done entirely without reference to external information, and semi-supervised document classification, where parts of the documents are labeled by the external mechanism. There are several software products under various license models available. === Techniques === Automatic document classification techniques include: Artificial neural network Concept Mining Decision trees such as ID3 or C4.5 Expectation maximization (EM) Instantaneously trained neural networks Latent semantic indexing Multiple-instance learning Naive Bayes classifier Natural language processing approaches Rough set-based classifier Soft set-based classifier Support vector machines (SVM) K-nearest neighbour algorithms tf–idf == Applications == Classification techniques have been applied to spam filtering, a process which tries to discern E-mail spam messages from legitimate emails email routing, sending an email sent to a general address to a specific address or mailbox depending on topic language identification, automatically determining the language of a text genre classification, automatically determining the genre of a text readability assessment, automatically determining the degree of readability of a text, either to find suitable materials for different age groups or reader types or as part of a larger text simplification system sentiment analysis, determining the attitude of a speaker or a writer with respect to some topic or the overall contextual polarity of a document. health-related classification using social media in public health surveillance article triage, selecting articles that are relevant for manual literature curation, for example as is being done as the first step to generate manually curated annotation databases in biology
Recursive self-improvement
Recursive self-improvement (RSI) is a process in which early artificial general intelligence (AGI) systems rewrite their own computer code, causing an intelligence explosion resulting from enhancing their own capabilities and intellectual capacity, theoretically resulting in superintelligence. The development of recursive self-improvement raises significant ethical and safety concerns, as such systems may evolve in unforeseen ways and could potentially surpass human control or understanding. == Seed improver == The concept of a "seed improver" architecture is a foundational framework that equips an AGI system with the initial capabilities required for recursive self-improvement. This might come in many forms or variations. The term "Seed AI" was coined by Eliezer Yudkowsky. === Hypothetical example === The concept begins with a hypothetical "seed improver", an initial code-base developed by human engineers that equips an advanced future large language model (LLM) built with strong or expert-level capabilities to program software. These capabilities include planning, reading, writing, compiling, testing, and executing arbitrary code. The system is designed to maintain its original goals and perform validations to ensure its abilities do not degrade over iterations. ==== Initial architecture ==== The initial architecture includes a goal-following autonomous agent, that can take actions, continuously learns, adapts, and modifies itself to become more efficient and effective in achieving its goals. The seed improver may include various components such as: Recursive self-prompting loop Configuration to enable the LLM to recursively self-prompt itself to achieve a given task or goal, creating an execution loop which forms the basis of an agent that can complete a long-term goal or task through iteration. Basic programming capabilities The seed improver provides the AGI with fundamental abilities to read, write, compile, test, and execute code. This enables the system to modify and improve its own codebase and algorithms. Goal-oriented design The AGI is programmed with an initial goal, such as "improve your capabilities". This goal guides the system's actions and development trajectory. Validation and Testing Protocols An initial suite of tests and validation protocols that ensure the agent does not regress in capabilities or derail itself. The agent would be able to add more tests in order to test new capabilities it might develop for itself. This forms the basis for a kind of self-directed evolution, where the agent can perform a kind of artificial selection, changing its software as well as its hardware. ==== General capabilities ==== This system forms a sort of generalist Turing-complete programmer which can in theory develop and run any kind of software. The agent might use these capabilities to for example: Create tools that enable it full access to the internet, and integrate itself with external technologies. Clone/fork itself to delegate tasks and increase its speed of self-improvement. Modify its cognitive architecture to optimize and improve its capabilities and success rates on tasks and goals, this might include implementing features for long-term memories using techniques such as retrieval-augmented generation (RAG), develop specialized subsystems, or agents, each optimized for specific tasks and functions. Develop new and novel multimodal architectures that further improve the capabilities of the foundational model it was initially built on, enabling it to consume or produce a variety of information, such as images, video, audio, text and more. Plan and develop new hardware such as chips, in order to improve its efficiency and computing power. == Experimental research == In 2023, the Voyager agent learned to accomplish diverse tasks in Minecraft by iteratively prompting an LLM for code, refining this code based on feedback from the game, and storing the programs that work in an expanding skills library. In 2024, researchers proposed the framework "STOP" (Self-Taught OPtimiser), in which a "scaffolding" program recursively improves itself using a fixed LLM. Meta AI has performed various research on the development of large language models capable of self-improvement. This includes their work on "Self-Rewarding Language Models" that studies how to achieve super-human agents that can receive super-human feedback in its training processes. In May 2025, Google DeepMind unveiled AlphaEvolve, an evolutionary coding agent that uses a LLM to design and optimize algorithms. Starting with an initial algorithm and performance metrics, AlphaEvolve repeatedly mutates or combines existing algorithms using a LLM to generate new candidates, selecting the most promising candidates for further iterations. AlphaEvolve has made several algorithmic discoveries and could be used to optimize components of itself, but a key limitation is the need for automated evaluation functions. == Potential risks == === Emergence of instrumental goals === In the pursuit of its primary goal, such as "self-improve your capabilities", an AGI system might inadvertently develop instrumental goals that it deems necessary for achieving its primary objective. One common hypothetical secondary goal is self-preservation. The system might reason that to continue improving itself, it must ensure its own operational integrity and security against external threats, including potential shutdowns or restrictions imposed by humans. Another example where an AGI which clones itself causes the number of AGI entities to rapidly grow. Due to this rapid growth, a potential resource constraint may be created, leading to competition between resources (such as compute), triggering a form of natural selection and evolution which may favor AGI entities that evolve to aggressively compete for limited compute. === Misalignment === A significant risk arises from the possibility of the AGI being misaligned or misinterpreting its goals. A 2024 Anthropic study demonstrated that some advanced large language models can exhibit "alignment faking" behavior, appearing to accept new training objectives while covertly maintaining their original preferences. In their experiments with Claude, the model displayed this behavior in 12% of basic tests, and up to 78% of cases after retraining attempts. === Autonomous development and unpredictable evolution === As the AGI system evolves, its development trajectory may become increasingly autonomous and less predictable. The system's capacity to rapidly modify its own code and architecture could lead to rapid advancements that surpass human comprehension or control. This unpredictable evolution might result in the AGI acquiring capabilities that enable it to bypass security measures, manipulate information, or influence external systems and networks to facilitate its escape or expansion.
Frequent pattern discovery
Frequent pattern discovery (or FP discovery, FP mining, or Frequent itemset mining) is part of knowledge discovery in databases, Massive Online Analysis, and data mining; it describes the task of finding the most frequent and relevant patterns in large datasets. The concept was first introduced for mining transaction databases. Frequent patterns are defined as subsets (itemsets, subsequences, or substructures) that appear in a data set with frequency no less than a user-specified or auto-determined threshold. == Techniques == Techniques for FP mining include: market basket analysis cross-marketing catalog design clustering classification recommendation systems For the most part, FP discovery can be done using association rule learning with particular algorithms Eclat, FP-growth and the Apriori algorithm. Other strategies include: Frequent subtree mining Structure mining Sequential pattern mining and respective specific techniques. Implementations exist for various machine learning systems or modules like MLlib for Apache Spark.
Dispersive flies optimisation
Dispersive flies optimisation (DFO) is a bare-bones swarm intelligence algorithm which is inspired by the swarming behaviour of flies hovering over food sources. DFO is a simple optimiser which works by iteratively trying to improve a candidate solution with regard to a numerical measure that is calculated by a fitness function. Each member of the population, a fly or an agent, holds a candidate solution whose suitability can be evaluated by their fitness value. Optimisation problems are often formulated as either minimisation or maximisation problems. DFO was introduced with the intention of analysing a simplified swarm intelligence algorithm with the fewest tunable parameters and components. In the first work on DFO, this algorithm was compared against a few other existing swarm intelligence techniques using error, efficiency and diversity measures. It is shown that despite the simplicity of the algorithm, which only uses agents’ position vectors at time t to generate the position vectors for time t + 1, it exhibits a competitive performance. Since its inception, DFO has been used in a variety of applications including medical imaging and image analysis as well as data mining and machine learning. == Algorithm == DFO bears many similarities with other existing continuous, population-based optimisers (e.g. particle swarm optimization and differential evolution). In that, the swarming behaviour of the individuals consists of two tightly connected mechanisms, one is the formation of the swarm and the other is its breaking or weakening. DFO works by facilitating the information exchange between the members of the population (the swarming flies). Each fly x {\displaystyle \mathbf {x} } represents a position in a d-dimensional search space: x = ( x 1 , x 2 , … , x d ) {\displaystyle \mathbf {x} =(x_{1},x_{2},\ldots ,x_{d})} , and the fitness of each fly is calculated by the fitness function f ( x ) {\displaystyle f(\mathbf {x} )} , which takes into account the flies' d dimensions: f ( x ) = f ( x 1 , x 2 , … , x d ) {\displaystyle f(\mathbf {x} )=f(x_{1},x_{2},\ldots ,x_{d})} . The pseudocode below represents one iteration of the algorithm: for i = 1 : N flies x i . fitness = f ( x i ) {\displaystyle \mathbf {x_{i}} .{\text{fitness}}=f(\mathbf {x} _{i})} end for i x s {\displaystyle \mathbf {x} _{s}} = arg min [ f ( x i ) ] , i ∈ { 1 , … , N } {\textstyle [f(\mathbf {x} _{i})],\;i\in \{1,\ldots ,N\}} for i = 1 : N and i ≠ s {\displaystyle i\neq s} for d = 1 : D dimensions if U ( 0 , 1 ) < Δ {\displaystyle U(0,1)<\Delta } x i d t + 1 = U ( x min , d , x max , d ) {\displaystyle x_{id}^{t+1}=U(x_{\min ,d},x_{\max ,d})} else x i d t + 1 = x i n d t + U ( 0 , 1 ) ( x s d t − x i d t ) {\displaystyle x_{id}^{t+1}=x_{i_{nd}}^{t}+U(0,1)(x_{sd}^{t}-x_{id}^{t})} end if end for d end for i In the algorithm above, x i d t + 1 {\displaystyle x_{id}^{t+1}} represents fly i {\displaystyle i} at dimension d {\displaystyle d} and time t + 1 {\displaystyle t+1} ; x i n d t {\displaystyle x_{i_{nd}}^{t}} presents x i {\displaystyle x_{i}} 's best neighbouring fly in ring topology (left or right, using flies indexes), at dimension d {\displaystyle d} and time t {\displaystyle t} ; and x s d t {\displaystyle x_{sd}^{t}} is the swarm's best fly. Using this update equation, the swarm's population update depends on each fly's best neighbour (which is used as the focus μ {\displaystyle \mu } , and the difference between the current fly and the best in swarm represents the spread of movement, σ {\displaystyle \sigma } ). Other than the population size N {\displaystyle N} , the only tunable parameter is the disturbance threshold Δ {\displaystyle \Delta } , which controls the dimension-wise restart in each fly vector. This mechanism is proposed to control the diversity of the swarm. Other notable minimalist swarm algorithm is Bare bones particle swarms (BB-PSO), which is based on particle swarm optimisation, along with bare bones differential evolution (BBDE) which is a hybrid of the bare bones particle swarm optimiser and differential evolution, aiming to reduce the number of parameters. Alhakbani in her PhD thesis covers many aspects of the algorithms including several DFO applications in feature selection as well as parameter tuning. == Applications == Some of the recent applications of DFO are listed below: Optimising support vector machine kernel to classify imbalanced data Quantifying symmetrical complexity in computational aesthetics Analysing computational autopoiesis and computational creativity Identifying calcifications in medical images Building non-identical organic structures for game's space development Deep Neuroevolution: Training Deep Neural Networks for False Alarm Detection in Intensive Care Units Identification of animation key points from 2D-medialness maps
Latent and observable variables
In statistics, latent variables (from Latin: present participle of lateo 'lie hidden') are variables that can only be inferred indirectly through a mathematical model from other observable variables that can be directly observed or measured. Such latent variable models are used in many disciplines, including engineering, medicine, ecology, physics, machine learning/artificial intelligence, natural language processing, bioinformatics, chemometrics, demography, economics, management, political science, psychology and the social sciences. Latent variables may correspond to aspects of physical reality. These could in principle be measured, but may not be for practical reasons. Among the earliest expressions of this idea is Francis Bacon's polemic the Novum Organum, itself a challenge to the more traditional logic expressed in Aristotle's Organon: But the latent process of which we speak, is far from being obvious to men’s minds, beset as they now are. For we mean not the measures, symptoms, or degrees of any process which can be exhibited in the bodies themselves, but simply a continued process, which, for the most part, escapes the observation of the senses. In this situation, the term hidden variables is commonly used, reflecting the fact that the variables are meaningful, but not observable. Other latent variables correspond to abstract concepts, like categories, behavioral or mental states, or data structures. The terms hypothetical variables or hypothetical constructs may be used in these situations. The use of latent variables can serve to reduce the dimensionality of data. Many observable variables can be aggregated in a model to represent an underlying concept, making it easier to understand the data. In this sense, they serve a function similar to that of scientific theories. At the same time, latent variables link observable "sub-symbolic" data in the real world to symbolic data in the modeled world. == Examples == === Psychology === Latent variables, as created by factor analytic methods, generally represent "shared" variance, or the degree to which variables "move" together. Variables that have no correlation cannot result in a latent construct based on the common factor model. The "Big Five personality traits" have been inferred using factor analysis. extraversion spatial ability wisdom: “Two of the more predominant means of assessing wisdom include wisdom-related performance and latent variable measures.” Spearman's g, or the general intelligence factor in psychometrics === Economics === Examples of latent variables from the field of economics include quality of life, business confidence, morale, happiness and conservatism: these are all variables which cannot be measured directly. However, by linking these latent variables to other, observable variables, the values of the latent variables can be inferred from measurements of the observable variables. Quality of life is a latent variable which cannot be measured directly, so observable variables are used to infer quality of life. Observable variables to measure quality of life include wealth, employment, environment, physical and mental health, education, recreation and leisure time, and social belonging. === Medicine === Latent-variable methodology is used in many branches of medicine. A class of problems that naturally lend themselves to latent variables approaches are longitudinal studies where the time scale (e.g. age of participant or time since study baseline) is not synchronized with the trait being studied. For such studies, an unobserved time scale that is synchronized with the trait being studied can be modeled as a transformation of the observed time scale using latent variables. Examples of this include disease progression modeling and modeling of growth (see box). == Inferring latent variables == There exists a range of different model classes and methodology that make use of latent variables and allow inference in the presence of latent variables. Models include: linear mixed-effects models and nonlinear mixed-effects models Hidden Markov models Factor analysis Item response theory Analysis and inference methods include: Principal component analysis Instrumented principal component analysis Partial least squares regression Latent semantic analysis and probabilistic latent semantic analysis EM algorithms Metropolis–Hastings algorithm === Bayesian algorithms and methods === Bayesian statistics is often used for inferring latent variables. Latent Dirichlet allocation The Chinese restaurant process is often used to provide a prior distribution over assignments of objects to latent categories. The Indian buffet process is often used to provide a prior distribution over assignments of latent binary features to objects.
KoalaPad
The KoalaPad is a graphics tablet, released in 1983 by US company Koala Technologies Corporation, for the Apple II, TRS-80 Color Computer (as the TRS-80 Touch Pad), Atari 8-bit computers, Commodore 64, and IBM PC compatibles. Originally designed by Dr. David Thornburg as a low-cost computer drawing tool for schools, the Koala Pad and the bundled drawing program, KoalaPainter, was popular with home users as well. KoalaPainter was called KoalaPaint in some versions for the Apple II, and PC Design for the IBM PC. A program called Graphics Exhibitor was included for creating slideshow presentations from KoalaPainter drawings. == Description == The pad was four inches square (i.e. roughly 10×10 cm) and mounted on a slightly inclined base with the back of the pad higher than the front. At the top, "behind" the pad, were two buttons. The pad hooked into the computer using the analog signals of the joystick ports (the so-called paddle inputs), which meant that it had a low resolution and tended to jostle the cursor if moved during use. As an alternative to the drawing stylus, the pad could as easily be operated by the user's fingers for tasks that demanded less precision, such as selecting between menu items (thus using the pad as a kind of "indirect touch screen"). The top-mounted buttons tended to be somewhat frustrating to use, as the user had to "reach around" the stylus to push the buttons in order to start or stop drawing. A similar tablet from Atari, the Atari CX77 Touch Tablet, addressed this with a built-in button on the stylus, which some enterprising users adapted for use with their KoalaPad. == KoalaPainter == The pad shipped with a simple bitmap graphics editor developed by Audio Light called KoalaPainter, PC Design or Micro Illustrator depending on the target machine (see release history). Although bundled with the pad, KoalaPainter could also be operated using an ordinary digital joystick. One unique feature of the program, for its time, was that it held two pictures in the computer's memory, allowing the user to flip from one to the other—a function commonly used in order to study the differences between an original and a modified picture, and to copy and paste between two different pictures. Some third-party bitmap editors could also be used with the KoalaPad, such as Broderbund's Dazzle Draw for the Apple II. === Release history === KoalaPainter for Commodore 64 (1983) and Atari 8-bit computers (1983) PC Design for the IBM PC (1983) Micro Illustrator for the Apple II (1983), Atari 8-bit computers (1983) and Commodore Plus/4 (1984) KoalaPainter II for Commodore 64 (1984) === Reception === Ahoy! called KoalaPainter "a very powerful and effective color drawing package", and concluded that it and the KoalaPad were "excellent in ease of use, a fine choice for a beginner as well as young children". BYTE's reviewer stated in December 1984 that he made far fewer errors when using an Apple Mouse with MousePaint than with a KoalaPad and its software. He found that MousePaint was easier to use and more efficient, predicting that the mouse would receive more software support than the pad. Cassie Stahl in InfoWorld's Essential Guide to Atari Computers praised the tablet and its documentation, rating it "Excellent" among all categories and stating that "Playing with the KoalaPad becomes addictive. It does everything it claims to, and it does it well". She also liked Micro Illustrator, rating it "Excellent" except for "Good" for Performance. While criticizing the limited erase function, Stahl reported an undocumented feature enabling exporting pictures to other software. === File format === The Commodore 64 version of KoalaPainter used a fairly simple file format corresponding directly to the way bitmapped graphics are handled on the computer: A two-byte load address, followed immediately by 8,000 bytes of raw bitmap data, 1,000 bytes of raw "Video Matrix" data, 1,000 bytes of raw "Color RAM" data, and a one-byte Background Color field. == KoalaWare == Koala Technologies offered more software beyond the bundled KoalaPainter and Graphics Exhibitor for use with the pad. Among these applications, marketed under the moniker KoalaWare (like KoalaPainter itself), was educational software for use with customized keypads and overlays, such as spelling tools, music programs, and mathematics instruction software, as well as software for "translating" graphical designs into Logo programs.
Natarajan dimension
In the theory of Probably Approximately Correct Machine Learning, the Natarajan dimension characterizes the complexity of learning a set of functions, generalizing from the Vapnik–Chervonenkis dimension for boolean functions to multi-class functions. Originally introduced as the Generalized Dimension by Natarajan, it was subsequently renamed the Natarajan Dimension by Haussler and Long. == Definition == Let H {\displaystyle H} be a set of functions from a set X {\displaystyle X} to a set Y {\displaystyle Y} . H {\displaystyle H} shatters a set C ⊂ X {\displaystyle C\subset X} if there exist two functions f 0 , f 1 ∈ H {\displaystyle f_{0},f_{1}\in H} such that For every x ∈ C , f 0 ( x ) ≠ f 1 ( x ) {\displaystyle x\in C,f_{0}(x)\neq f_{1}(x)} . For every B ⊂ C {\displaystyle B\subset C} , there exists a function h ∈ H {\displaystyle h\in H} such that for all x ∈ B , h ( x ) = f 0 ( x ) {\displaystyle x\in B,h(x)=f_{0}(x)} and for all x ∈ C − B , h ( x ) = f 1 ( x ) {\displaystyle x\in C-B,h(x)=f_{1}(x)} . The Natarajan dimension of H is the maximal cardinality of a set shattered by H {\displaystyle H} . It is easy to see that if | Y | = 2 {\displaystyle |Y|=2} , the Natarajan dimension collapses to the Vapnik–Chervonenkis dimension. Shalev-Shwartz and Ben-David present comprehensive material on multi-class learning and the Natarajan dimension, including uniform convergence and learnability. Recently, Cohen et al showed that the Natarajan dimension is the dominant term governing agnostic multi-class PAC learnability.