Algorithmic learning theory

Algorithmic learning theory

Algorithmic learning theory is a mathematical framework for analyzing machine learning problems and algorithms. Synonyms include formal learning theory and algorithmic inductive inference. Algorithmic learning theory is different from statistical learning theory in that it does not make use of statistical assumptions and analysis. Both algorithmic and statistical learning theory are concerned with machine learning and can thus be viewed as branches of computational learning theory. == Distinguishing characteristics == Unlike statistical learning theory and most statistical theory in general, algorithmic learning theory does not assume that data are random samples, that is, that data points are independent of each other. This makes the theory suitable for domains where observations are (relatively) noise-free but not random, such as language learning and automated scientific discovery. The fundamental concept of algorithmic learning theory is learning in the limit: as the number of data points increases, a learning algorithm should converge to a correct hypothesis on every possible data sequence consistent with the problem space. This is a non-probabilistic version of statistical consistency, which also requires convergence to a correct model in the limit, but allows a learner to fail on data sequences with probability measure 0 . Algorithmic learning theory investigates the learning power of Turing machines. Other frameworks consider a much more restricted class of learning algorithms than Turing machines, for example, learners that compute hypotheses more quickly, for instance in polynomial time. An example of such a framework is probably approximately correct learning . == Learning in the limit == The concept was introduced in E. Mark Gold's seminal paper "Language identification in the limit". The objective of language identification is for a machine running one program to be capable of developing another program by which any given sentence can be tested to determine whether it is "grammatical" or "ungrammatical". The language being learned need not be English or any other natural language - in fact the definition of "grammatical" can be absolutely anything known to the tester. In Gold's learning model, the tester gives the learner an example sentence at each step, and the learner responds with a hypothesis, which is a suggested program to determine grammatical correctness. It is required of the tester that every possible sentence (grammatical or not) appears in the list eventually, but no particular order is required. It is required of the learner that at each step the hypothesis must be correct for all the sentences so far. A particular learner is said to be able to "learn a language in the limit" if there is a certain number of steps beyond which its hypothesis no longer changes. At this point it has indeed learned the language, because every possible sentence appears somewhere in the sequence of inputs (past or future), and the hypothesis is correct for all inputs (past or future), so the hypothesis is correct for every sentence. The learner is not required to be able to tell when it has reached a correct hypothesis, all that is required is that it be true. Gold showed that any language which is defined by a Turing machine program can be learned in the limit by another Turing-complete machine using enumeration. This is done by the learner testing all possible Turing machine programs in turn until one is found which is correct so far - this forms the hypothesis for the current step. Eventually, the correct program will be reached, after which the hypothesis will never change again (but note that the learner does not know that it won't need to change). Gold also showed that if the learner is given only positive examples (that is, only grammatical sentences appear in the input, not ungrammatical sentences), then the language can only be guaranteed to be learned in the limit if there are only a finite number of possible sentences in the language (this is possible if, for example, sentences are known to be of limited length). Language identification in the limit is a highly abstract model. It does not allow for limits of runtime or computer memory which can occur in practice, and the enumeration method may fail if there are errors in the input. However the framework is very powerful, because if these strict conditions are maintained, it allows the learning of any program known to be computable. This is because a Turing machine program can be written to mimic any program in any conventional programming language. See Church-Turing thesis. == Other identification criteria == Learning theorists have investigated other learning criteria, such as the following. Efficiency: minimizing the number of data points required before convergence to a correct hypothesis. Mind Changes: minimizing the number of hypothesis changes that occur before convergence. Mind change bounds are closely related to mistake bounds that are studied in statistical learning theory. Kevin Kelly has suggested that minimizing mind changes is closely related to choosing maximally simple hypotheses in the sense of Occam’s Razor. == Annual conference == Since 1990, there is an International Conference on Algorithmic Learning Theory (ALT), called Workshop in its first years (1990–1997). Between 1992 and 2016, proceedings were published in the LNCS series. Starting from 2017, they are published by the Proceedings of Machine Learning Research. The 34th conference will be held in Singapore in Feb 2023. The topics of the conference cover all of theoretical machine learning, including statistical and computational learning theory, online learning, active learning, reinforcement learning, and deep learning.

Landmark point

In morphometrics, landmark point or shortly landmark is a point in a shape object in which correspondences between and within the populations of the object are preserved. In other disciplines, landmarks may be known as vertices, anchor points, control points, sites, profile points, 'sampling' points, nodes, markers, fiducial markers, etc. Landmarks can be defined either manually by experts or automatically by a computer program. There are three basic types of landmarks: anatomical landmarks, mathematical landmarks or pseudo-landmarks. An anatomical landmark is a biologically-meaningful point in an organism. Usually experts define anatomical points to ensure their correspondences within the same species. Examples of anatomical landmark in shape of a skull are the eye corner, tip of the nose, jaw, etc. Anatomical landmarks determine homologous parts of an organism, which share a common ancestry. Mathematical landmarks are points in a shape that are located according to some mathematical or geometrical property, for instance, a high curvature point or an extreme point. A computer program usually determines mathematical landmarks used for an automatic pattern recognition. Pseudo-landmarks are constructed points located between anatomical or mathematical landmarks. A typical example is an equally spaced set of points between two anatomical landmarks to get more sample points from a shape. Pseudo-landmarks are useful during shape matching, when the matching process requires a large number of points.

Software requirements

Software requirements for a system are the description of what the system should do, the service or services that it provides and the constraints on its operation. The IEEE Standard Glossary of Software Engineering Terminology defines a requirement as: A condition or capability needed by a user to solve a problem or achieve an objective A condition or capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed document A documented representation of a condition or capability as in 1 or 2 The activities related to working with software requirements can broadly be broken down into elicitation, analysis, specification, and management. Note that the wording Software requirements is additionally used in software release notes to explain, which depending on software packages are required for a certain software to be built/installed/used. == Elicitation == Elicitation is the gathering and discovery of requirements from stakeholders and other sources. A variety of techniques can be used such as joint application design (JAD) sessions, interviews, document analysis, focus groups, etc. Elicitation is the first step of requirements development. == Analysis == Analysis is the logical breakdown that proceeds from elicitation. Analysis involves reaching a richer and more precise understanding of each requirement and representing sets of requirements in multiple, complementary ways. Requirements Triage or prioritization of requirements is another activity which often follows analysis. This relates to Agile software development in the planning phase, e.g. by Planning poker, however it might not be the same depending on the context and nature of the project and requirements or product/service that is being built. == Specification == Specification involves representing and storing the collected requirements knowledge in a persistent and well-organized fashion that facilitates effective communication and change management. Use cases, user stories, functional requirements, and visual analysis models are popular choices for requirements specification. == Validation == Validation involves techniques to confirm that the correct set of requirements has been specified to build a solution that satisfies the project's business objectives, and to detect and correct errors in the requirements before implementation. == Management == Requirements change during projects and there are often many of them. Management of this change becomes paramount to ensuring that the correct software is built for the stakeholders. == Tool support for Requirements Engineering == === Tools for Requirements Elicitation, Analysis and Validation === Taking into account that these activities may involve some artifacts such as observation reports (user observation), questionnaires (interviews, surveys and polls), use cases, user stories; activities such as requirement workshops (charrettes), brainstorming, mind mapping, role-playing; and even, prototyping; software products providing some or all of these capabilities can be used to help achieve these tasks. There is at least one author who advocates, explicitly, for mind mapping tools such as FreeMind; and, alternatively, for the use of specification by example tools such as Concordion. Additionally, the ideas and statements resulting from these activities may be gathered and organized with wikis and other collaboration tools such as Trello. The features actually implemented and standards compliance vary from product to product. === Tools for Requirements Specification === A Software requirements specification (SRS) document might be created using general-purpose software like a word processor or one of several specialized tools. Some of these tools can import, edit, export and publish SRS documents. It may help to make SRS documents while following a standardised structure and methodology, such as ISO/IEC/IEEE 29148:2018. Likewise, software may or not use some standard to import or export requirements (such as ReqIF) or not allow these exchanges at all. === Tools for Requirements Document Verification === Tools of this kind verify if there are any errors in a requirements document according to some expected structure or standard. === Tools for Requirements Comparison === Tools of this kind compare two requirement sets according to some expected document structure and standard. === Tools for Requirements Merge and Update === Tools of this kind allow the merging and update of requirement documents. === Tools for Requirements Traceability === Tools of this kind allow tracing requirements to other artifacts such as models and source code (forward traceability) or, to previous ones such as business rules and constraints (backwards traceability). === Tools for Model-Based Software or Systems Requirement Engineering === Model-based systems engineering (MBSE) is the formalised application of modelling to support system requirements, design, analysis, verification and validation activities beginning in the conceptual design phase and continuing throughout development and later lifecycle phases. It is also possible to take a model-based approach for some stages of the requirements engineering and, a more traditional one, for others. Very many combinations might be possible. The level of formality and complexity depends on the underlying methodology involved (for instance, i is much more formal than SysML and, even more formal than UML) === Tools for general Requirements Engineering === Tools in this category may provide some mix of the capabilities mentioned previously and others such as requirement configuration management and collaboration. The features actually implemented and standards compliance vary from product to product. There are even more capable or general tools that support other stages and activities. They are classified as ALM tools.

Business Controls Corporation

Business Controls Corporation is a privately held computer company that developed an application-program-generator and also a series of accounting software packages. These packages were widely enough used for various business magazines to have back-of-the-book ads for companies seeking accountants with experience in one or more of them. Computer magazines ran coverage for their SB-5 application-program-generator as from time to time new versions were released, each with new or improved features. == Early days == The company's initial offerings were packages for the DEC PDP-8, although Business Controls Corporation also wrote custom-written programs for customers. Large customers with mainframes who also used smaller systems for departmental use and distributed processing also used BCC's services. == SB-5 == The addition of an application-program-generator named SB-5 that, from specifications, could generate COBOL code was a major step forward. Although this began with supporting the DEC PDP-11, they subsequently began to support COBOL on DEC's DECsystem-10 & DECSYSTEM-20. VAX support came later. The specifications also permitted COBOL inserts and overrides: SB-5 could build an application that was all COBOL, yet only code the portions that varied from BCC's "vanilla" accounting packages. === Similar offerings === A similar idea was done for the IBM mainframe world in the form of a series of application-program-generators from Dylakor Corporation. They were named DYL-250, DYL-260, DYL-270 & DYL-280. Dylakor was acquired by Computer Associates. The specific syntax was different, but it had wider use, and - a mark of success and recognition in the industry - syntax-compatible implementations were released by a competitor. Still another alternative was Peat Marwick Mitchell's PMM2170 application-program-generator package. Like the others, it supported COBOL inserts and overrides. === Extended integration === Business Controls Corporation subsequently extended SB-5's feature set to provide support for System 1022, a product for the DECsystem-10 & DECSYSTEM-20; 1022's vendor also had a VAX/VMS (later OpenVMS) product, System 1032.

Clone tool

The clone tool, as it is known in Adobe Photoshop, Inkscape, GIMP, and Corel PhotoPaint, is used in digital image editing to replace information for one part of a picture with information from another part. In other image editing software, its equivalent is sometimes called a rubber stamp tool or a clone brush. == Applications == The clone tool can remove objects by copying a nearby background. The user selects a matching location as the source, then paints over the element to be hidden. A typical use for the tool is in object removal – more colloquially, "airbrushing" or "photoshopping" out an unwanted part of the image. If a part of an image is removed simply by cutting it out, then a hole is left in the background. The Clone tool can fill in this hole convincingly with a copy of the existing background from elsewhere in the image. A common use for this tool is to retouch skin, particularly in portraits, to remove blemishes and make skin tones more even. Cloning can also be used to remove other unwanted elements, such as telephone wires, an unwanted bird in the sky, and the like. A more automated method of object removal uses texture synthesis to fill in gaps. Of these, patch-based texture synthesis or "image quilting" is essentially an automated application of the clone tool, choosing the optimal source area so as to patch over with a minimal seam. In some cases, the undesired object is mixed with the remainder of the image, and a simple circular brush, even with feathering, would not work. For these cases, some programs allow an object to be selected by color/outline so other areas are not affected. Other programs allow edge/color sensitive brushes to deal with such objects. == Healing tool == A similar tool is the healing tool, which occurs in variants such as the healing brush or spot healing tool. These incorporate the existing texture, rather than painting it over.

Database

In computing, a database is an organized collection of data or a type of data store based on the use of a database management system (DBMS), the software that interacts with end users, applications, and the database itself to capture and analyze the data. The DBMS additionally encompasses the core facilities provided to administer the database. The sum total of the database, the DBMS and the associated applications can be referred to as a database system. Often the term "database" is also used loosely to refer to any of the DBMS, the database system or an application associated with the database. Before digital storage and retrieval of data became widespread, index cards were used for data storage in a wide range of applications and environments: in the home to record and store recipes, shopping lists, contact information and other organizational data; in business to record presentation notes, project research and notes, and contact information; in schools as flash cards or other visual aids; and in academic research to hold data such as bibliographical citations or notes in a card file. Professional book indexers used index cards in the creation of book indexes until they were replaced by indexing software in the 1980s and 1990s. Small databases can be stored on a file system, while large databases are hosted on computer clusters or cloud storage. The design of databases spans formal techniques and practical considerations, including data modeling, efficient data representation and storage, query languages, security and privacy of sensitive data, and distributed computing issues, including supporting concurrent access and fault tolerance. Computer scientists may classify database management systems according to the database models that they support. Relational databases became dominant in the 1980s. These model data as rows and columns in a series of tables, and the vast majority use SQL for writing and querying data. In the 2000s, non-relational databases became popular, collectively referred to as NoSQL, because they use different query languages. == Terminology and overview == Formally, a "database" refers to a set of related data accessed through the use of a "database management system" (DBMS), which is an integrated set of computer software that allows users to interact with one or more databases and provides access to all of the data contained in the database (although restrictions may exist that limit access to particular data). The DBMS provides various functions that allow entry, storage and retrieval of large quantities of information and provides ways to manage how that information is organized. Because of the close relationship between them, the term "database" is often used casually to refer to both a database and the DBMS used to manipulate it. Outside the world of professional information technology, the term database is often used to refer to any collection of related data (such as a spreadsheet or a card index) as size and usage requirements typically necessitate use of a database management system. Existing DBMSs provide various functions that allow management of a database and its data which can be classified into four main functional groups: Data definition – Creation, modification and removal of definitions that detail how the data is to be organized. Update – Insertion, modification, and deletion of the data itself. Retrieval – Selecting data according to specified criteria (e.g., a query, a position in a hierarchy, or a position in relation to other data) and providing that data either directly to the user, or making it available for further processing by the database itself or by other applications. The retrieved data may be made available in a more or less direct form without modification, as it is stored in the database, or in a new form obtained by altering it or combining it with existing data from the database. Administration – Registering and monitoring users, enforcing data security, monitoring performance, maintaining data integrity, dealing with concurrency control, and recovering information that has been corrupted by some event such as an unexpected system failure. Both a database and its DBMS conform to the principles of a particular database model. "Database system" refers collectively to the database model, database management system, and database. Physically, database servers are dedicated computers that hold the actual databases and run only the DBMS and related software. Database servers are usually multiprocessor computers, with generous memory and RAID disk arrays used for stable storage. Hardware database accelerators, connected to one or more servers via a high-speed channel, are also used in large-volume transaction processing environments. DBMSs are found at the heart of most database applications. DBMSs may be built around a custom multitasking kernel with built-in networking support, but modern DBMSs typically rely on a standard operating system to provide these functions. Since DBMSs comprise a significant market, computer and storage vendors often take into account DBMS requirements in their own development plans. Databases and DBMSs can be categorized according to the database model(s) that they support (such as relational or XML), the type(s) of computer they run on (from a server cluster to a mobile phone), the query language(s) used to access the database (such as SQL or XQuery), and their internal engineering, which affects performance, scalability, resilience, and security. == History == The sizes, capabilities, and performance of databases and their respective DBMSs have grown in orders of magnitude. These performance increases were enabled by the technology progress in the areas of processors, computer memory, computer storage, and computer networks. The concept of a database was made possible by the emergence of direct access storage media such as magnetic disks, which became widely available in the mid-1960s; earlier systems relied on sequential storage of data on magnetic tape. The subsequent development of database technology can be divided into three eras based on data model or structure: navigational, SQL/relational, and post-relational. The two main early navigational data models were the hierarchical model and the CODASYL model (network model). These were characterized by the use of pointers (often physical disk addresses) to follow relationships from one record to another. The relational model, first proposed in 1970 by Edgar F. Codd, departed from this tradition by insisting that applications should search for data by content, rather than by following links. The relational model employs sets of ledger-style tables, each used for a different type of entity. Only in the mid-1980s did computing hardware become powerful enough to allow the wide deployment of relational systems (DBMSs plus applications). By the early 1990s, however, relational systems dominated in all large-scale data processing applications, and as of 2018 they remain dominant: IBM Db2, Oracle, MySQL, and Microsoft SQL Server are the most searched DBMS. The dominant database language, standardized SQL for the relational model, has influenced database languages for other data models. Object databases were developed in the 1980s to overcome the inconvenience of object–relational impedance mismatch, which led to the coining of the term "post-relational" and also the development of hybrid object–relational databases. The next generation of post-relational databases in the late 2000s became known as NoSQL databases, introducing fast key–value stores and document-oriented databases. A competing "next generation" known as NewSQL databases attempted new implementations that retained the relational/SQL model while aiming to match the high performance of NoSQL compared to commercially available relational DBMSs. === 1960s, navigational DBMS === The introduction of the term database coincided with the availability of direct-access storage (disks and drums) from the mid-1960s onwards. The term represented a contrast with the tape-based systems of the past, allowing shared interactive use rather than daily batch processing. The Oxford English Dictionary cites a 1962 report by the System Development Corporation of California as the first to use the term "data-base" in a specific technical sense. As computers grew in speed and capability, a number of general-purpose database systems emerged; by the mid-1960s a number of such systems had come into commercial use. Interest in a standard began to grow, and Charles Bachman, author of one such product, the Integrated Data Store (IDS), founded the Database Task Group within CODASYL, the group responsible for the creation and standardization of COBOL. In 1971, the Database Task Group delivered their standard, which generally became known as the CODASYL approach, and soon a number of commercial products based on this approach entered the market. The CODASYL approach of

Skipper (computer software)

Skipper is a visualization tool and code/schema generator for PHP ORM frameworks like Doctrine2, Doctrine, Propel, and CakePHP, which are used to create database abstraction layer. Skipper is developed by Czech company Inventic, s.r.o. based in Brno, and was known as ORM Designer prior to rebranding in 2014. == Overview == Generates visual model from the schema definition files Repetitive import/export of schema definitions in supported formats (XML, YML, PHP annotations) Schema definition files are automatically generated from the visual model Visual representation uses ER diagram extended by concepts of inheritance and many-to-many Supports customization using .xml configuration files and JavaScript Does not support direct connections to the database Crude and simplistic visual representation and menus == Architecture == Skipper was built on the Qt framework. Import/export of the schema definitions uses XSL transformations powered by LibXslt library. Imported source files are first converted to XML format: no conversion for XML, simple conversion for YML, creating the Abstract Syntax Tree and its subsequent conversion to XML for PHP annotations. The import/export scripts are configured in JavaScript and can be freely customized. == Supported ORM frameworks == Frameworks supported for visual model and schema files generation: Doctrine2 Doctrine CakePHP == History == Skipper was created as an internal tool for the web applications developed by Inventic. It was first published as a commercial tool under the name ORM Designer in 2009. Application was reworked and optimized in January 2013, and released as ORM Designer 2. In May 2013 ORM Designer became part of the South Moravian Innovation Center Incubator program (support program for innovative technological startups). In June 2014, ORM Designer version 3 was released and rebranded under the name of Skipper