AI Coding Vscode Extension

AI Coding Vscode Extension — independent reviews, comparisons, pricing and step-by-step guides on Aizhi.

• 

  Apptek

  Applications Technology (AppTek) is a U.S. company headquartered in McLean, Virginia that specializes in artificial intelligence and machine learning for human language technologies. The company provides both managed and professional services for natural language processing (NLP) technologies including automatic speech recognition (ASR), neural machine translation (MT), natural-language understanding (NLU) and neural speech synthesis. AppTek's Head of Science, Prof. Dr. -Ing Hermann Ney, was awarded the IEEE James L. Flanagan Speech and Audio Processing Award in 2019 and the ISCA Medal for Scientific Achievement in 2021 for his work in natural language processing. == History == AppTek was acquired in 1998 by Lernout & Hauspie (at the time a NASDAQ publicly traded company), AppTek organized a management buy-out and went private again in 2001. In 2014, the company sold its hybrid machine translation technology to eBay and has since rebuilt the platform to modern neural-based approaches for machine translation. In 2020, SOSi acquired non-controlling interest in AppTek and became an exclusive reseller of AppTek products for U.S. federal, state, and local government entities.

  Read more →
• 

  Artificial intelligence in architecture

  Artificial intelligence in architecture is the use of artificial intelligence in automation, design, and planning in the architectural process or in assisting human skills in the field of architecture. AI has been used by some architects for design, and has been proposed as a way to automate planning and routine tasks in the field. == Implications == === Benefits === Artificial intelligence, according to ArchDaily, is said to potentially significantly augment the architectural profession through its ability to improve the design and planning process as well as increasing productivity. Through its ability to handle a large amount of data, AI is said to potentially allow architects a range of design choices with criteria considerations such as budget, requirements adjusted to space, and sustainability goals calculated as part of the design process. ArchDaily said this may allow the design of optimized alternatives that can then undergo human review. AI tools are also said to potentially allow architects to assimilate urban and environmental data to inform their designs, streamlining initial stages of project planning and increasing efficiency and productivity. The advances in generative design through the input of specific prompts allow architects to produce visual designs, including photorealistic images, and thus render and explore various material choices and spatial configurations. ArchDaily noted this could speed the creative process as well as allow for experimentation and sophistication in the design. Additionally, AI's capacity for pattern recognition and coding could aid architects in organizing design resources and developing custom applications, thus enhancing the efficiency and collaboration between both architects and AI. AI is thought to also be able to contribute to the sustainability of buildings by analyzing various factors and following recommended energy-efficient modifications, thus pushing the industry towards greener practices. The use of AI in building maintenance, project management, and the creation of immersive virtual reality experiences are also thought of as potentially augmenting the architectural design process and workflow. Examples include the use of text-to-image systems such as Midjourney to create detailed architectural images, and the use of AI optimization systems from companies such as Finch3D and Autodesk to automatically generate floor plans from simple programmatic inputs. In contrast to digital-only creative practices, the high materiality of architectural outputs requires transitions from ephemeral digital files to permanent physical structures that are subject to strict safety regulations, material constraints, sensory intuition, and site-specific cultural contexts, making full automation difficult. Early adopters such as architect Stephen Coorlas have actively challenged the boundaries of architectural practice through AI. His early experimental initiative, Speculations on AI and Architecture, confronts the discipline's traditional workflows by training text-to-image AI tools such as Midjourney, Luma AI, and PromeAI to generate more nuanced architectural illustrations including construction documents, architectural details, and assembly sequences for various structures. Coorlas inputs precise terminology and architectural language to provoke the AI into producing axonometric drawings that resemble conventional documentation, then experiments with animating the outputs using AI generated depth maps and other AI image-to-3D wireframe tools. Stephen's inventive process invites architects and designers to reconsider authorship, automation, and the future of visual communication in the built environment. Rather than treating AI as a peripheral tool, Stephen has advocated for AI to be a speculative collaborator capable of engaging with discipline-specific challenges. His work contributes to the growing discourse on generative design, parametric optimization, and the philosophical implications of machine-assisted creativity raising urgent questions about how such technologies will reshape architectural agency, precision, and pedagogy. Another prominent advocate is Architect Andrew Kudless, who in an interview to Dezeen recounted that he uses AI to innovate in architectural design by incorporating materials and scenes not usually present in initial plans, which he believes can significantly alter client presentations. He told Dezeen he believes one should show clients renderings from the onset, with AI assisting in this work, arguing that changes in design should be a positive aspect of the client-designer relationship by actively involving clients in the process. Additionally, Kudless highlighted the AI's potential to facilitate labor in architectural firms, particularly in automating rendering tasks, thus reducing the workload on junior staff while maintaining control over the creative output. === Emergent aesthetics === In an interview for the AItopia series to Dezeen, designer Tim Fu discussed the transformative potential of AI in architecture, and proposed a future where AI could herald a "neoclassical futurist" style, blending the grandeur of classical aesthetics with futuristic design. Through his collaborative project, The AI Stone Carver, Fu showcased how AI can innovate traditional practices by generating design concepts that are then realized through human craftsmanship, such as stone carving by mason Till Apfel. This approach, he believed, celebrated the fusion of diverse architectural styles and also emphasized the unique capabilities of AI in enhancing creative design processes. Fu told Dezeen he envisions the integration of AI in design as a means to revive the ornamentation and detailed aesthetics characteristic of classical architecture, moving away from minimalism, which he said dominates contemporary architecture. He argued that AI's involvement in the ideation phase of design allows for a reversal in the roles of machine and human, enabling architects and designers to focus on creating more intricate and ornamental structures. Fu's optimistic outlook extended to the broader impact of AI on the architectural field, seeing it as an indispensable tool that will shift rather than replace human roles, enriching the field with innovative designs that pay homage to the beauty and qualities of classical architecture not present in contemporary architecture while embracing new technologies. This perspective resonates with designers like Manas Bhatia, whose explorations similarly embrace generative AI as a co-creator and a medium to express ideas, blend architectural traditions, and speculate spatial futures. === Concerns === As AI continues to expand its presence across various industries, its impact on the architectural profession has become a topic of growing discussion. These discussions focus on how AI processes may influence traditional architectural practices, potentially altering job roles, and shaping the nature of creativity. While AI-driven processes may increase efficiency in some aspects of the profession, they also raise questions about the potential loss of unique design perspectives. These thoughts have been countered by many prominent creative figures in the realm of AI architecture, such as Stephen Coorlas, Tim Fu, Hassan Ragab, and Manas Bhatia who have showcased the amplification of creativity in design and potential benefits in terms of restoring creative power to the designer. A key concern is that AI-powered tools could diminish the need for human involvement in specific tasks traditionally performed by architects. This has led to speculation that the profession may increasingly shift toward roles focused on oversight, coordination, and strategic decision-making rather than hands-on design work. In some design scenarios, algorithmically generated solutions can be adjusted to prioritize efficiency and cost-effectiveness, which some argue may overshadow the creative and contextual nuances that define individual architectural styles. As with any discipline though, it has been determined that AI can be configured to provide beneficial results based on inputs and end goals the architect or designer assigns it. There are also concerns about the potential for AI to exacerbate inequalities within the architectural profession. For instance, larger firms with greater resources to invest in advanced AI technologies may gain a competitive edge over smaller firms and independent architects. This dynamic could contribute to industry consolidation, potentially limiting the diversity of architectural practice and stifling innovation. Ethical considerations in regard to cultural sensitivity have also been raised due to the datasets used to train AI. Without proper vetting of data or implementing failsafe overrides, AI generated outcomes can trend toward overly documented and prioritized content.

  Read more →
• 

  Living lab

  The concept of the living lab has been defined in multiple ways. A definition from the European Network of Living Labs (ENoLL) is used most widely, describing them as "user-centred open innovation ecosystems” that integrate research and innovation through co-creation in real-world environments.[1] Emerging at the intersection of ambient intelligence research and user experience methodologies in the late 1990s, the concept was pioneered at the Massachusetts Institute of Technology (MIT) as a way to study human interaction with new technologies in natural settings. Over time, living labs have evolved beyond their origins as controlled research environments, becoming dynamic platforms for participatory design, collaborative experimentation, and iterative innovation across various domains, including urban development, healthcare, sustainability, and digital technology. Characterized by principles such as real-world experimentation, active user involvement, and multi-stakeholder collaboration, living labs enable the continuous adaptation and validation of solutions in everyday contexts. Today, they are implemented globally, supported by networks like the European Network of Living Labs (ENoLL), and increasingly recognized as vital tools for addressing local and global transformation agendas. == Background == The term "living lab" has emerged in parallel from the ambient intelligence (AmI) research communities context and from the discussion on experience and application research (EAR). The emergence of the term is based on the concept of user experience and ambient intelligence. The term dates back to the late 1990s when Professor William J. Mitchell, Kent Larson, and Alex (Sandy) Pentland at the Massachusetts Institute of Technology were credited with first exploring the concept of a living laboratory. It was first associated with MIT's Media Lab as a concept for studying real-life contexts, where they described a living lab as a controlled environment designed to test new information and communication technology (ICT) innovations in a simulated home setting. This was also when some of the key characteristics often assigned to living labs today began to take shape. They argued that a living lab represents a user-centric research methodology for sensing, prototyping, validating and refining complex solutions in multiple and evolving real-life contexts. Research on living labs has expanded since the 1990s, especially in the 2010s, with growing interest in co-creation and participatory design. Particularly in Europe, the living lab evolved into a model that focused on studying user interactions with technology in real-world environments. This shift was influenced by earlier experiences in participatory design and social experiments with ICT. As interest grew, the term began to encompass a broader array of initiatives and projects, leading to variations in its interpretation and implementation. Today, living labs are used in various fields, such as technology, healthcare, and urban sustainability, showing a transition from a narrow focus on their role as controlled environments to a more wide-ranging understanding of collaborative innovation addressing real societal challenges, while also being referred to with various descriptions and definitions available from different sources. == Description == The ENoLL definition that refers to living labs as "user-centred open innovation ecosystems” that integrate research and innovation through co-creation in real-world environments is the most widely accepted description of living labs in academic literature. In simple terms, living labs can be described as an organization or experimental space, that can be both virtually or physically located, bringing different stakeholders from research, business, government, and citizens together to design and test solutions to be implemented in a real world environment. A common definition for the living lab term still does not exist to this day, which is due to the fact that living labs are interpreted and implemented across different contexts and can cover a wide range of activities and organizations, leading to different understandings of how living labs should function. Living labs also often operate in various territorial contexts (e.g. city, agglomeration, region, campus), and can vary in their methodological approach integrating concurrent research and innovation processes within a public-private-people partnership. Despite these variations, common characteristics include user-centricity, real-world experimentation, multi-stakeholder collaboration, and iterative innovation processes. The systematic user co-creation approach refers to integrating research and innovation processes through the co-creation, exploration, experimentation and evaluation of innovative ideas, scenarios, concepts and related technological artefacts in real life use cases. Such use cases involve user communities, not only as observed subjects but also as a source of creation. This approach allows all involved stakeholders to concurrently consider both the global performance of a product or service and its potential adoption by users. This consideration may be made at the earlier stage of research and development and through all elements of the product life-cycle, from design up to recycling. User-centred research methods, such as action research, community informatics, contextual design, user-centered design, participatory design, empathic design, emotional design, and other usability methods, already exist but fail to sufficiently empower users for co-creating into open development environments. More recently, the Web 2.0 has demonstrated the positive impact of involving user communities in new product development (NPD) such as mass collaboration projects (e.g. crowdsourcing, Wisdom of Crowds) in collectively creating new contents and applications. Real-world experimentation emphasizes conducting activities in real-life settings to ensure that the results of the projects and solutions are applicable to actual market conditions. Multi-stakeholder collaboration refers to an approach that involved various stakeholders, such as users, businesses, researchers, and government entities, working together towards a common goal. This is an important characteristics of living lab because collaboration of these diverse groups allows for exchange of ideas and perspectives, which are thought to enhance innovation processes. Iterative innovation processes involve a cyclical method of developing products or services, where stages such as research, development, testing, and implementation are revisited multiple times based on feedback and evaluation. This process allows for continuous improvement of the innovation, product, or service being developed. In particular, the ongoing involvement of the user creates feedback mechanisms that are ultimately key to successful development and implementation of products and services. A living lab is not similar to a testbed as its philosophy is to turn users, from being traditionally considered as observed subjects for testing modules against requirements, into value creation in contributing to the co-creation and exploration of emerging ideas, breakthrough scenarios, innovative concepts and related artefacts. Hence, a living lab rather constitutes an experiential environment, which could be compared to the concept of experiential learning, where users are immersed in a creative social space for designing and experiencing their own future. Living labs could also be used by policy makers and users/citizens for designing, exploring, experiencing and refining new policies and regulations in real-life scenarios for evaluating their potential impacts before their implementations. == European Network of Living Labs (ENoLL) == The European Network of Living Labs (ENoLL) is an international, non-profit, independent association of certified living labs, which popularized the living lab concept in the aim to increase user involvement in innovation. Formed in November 2006 under the guidance of the Finnish European Presidency, ENoLL is composed of a variety of stakeholders, including municipalities and research institutes, businesses, and users. Its primary role is to support the collaboration among living labs across Europe and includes many living labs focused on user-driven innovation across sectors. ENoLL focuses on facilitating knowledge exchange, joint actions and project partnerships among its historically labelled +/- 500 members, influencing EU policies, promoting living labs and enabling their implementation worldwide. ENoLL serves as a platform for linking living labs around the globe, which enables knowledge sharing and collaborative learning among diverse cultural environments. Membership to the platform is open to organizations worldwide, and ENoLL has expanded beyond Europe to include global members. ENoLL follows an application and accreditation pro

  Read more →
• 

  Relational data stream management system

  A relational data stream management system (RDSMS) is a distributed, in-memory data stream management system (DSMS) that is designed to use standards-compliant SQL queries to process unstructured and structured data streams in real-time. Unlike SQL queries executed in a traditional RDBMS, which return a result and exit, SQL queries executed in a RDSMS do not exit, generating results continuously as new data become available. Continuous SQL queries in a RDSMS use the SQL Window function to analyze, join and aggregate data streams over fixed or sliding windows. Windows can be specified as time-based or row-based. == RDSMS SQL Query Examples == Continuous SQL queries in a RDSMS conform to the ANSI SQL standards. The most common RDSMS SQL query is performed with the declarative SELECT statement. A continuous SQL SELECT operates on data across one or more data streams, with optional keywords and clauses that include FROM with an optional JOIN subclause to specify the rules for joining multiple data streams, the WHERE clause and comparison predicate to restrict the records returned by the query, GROUP BY to project streams with common values into a smaller set, HAVING to filter records resulting from a GROUP BY, and ORDER BY to sort the results. The following is an example of a continuous data stream aggregation using a SELECT query that aggregates a sensor stream from a weather monitoring station. The SELECTquery aggregates the minimum, maximum and average temperature values over a one-second time period, returning a continuous stream of aggregated results at one second intervals. RDSMS SQL queries also operate on data streams over time or row-based windows. The following example shows a second continuous SQL query using the WINDOW clause with a one-second duration. The WINDOW clause changes the behavior of the query, to output a result for each new record as it arrives. Hence the output is a stream of incrementally updated results with zero result latency.

  Read more →
• 

  AppyStore

  AppyStore is a comprehensive learning videos and games app for kids up to the age of 8 years. The platform developed by Mauj Mobile, a mobile value-added services (VAS) provider curates content to help in child development by leveraging technology. Mauj is funded by Sequoia Capital, Westbridge Capital and Intel Capital. == Background == AppyStore was launched in 2014 as a platform providing content for kids between the ages of 1.5 and 6 years. AppyStore subsequently extended its services for kids up to 8 years of age. The company operates on a subscription-based model and claims to have 5,000 learning games and videos segregated in 18 learning areas developed to help children gain optimal skills and qualities. According to an article published in Business Standard, the application is claimed to be one of the top 5 apps that help to enhance the logical and imaginative capabilities of children. AppyStore was awarded the Best app for kids by Google Play in December 2017. == Service == The company provides content via a website and an Android app. The website and android app provide learning games, rhymes, phonics, reading, stories, science, numbers, maths, logic videos comprising puzzles, worksheets, videos and fun activities and the premium subscription also includes physical worksheets which are home delivered. This content is educational and has been handpicked by teachers and experts with an understanding of the major areas of child development milestones for children up to 8 years of age. The mobile application also allows parents to track the progress of their child on the basis of the number of videos viewed.

  Read more →
• 

  Information logistics

  Information Logistics (IL) deals with the flow of information between human or machine actors within or between any number of organizations that in turn form a value creating network (see, e.g.). IL is closely related to information management, information operations and information technology. == Definition == The term Information Logistics (IL) may be used in either of two ways: Firstly, it can be defined as "managing and controlling information handling processes optimally with respect to time (flow time and capacity), storage, distribution and presentation in such a way that it contributes to company results in concurrence with the costs of capturing (creation, searching, maintenance etc)." (Petri,2017) Thus IL utilizes logistic principles to optimize information handling. Secondly, IL can be seen as a concept using information technology to optimize logistics. A term which is closely related to the first meaning of Information Logistics is Data Logistics, a concept used in Computer Networking. "The study of solutions to problems in Computer Systems that flexibly span resources and services relating to Data Movement, Data Storage and Data Processing." [ref?] Systems that support general Data Logistics solutions thus must span the traditionally separate fields of Networking, File/Database Systems and Process Management. Data Logistics is a more general form of the term Logistical Networking, used as the name of a particular network storage architecture and software stack. == Goal == The goal of Information Logistics is to deliver the right product, consisting of the right information element, in the right format, at the right place at the right time for the right people at the right price and all of this is customer demand driven. If this goal is to be achieved, knowledge workers are best equipped with information for the task at hand for improved interaction with its customers and machines are enabled to respond automatically to meaningful information. Methods for achieving the goal are: the analysis of information demand intelligent information storage the optimization of the flow of information maintaining both security and organizational flexibility integrated information and billing solutions The expression was formed by the Indian mathematician and librarian S. R. Ranganathan . The supply of a product is part of the discipline Logistics. The purpose of this discipline is described as follows: Logistics is the teachings of the plans and the effective and efficient run of supply. The contemporary logistics focuses on the organization, planning, control and implementation of the flow of goods, money, information and people. Information Logistics focusses on information. Information (from Latin informare: "shape, shapes, instruct") means in a general sense everything that adds knowledge and thus reduce ignorance or lack of precision. In a stricter sense, raw data only becomes information to those who can interpret it. Interpreting relevant, related information produces insight that either leads to existing, or eventually builds new, knowledge. == Information element == An information element (IE) is an information component that is located in the organizational value chain. The combination of certain IEs leads to an information product (IP), which is any final product in the form of information that a person needs to have. When a higher number of different IEs are required, it often results in more planning problems in capacity and inherently leads to a non-delivery of the IP. To illustrate the concept of an IP, an example is shown of a bottleneck analysis in HR (by J. Willems 2008). Here, the illustration shows how the information elements (e.g. qualifications) build up the information product (e.g. HR file). == Data logistics == Data logistics is a concept that developed independently of information logistics in the 1990s, in response to the explosion of Internet content and traffic due to the invention of the World Wide Web (WWW). Some motivations for the emergence of interest in Data Logistics included: The incorporation of network hyperlinks into content encoded in HTML encouraged users to freely dereference those links without regard to, or in many cases without even having any knowledge of, the identity (much less the geographical or network topological location of) the target Web server. The growth in the volume of Web hits, combined with the steady increase in the size of Web-delivered objects such as images, audio and video clips resulted in the localized overloading of the bandwidth and processing resources of the local and/or wide area network and/or the Web server infrastructure. The resulting Internet bottleneck can cause Web clients to experience poor performance or complete denial of access to servers that host high volume sites (the so-called Slashdot effect). The growth in all Internet traffic, especially across international telecommunication links, resulted in stress to institutional infrastructure and high costs on networks that billed Internet traffic on a per-use basis. Much of this traffic was redundant, the results of repeated requests by many independent users to access the same stored files and content. Large files and content retrieved from distant Web servers was often delayed due to high delays experienced over long and complex Internet paths. These factors led to interest in the use of large scale storage (and to a lesser extent, processing) resources to cache the response to network requests, first at the Internet endpoint using a Web browser cache and later at intermediate network locations using shared network caches. This line of development also gave rise to Web server replication and other techniques for offloading and distributing the work of delivering large volume Web services to widely dispersed client communities, ultimately resulting in the creation of modern Content delivery networks. At the same time, research efforts in server replication and content delivery gave rise to a number of related projects and strategies, including Logistical Networking (LN). The name LN was intended as an analogy to physical supply chain logistics, in which goods are not only carried from source to destination on networks of roads, but are also stored at warehouses located throughout the transportation infrastructure. This led to a nomenclature in which LN network storage resources are termed "storage depots". The principles that underpin LN have been abstracted into the more general study of scheduling and optimization across the traditional infrastructure silos of Storage, Networking and Processing which was named Data Logistics. === Illustrative examples of data logistics === Data Caching and Replication are classic examples of Data Logistics solutions to problems in Computer Systems and Networking with high data access latencies or data transfer resource limitations. It works mainly across the areas of data transfer and data storage. Dynamic Compression in data transfer is another example which uses computational resources to minimize the bandwidth requirements of data transfer.

  Read more →
• 

  Timeline of algorithms

  The following timeline of algorithms outlines the development of algorithms (mainly "mathematical recipes") since their inception. == Antiquity == Before – writing about "recipes" (on cooking, rituals, agriculture and other themes) c. 1700–2000 BC – Egyptians develop earliest known algorithms for multiplying two numbers c. 1600 BC – Babylonians develop earliest known algorithms for factorization and finding square roots c. 300 BC – Euclid's algorithm c. 200 BC – the Sieve of Eratosthenes 263 AD – Gaussian elimination described by Liu Hui == Medieval Period == 628 – Chakravala method described by Brahmagupta c. 820 – Al-Khawarizmi described algorithms for solving linear equations and quadratic equations in his Algebra; the word algorithm comes from his name 825 – Al-Khawarizmi described the algorism, algorithms for using the Hindu–Arabic numeral system, in his treatise On the Calculation with Hindu Numerals, which was translated into Latin as Algoritmi de numero Indorum, where "Algoritmi", the translator's rendition of the author's name gave rise to the word algorithm (Latin algorithmus) with a meaning "calculation method" c. 850 – cryptanalysis and frequency analysis algorithms developed by Al-Kindi (Alkindus) in A Manuscript on Deciphering Cryptographic Messages, which contains algorithms on breaking encryptions and ciphers c. 1025 – Ibn al-Haytham (Alhazen), was the first mathematician to derive the formula for the sum of the fourth powers, and in turn, he develops an algorithm for determining the general formula for the sum of any integral powers c. 1400 – Ahmad al-Qalqashandi gives a list of ciphers in his Subh al-a'sha which include both substitution and transposition, and for the first time, a cipher with multiple substitutions for each plaintext letter; he also gives an exposition on and worked example of cryptanalysis, including the use of tables of letter frequencies and sets of letters which can not occur together in one word == Before 1940 == 1540 – Lodovico Ferrari discovered a method to find the roots of a quartic polynomial 1545 – Gerolamo Cardano published Cardano's method for finding the roots of a cubic polynomial 1614 – John Napier develops method for performing calculations using logarithms 1671 – Newton–Raphson method developed by Isaac Newton 1690 – Newton–Raphson method independently developed by Joseph Raphson 1706 – John Machin develops a quickly converging inverse-tangent series for π and computes π to 100 decimal places 1768 – Leonhard Euler publishes his method for numerical integration of ordinary differential equations in problem 85 of Institutiones calculi integralis 1789 – Jurij Vega improves Machin's formula and computes π to 140 decimal places, 1805 – FFT-like algorithm known by Carl Friedrich Gauss 1842 – Ada Lovelace writes the first algorithm for a computing engine 1903 – A fast Fourier transform algorithm presented by Carle David Tolmé Runge 1918 - Soundex 1926 – Borůvka's algorithm 1926 – Primary decomposition algorithm presented by Grete Hermann 1927 – Hartree–Fock method developed for simulating a quantum many-body system in a stationary state. 1934 – Delaunay triangulation developed by Boris Delaunay 1936 – Turing machine, an abstract machine developed by Alan Turing, with others developed the modern notion of algorithm. == 1940s == 1942 – A fast Fourier transform algorithm developed by G.C. Danielson and Cornelius Lanczos 1945 – Merge sort developed by John von Neumann 1947 – Simplex algorithm developed by George Dantzig == 1950s == 1950 – Hamming codes developed by Richard Hamming 1952 – Huffman coding developed by David A. Huffman 1953 – Simulated annealing introduced by Nicholas Metropolis 1954 – Radix sort computer algorithm developed by Harold H. Seward 1964 – Box–Muller transform for fast generation of normally distributed numbers published by George Edward Pelham Box and Mervin Edgar Muller. Independently pre-discovered by Raymond E. A. C. Paley and Norbert Wiener in 1934. 1956 – Kruskal's algorithm developed by Joseph Kruskal 1956 – Ford–Fulkerson algorithm developed and published by R. Ford Jr. and D. R. Fulkerson 1957 – Prim's algorithm developed by Robert Prim 1957 – Bellman–Ford algorithm developed by Richard E. Bellman and L. R. Ford, Jr. 1959 – Dijkstra's algorithm developed by Edsger Dijkstra 1959 – Shell sort developed by Donald L. Shell 1959 – De Casteljau's algorithm developed by Paul de Casteljau 1959 – QR factorization algorithm developed independently by John G.F. Francis and Vera Kublanovskaya 1959 – Rabin–Scott powerset construction for converting NFA into DFA published by Michael O. Rabin and Dana Scott == 1960s == 1960 – Karatsuba multiplication 1961 – CRC (Cyclic redundancy check) invented by W. Wesley Peterson 1962 – AVL trees 1962 – Quicksort developed by C. A. R. Hoare 1962 – Bresenham's line algorithm developed by Jack E. Bresenham 1962 – Gale–Shapley 'stable-marriage' algorithm developed by David Gale and Lloyd Shapley 1964 – Heapsort developed by J. W. J. Williams 1964 – multigrid methods first proposed by R. P. Fedorenko 1965 – Cooley–Tukey algorithm rediscovered by James Cooley and John Tukey 1965 – Levenshtein distance developed by Vladimir Levenshtein 1965 – Cocke–Younger–Kasami (CYK) algorithm independently developed by Tadao Kasami 1965 – Buchberger's algorithm for computing Gröbner bases developed by Bruno Buchberger 1965 – LR parsers invented by Donald Knuth 1966 – Dantzig algorithm for shortest path in a graph with negative edges 1967 – Viterbi algorithm proposed by Andrew Viterbi 1967 – Cocke–Younger–Kasami (CYK) algorithm independently developed by Daniel H. Younger 1968 – A graph search algorithm described by Peter Hart, Nils Nilsson, and Bertram Raphael 1968 – Risch algorithm for indefinite integration developed by Robert Henry Risch 1969 – Strassen algorithm for matrix multiplication developed by Volker Strassen == 1970s == 1970 – Dinic's algorithm for computing maximum flow in a flow network by Yefim (Chaim) A. Dinitz 1970 – Knuth–Bendix completion algorithm developed by Donald Knuth and Peter B. Bendix 1970 – BFGS method of the quasi-Newton class 1970 – Needleman–Wunsch algorithm published by Saul B. Needleman and Christian D. Wunsch 1972 – Edmonds–Karp algorithm published by Jack Edmonds and Richard Karp, essentially identical to Dinic's algorithm from 1970 1972 – Graham scan developed by Ronald Graham 1972 – Red–black trees and B-trees discovered 1973 – RSA encryption algorithm discovered by Clifford Cocks 1973 – Jarvis march algorithm developed by R. A. Jarvis 1973 – Hopcroft–Karp algorithm developed by John Hopcroft and Richard Karp 1974 – Pollard's p − 1 algorithm developed by John Pollard 1974 – Quadtree developed by Raphael Finkel and J.L. Bentley 1975 – Genetic algorithms popularized by John Holland 1975 – Pollard's rho algorithm developed by John Pollard 1975 – Aho–Corasick string matching algorithm developed by Alfred V. Aho and Margaret J. Corasick 1975 – Cylindrical algebraic decomposition developed by George E. Collins 1976 – Salamin–Brent algorithm independently discovered by Eugene Salamin and Richard Brent 1976 – Knuth–Morris–Pratt algorithm developed by Donald Knuth and Vaughan Pratt and independently by J. H. Morris 1977 – Boyer–Moore string-search algorithm for searching the occurrence of a string into another string. 1977 – RSA encryption algorithm rediscovered by Ron Rivest, Adi Shamir, and Len Adleman 1977 – LZ77 algorithm developed by Abraham Lempel and Jacob Ziv 1977 – multigrid methods developed independently by Achi Brandt and Wolfgang Hackbusch 1978 – LZ78 algorithm developed from LZ77 by Abraham Lempel and Jacob Ziv 1978 – Bruun's algorithm proposed for powers of two by Georg Bruun 1979 – Khachiyan's ellipsoid method developed by Leonid Khachiyan 1979 – ID3 decision tree algorithm developed by Ross Quinlan == 1980s == 1980 – Brent's Algorithm for cycle detection Richard P. Brendt 1981 – Quadratic sieve developed by Carl Pomerance 1981 – Smith–Waterman algorithm developed by Temple F. Smith and Michael S. Waterman 1983 – Simulated annealing developed by S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi 1983 – Classification and regression tree (CART) algorithm developed by Leo Breiman, et al. 1984 – LZW algorithm developed from LZ78 by Terry Welch 1984 – Karmarkar's interior-point algorithm developed by Narendra Karmarkar 1984 – ACORN PRNG discovered by Roy Wikramaratna and used privately 1985 – Simulated annealing independently developed by V. Cerny 1985 – Car–Parrinello molecular dynamics developed by Roberto Car and Michele Parrinello 1985 – Splay trees discovered by Sleator and Tarjan 1986 – Blum Blum Shub proposed by L. Blum, M. Blum, and M. Shub 1986 – Push relabel maximum flow algorithm by Andrew Goldberg and Robert Tarjan 1986 – Barnes–Hut tree method developed by Josh Barnes and Piet Hut for fast approximate simulation of n-body problems 1987 – Fast multipole method developed by Leslie Greengard and Vladimir

  Read more →
• 

  Regulation of algorithms

  Regulation of algorithms, or algorithmic regulation, is the creation of laws, rules and public sector policies for promotion and regulation of algorithms, particularly in artificial intelligence and machine learning. For the subset of AI algorithms, the term regulation of artificial intelligence is used. The regulatory and policy landscape for artificial intelligence (AI) is an emerging issue in jurisdictions globally, including in the European Union. Regulation of AI is considered necessary to both encourage AI and manage associated risks, but challenging. Another emerging topic is the regulation of blockchain algorithms (Use of the smart contracts must be regulated) and is mentioned along with regulation of AI algorithms. Many countries have enacted regulations of high frequency trades, which is shifting due to technological progress into the realm of AI algorithms. The motivation for regulation of algorithms is the apprehension of losing control over the algorithms, whose impact on human life increases. Multiple countries have already introduced regulations in case of automated credit score calculation—right to explanation is mandatory for those algorithms. For example, The IEEE has begun developing a new standard to explicitly address ethical issues and the values of potential future users. Bias, transparency, and ethics concerns have emerged with respect to the use of algorithms in diverse domains ranging from criminal justice to healthcare—many fear that artificial intelligence could replicate existing social inequalities along race, class, gender, and sexuality lines. == Regulation of artificial intelligence == === Public discussion === In 2016, Joy Buolamwini founded Algorithmic Justice League after a personal experience with biased facial detection software in order to raise awareness of the social implications of artificial intelligence through art and research. In 2017 Elon Musk advocated regulation of algorithms in the context of the existential risk from artificial general intelligence. According to NPR, the Tesla CEO was "clearly not thrilled" to be advocating for government scrutiny that could impact his own industry, but believed the risks of going completely without oversight are too high: "Normally the way regulations are set up is when a bunch of bad things happen, there's a public outcry, and after many years a regulatory agency is set up to regulate that industry. It takes forever. That, in the past, has been bad but not something which represented a fundamental risk to the existence of civilisation." In response, some politicians expressed skepticism about the wisdom of regulating a technology that is still in development. Responding both to Musk and to February 2017 proposals by European Union lawmakers to regulate AI and robotics, Intel CEO Brian Krzanich has argued that artificial intelligence is in its infancy and that it is too early to regulate the technology. Instead of trying to regulate the technology itself, some scholars suggest to rather develop common norms including requirements for the testing and transparency of algorithms, possibly in combination with some form of warranty. One suggestion has been for the development of a global governance board to regulate AI development. In 2020, the European Union published its draft strategy paper for promoting and regulating AI. Algorithmic tacit collusion is a legally dubious antitrust practise committed by means of algorithms, which the courts are not able to prosecute. This danger concerns scientists and regulators in EU, US and beyond. European Commissioner Margrethe Vestager mentioned an early example of algorithmic tacit collusion in her speech on "Algorithms and Collusion" on March 16, 2017, described as follows: "A few years ago, two companies were selling a textbook called The Making of a Fly. One of those sellers used an algorithm which essentially matched its rival’s price. That rival had an algorithm which always set a price 27% higher than the first. The result was that prices kept spiralling upwards, until finally someone noticed what was going on, and adjusted the price manually. By that time, the book was selling – or rather, not selling – for 23 million dollars a copy." In 2018, the Netherlands employed an algorithmic system SyRI (Systeem Risico Indicatie) to detect citizens perceived being high risk for committing welfare fraud, which quietly flagged thousands of people to investigators. This caused a public protest. The district court of Hague shut down SyRI referencing Article 8 of the European Convention on Human Rights (ECHR). In 2020, algorithms assigning exam grades to students in the UK sparked open protest under the banner "Fuck the algorithm." This protest was successful and the grades were taken back. In 2024, the Munich Convention on AI, Data and Human Rights was introduced as part of growing international efforts to regulate artificial intelligence through a human rights lens. Developed through a collaborative drafting process involving scholars from the Technical University of Munich, Stellenbosch University, Ulster University, and KNUST, the initiative calls for an international conversation on a binding treaty to safeguard human rights and the principles enshrined in the UN Charter in the age of AI. === Implementation === AI law and regulations can be divided into three main topics, namely governance of autonomous intelligence systems, responsibility and accountability for the systems, and privacy and safety issues. The development of public sector strategies for management and regulation of AI has been increasingly deemed necessary at the local, national, and international levels and in fields from public service management to law enforcement, the financial sector, robotics, the military, and international law. There are many concerns that there is not enough visibility and monitoring of AI in these sectors. In the United States financial sector, for example, there have been calls for the Consumer Financial Protection Bureau to more closely examine source code and algorithms when conducting audits of financial institutions' non-public data. In the United States, on January 7, 2019, following an Executive Order on 'Maintaining American Leadership in Artificial Intelligence', the White House's Office of Science and Technology Policy released a draft Guidance for Regulation of Artificial Intelligence Applications, which includes ten principles for United States agencies when deciding whether and how to regulate AI. In response, the National Institute of Standards and Technology has released a position paper, the National Security Commission on Artificial Intelligence has published an interim report, and the Defense Innovation Board has issued recommendations on the ethical use of AI. In April 2016, for the first time in more than two decades, the European Parliament adopted a set of comprehensive regulations for the collection, storage, and use of personal information, the General Data Protection Regulation (GDPR)1 (European Union, Parliament and Council 2016). The GDPR's policy on the right of citizens to receive an explanation for algorithmic decisions highlights the pressing importance of human interpretability in algorithm design. In 2016, China published a position paper questioning the adequacy of existing international law to address the eventuality of fully autonomous weapons, becoming the first permanent member of the U.N. Security Council to broach the issue, and leading to proposals for global regulation. In the United States, steering on regulating security-related AI is provided by the National Security Commission on Artificial Intelligence. In 2017, the U.K. Vehicle Technology and Aviation Bill imposes liability on the owner of an uninsured automated vehicle when driving itself and makes provisions for cases where the owner has made "unauthorized alterations" to the vehicle or failed to update its software. Further ethical issues arise when, e.g., a self-driving car swerves to avoid a pedestrian and causes a fatal accident. In 2021, the European Commission proposed the Artificial Intelligence Act. == Algorithm certification == There is a concept of algorithm certification emerging as a method of regulating algorithms. Algorithm certification involves auditing whether the algorithm used during the life cycle 1) conforms to the protocoled requirements (e.g., for correctness, completeness, consistency, and accuracy); 2) satisfies the standards, practices, and conventions; and 3) solves the right problem (e.g., correctly model physical laws), and satisfies the intended use and user needs in the operational environment. == Regulation of blockchain algorithms == Blockchain systems provide transparent and fixed records of transactions and hereby contradict the goal of the European GDPR, which is to give individuals full control of their private data. By implementing the Decree on Development of Digital Economy, Bel

  Read more →
• 

  SimSimi

  SimSimi is an artificial intelligence conversation program created in 2002 by ISMaker. It grows its artificial intelligence day by day assisted by a feature that allows users to teach it to respond correctly. SimSimi, pronounced as "shim-shimi", is from a Korean word simsim (심심) which means "bored". It has an application designed for Android, Windows Phone and iOS. The application was banned in Thailand in 2012 after users taught it to make responses containing profanity, and to criticise leading politicians. In April 2018, SimSimi was suspended in Brazil due to accusations of sending inappropriate messages, such as sexual language, bullying and even death threats, being labeled as "dangerous" mainly due to its popularity among children, and according to its developer, the suspension of the app in the country "was inevitable because the SimSimi app, at least in the last few days, had a significant negative social impact in Brazil.”

  Read more →
• 

  Artificial intelligence industry in Canada

  The artificial intelligence industry in Canada is a rapidly expanding sector. Although Canada held a pioneering role in the early development of artificial intelligence, transforming research excellence into broad commercial adoption has proven challenging. Despite globally recognized scientific achievements and a deep pool of skilled experts, by June 2024, Canada recorded the lowest rate of AI integration among OECD countries, with only 12% of firms implementing AI in their products or services. However, AI adoption has shown significant momentum—doubling from mid-2024 to mid-2025, rising from 6.1% to 12.2%. As of September 2025, Statistics Canada indicated that while about one-third of Canadian businesses had no plans to adopt artificial intelligence in the next year, 14.5% reported intentions to begin using AI for producing goods or delivering services. The primary reasons for not moving forward with AI were lack of relevance, insufficient knowledge, and privacy concerns. According to Public Works Canada (PwC), the pace of AI adoption in Canada is roughly three-quarters of the United States rate, highlighting a notable gap between the two countries in business integration of this technology. British-Canadian computer scientist Geoffrey Hinton stated in 2025 that Canadian companies are adopting artificial intelligence at a slower pace, which may result in the loss of the country's early advantages in the field. At the "All In AI" conference held in Montreal in September 2025, the Minister of Artificial Intelligence and Digital Innovation Evan Solomon, described "Building digital sovereignty" as the most pressing democratic issue of the time. He introduced a 26-person task force focused on updating Canada's AI strategy. In their 2024 report " "Learning Together for Responsible Artificial Intelligence" report, the Innovation, Science, and Economic Development Canada stressed that public awareness, trust, and AI literacy are essential for the responsible adoption and governance of AI in Canada. Montreal workshops in 2021 expanded the OECD's 2019 definition of AI as "the set of computer techniques that enable a machine (e.g., a computer or telephone) to perform tasks that typically require intelligence, such as reasoning or learning. It is also referred to as the automation of intelligent tasks. Scientific developments in AI, such as deep-learning techniques, have made it possible to design access to huge amounts of data and ever-increasing computing power. These new techniques have been rapidly deployed on a large scale in all areas of social life, in transport, education, culture and health." == Federal investments and policy == The 2025 federal budget allocates over $1 billion over the next five years to bolster Canada's artificial intelligence and quantum computing ecosystem. == Industry landscape or research hubs == AlexNet, an influential deep convolutional neural network developed at the University of Toronto by Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton, marked a pivotal turning point in modern artificial intelligence. In 2012, it achieved a dramatic reduction in error rates for the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), showcasing the practical power of deep learning and GPU acceleration. The success of AlexNet helped cement Canada’s reputation for AI leadership and inspired rapid adoption of deep learning across the technology sector, with ongoing impact in both academic and commercial domains. In healthcare, AlexNet has been adapted for medical imaging to assist with analyzing radiographs, mammograms, and other scans, including identifying abnormalities and supporting clinical diagnosis. In 2015, the Ottawa-based start-up Advanced Symbolics Inc. (ASI) began developing Polly, an artificial intelligence system designed to analyze and anticipate how target audiences behave—enabling more effective communication strategies and advertising campaigns. Polly was named after its first assignment analyzing the politics of Brexit. The AI gained widespread attention in 2016 for accurately forecasting both the Brexit referendum and the 2016 U.S. presidential election won by Donald Trump. The company states that Polly is used by organizations in diverse sectors—including healthcare, politics, entertainment, and mental health research—to support decision-making based on predictive analytics. Chartwatch, an AI tool developed in Canada, has been shown to reduce unexpected hospital deaths by 26% according to a 2024 study. The system analyzes patient data to detect subtle signs of deterioration, supporting healthcare teams in providing timely interventions. === Notable figures in AI in Canada === Geoffrey Hinton's decades-long work eventually formed the foundation of artificial intelligence, which earned him the Nobel Prize for physics in 2024. Yoshua Bengio, who won the Turing Award in 2018 for his pioneering work in deep learning, founded what would become Mila in 1993. Mila, is currently a collaboration between four Montreal-based academic partners.—the Pan-Canadian Artificial Intelligence Strategy includes Alberta's Amii, Toronto's Vector Institute, and Mila. Fakhreddine Karray's work on operational AI has had tangible impact across several Canadian-relevant sectors, notably intelligent transportation systems, virtual healthcare, and driver safety. === AI in the oil and gas industry === According to a 2020 Ernst & Young report the oil and gas industry in Canada is using AI in automating routine, repetitive, and dangerous tasks with technologies like robotic process automation and machine learning; optimizing production and processing; enhancing transportation logistics; improving equipment operation and monitoring; and enabling preventative maintenance. AI is also deployed for data analysis to improve prediction and decision-making, and is expected to automate up to 50% of job competencies in upstream oil and gas by 2040. Oilsands giant Suncor Energy operates a large fleet of autonomous trucks and has started using AI in its dispatch system at the Mildred Lake mine. As of 2024, AI manages routine tasks such as allocating trucks to dump stations and sending them to refuelling locations. === Indigenous and Inuit Innovation in AI === Indigenous organizations have been working on the creation of new technologies for language revitalization in partnership with National Research Council of Canada since the mid-2010s. In 2025, Inuit researchers and technology partners launched an AI-powered initiative to support the revitalization and preservation of Inuktitut, demonstrating how artificial intelligence can be adapted for Indigenous language and cultural priorities. A 2025 CBC article notes that, while AI can help revitalize Inuktitut, Inuit leaders emphasize concerns about data sovereignty, information ownership, and the need for Indigenous leadership to ensure transparency, privacy, and accountability in AI development. == Regulation == Canada's Artificial Intelligence and Data Act (AIDA) was proposed in November 2022, as part of the Digital Charter Implementation Act (Bill C-27). As well voluntary codes, such as the September 2023 Code of Conduct for Generative AI, and landmark investments in advanced computing infrastructure and the Canadian Artificial Intelligence Safety Institute (CAISI) reflect Canada's commitment to both safety and global competitiveness. == AI infrastructure == Canada has undertaken efforts to expand its AI computing infrastructure at both provincial and federal levels. The federal government's Canadian Sovereign AI Compute Strategy, allocated up to C$2 billion in Budget 2024, aims to enhance computing capacity to support domestic AI industry growth and AI adoption across the economy, with up to C$700 million designated to mobilize private sector investment in new or expanded data centres. Alberta has introduced an AI Data Centres Strategy to position itself as a leading North American destination for data centre investment, targeting C$100 billion worth of AI data centres under development by 2030. One major project under Alberta's strategy is the Wonder Valley AI Data Centre Park near Grande Prairie, which was exempted from provincial environmental impact assessment in April 2026 but still requires permits demonstrating safe construction and operation. According to Statista, as of April 2026, Canada has 287 data centres.

  Read more →
• 

  Online analytical processing

  In computing, online analytical processing (OLAP) (), is an approach to quickly answer multi-dimensional analytical (MDA) queries. The term OLAP was created as a slight modification of the traditional database term online transaction processing (OLTP). OLAP is part of the broader category of business intelligence, which also encompasses relational databases, report writing and data mining. Typical applications of OLAP include business reporting for sales, marketing, management reporting, business process management (BPM), budgeting and forecasting, financial reporting and similar areas, with new applications emerging, such as agriculture. OLAP tools enable users to analyse multidimensional data interactively from multiple perspectives. OLAP consists of three basic analytical operations: consolidation (roll-up), drill-down, and slicing and dicing. Consolidation involves the aggregation of data that can be accumulated and computed in one or more dimensions. For example, all sales offices are rolled up to the sales department or sales division to anticipate sales trends. By contrast, the drill-down is a technique that allows users to navigate through the details. For instance, users can view the sales by individual products that make up a region's sales. Slicing and dicing is a feature whereby users can take out (slicing) a specific set of data of the OLAP cube and view (dicing) the slices from different viewpoints. These viewpoints are sometimes called dimensions (such as looking at the same sales by salesperson, or by date, or by customer, or by product, or by region, etc.). Databases configured for OLAP use a multidimensional data model, allowing for complex analytical and ad hoc queries with a rapid execution time. They borrow aspects of navigational databases, hierarchical databases and relational databases. OLAP is typically contrasted to OLTP (online transaction processing), which is generally characterized by much less complex queries, in a larger volume, to process transactions rather than for the purpose of business intelligence or reporting. Whereas OLAP systems are mostly optimized for read, OLTP has to process all kinds of queries (read, insert, update and delete). == Overview of OLAP systems == At the core of any OLAP system is an OLAP cube (also called a 'multidimensional cube' or a hypercube). It consists of numeric facts called measures that are categorized by dimensions. The measures are placed at the intersections of the hypercube, which is spanned by the dimensions as a vector space. The usual interface to manipulate an OLAP cube is a matrix interface, like Pivot tables in a spreadsheet program, which performs projection operations along the dimensions, such as aggregation or averaging. The cube metadata is typically created from a star schema or snowflake schema or fact constellation of tables in a relational database. Measures are derived from the records in the fact table and dimensions are derived from the dimension tables. Each measure can be thought of as having a set of labels, or meta-data associated with it. A dimension is what describes these labels; it provides information about the measure. A simple example would be a cube that contains a store's sales as a measure, and Date/Time as a dimension. Each Sale has a Date/Time label that describes more about that sale. For example: Sales Fact Table +-------------+----------+ | sale_amount | time_id | +-------------+----------+ Time Dimension | 930.10| 1234 |----+ +---------+-------------------+ +-------------+----------+ | | time_id | timestamp | | +---------+-------------------+ +---->| 1234 | 20080902 12:35:43 | +---------+-------------------+ === Multidimensional databases === Multidimensional structure is defined as "a variation of the relational model that uses multidimensional structures to organize data and express the relationships between data". The structure is broken into cubes and the cubes are able to store and access data within the confines of each cube. "Each cell within a multidimensional structure contains aggregated data related to elements along each of its dimensions". Even when data is manipulated it remains easy to access and continues to constitute a compact database format. The data still remains interrelated. Multidimensional structure is quite popular for analytical databases that use online analytical processing (OLAP) applications. Analytical databases use these databases because of their ability to deliver answers to complex business queries swiftly. Data can be viewed from different angles, which gives a broader perspective of a problem unlike other models. === Aggregations === It has been claimed that for complex queries OLAP cubes can produce an answer in around 0.1% of the time required for the same query on OLTP relational data. The most important mechanism in OLAP which allows it to achieve such performance is the use of aggregations. Aggregations are built from the fact table by changing the granularity on specific dimensions and aggregating up data along these dimensions, using an aggregate function (or aggregation function). The number of possible aggregations is determined by every possible combination of dimension granularities. The combination of all possible aggregations and the base data contains the answers to every query which can be answered from the data. Because usually there are many aggregations that can be calculated, often only a predetermined number are fully calculated; the remainder are solved on demand. The problem of deciding which aggregations (views) to calculate is known as the view selection problem. View selection can be constrained by the total size of the selected set of aggregations, the time to update them from changes in the base data, or both. The objective of view selection is typically to minimize the average time to answer OLAP queries, although some studies also minimize the update time. View selection is NP-complete. Many approaches to the problem have been explored, including greedy algorithms, randomized search, genetic algorithms and A search algorithm. Some aggregation functions can be computed for the entire OLAP cube by precomputing values for each cell, and then computing the aggregation for a roll-up of cells by aggregating these aggregates, applying a divide and conquer algorithm to the multidimensional problem to compute them efficiently. For example, the overall sum of a roll-up is just the sum of the sub-sums in each cell. Functions that can be decomposed in this way are called decomposable aggregation functions, and include COUNT, MAX, MIN, and SUM, which can be computed for each cell and then directly aggregated; these are known as self-decomposable aggregation functions. In other cases, the aggregate function can be computed by computing auxiliary numbers for cells, aggregating these auxiliary numbers, and finally computing the overall number at the end; examples include AVERAGE (tracking sum and count, dividing at the end) and RANGE (tracking max and min, subtracting at the end). In other cases, the aggregate function cannot be computed without analyzing the entire set at once, though in some cases approximations can be computed; examples include DISTINCT COUNT, MEDIAN, and MODE; for example, the median of a set is not the median of medians of subsets. These latter are difficult to implement efficiently in OLAP, as they require computing the aggregate function on the base data, either computing them online (slow) or precomputing them for possible rollouts (large space). == Types == OLAP systems have been traditionally categorized using the following taxonomy. === Multidimensional OLAP (MOLAP) === MOLAP (multi-dimensional online analytical processing) is the classic form of OLAP and is sometimes referred to as just OLAP. MOLAP stores this data in an optimized multi-dimensional array storage, rather than in a relational database. Some MOLAP tools require the pre-computation and storage of derived data, such as consolidations – the operation known as processing. Such MOLAP tools generally utilize a pre-calculated data set referred to as a data cube. The data cube contains all the possible answers to a given range of questions. As a result, they have a very fast response to queries. On the other hand, updating can take a long time depending on the degree of pre-computation. Pre-computation can also lead to what is known as data explosion. Other MOLAP tools, particularly those that implement the functional database model do not pre-compute derived data but make all calculations on demand other than those that were previously requested and stored in a cache. Advantages of MOLAP Fast query performance due to optimized storage, multidimensional indexing and caching. Smaller on-disk size of data compared to data stored in relational database due to compression techniques. Automated computation of higher-level aggregates of the data. It is very compact for low dimension data se

  Read more →
• 

  Leiden algorithm

  The Leiden algorithm is a community detection algorithm developed by Traag et al at Leiden University. It was developed as a modification of the Louvain method. Like the Louvain method, the Leiden algorithm attempts to optimize modularity in extracting communities from networks; however, it addresses key issues present in the Louvain method, namely poorly connected communities and the resolution limit of modularity. == Improvement over Louvain method == Broadly, the Leiden algorithm uses the same two primary phases as the Louvain algorithm: a local node moving step (though, the method by which nodes are considered in Leiden is more efficient) and a graph aggregation step. However, to address the issues with poorly-connected communities and the merging of smaller communities into larger communities (the resolution limit of modularity), the Leiden algorithm employs an intermediate refinement phase in which communities may be split to guarantee that all communities are well-connected. Consider, for example, the following graph: Three communities are present in this graph (each color represents a community). Additionally, the center "bridge" node (represented with an extra circle) is a member of the community represented by blue nodes. Now consider the result of a node-moving step which merges the communities denoted by red and green nodes into a single community (as the two communities are highly connected): Notably, the center "bridge" node is now a member of the larger red community after node moving occurs (due to the greedy nature of the local node moving algorithm). In the Louvain method, such a merging would be followed immediately by the graph aggregation phase. However, this causes a disconnection between two different sections of the community represented by blue nodes. In the Leiden algorithm, the graph is instead refined: The Leiden algorithm's refinement step ensures that the center "bridge" node is kept in the blue community to ensure that it remains intact and connected, despite the potential improvement in modularity from adding the center "bridge" node to the red community. == Graph components == Before defining the Leiden algorithm, it will be helpful to define some of the components of a graph. === Vertices and edges === A graph is composed of vertices (nodes) and edges. Each edge is connected to two vertices, and each vertex may be connected to zero or more edges. Edges are typically represented by straight lines, while nodes are represented by circles or points. In set notation, let V {\displaystyle V} be the set of vertices, and E {\displaystyle E} be the set of edges: V := { v 1 , v 2 , … , v n } E := { e i j , e i k , … , e k l } {\displaystyle {\begin{aligned}V&:=\{v_{1},v_{2},\dots ,v_{n}\}\\E&:=\{e_{ij},e_{ik},\dots ,e_{kl}\}\end{aligned}}} where e i j {\displaystyle e_{ij}} is the directed edge from vertex v i {\displaystyle v_{i}} to vertex v j {\displaystyle v_{j}} . We can also write this as an ordered pair: e i j := ( v i , v j ) {\displaystyle {\begin{aligned}e_{ij}&:=(v_{i},v_{j})\end{aligned}}} === Community === A community is a unique set of nodes: C i ⊆ V C i ⋂ C j = ∅ ∀ i ≠ j {\displaystyle {\begin{aligned}C_{i}&\subseteq V\\C_{i}&\bigcap C_{j}=\emptyset ~\forall ~i\neq j\end{aligned}}} and the union of all communities must be the total set of vertices: V = ⋃ i = 1 C i {\displaystyle {\begin{aligned}V&=\bigcup _{i=1}C_{i}\end{aligned}}} === Partition === A partition is the set of all communities: P = { C 1 , C 2 , … , C n } {\displaystyle {\begin{aligned}{\mathcal {P}}&=\{C_{1},C_{2},\dots ,C_{n}\}\end{aligned}}} == Partition quality == How communities are partitioned is an integral part on the Leiden algorithm. How partitions are decided can depend on how their quality is measured. Additionally, many of these metrics contain parameters of their own that can change the outcome of their communities. === Modularity === Modularity is a highly used quality metric for assessing how well a set of communities partition a graph. The equation for this metric is defined for an adjacency matrix, A, as: Q = 1 2 m ∑ i j ( A i j − k i k j 2 m ) δ ( c i , c j ) {\displaystyle Q={\frac {1}{2m}}\sum _{ij}(A_{ij}-{\frac {k_{i}k_{j}}{2m}})\delta (c_{i},c_{j})} where: A i j {\displaystyle A_{ij}} represents the edge weight between nodes i {\displaystyle i} and j {\displaystyle j} ; see Adjacency matrix; k i {\displaystyle k_{i}} and k j {\displaystyle k_{j}} are the sum of the weights of the edges attached to nodes i {\displaystyle i} and j {\displaystyle j} , respectively; m {\displaystyle m} is the sum of all of the edge weights in the graph; c i {\displaystyle c_{i}} and c j {\displaystyle c_{j}} are the communities to which the nodes i {\displaystyle i} and j {\displaystyle j} belong; and δ {\displaystyle \delta } is Kronecker delta function: δ ( c i , c j ) = { 1 if c i and c j are the same community 0 otherwise {\displaystyle {\begin{aligned}\delta (c_{i},c_{j})&={\begin{cases}1&{\text{if }}c_{i}{\text{ and }}c_{j}{\text{ are the same community}}\\0&{\text{otherwise}}\end{cases}}\end{aligned}}} === Reichardt Bornholdt Potts Model (RB) === One of the most well used metrics for the Leiden algorithm is the Reichardt Bornholdt Potts Model (RB). This model is used by default in most mainstream Leiden algorithm libraries under the name RBConfigurationVertexPartition. This model introduces a resolution parameter γ {\displaystyle \gamma } and is highly similar to the equation for modularity. This model is defined by the following quality function for an adjacency matrix, A, as: Q = ∑ i j ( A i j − γ k i k j 2 m ) δ ( c i , c j ) {\displaystyle Q=\sum _{ij}(A_{ij}-\gamma {\frac {k_{i}k_{j}}{2m}})\delta (c_{i},c_{j})} where: γ {\displaystyle \gamma } represents a linear resolution parameter === Constant Potts Model (CPM) === Another metric similar to RB is the Constant Potts Model (CPM). This metric also relies on a resolution parameter γ {\displaystyle \gamma } The quality function is defined as: H = − ∑ i j ( A i j w i j − γ ) δ ( c i , c j ) {\displaystyle H=-\sum _{ij}(A_{ij}w_{ij}-\gamma )\delta (c_{i},c_{j})} === Understanding Potts Model resolution parameters/Resolution limit === Typically Potts models such as RB or CPM include a resolution parameter in their calculation. Potts models are introduced as a response to the resolution limit problem that is present in modularity maximization based community detection. The resolution limit problem is that, for some graphs, maximizing modularity may cause substructures of a graph to merge and become a single community and thus smaller structures are lost. These resolution parameters allow modularity adjacent methods to be modified to suit the requirements of the user applying the Leiden algorithm to account for small substructures at a certain granularity. The figure on the right illustrates why resolution can be a helpful parameter when using modularity based quality metrics. In the first graph, modularity only captures the large scale structures of the graph; however, in the second example, a more granular quality metric could potentially detect all substructures in a graph. == Algorithm == The Leiden algorithm starts with a graph of disorganized nodes (a) and sorts it by partitioning them to maximize modularity (the difference in quality between the generated partition and a hypothetical randomized partition of communities). The method it uses is similar to the Louvain algorithm, except that after moving each node it also considers that node's neighbors that are not already in the community it was placed in. This process results in our first partition (b), also referred to as P {\displaystyle {\mathcal {P}}} . Then the algorithm refines this partition by first placing each node into its own individual community and then moving them from one community to another to maximize modularity. It does this iteratively until each node has been visited and moved, and each community has been refined - this creates partition (c), which is the initial partition of P refined {\displaystyle {\mathcal {P}}_{\text{refined}}} . Then an aggregate network (d) is created by turning each community into a node. P refined {\displaystyle {\mathcal {P}}_{\text{refined}}} is used as the basis for the aggregate network while P {\displaystyle {\mathcal {P}}} is used to create its initial partition. Because we use the original partition P {\displaystyle {\mathcal {P}}} in this step, we must retain it so that it can be used in future iterations. These steps together form the first iteration of the algorithm. In subsequent iterations, the nodes of the aggregate network (which each represent a community) are once again placed into their own individual communities and then sorted according to modularity to form a new P refined {\displaystyle {\mathcal {P}}_{\text{refined}}} , forming (e) in the above graphic. In the case depicted by the graph, the nodes were already sorted optimally, so no change too

  Read more →
• 

  Normal distributions transform

  The normal distributions transform (NDT) is a point cloud registration algorithm introduced by Peter Biber and Wolfgang Straßer in 2003, while working at University of Tübingen. The algorithm registers two point clouds by first associating a piecewise normal distribution to the first point cloud, that gives the probability of sampling a point belonging to the cloud at a given spatial coordinate, and then finding a transform that maps the second point cloud to the first by maximising the likelihood of the second point cloud on such distribution as a function of the transform parameters. Originally introduced for 2D point cloud map matching in simultaneous localization and mapping (SLAM) and relative position tracking, the algorithm was extended to 3D point clouds and has wide applications in computer vision and robotics. NDT is very fast and accurate, making it suitable for application to large scale data, but it is also sensitive to initialisation, requiring a sufficiently accurate initial guess, and for this reason it is typically used in a coarse-to-fine alignment strategy. == Formulation == The NDT function associated to a point cloud is constructed by partitioning the space in regular cells. For each cell, it is possible to define the mean q = 1 n ∑ i x i {\displaystyle \textstyle \mathbf {q} ={\frac {1}{n}}\sum _{i}\mathbf {x_{i}} } and covariance S = 1 n ∑ i ( x i − q ) ( x i − q ) ⊤ {\displaystyle \textstyle \mathbf {S} ={\frac {1}{n}}\sum _{i}\left(\mathbf {x} _{i}-\mathbf {q} \right)\left(\mathbf {x} _{i}-\mathbf {q} \right)^{\top }} of the n {\displaystyle n} points of the cloud x 1 , … , x n {\displaystyle \mathbf {x} _{1},\dots ,\mathbf {x} _{n}} that fall within the cell. The probability density of sampling a point at a given spatial location x {\displaystyle \mathbf {x} } within the cell is then given by the normal distribution e − 1 2 ( x − q ) ⊤ S − 1 ( x − q ) {\displaystyle e^{-{\frac {1}{2}}\left(\mathbf {x} -\mathbf {q} \right)^{\top }\mathbf {S} ^{-1}\left(\mathbf {x} -\mathbf {q} \right)}} . Two point clouds can be mapped by a Euclidean transformation f {\displaystyle f} with rotation matrix R {\displaystyle \mathbf {R} } and translation vector t {\displaystyle \mathbf {t} } f R , t ( x ) = R x + t {\displaystyle f_{\mathbf {R} ,\mathbf {t} }(\mathbf {x} )=\mathbf {R} \mathbf {x} +\mathbf {t} } that maps from the second cloud to the first, parametrised by the rotation angles and translation components. The algorithm registers the two point clouds by optimising the parameters of the transformation that maps the second cloud to the first, with respect to a loss function based on the NDT of the first point cloud, solving the following problem arg ⁡ min R , t { − ∑ i NDT ⁡ ( f R , t ( x i ) ) } {\displaystyle \arg \min _{\mathbf {R} ,\mathbf {t} }\left\{-\sum _{i}\operatorname {NDT} \left(f_{\mathbf {R} ,\mathbf {t} }\left(\mathbf {x_{i}} \right)\right)\right\}} where the loss function represents the negated likelihood, obtained by applying the transformation to all points in the second cloud and summing the value of the NDT at each transformed point f R , t ( x ) {\displaystyle f_{\mathbf {R} ,\mathbf {t} }(\mathbf {x} )} . The loss is piecewise continuous and differentiable, and can be optimised with gradient-based methods (in the original formulation, the authors use Newton's method). In order to reduce the effect of cell discretisation, a technique consists of partitioning the space into multiple overlapping grids, shifted by half cell size along the spatial directions, and computing the likelihood at a given location as the sum of the NDTs induced by each grid.

  Read more →
• 

  FAIR data

  FAIR data is data which meets the 2016 FAIR principles of findability, accessibility, interoperability, and reusability (FAIR). The FAIR principles emphasize machine-actionability (i.e., the capacity of computational systems to find, access, interoperate, and reuse data with none or minimal human intervention) because humans increasingly rely on computational support to deal with data as a result of the increase in the volume, complexity, and rate of production of data. The abbreviation FAIR/O data is sometimes used to indicate that the dataset or database in question complies with the FAIR principles and also carries an explicit data‑capable open license. == FAIR principles published by GO FAIR == Findable The first step in (re)using data is to find them. Metadata and data should be easy to find for both humans and computers. Machine-readable metadata are essential for automatic discovery of datasets and services, so this is an essential component of the FAIRification process. F1. (Meta)data are assigned a globally unique and persistent identifier F2. Data are described with rich metadata (defined by R1 below) F3. Metadata clearly and explicitly include the identifier of the data they describe F4. (Meta)data are registered or indexed in a searchable resource Accessible Once the user finds the required data, they need to know how they can be accessed, possibly including authentication and authorisation. A1. (Meta)data are retrievable by their identifier using a standardised communications protocol A1.1 The protocol is open, free, and universally implementable A1.2 The protocol allows for an authentication and authorisation procedure, where necessary A2. Metadata are accessible, even when the data are no longer available Interoperable The data usually need to be integrated with other data. In addition, the data need to interoperate with applications or workflows for analysis, storage, and processing. I1. (Meta)data use a formal, accessible, shared, and broadly applicable language for knowledge representation I2. (Meta)data use vocabularies that follow FAIR principles I3. (Meta)data include qualified references to other (meta)data Reusable The ultimate goal of FAIR is to optimise the reuse of data. To achieve this, metadata and data should be well-described so that they can be replicated and/or combined in different settings. R1. (Meta)data are richly described with a plurality of accurate and relevant attributes R1.1. (Meta)data are released with a clear and accessible data usage license R1.2. (Meta)data are associated with detailed provenance R1.3. (Meta)data meet domain-relevant community standards The principles refer to three types of entities: data (or any digital object), metadata (information about that digital object), and infrastructure. For instance, principle F4 defines that both metadata and data are registered or indexed in a searchable resource (the infrastructure component). === Acceptance and implementation === Before FAIR, a 2007 OECD report was the most influential paper discussing similar ideas related to data accessibility. In January 2014, the Lorentz Centre at Leiden University hosted a workshop entitled "Jointly designing a data FAIRPORT" where the participants first formulated the FAIR principles. After further discussions, they were published in the March 2016 issue of Scientific Data. At the 2016 G20 Hangzhou summit, the G20 leaders issued a statement endorsing the application of FAIR principles to research. Also in 2016, a group of Australian organisations developed a Statement on FAIR Access to Australia's Research Outputs, which aimed to extend the principles to research outputs more generally. In 2017, Germany, Netherlands and France agreed to establish an international office to support the FAIR initiative, the GO FAIR International Support and Coordination Office. Other international organisations active in the research data ecosystem, such as CODATA or Research Data Alliance (RDA) also support FAIR implementations by their communities. FAIR principles implementation assessment is being explored by FAIR Data Maturity Model Working Group of RDA, CODATA's strategic Decadal Programme "Data for Planet: Making data work for cross-domain challenges" mentions FAIR data principles as a fundamental enabler of data driven science. The Association of European Research Libraries recommends the use of FAIR principles. A 2017 paper by advocates of FAIR data reported that awareness of the FAIR concept was increasing among various researchers and institutes, but also, understanding of the concept was becoming confused as different people apply their own differing perspectives to it. Guides on implementing FAIR data practices state that the cost of a data management plan in compliance with FAIR data practices should be 5% of the total research budget. In 2019 the Global Indigenous Data Alliance (GIDA) released the CARE Principles for Indigenous Data Governance as a complementary guide. The CARE principles extend principles outlined in FAIR data to include Collective benefit, Authority to control, Responsibility, and Ethics to ensure data guidelines address historical contexts and power differentials. The CARE Principles for Indigenous Data Governance were drafted at the International Data Week and Research Data Alliance Plenary co-hosted event, "Indigenous Data Sovereignty Principles for the Governance of Indigenous Data Workshop", held 8 November 2018, in Gaborone, Botswana. The lack of information on how to implement the guidelines have led to inconsistent interpretations of them. In January 2020, representatives of nine groups of universities around the world produced the Sorbonne declaration on research data rights, which included a commitment to FAIR data, and called on governments to provide support to enable it. In 2021, researchers identified the FAIR principles as a conceptual component of data catalog software tools, with the other components being metadata management, business context and data responsibility roles. In April 2022, Matthias Scheffler and colleagues argued in Nature that FAIR principles are "a must" so that data mining and artificial intelligence can extract useful scientific information from the data. There have been moves in the geosciences to establish FAIR data by use of decimal georeferencing However, making data (and research outcomes) FAIR is a challenging task, and it is challenging to assess the FAIRness. In 2020, the FAIR Data Maturity Model Working Group published a set of guidelines for assessing "FAIRness".

  Read more →
• 

  Investigative Data Warehouse

  Investigative Data Warehouse (IDW) is a searchable database operated by the FBI. It was created in 2004. Much of the nature and scope of the database is classified. The database is a centralization of multiple federal and state databases, including criminal records from various law enforcement agencies, the U.S. Department of the Treasury's Financial Crimes Enforcement Network (FinCEN), and public records databases. According to Michael Morehart's testimony before the House Committee on Financial Services in 2006, the "IDW is a centralized, web-enabled, closed system repository for intelligence and investigative data. This system, maintained by the FBI, allows appropriately trained and authorized personnel throughout the country to query for information of relevance to investigative and intelligence matters." == Overview == In 2004, according to a government solicitation for bids to manage the project, it was approximately 10TB in size. In 2005, according to one FBI official, the IDW contained approximately 100 million documents. In 2006 it contained more than 560 million documents and was accessible by more than 12,000 individuals. According to the FBI's website, as of August 22, 2007, the database contained 700 million records from 53 databases and was accessible by 13,000 individuals around the world. As of 2007, the FBI was the subject of a lawsuit brought by the EFF (Electronic Frontier Foundation) because of a lack of public notice describing the database and the criteria for including personal information, as required by the Privacy Act of 1974. The lawsuits were a result of two Freedom of Information Act requests filed by the EFF in 2006. It was built in part by Chiliad corporation, the FBI Office of the Chief Technology Officer, and others. Companies listed on the FOIA files include Northrop Grumman . == Purpose == Investigative Data Warehouse–Secret (IDW-S) "provides data and data processing/analysis services to FBI agents and analysts as they perform counter-terrorism, counter-intelligence, and law enforcement missions". The core subsystem supports the Counter-Terrorism Division (CTD), the Special Event Unit, and via DOCLAB-S, the Joint Intelligence Committee Investigation (JICI) and IntelPlus. According to a 2005 email, "IDW will also be used for criminal and other authorized non-CT investigations as it evolves." (CT being counter terrorism) == Subsystems == Within the system, there were subsystems named IDW-S Core, SPT, and DOCLAB-S The special projects team (SPT): allows for the rapid import of new specialized data sources. These data sources are not made available to the general IDW users but instead are provided to a small group of users who have a demonstrated "need-to-know". The SPT System is similar in function to the IDW-S system, with the main difference is a different set of data sources. The SPT System allows its users to access not only the standard IDW Data Store but the specialized SPT Data Store. == Privacy == According to internal emails, the FBI performed several Privacy Impact Assessments (PIAs) of the IDW system. They worked with lawyers from their National Security Law Branch (NSLB) to attempt to make sure their system was complying with various laws regarding sharing of information and secrecy (for example, rule 6e of the Federal Rules of Criminal Procedure, regarding the secrecy of Grand Jury material ). The Information Sharing Policy Group (ISPG) formed a Discretionary Access Control Team (DACT), to work on "approval of data sets" and "access control requirements" for IDW and DataMart, and responding to other Intelligence Community agencies requesting access. The EFF FOIA IDW website states "Despite the vast amount of personal information contained in the IDW, the FBI has never published a Privacy Act notice describing the system or explaining the ways in which the records might be used." There was also a 2005 email from someone on the Office of General Council (OGC) about "preliminary staff musings that maybe we should limit FBI PIA requirements to non-NS systems" (NS being National Security). There was also an email from 2006 saying that 'national security systems are exempt from E-Gov', apparently referring to the E-Government Act of 2002, which has a section that deals with privacy. == Data sources == The IDW used many data sources. The FOIA documents from EFF are heavily redacted, but some of the sources are as follows: FBI Automated Case Support system (ACS), subset of the Electronic Case File (ECF) system Joint Intelligence Committee Investigation documents (JICI), with OCR text "Open Source News" (public websites, such as the Washington Post and others) Secure Automated Messaging Network (SAMNet) Violent Gang and Terrorist Organizing File (VGTOF) DARPA TIDES program ('open source news' that has been organized and collected) IntelPlus Filerooms, with OCR text FBI National Crime Information Center (NCIC) FBI Records Management Division (RMD), Document Laboratory (DocLab), FBIHQ MiTAP (collects data from public sources, websites, etc.) SPT-Specific data sources (partial list, FOIA files have large parts redacted): Unified Name Index (UNI) extracts Financial Center (FinCen), including Bank Secrecy Act data "Various Sources", including the Transportation Security Administration FBI Counterterrorism Division (CTD) Telephone numbers / addresses from ACS Case data from ACS Terrorist Watch List (TWL) "Other NJTTF data" DoS ... Lost/Stolen Passport data No Fly List, from TSA Selectee list, from TSA ACS/ECF with some case types excluded CIA non-TS/non-SCI Technical Discussions (TDs) and Intelligence Information Reports (IIRs) from 1978 to the May 2004 There was also talk of linking the FTTTF "Data Mart" with IDW. The data in IDW is classified at the 'Secret' level or lower. Higher classifications are not allowed, and can be removed

  Read more →